Cross Compiling Node.js for ARM on Ubuntu

Node.js is available for many different operating systems and architectures. We can run it on our Macbooks locally, on full server grade systems, and on IoT devices with ease. The Node Foundation publishes binary tarballs for a broad spectrum of target platforms. Over at NodeSource we package those builds up for Linux to make them easy to install. However, building Node.js for architectures other than Intel x86 or x64 can be very time-consuming. A clean build of a release from the 10.x line on a Raspberry Pi 3 takes around five hours. It would be nicer if we could cross compile from a beefy Intel machine with the target being the ARM chips that are used in the Raspberry Pi. This is possible, but not as easy as I would have thought when using a recent 64bit Ubuntu system. I spent the better part of my Friday night figuring out exactly how to do it, so I thought I’d share what I learned so you don’t have to jump through all of the hoops I did.

Challenges With Cross Compiling Node.js

In theory cross compiling is easy. You install the relevant cross compiler(s), point them at your source code, and do something like

% CC=<your C cross compiler> CXX=<your C++ cross compiler> ./configure && make

and you should be good to go. Unfortunately, the situation with Node.js isn’t as straightforward. As part of its own build process, Node.js makes some binaries that need to be executed along the way. Consequently, if you build those tools with a cross compiler for ARM and try to run them on the Intel machine they’re sitting on, the build fails at that point because of the architecture mismatch.

There are tools such as sbuild that make it easy to get around that problem, but they rely on emulation which makes the build process extremely slow again. Fortunately, the Node.js project has given us some other options.

Built-in Cross Compiling Features

Due to the inability to build Node.js without having to run internal executables, the Node project gives us some tricks to help us out. I wasn’t able to find much documentation about them, but the environment variables

CC
CXX
CC_host
CXX_host

all affect the build process. They allow you to specify different compilers for the target and host architectures, in turn allowing the system to make the local executables runnable on the host machine. If it seems like you’re all set to cross compile at this point, you’re close to correct. But not quite.

Challenges With Cross Compiling on Ubuntu

Ubuntu has been my go-to Linux distribution for years now. One of its greatest strengths is the amount of software that’s available for it, and cross compilers are no exception. However, the sheer number of available compilers effectively creates its own set of issues. Essentially, it can be tricky to get the needed header files where the various compilers need them to be in order for all of this to work. I’m going to spare you the yak shaving I had to go through to get this part sorted out, but if you want to read all about it, this bug report is where I’d start.

The crux of the solution is that to cross compile for armhf, which is a 32bit architecture, from a 64bit Intel host, you need to install some 32bit Intel libraries and invoke the host compiler in 32bit mode. We’ll see exactly what do to in the next few sections.

Packages to Install

On a standard 64bit installation of Ubuntu Bionic, we need to install some 32bit libraries as well as the ARM C and C++ compilers. First, we need to add the 32bit architecture to dpkg with the command

% sudo dpkg --add-architecture i386

Next up is installing the libraries, utilities, and compilers that we’ll need.

% sudo apt install build-essential \
> binutils-multiarch \
> libc6-dev:i386 \
> libstdc++-7-dev:i386 \
> gcc-arm-linux-gnueabihf \
> g++-arm-linux-gnueabihf \
> gcc-7-multilib \
> gcc-7-multilib-arm-linux-gnueabihf \
> g++-7-multilib \
> g++-7-multilib-arm-linux-gnueabihf

This should get everything you need onto your system. Take note that the “7” in packages such as gcc-7-multilib corresponds to the major version of the default installation of gcc on the host machine. If you’re reading this in the future, you may well want to change this to “8”, or whatever gcc version your distribution defaults to.

Configuring the Cross Compile Job

Now that we have everything installed, it’s time to set up the build. At the time of this writing, the newest supported Node.js release is 10.9.0, so we’ll build that. First, download the source and put it in a directory to build things in. Something like this should work:

% mkdir ~/NodeBuild && cd ~/NodeBuild
% mkdir install
% curl -RLO https://nodejs.org/dist/v10.9.0/node-v10.9.0.tar.xz
% tar xvJf node-v10.9.0.tar.xz
% cd node-v10.9.0

Next, we need to run the configure script using the environment variables we noted above to set up things for cross compilation. Assuming that the target is for an armhf system such as the aforementioned Raspberry Pi 3, this should work:

% CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ CC_host="gcc -m32" CXX_host="g++ -m32" \
> ./configure --prefix=../install --dest-cpu=arm --cross-compiling --dest-os=linux \
> --with-arm-float-abi=hard --with-arm-fpu=neon

It’s critical to note that we invoke the host compilers with the -m32 flag. Doing this invokes the local gcc in 32bit mode, which consequently makes everything find and use the 32bit headers as they are supposed to. The resulting build will install into the cleverly named install directroy that we created parallel to the source code directory. The output JSON from running this on my machine looks like this:

{ 'target_defaults': { 'cflags': [],
                       'default_configuration': 'Release',
                       'defines': [],
                       'include_dirs': [],
                       'libraries': []},
  'variables': { 'arm_float_abi': 'hard',
                 'arm_fpu': 'neon',
                 'arm_thumb': 0,
                 'arm_version': '7',
                 'asan': 0,
                 'build_v8_with_gn': 'false',
                 'coverage': 'false',
                 'debug_nghttp2': 'false',
                 'enable_lto': 'false',
                 'force_dynamic_crt': 0,
                 'gas_version': '2.30',
                 'host_arch': 'ia32',                 'icu_data_in': '../../deps/icu-small/source/data/in/icudt62l.dat',
                 'icu_endianness': 'l',
                 'icu_gyp_path': 'tools/icu/icu-generic.gyp',
                 'icu_locales': 'en,root',
                 'icu_path': 'deps/icu-small',
                 'icu_small': 'true',
                 'icu_ver_major': '62',
                 'llvm_version': 0,
                 'node_byteorder': 'little',
                 'node_debug_lib': 'false',
                 'node_enable_d8': 'false',
                 'node_enable_v8_vtunejit': 'false',
                 'node_install_npm': 'true',
                 'node_module_version': 64,
                 'node_no_browser_globals': 'false',
                 'node_prefix': '../build',
                 'node_release_urlbase': '',
                 'node_shared': 'false',
                 'node_shared_cares': 'false',
                 'node_shared_http_parser': 'false',
                 'node_shared_libuv': 'false',
                 'node_shared_nghttp2': 'false',
                 'node_shared_openssl': 'false',
                 'node_shared_zlib': 'false',
                 'node_tag': '',
                 'node_target_type': 'executable',
                 'node_use_bundled_v8': 'true',
                 'node_use_dtrace': 'false',
                 'node_use_etw': 'false',
                 'node_use_openssl': 'true',
                 'node_use_pch': 'false',
                 'node_use_perfctr': 'false',
                 'node_use_v8_platform': 'true',
                 'node_with_ltcg': 'false',
                 'node_without_node_options': 'false',
                 'openssl_fips': '',
                 'openssl_no_asm': 0,
                 'shlib_suffix': 'so.64',
                 'target_arch': 'arm',
                 'v8_enable_gdbjit': 0,
                 'v8_enable_i18n_support': 1,
                 'v8_enable_inspector': 1,
                 'v8_no_strict_aliasing': 1,
                 'v8_optimized_debug': 0,
                 'v8_promise_internal_field_count': 1,
                 'v8_random_seed': 0,
                 'v8_trace_maps': 0,
                 'v8_typed_array_max_size_in_heap': 0,
                 'v8_use_snapshot': 'true',
                 'want_separate_host_toolset': 1}}

You’ll note I’ve highlighted that host_arch is ia32. Again, this is important so that everything finds the 32bit headers, even though the host OS is actually 64bit.

From here a simple

% make && make install

should cross compile Node.js at machine native speed, and put everything in your install directory. The results look like this for me:

% cd ~/NodeBuild/install
% ls
bin  include  lib  share
% cd bin
% file node
node: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=6cad42017fc1be2138aee900f0219890584268c7, with debug_info, not stripped

Success! We’ve cross compiled a 32bit ARM binary from our 64bit Intel machine. You can now go crack open a cold one, as you’ve earned it. However, if you’d like just a bit more cross compiling fun, you can keep going to the next section.

Changes for 64bit ARM

If your ARM-fu is strong enough that you’ve moved past armhf then I applaud your mojo. If you have a 64bit ARM machine and you want to build Node.js for it, the process is very similar to what we just did. As before, step one is to install the needed compilers and aarch64 utilities:

% sudo apt install binutils-aarch64-linux-gnu \
> gcc-aarch64-linux-gnu \
> g++-aarch64-linux-gnu

Next, create a clean source code and install directory as we did earlier. Once you’re in there, the configure command is similar, though with some key changes:

% CC=aarch64-linux-gnu-gcc CXX=aarch64-linux-gnu-g++ CC_host="gcc" CXX_host="g++" \
> ./configure --prefix=../install --dest-cpu=arm64 --cross-compiling --dest-os=linux \
> --with-arm-float-abi=hard --with-arm-fpu=neon

The key changes are that we use the aarch64 cross compilers, we don’t append the -m32 flags to the host compilers because we’re doing everything in 64bit mode here, and the --dest-cpu flag has changed from arm to arm64.

From here, another

% make && make install

will build and install everything as before, but with a 64bit aarch64 binary as the result.

In Conclusion

With a bit of work, you can set things up on a standard 64bit Ubuntu installation to compile things for a lot of different targets. Building Node.js for alternate architectures is a bit tricker than normal due to how its build process works, but it’s perfectly doable. I don’t know if these instructions will be valid or useful for other distros, as some of the hurdles here were tied to how Ubuntu toolchains are packaged up, but if you’re using a new-ish Ubuntu (or, I expect Debian) release, this should serve to get you on your way. Happy building!

Also published on Medium.