Introduction
There are several reasons to cross-compile the Linux kernel:-
Compiling on a native CPU and hardware requires a wide range of
devices, which is not so practical. Furthermore, the actual hardware
is often not suitable for the workload of kernel compilation due to
a slow CPU, small memory, etc.
-
Hardware emulation (e.g., with
qemu) can be a viable substitution if the slowness can be tolerated. However, how to setup the compilation environment is another challenge, as it requires at least a preferably Linux-flavored OS, thebashandmakeecosystem, and most importantly, the toolchain.
-
Cross-compiling on a powerful host enables quick detection of kernel
compile errors and config errors, as shown in the
stable queue builds project.
Coupled with emulation, testing on non-native architectures becomes
easier as well.
Kconfig and
Makefile, probably the best way is to actually build the kernel
and dump the verbose version of the build log.Fortunately, with a modern Linux release (like Ubuntu or Fedora), all we need to cross-compile the kernel is a toolchain that can produce binaries for another CPU architecture.
Toolchain
On cross-compiling the kernel, we only need two things:binutils,
and gcc, and there are generally two ways to obtain them: - install pre-compiled packages
- build from source
apt-get from the official repositories.
For example, apt-get install binutils-aarch64-linux-gnu gcc-aarch64-linux-gnu
However, in case you want to compile the toolchain from source, for whatever reasons you might have, such as using the latest version or a customized version of
gcc, you might follow the following steps.As a head up, I am running a standard Ubuntu 16.04.3 LTS release with kernel version 4.4.0. The host
gcc and binutils (i.e., the gcc
and binutils used to build the toolchain) are the default ones with the
Ubuntu release, which is in version 5.4.0 and 2.27 respectively.
However, I don't see major obstacles in applying it to other combinations
of OS, gcc, and binutils versions as long as they are recent enough
to build the toolchain.Before diving into the building process, let's setup some environment variables. Part of them are for convenience reasons, while others are necessary in the build process.
export TARGET=aarch64-unknown-linux-gnu # replace with your intended target
export PREFIX="$HOME/opt/cross/aarch64" # replace with your intended path
export PATH="$PREFIX/bin:$PATH"
TARGET variable should be a target triplet which is used by the
autoconf system. Their valid values can be found in the
config.guess script.The last step is necessary. By adding the installation prefix to the the
PATH of the current shell session, we ensure the gcc is able to detect
our new binutils once we have built them.binutils
The source code forbinutils can be obtained from the GNU servers:# for stable releases
BINUTILS_VERSION=2.29.1
wget ftp://ftp.gnu.org/gnu/binutils/binutils-$BINUTILS_VERSION.tar.xz
# for development branch
git clone git://sourceware.org/git/binutils-gdb.git
cd $BINUTILS_BUILD
$BINUTILS_SOURCE/configure \
--target=$TARGET \
--prefix=$PREFIX \
--with-sysroot \
--disable-nls \
--disable-werror
make
make install
--disable-nls tells
binutils not to include native language
support. This is optional, but reduces dependencies and compile time.--with-sysroot tells
binutils to enable sysroot support in
the cross-compiler by pointing it to a default empty directory.
By default the linker refuses to use sysroots for no good technical reason,
while gcc is able to handle both cases at runtime. --disable-werror behaves exactly as the name suggests, do not add
-Werror in the flag.After installation, the binaries
aarch64-unknown-linux-gnu-{as/ar/ld/...}
should exist in $PATH.These instructions apply to
binutils version 2.29.1, which is the latest
release at the time of writing. gcc
Similar tobinutils. the source code for gcc can be obtained from the
GNU servers too:# for stable releases
GCC_VERSION=7.2.0
wget ftp://ftp.gnu.org/gnu/gcc/gcc-$GCC_VERSION/gcc-$GCC_VERSION.tar.xz
# for development branch
git clone git://gcc.gnu.org/git/gcc.git
cd $GCC_SOURCE
./contrib/download_prerequisites
cd $GCC_BUILD
$GCC_SOURCE/configure \
--target=$TARGET \
--prefix=$PREFIX \
--enable-languages=c,c++ \
--without-headers \
--disable-nls \
--disable-shared \
--disable-decimal-float \
--disable-threads \
--disable-libmudflap \
--disable-libssp \
--disable-libgomp \
--disable-libquadmath \
--disable-libatomic \
--disable-libmpx \
--disable-libcc1
make all-gcc
make install-gcc
gcc needs these packages for compilation. This is handled
seamlessly with the download_prerequisites script. --enable-languages=c,c++ tells
gcc not to build frontends
for other languages like Fortran, Java, etc.--without-headers tells
gcc not to rely on any C library
being present for the target.--disable-<package> tells
gcc not to build those packages
as they will not be needed in kernel compilation.More importantly, we should not simply
make all as that would
build way too much (for example, the libgcc, libc, libstdc++,
etc), which are not needed at all. All we need is the compiler
itself which can be built by make all-gcc.Another important note is that in order for
gcc to lookup the
correct set of binutils, both $TARGET and $PREFIX must be
exactly the same when configuring binutils and gcc.
For example, aarch64-unknown-linux-gnu-gcc will lookup
aarch64-unknown-linux-gnu-as in the same directory:
merely putting aarch64-unknown-linux-gnu-as in $PATHis not enough.The instructions apply to
gcc version 7.2.0, which is the latest
release at the time of writing.After the whole process, the following files should present in the
$PREFIX/bin directory, and also in the $PATH.aarch64-unknown-linux-gnu-addr2line
aarch64-unknown-linux-gnu-ar
aarch64-unknown-linux-gnu-as
aarch64-unknown-linux-gnu-c++
aarch64-unknown-linux-gnu-c++filt
aarch64-unknown-linux-gnu-cpp
aarch64-unknown-linux-gnu-elfedit
aarch64-unknown-linux-gnu-g++
aarch64-unknown-linux-gnu-gcc
aarch64-unknown-linux-gnu-gcc-7.2.0
aarch64-unknown-linux-gnu-gcc-ar
aarch64-unknown-linux-gnu-gcc-nm
aarch64-unknown-linux-gnu-gcc-ranlib
aarch64-unknown-linux-gnu-gcov
aarch64-unknown-linux-gnu-gcov-dump
aarch64-unknown-linux-gnu-gcov-tool
aarch64-unknown-linux-gnu-gprof
aarch64-unknown-linux-gnu-ld
aarch64-unknown-linux-gnu-ld.bfd
aarch64-unknown-linux-gnu-nm
aarch64-unknown-linux-gnu-objcopy
aarch64-unknown-linux-gnu-objdump
aarch64-unknown-linux-gnu-ranlib
aarch64-unknown-linux-gnu-readelf
aarch64-unknown-linux-gnu-size
aarch64-unknown-linux-gnu-strings
aarch64-unknown-linux-gnu-strip
Kernel Build
With the toolchain ready, cross-compiling the kernel involves two extra steps:- Find the architecture name (
$KERNEL_ARCH) in kernel source tree- they are typically Located in
arch/*in the kernel source tree.
- they are typically Located in
- Find the configuration (
$KERNEL_CONF) for each sub-arch (e.g., 32-bit vs 64-bit, big endian vs little endian, etc)make ARCH=$KERNEL_ARCH helpwill show some hints.- if there is no specific machine config, the
defconfigshould work. - if there are available machine configs, choosing from an existing
config seems to be more hassle free compared with doing a manual
menuconfig.
$KERNEL_ARCH and $KERNEL_CONF,
cross-compiling the kernel is as easy as the following:cd $KERNEL_SOURCE
make ARCH=$KERNEL_ARCH O=$KERNEL_BUILD $KERNEL_CONF
cd $KERNEL_BUILD
make ARCH=$KERNEL_ARCH CROSS_COMPILE=$TARGET- V=1 vmlinux modules
cd $KERNEL_SOURCE
make ARCH=arm64 O=$KERNEL_BUILD defconfig
cd $KERNEL_BUILD
make ARCH=arm64 CROSS_COMPILE=aarch64-unknown-linux-gnu- V=1 vmlinux modules
make -j flags.The instructions apply to Linux kernel version 4.13.5, which is the latest release at the time of writing.
Results
In total, I have currently cross-compiled the kernel for 7 architectures and 12 sub-archs, with the procedure described above. The result is summarized in the following table:| Architecture | Name | $TARGET |
$KERNEL_ARCH |
$KERNEL_CONF |
|---|---|---|---|---|
| x86 (32-bit) | i386 | i386-pc-linux-gnu | x86 | i386_defconfig |
| x86 (64-bit) | x86_64 | x86_64-pc-linux-gnu | x86 | x86_64_defconfig |
| arm (32-bit) | arm | armv7-unknown-linux-gnueabi | arm | multi_v7_defconfig |
| arm (64-bit) | aarch64 | aarch64-unknown-linux-gnu | arm64 | defconfig |
| powerpc (32-bit) | ppc | powerpcle-unknown-linux-gnu | powerpc | pmac32_defconfig |
| powerpc (64-bit) | ppc64 | powerpc64le-unknown-linux-gnu | powerpc | ppc64le_defconfig |
| sparc (32-bit) | sparc | sparc-unknown-linux-gnu | sparc | sparc32_defconfig |
| sparc (64-bit) | sparc64 | sparc64-unknown-linux-gnu | sparc | sparc64_defconfig |
| mips (32-bit) | mips | mips-unknown-linux-gnu | mips | 32r6_defconfig |
| mips (64-bit) | mips64 | mips64-unknown-linux-gnu | mips | 64r6_defconfig |
| s390x (64-bit) | s390x | s390x-ibm-linux-gnu | s390 | default_defconfig |
| ia64 (64-bit) | ia64 | ia64-unknown-linux-gnu | ia64 | generic_defconfig |
allyesconfig kernel and succeeded in 7
architectures, as shown in the following table.
Unfortunately, ia64 failed miserably with error message
Error: Operand 2 of 'adds' should be a 14-bit integer (-8192-8191),
and the error seems to have been there for
more than 7 months.| Architecture | Name | $TARGET |
$KERNEL_ARCH |
$KERNEL_CONF |
|---|---|---|---|---|
| x86 (64-bit) | x86_64 | x86_64-pc-linux-gnu | x86 | allyesconfig |
| arm (32-bit) | arm | armv7-unknown-linux-gnueabi | arm | allyesconfig |
| arm (64-bit) | aarch64 | aarch64-unknown-linux-gnu | arm64 | allyesconfig |
| powerpc (64-bit) | ppc64 | powerpc64-unknown-linux-gnu | powerpc | allyesconfig |
| sparc (64-bit) | sparc64 | sparc64-unknown-linux-gnu | sparc | allyesconfig |
| mips (64-bit) | mips64 | mips64-unknown-linux-gnu | mips | allyesconfig |
| s390x (64-bit) | s390x | s390x-ibm-linux-gnu | s390 | allyesconfig |
$TARGET, $KERNEL_ARCH, and $KERNEL_CONF into the scripts
above and test.
periodically checks the health of the spacecraft systems and software.
processes image data.
performs an occasional test on equipment status.
–Normal
Execution 
– Execution
in Critical Section 
– Unbounded
Priority Inversion