Tools for building Rump Kernels

The buildrump.sh script builds unmodified NetBSD kernel drivers such as file systems and the TCP/IP stack as components which can be linked to form rump kernels. These lightweight rump kernels run on top of a high-level hypercall interface which is straightforward to implement for most environments. This repository includes the hypercall implementation for running in userspace on POSIX hosts, with alternative implementations such as for the Xen hypervisor and the Linux kernel being hosted elsewhere.

In other words, rump kernels enable embedding unmodified kernel drivers in various environments and using the drivers as services. Some examples of how to use these services are as follows:

• fs-utils uses file system drivers to provide applications for accessing file system images
• TCP/IP stack for the DPDK and netmap userspace packet processing frameworks
• ljsyscall provides a Lua interface to rump kernels, allowing easy access from applications written in Lua
• rumpuser-xen enables running applications as standalone Xen DomU's (includes libc support)

For full details on rump kernels, read http://www.NetBSD.org/docs/rump/ and follow the links. For a video overview including various demos, watch the FOSDEM 2013 presentation.

Discuss buildrump.sh on [email protected] (subscribe before posting), or join #rumpkernel on irc.freenode.net.

Running buildrump.sh requires a network connection for fetching NetBSD kernel driver source code. Self-contained archives can be created using the tarup.sh script, and snapshots are available for download from here.

Installation Instructions

The easiest way to install rump kernel components is to use a binary package for your OS/distribution/architecture. These packages will also install the POSIX hypercall implementation.

• Void Linux: xbps-install -S netbsd-rumpkernel
• Arch Linux: pacman (OBS), AUR
• OpenSUSE Linux: 12.3 RPM (OBS) || Tumbleweed RPM (OBS) || Factory RPM (OBS) || SLE_11_SP2 RPM (OBS)
• Fedora Linux: 17 RPM (OBS) || 18 RPM (OBS) || RHEL 6 RPM (OBS) || CentOS 6 RPM (OBS)
• Mandriva Linux 2011: RPM (OBS)
• Debian Linux: 7 DEB (OBS)
• Ubuntu Linux: 13.04 DEB (OBS)
• NetBSD: pkgsrc/misc/rump
• DragonFly BSD: pkgsrc/misc/rump
• Solaris: pkgsrc/misc/rump

The links for some of packages are provided by the openSUSE Build Service. You can download and install the packages manually, but it is highly recommended to add the OBS repositories for the right distro and architecture to the package manager. This way, updates and dependencies will be automatically resolved for other packages depending on rump kernels.

Building from Source Code

Building from source may be necessary of there are no binary packages for your systems, or if you wish to make source level modifications to the rump kernel components.

Build dependencies

The following are required for building from source:

• cc (gcc and clang are known to work)
• ld (GNU or Solaris ld required)
• binutils (ar, nm, objcopy)

The short version

Clone the repository and run:

• ./buildrump.sh

You will now find the kernel drivers and necessary headers in ./rump ready for use. Examples on how to use the resulting drivers are available in the tests and examples directories.

The long(er) version

When run without parameters, buildrump.sh implicitly assumes that the given commands were checkout fullbuild tests. You can override this default by giving explicit commands.

The checkout command will fetch the necessary subset of the NetBSD source tree from github into ./src (the location can be changed using the -s parameter). You are free to use any method for fetching NetBSD sources, though the only officially supported way is to use the checkout command. Note that the NetBSD sources and their timestamps may vary from one buildrump.sh revision to another. By default, the script checks that you have the appropriate set of sources even if you do not run checkout.

The fullbuild command will then instruct the script to to build the necessary set of tools for building rump kernels, e.g. the BSD version of make, after which it will build the rump kernels. By default, cc from path is used along with other host tools such as nm. Crosscompilation is documented further below.

If the command tests is given, the script will run simple tests to check that e.g. file systems and the TCP/IP stack work correctly. If everything was successfully completed, the final output from the script is "buildrump.sh ran successfully". Note that tests cannot be run when buildrump.sh is used with a crosscompiler or in kernel-only mode (see below).

To learn more about command line parameters, run the buildrump.sh script with the -h flag.

Crosscompiling

If the environment variable $CC is set, its value is used as the compiler instead of cc. This allows not only to select between compiling with gcc or clang, but also allows to specify a crosscompiler. If $CC is set and does not contain the value cc, gcc, or clang the script assumes a crosscompiler and will by default use tools with names based on the target of $CC with the format target-tool (e.g. target-nm).

Crosscompiling for an ARM system might look like this (first command is purely informational):

$ arm-linux-gnueabihf-gcc -dumpmachine
arm-linux-gnueabihf
$ env CC=arm-linux-gnueabihf-gcc ./buildrump.sh [params]

Since the target is arm-linux-gnueabihf, arm-linux-gnueabihf-nm etc. must be found from $PATH. The assumption is that the crosscompiler can find the target platform headers and libraries which are required for building the hypercall library. You can override the defaults by setting $AR, $NM and/or $OBJCOPY in the environment before running the script.

Kernel-only mode

If the -k kernel-only parameter is specified, the script will omit building the POSIX hypercall implementation. This is useful if you are developing your own hypercall layer implementation. See the rumpuser-xen repository for the canonical example of using -k.

Tested hosts

Continuous testing on Linux/amd64 + gcc/clang is done by Travis CI for every commit.

There is a broader platform CI testing for Linux x86 (32/64bit), ARM, PowerPC (32/64bit), MIPS (o32 ABI) and NetBSD x86 (32/64bit) using buildbot.

Tested machine architectures include x86 (32/64bit), ARM, PowerPC (32/64bit), MIPS (32bit) and UltraSPARC (32/64bit).

Examples of hosts buildrump.sh has manually been tested on are as follows:

• Linux

  • Linux Gallifrey 2.6.35.14-106.fc14.x86_64 #1 SMP Wed Nov 23 13:07:52 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux (with seLinux in permissive mode, amd64)
  • Linux vyrnwy 3.6.2-1.fc16.x86_64 #1 SMP Wed Oct 17 05:30:01 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux (Fedora release 16 with read-only /usr/src via NFS)
  • Linux void-rpi 3.6.11_1 #1 PREEMPT Tue Feb 19 17:40:24 CET 2013 armv6l GNU/Linux (Void, Raspberry Pi, evbarm)
  • Linux braniac 3.9.9-1-ARCH #1 SMP PREEMPT Wed Jul 3 22:45:16 CEST 2013 x86_64 GNU/Linux (Arch Linux, amd64, gcc 4.8.1)
  • Linux pike 3.6.7-4.fc17.ppc64 #1 SMP Thu Dec 6 06:41:58 MST 2012 ppc64 ppc64 ppc64 GNU/Linux (Fedora, ppc64)
  • Linux 172-29-171-95.dal-ebis.ihost.com 2.6.32-358.el6.ppc64 #1 SMP Tue Jan 29 11:43:27 EST 2013 ppc64 ppc64 ppc64 GNU/Linux (RHEL6, ppc64, 64 and 32 bit builds, IBM Virtual Loaner Program)
  • Linux fuloong 3.11.6-gnu #8 PREEMPT Mon Oct 28 23:28:22 GMT 2013 mips64 ICT Loongson-2 V0.3 FPU V0.1 lemote-fuloong-2f-box GNU/Linux (Gentoo, mips o32)

• DragonFly BSD

  • DragonFly 3.2-RELEASE DragonFly v3.2.1.9.g80b03f-RELEASE #2: Wed Oct 31 20:17:57 PDT 2012 [email protected]:/usr/obj/build/home/justin/src/sys/GENERIC i386

• FreeBSD

  • FreeBSD frab 9.1-PRERELEASE FreeBSD 9.1-PRERELEASE #5 r243866: Wed Dec 5 02:15:02 CET 2012 root@vetinari:/usr/obj/usr/src/sys/RINCEWIND amd64 (static rump kernel components, with thanks to Philip for test host access)

• NetBSD

  • NetBSD pain-rustique.localhost 5.1_STABLE NetBSD 5.1_STABLE (PAIN-RUSTIQUE) #5: Wed Feb 16 13:34:14 CET 2011 [email protected]:/objs/kobj.i386/PAIN-RUSTIQUE i386

• Solaris

  • SunOS hutcs 5.10 Generic_142900-15 sun4v sparc SUNW,T5240 Solaris (needs xpg4/bin/sh, sparc64 in 64bit mode, sparc in 32bit mode)
  • SunOS pkgsrc-dev 5.11 joyent_20120126T071347Z i86pc i386 i86pc (with thanks to Jonathan for test host access, amd64 in 64bit mode, i386 in 32bit mode)

There is also initial support for Cygwin, but it will not work out-of-the-box due to object format issues (ELF vs. PE-COFF). Mac OS X is likely to require support for its linker.

Tips for advanced users

• Place your buildtools in a separate directory, e.g. $HOME/rumptools using ./buildrump.sh -T $HOME/rumptools fullbuild. Put that directory in $PATH. You can now do fast build iteration for kernel components by going to the appropriate directory and running rumpmake dependall && rumpmake install.

• You can list the NetBSD source dates used by ./buildrump.sh checkout by running ./checkout.sh listdates.

• Assuming you have a commit bit to NetBSD, you can use HEAD from NetBSD src and be able to commit your changes to NetBSD from src with the following setup:

  • [email protected]:/cvsroot ./checkout.sh cvs nbcvs HEAD
  • ./buildrump.sh -s nbcvs fullbuild

  Of course, replace dev with your NetBSD account name. Equally "of course", this operating mode is not officially supported by buildrump.sh. However, if you run into problems that will affect buildrump.sh after the checkout date is bumped, report the problems using your discretion.

• You can override the compiler optimization flags by setting BUILDRUMP_DBG in the env before running the script. For example, BUILDRUMP_DBG=-Os ./buildrump.sh will build with -Os instead of the default -O2 -g.