From 4dc685647e461ec367de36baa67fe12dee87249f Mon Sep 17 00:00:00 2001 From: storypku Date: Wed, 2 Dec 2020 21:46:21 +0800 Subject: [PATCH] Docs: updated "How to Install Low-Latency/Realtime Kernel" Added workaround to build Nvidia driver for PREEMPT_RT kernel --- docs/howto/how_to_install_apollo_kernel.md | 126 +++++++++++++-------- 1 file changed, 80 insertions(+), 46 deletions(-) diff --git a/docs/howto/how_to_install_apollo_kernel.md b/docs/howto/how_to_install_apollo_kernel.md index 9c5a88a8e8a..1dbbee7e5d1 100644 --- a/docs/howto/how_to_install_apollo_kernel.md +++ b/docs/howto/how_to_install_apollo_kernel.md @@ -1,57 +1,38 @@ -# How to Install Low-Latency Kernel for Apollo +# How to Install Low-Latency / Realtime Kernel -This document describes the steps to install stock Ubuntu low-latency kernel -with Nvidia driver on Ubuntu 18.04.x. +This document describes the steps to install low-latency/realtime kernel with +Nvidia driver on Ubuntu 18.04. ## Before You Begin -There was once an -[Apollo Kernel](https://github.com/ApolloAuto/apollo-kernel.git) project based -on the Linux 4.4.32 source tree, which provides real-time kernel support for -running Apollo software stack in the vehicle. However, that project hasn't been -updated for long, causing a lot of trouble for the community. The Apollo team -thus decided to find alternative ways to provide low-latency/real-time kernel -support, which, should be easy to migrate to newer Linux kernels. +Low-Latency/Realtime kernel is only required for running Apollo software stack +in the vehicle. If your sole purpose was to develop/test your algorithms or to +run simulation software (e.g., LGSVL simulator) on top of Apollo, then maybe you +don't need to install low-latency/real-time kernels described at all. -The low-latency alternative they found was stock Ubuntu low-latency kernel -(available in Ubuntu repositories). Ubuntu low-latency kernel is completely -capable of low- to no- latency for running Apollo in the vehicle. Preempt -optimization is enabled in kernel configuration, and latency as low as 0.1 -millisecond can and has been achieved using it. +## Ubuntu Low-Latency Kernel -In the section below, we will describe the steps to install the low latency -kernel on Ubuntu. +Ubuntu low-latency kernel (availabe in Ubuntu repository) is completely capable +of low- to no- latency for running Apollo in the vehicle. Preempt optimization +is enabled in its kernel configuration, and latency as low as 0.1 millisecond +can and has been achieved using it. -**Note**: - -> If your sole purpose was to develop/test on Apollo platform, or to run -> simulation software (e.g., LGSVL simulator), then you don't need to install -> low-latency/real-time kernels at all. - -## Installing Ubuntu Low-Latency Kernel - -The steps required to install low-latency kernel on Ubuntu is as follows: +The steps required to install Ubuntu low-latency kernel are described below: 1. Install the latest low-latency kernel and its headers. ```bash -# Sync package index with upstream sources.list sudo apt-get update - -# Print your current kernel release -uname -r - -# Check if newer Linux image is available. -apt-cache search linux-image | grep "Signed kernel image lowlatency" - # Install the latest low-latency kernel & headers. sudo apt-get install linux-image-$(uname -r)-lowlatency linux-headers-$(uname -r)-lowlatency ``` **Note**: -> Please change `$(uname -r)` to the latest kernel image if availabe. The latest -> kernel version on Ubuntu 18.04 as of this writing (Nov 17, 2020) is `5.4.0-54` +> Please change `$(uname -r)` to the latest kernel should there be newer kernel +> packages available. (You can view newer packages availabe with +> `apt list --upgradable`.) The latest kernel version on Ubuntu 18.04 as of this +> writing (Dec 02, 2020) is `5.4.0-56`. 2. Reboot to start the low-latency kernel. @@ -59,8 +40,21 @@ sudo apt-get install linux-image-$(uname -r)-lowlatency linux-headers-$(uname -r sudo reboot ``` +## Install Realtime Kernel + +There is a community-contributed +[Building Realtime Linux for ROS2](https://index.ros.org/doc/ros2/Tutorials/Building-Realtime-rt_preempt-kernel-for-ROS-2) +document for building/installing PREEMPT_RT kernel on Ubuntu 20.04, which is +applicable to Ubuntu 18.04 also. Please follow the instructions there to install +the latest stable realtime kernel first. + ## Install Nvidia Driver +### Install Nvidia Driver for Low-Latency Kernel + +For Ubuntu low-latency kernel, the steps to install Nvidia driver is relatively +straightforward. + 1. Download and install the latest Nvidia driver from the [CUDA Toolkit Downloads](https://developer.nvidia.com/cuda-downloads?target_os=Linux) Page. @@ -75,7 +69,7 @@ sudo reboot **Note** -> You may need to regist and sign CUDA EULA before continue. +> You may need to regist and sign CUDA EULA to download Nvidia packages. For example, below is the instructions installing Nvidia driver on x86_64 Ubuntu 18.04.5 with the "deb[local]" approach: @@ -99,15 +93,55 @@ sudo apt-get install nvidia-driver-455 3. Run `nvidia-smi` to check if everything is OK. -## Build and Install ESD-CAN Driver (Optional) +### Install Nvidia Driver for Realtime Kernel -You should follow the -[instructions](https://github.com/ApolloAuto/apollo-kernel/blob/master/linux/ESDCAN-README.md#build--install-out-of-tree-esd-kernel-driver) -to build ESD-CAN driver if necessary. +Please follow the first step listed above first. However, as the Nvidia driver +does not support real-time kernels, running +`sudo apt-get install nvidia-driver-455` on a PREEMPT_RT kernel should partially +fail with the following error message: -## Final Words +```text +The kernel you are installing for is a PREEMPT_RT kernel! -In this article, we described briefly the steps to install Ubuntu low-latency -kernel and integrate Nvidia GPU driver with it. This kernel should be sufficient -for Apollo runtime in vehicle. However, we haven't fully test it on real -vehicles. Please use it with caution and report any issues to us. +The NVIDIA driver does not support real-time kernels. If you +are using a stock distribution kernel, please install +a variant of this kernel that does not have the PREEMPT_RT +patch set applied; if this is a custom kernel, please +install a standard Linux kernel. Then try installing the +NVIDIA kernel module again. + +*** Failed PREEMPT_RT sanity check. Bailing out! *** +``` + +We can set `IGNORE_PREEMPT_RT_PRESENCE=1` when build Nvidia driver as a +workaround: + +1. Run the following commands to build Nvidia driver: + +```bash +# Change to Nvidia driver source directory +cd "$(dpkg -L nvidia-kernel-source-455 | grep -m 1 "nvidia-drm" | xargs dirname)" + +# Build Nvidia driver with IGNORE_PREEMPT_RT_PRESENCE=1 +sudo env NV_VERBOSE=1 \ + make -j8 NV_EXCLUDE_BUILD_MODULES='' \ + KERNEL_UNAME=$(uname -r) \ + IGNORE_XEN_PRESENCE=1 \ + IGNORE_CC_MISMATCH=1 \ + IGNORE_PREEMPT_RT_PRESENCE=1 \ + SYSSRC=/lib/modules/$(uname -r)/build \ + LD=/usr/bin/ld.bfd \ + modules + +sudo mv *.ko /lib/modules/$(uname -r)/updates/dkms/ +sudo depmod -a +``` + +2. Reboot the system +3. Run `nvidia-smi` to check if everything is OK. + +## Install ESD-CAN Driver (Optional) + +You should follow the +[instructions](https://github.com/ApolloAuto/apollo-kernel/blob/master/linux/ESDCAN-README.md) +to build ESD-CAN driver if necessary.