Skip to content

Commit

Permalink
script: Support installation for kernel with or without XDP_TBS
Browse files Browse the repository at this point in the history
Changes List:
- Support installation for kernel with or without XDP_TBS
  for libbpf and open62541
- Add helpers.sh to support the installers
- Update DEPENDENCIES.md

Signed-off-by: Goh Wei Sheng <[email protected]>
  • Loading branch information
ws-intel committed Nov 16, 2022
1 parent a4a01ee commit 581ca86
Show file tree
Hide file tree
Showing 30 changed files with 2,187 additions and 195 deletions.
17 changes: 14 additions & 3 deletions DEPENDENCIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,16 +40,27 @@ In order to compile under Ubuntu, these packages need to be installed first:
In order to ease the installation of the customized helper libraries needed, we have included a few installation scripts in a specific branch.
These scripts will check out a specific version of the libraries from upstream git and apply our patches on top of it.

Branch name : **experimental/dependencies_installer**
The dependencies installer branch : https://github.com/intel/iotg_tsn_ref_sw/tree/experimental/dependencies_installer/
Branch name : **master**
Folder name : **depedencies**
The dependencies installer branch : https://github.com/intel/iotg_tsn_ref_sw

Refer to the README.md to understand the differences between the scripts provided:

https://github.com/intel/iotg_tsn_ref_sw/blob/experimental/dependencies_installer/dependencies/README.md
https://github.com/intel/iotg_tsn_ref_sw/blob/master/dependencies/README.md

Suggestion is to use the overwriting installer script (after having kept the original libbpf and libopen62541 in a safe place).
This will ensure the tsn ref sw will be able to find/use correct libraries.

NOTE: If your kernel support for Intel XDP+TBS, please ensure the variable 'txtime' is available in struct xdp_desc() in /usr/include/linux/if_xdp.h as example below:

/* Rx/Tx descriptor */
struct xdp_desc {
__u64 addr;
__u32 len;
__u32 options;
__u64 txtime;
};

NOTE: If you are installing the libbpf and libopen62541-iotg manually, there is a chance the related open62541-iotg.pc file is not there. Hence, you might have to comment out this line in **configure.ac**.

**AM_CONDITIONAL([WITH_OPCUA], [test "x$no_open62541_iotg_fork" = "x0"] && test "x${HAVE_XDP}" = "xyes")**
Expand Down
179 changes: 179 additions & 0 deletions dependencies/helpers.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,179 @@
#!/bin/sh
#/******************************************************************************
# Copyright (c) 2020, Intel Corporation
# All rights reserved.

# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:

# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.

# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.

# 3. Neither the name of the copyright holder nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.

# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
# *****************************************************************************/

# Helper functions. This script executes nothing.

###############################################################################

# Local Variable Declartion
LOCAL_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)

check_distro(){
# Check OS distro

echo -e "\nINSTALL-DEPENDENCIES.SH: Checking OS Distro"
os_distro=$(cat /etc/os-release | grep -w NAME | cut -c 6-)
echo "OS Distro: $os_distro"
}

config_proxy(){
# Configure proxy

echo -e "\nINSTALL-DEPENDENCIES.SH: Configuring proxy"
echo -e export https_proxy=http://proxy.png.intel.com:911
echo -e git config --global https.proxy http://proxy.jf.intel.com:911
export https_proxy=http://proxy.png.intel.com:911
git config --global https.proxy http://proxy.jf.intel.com:911
}

check_xdp_tbs(){
# Check whether the kernel support XDP_TBS

echo -e "\nINSTALL-DEPENDENCIES.SH: Checking XDP+TBS Availability"

if [[ -f /usr/include/linux/if_xdp.h ]]; then
txtime=$(cat /usr/include/linux/if_xdp.h | grep -i txtime > /dev/null && echo yes || echo no)
else
echo -e "WARNING: Header file if_xdp.h is not found"
echo -e "System proceed to exit the installation as if_xdp.h not found"
exit 0
fi

if [[ -f /proc/config.gz ]]; then
cat /proc/config.gz | gunzip > $LOCAL_DIR/kernel.config
xdp_supported=$(cat $LOCAL_DIR/kernel.config | grep -i CONFIG_XDP_SOCKETS=y > /dev/null && echo yes || echo no)
if [[ "$xdp_supported" == "no" ]]; then
echo -e "WARNING: CONFIG_XDP_SOCKETS is not enabled in the kernel."
echo -e "Installation will proceed as if_xdp.h exist. However it might crash when run the program"
fi
elif [[ -f /boot/config-$(uname -r) ]]; then
xdp_supported=$(cat /boot/config-$(uname -r)* | grep -i CONFIG_XDP_SOCKETS=y > /dev/null && echo yes || echo no)
if [[ "$xdp_supported" == "no" ]]; then
echo -e "WARNING: CONFIG_XDP_SOCKETS is not enabled in the kernel."
echo -e "Installation will proceed as if_xdp.h exist. However it might crash when run the program"
fi
else
xdp_supported=no
echo -e "WARNING: Kernel config is not found."
echo -e "Installation will proceed as if_xdp.h exist. However it might crash when run the program"
fi

if [[ "$xdp_supported" == "yes" && "$txtime" == "yes" ]]; then
echo -e "Checking for XDP+TBS availability in kernel... yes"
else
echo -e "Checking for XDP+TBS availability in kernel... no"
fi
}

clone_libbpf(){
# Git clone libbpf

echo -e "\n============================================="
echo -e "INSTALL-DEPENDENCIES.SH: Installing libbpf"
echo -e "============================================="
cd libbpf
rm -rf libbpf
git clone https://github.com/libbpf/libbpf.git
cd libbpf

echo -e "\nINSTALL-DEPENDENCIES.SH: Switching branch"
git checkout d1fd50d475779f64805fdc28f912547b9e3dee8a

if [[ "$xdp_supported" == "yes" && "$txtime" == "yes" ]]; then
echo -e "\nINSTALL-DEPENDENCIES.SH: Applying patches to libbpf"
EMAIL=root@localhost git am ../patches/*.patch
fi
}

clone_open62541(){
# Git clone libopen62541

echo -e "\n============================================="
echo -e "INSTALL-DEPENDENCIES.SH: Installing open62541"
echo -e "============================================="
cd $LOCAL_DIR
cd open62541
rm -rf open62541
git clone https://github.com/open62541/open62541.git
cd open62541

echo -e "\nINSTALL-DEPENDENCIES.SH: Switching branch"
git checkout a77b20ff940115266200d31d30d3290d6f2d57bd

# Apply patches for open62541
if [[ "$xdp_supported" == "yes" && "$txtime" == "yes" ]]; then
echo -e "\nINSTALL-DEPENDENCIES.SH: Applying patches with XDP_TBS to open62541"
EMAIL=root@localhost git am ../patches/patches_w_xdp_tbs/*.patch
else
echo -e "\nINSTALL-DEPENDENCIES.SH: Applying patches without XDP_TBS to open62541"
EMAIL=root@localhost git am ../patches/patches_wo_xdp_tbs/*.patch
fi
}

compile_open62541(){
# Compile open62541

echo -e "\nINSTALL-DEPENDENCIES.SH: Compiling open62541"

mkdir build
cd build

if [ "$xdp_support" == "no" ]; then
# Without XDP
echo -e "Compiling open62541 without XDP...."
cmake -DUA_BUILD_EXAMPLES=OFF \
-DUA_ENABLE_PUBSUB=ON \
-DUA_ENABLE_PUBSUB_CUSTOM_PUBLISH_HANDLING=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_ETF=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_XDP=OFF \
-DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_SHARED_LIBS=ON ..
else
# With XDP
echo -e "Compiling open62541 with XDP...."
cmake -DUA_BUILD_EXAMPLES=OFF \
-DUA_ENABLE_PUBSUB=ON \
-DUA_ENABLE_PUBSUB_CUSTOM_PUBLISH_HANDLING=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_ETF=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_XDP=ON \
-DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_SHARED_LIBS=ON ..
fi

# Modify CMakeFiles to ignore warnings
sed -i '/C_FLAGS/ s/$/ -Wno-error=conversion/' CMakeFiles/open62541-plugins.dir/flags.make
sed -i '/C_FLAGS/ s/$/ -Wno-error=maybe-uninitialized/' CMakeFiles/open62541-object.dir/flags.make

make && make install
}
115 changes: 15 additions & 100 deletions dependencies/install_dependencies.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,60 +30,23 @@
# POSSIBILITY OF SUCH DAMAGE.
# *****************************************************************************/

# Check OS distro
echo -e "\nINSTALL-DEPENDENCIES.SH: Checking OS Distro"
os_distro=$(cat /etc/os-release | grep -w NAME | cut -c 6-)
echo "OS Distro: $os_distro"


# Configure proxy
echo -e "\nINSTALL-DEPENDENCIES.SH: Configuring proxy"
echo -e export https_proxy=http://proxy.png.intel.com:911
echo -e git config --global https.proxy http://proxy.jf.intel.com:911
export https_proxy=http://proxy.png.intel.com:911
git config --global https.proxy http://proxy.jf.intel.com:911


# Check if if_xdp.h have XDP header
echo -e "\nINSTALL-DEPENDENCIES.SH: Checking XDP+TBS Availability"
check_xdp=$(cat /boot/config-$(uname -r)* | grep -i CONFIG_XDP_SOCKETS=y > /dev/null && echo 0 || echo 1)
txtime=$(cat /usr/include/linux/if_xdp.h | grep -i txtime > /dev/null && echo 0 || echo 1)
if [[ "$check_xdp" == "0" && "$txtime" == "0" ]]; then
echo -e "Checking for XDP+TBS availability in kernel... yes"
else
echo -e "Checking for XDP+TBS availability in kernel... no"
echo -e "Currently TSN Ref Sw App only support on kernel that support XDP+TBS"
exit 0
fi


# Add XDP header in if_xdp.h
#xdp_header=$(cat /usr/include/linux/if_xdp.h | grep -i xdp_desc > /dev/null && echo 0 || echo 1)
#txtime=$(cat /usr/include/linux/if_xdp.h | grep -i txtime > /dev/null && echo 0 || echo 1)
#if [[ "$xdp_header" == "0" && "$txtime" == "1" && "$check_xdp" == "0" ]]; then
# sed -i '107i \\t__u64 txtime;' /usr/include/linux/if_xdp.h
#fi

DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
source $DIR/helpers.sh

# Local Variable Declaration
LOCAL_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)

# Check OS distro
check_distro

# Install libbpf
SRCREV_LIBBPF="d1fd50d475779f64805fdc28f912547b9e3dee8a"
echo -e "\n============================================="
echo -e "INSTALL-DEPENDENCIES.SH: Installing libbpf"
echo -e "============================================="
cd libbpf
rm -rf libbpf
git clone https://github.com/libbpf/libbpf.git
cd libbpf
# Check whether the kernel support XDP_TBS
check_xdp_tbs

echo -e "\nINSTALL-DEPENDENCIES.SH: Switch to branch: '$SRCREV_LIBBPF'"
git checkout $SRCREV_LIBBPF
# Configure proxy
config_proxy

echo -e "\nINSTALL-DEPENDENCIES.SH: Applying patches to libbpf"
EMAIL=root@localhost git am ../patches/*.patch
# Git clone libbpf
clone_libbpf

if [[ "$1" == "--overwrite" ]]; then
echo -e "\nNOTE: The dependencies installer will overwrite the original libbpf"
Expand All @@ -107,23 +70,8 @@ NO_PKG_CONFIG=1 make -j$(nproc) -C src
DESTDIR=/ make install -j$(nproc) -C src
ldconfig


# Install libopen62541
SRCREV_LIBOPEN62541="a77b20ff940115266200d31d30d3290d6f2d57bd"
echo -e "\n============================================="
echo -e "INSTALL-DEPENDENCIES.SH: Installing open62541"
echo -e "============================================="
cd $LOCAL_DIR
cd open62541
rm -rf open62541
git clone https://github.com/open62541/open62541.git
cd open62541

echo -e "\nINSTALL-DEPENDENCIES.SH: Switch to branch: '$SRCREV_LIBOPEN62541'"
git checkout $SRCREV_LIBOPEN62541

echo -e "\nINSTALL-DEPENDENCIES.SH: Applying patches to open62541"
EMAIL=root@localhost git am ../patches/*.patch
# Git clone libopen62541
clone_open62541

if [[ "$1" == "--overwrite" ]]; then
echo -e "\nNOTE: The dependencies installer will overwrite the original open62541"
Expand All @@ -140,39 +88,8 @@ else
sed -i 's/Cflags: -I${includedir}\/open62541-iotg/Cflags: -I${includedir}\/open62541-iotg-custom/g' open62541.pc.in
fi

echo -e "\nINSTALL-DEPENDENCIES.SH: Compiling open62541"
mkdir build
cd build
if [ "$check_xdp" == "1" ]; then
# Without XDP
echo -e "Compiling open62541 without XDP...."
cmake -DUA_BUILD_EXAMPLES=OFF \
-DUA_ENABLE_PUBSUB=ON \
-DUA_ENABLE_PUBSUB_CUSTOM_PUBLISH_HANDLING=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_ETF=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_XDP=OFF \
-DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_SHARED_LIBS=ON ..
else
# With XDP
echo -e "Compiling open62541 with XDP...."
cmake -DUA_BUILD_EXAMPLES=OFF \
-DUA_ENABLE_PUBSUB=ON \
-DUA_ENABLE_PUBSUB_CUSTOM_PUBLISH_HANDLING=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_ETF=ON \
-DUA_ENABLE_PUBSUB_ETH_UADP_XDP=ON \
-DCMAKE_INSTALL_PREFIX=/usr \
-DBUILD_SHARED_LIBS=ON ..
fi

# Modify CMakeFiles to ignore warnings
sed -i '/C_FLAGS/ s/$/ -Wno-error=conversion/' CMakeFiles/open62541-plugins.dir/flags.make
sed -i '/C_FLAGS/ s/$/ -Wno-error=maybe-uninitialized/' CMakeFiles/open62541-object.dir/flags.make

make && make install

# Compile open62541
compile_open62541

# Redirect to default directory
cd $LOCAL_DIR
Expand All @@ -184,6 +101,7 @@ check_rpath=$(cat Makefile.am | grep -i "rpath" > /dev/null && echo 0 || echo 1)
check_txrx_afxdp_c=$(cat src/txrx-afxdp.c | grep -i "bpf-iotg-custom" > /dev/null && echo 0 || echo 1)
check_txrx_h=$(cat src/txrx.h | grep -i "bpf-iotg-custom" > /dev/null && echo 0 || echo 1)

# Link up the libraries
if [[ "$1" == "--overwrite" ]]; then
echo -e "\nNOTE: The dependencies installer will link to the original libraries path"
else
Expand Down Expand Up @@ -236,7 +154,6 @@ if [[ "$1" == "--overwrite" ]]; then
ln -sf libopen62541-iotg.so.1.1.0 $open62541_so_path
fi


# Echo libraries path to user
echo -e "\n============================================="
echo -e "Installed Libraries Information:"
Expand All @@ -252,8 +169,6 @@ echo -e "libopen62541.so.1.1.0"
echo -e "\nLibrary Install Folder:"
echo -e "$open62541_path"



echo -e "\nSetup Successful."
echo -e "Please Re-login or source /etc/environment for libraries to link up"

Expand Down
Loading

0 comments on commit 581ca86

Please sign in to comment.