Skip to content

Commit

Permalink
Add new UPG E2E tests (travelping#23)
Browse files Browse the repository at this point in the history
* e2e: quick and dirty prototype of 'captive VPP'

* e2e: use http.Client for downloading

* e2e: add framework package

* e2e: copy netns stuff from CNI plugins

* e2e: refactor netns usage a bit

* e2e: add hs_proxy based test and remove main.go for now

* e2e: make VPP config more flexible

* e2e: add basic PFCP Association test

* e2e: store PCAPs

* e2e: establish PFCP session

* e2e: update base image name

* e2e: add names for namespaces

* e2e: add routes

* e2e: add ping to the image

* e2e: download some data through UPG

* e2e: disable offload for veths

This gives a substantial speed boost so we also increase download size

* e2e: disable http_static use for now as it breaks on Mac Docker

* e2e: use gopacket instead of tcpdump to capture the packets

* e2e: use patched go-pfcp from github

* e2e: fix method name

* e2e: improve docker env

Specify base image as a build arg.
Also, use dumb-init to avoid having zombies.

* e2e: add README.md

* e2e: reduce noise from NetNS.disableOffloading()

* e2e: improve TrafficGen config and skip VPP WS test on Mac Docker

* e2e: issue SessionModificationRequests with QueryURR and handle responses

* e2e: refactor PFCPConnection

* e2e: reproduce the crash on bad pdr_idx in the frame

* e2e: fail when VPP dies

* e2e: add a separate test case for PDR replacement crash

* e2e: use saner PFCPConnection interface that suports multiple sessions

* e2e: add PDR replacement test

Crashes UPG if `idBase ^= 9` is uncommented

* e2e: fix stopping upon VPP crash

* e2e: fix traffic volume checks

* e2e: make VPP binary and plugin paths configurable

This helps with running the tests against VPP built using its 'make'

* e2e: add UDP test cases

* e2e: do actually change PDR Ids in TestPDRReplacement

* e2e: remove obsolete comments

* e2e: test app detection

* e2e: test switching proxy on/off

* e2e: refactor and add UPG_TEST_QUICK env var

* e2e: add redirect test

* e2e: switch to Ginkgo/Gomega

* e2e: switch to govpp master to avoid panics upon VPP crash

* e2e: update README.md after switching to Ginkgo/Gomega

* e2e: add PGW mode

* e2e: make PGW tests pass and enable PDR replacement for PGW

* e2e: port newer uplane changes from sgwsimulator

This fixes some of the unneeded error logging
(that wasn't causing any test failures, though).

* e2e: fix handling GTPU issues

* e2e: fix silly sleep

* e2e: fix kernel GTPU and re-do GTPU close fix

Some of the things need to be done inside the
respective network namespaces.
Previous "GTPU issues" fix was a red herring, what
we really have here is net conn's File() method
breaking things after Fd() is retrieved from it.

Ref:
golang/go#29277
https://github.com/golang/go/blob/2291cae2af659876e93a3e1f95c708abb1475d02/src/os/file_unix.go#L76-L80

* e2e: add IPv6 tests

* e2e: fix logging

* e2e: fix SEID logging

* e2e: add extra checks to ensure proper netns for Dial/Listen

* e2e: support running tests in parallel

* e2e: split framework package

* e2e: add conn flood (+netem) tests

* e2e: relax UDP content checks when retries are enabled

* e2e: add icmp ping test

* e2e: add session deletion loop and fix conn flood test

* e2e: add MTU test

* e2e: add artifacts dir

* e2e: add junit reporting

* Integrate new e2e tests with the rest of UPG

* e2e: fix ICMP test name

* e2e: make tests lighter (especially http)

* e2e: use Ginkgo --flakeAttempts (mis)feature to cope with flakes

Apparently, CI environment is not powerful enough to handle the tests
atm.

* e2e: fix http trafficgen flakes in persist mode

* e2e: fix excessive resource consumption

Had to tune down 'excessive' tests for now.  Eventually, need to
update trafficgen routines so that they don't allocate a new buffer
for each goroutine.

* e2e: fix compat with earlier UPF builds

* e2e: try using 8 parallel nodes for e2e

* e2e: add minimal docs

* e2e: use Tomb for easier goroutine lifecycle control
  • Loading branch information
ivan4th authored Nov 17, 2020
1 parent 778bad9 commit a125adf
Show file tree
Hide file tree
Showing 60 changed files with 8,289 additions and 103 deletions.
124 changes: 28 additions & 96 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ prepare:
# build container image
.build:
stage: build
image: quay.io/travelping/upg-build:3ba60fbb1e5584fac28ff8351b37c8e6 # XX_DO_NOT_REMOVE_THIS_COMMENT
image: quay.io/travelping/upg-build:c0e8ed0e5ab1e1160f104c77fc4b5654 # XX_DO_NOT_REMOVE_THIS_COMMENT
script:
- |
export REGISTRY_LOGIN=${QUAY_USER_ID}
Expand Down Expand Up @@ -91,7 +91,7 @@ build:debug:
.test:
stage: test
# the following is updated automatically by make update-build-image-tag
image: quay.io/travelping/upg-build:3ba60fbb1e5584fac28ff8351b37c8e6 # XX_DO_NOT_REMOVE_THIS_COMMENT
image: quay.io/travelping/upg-build:c0e8ed0e5ab1e1160f104c77fc4b5654 # XX_DO_NOT_REMOVE_THIS_COMMENT
script:
- make update-vpp
- tar -C vpp -xvzf ${TEST_ARCHIVE}
Expand All @@ -114,7 +114,7 @@ build:debug:
test:patch-style:
stage: test
# the following is updated automatically by make update-build-image-tag
image: quay.io/travelping/upg-build:3ba60fbb1e5584fac28ff8351b37c8e6 # XX_DO_NOT_REMOVE_THIS_COMMENT
image: quay.io/travelping/upg-build:c0e8ed0e5ab1e1160f104c77fc4b5654 # XX_DO_NOT_REMOVE_THIS_COMMENT
script:
- hack/checkstyle.sh

Expand All @@ -134,108 +134,40 @@ test:release:
dependencies:
- build:release

.test-ext:
.test-e2e:
stage: test
image: quay.io/travelping/upf-test:12b024a
services:
- docker:19.03.8-dind
variables:
SKIP_VPP_SOURCE_CHECK: "y"
# the following is updated automatically by make update-build-image-tag
image: quay.io/travelping/upg-build:c0e8ed0e5ab1e1160f104c77fc4b5654 # XX_DO_NOT_REMOVE_THIS_COMMENT
script:
# https://github.com/docker/compose/issues/5696#issuecomment-377906769
- export COMPOSE_INTERACTIVE_NO_CLI=1
- IMAGE_VARIANT=debug
- IMAGE_BASE_NAME=${REGISTRY}/${IMAGE_NAME}
- |
. hack/version.sh
IMAGE_BASE_TAG=${UPG_IMAGE_TAG}
- IMAGE_FULL_NAME=${IMAGE_BASE_NAME}:${IMAGE_BASE_TAG}_${IMAGE_VARIANT}
- cd /test
# temporary workaround for the hugepages issue
- tar -C vpp -xvzf ${TEST_ARCHIVE}
- sysctl vm.nr_hugepages=0
- mkdir artifacts junit-output
# TODO: nightly run without E2E_QUICK
# TODO: get rid of E2E_FLAKE_ATTEMPTS. The tests need to be less flaky.
- |
if bash -x scripts/test.sh "${IMAGE_FULL_NAME}" ${TEST_PROFILES}; then
r=0
else
r=1
if ! make ${TEST_TARGET} E2E_RETEST=y E2E_ARTIFACTS_DIR=$PWD/artifacts E2E_JUNIT_DIR=$PWD/junit-output E2E_PARALLEL=y E2E_PARALLEL_NODES=8 E2E_QUICK=y; then
mkdir test-out
tar -cvzf test-out/vpp-test.tar.gz artifacts || true
ls -l test-out/
exit 1
fi
mv /test/pcaps "/builds/${CI_PROJECT_PATH}/pcaps"
ls -l "/builds/${CI_PROJECT_PATH}/pcaps"
exit ${r}
dependencies:
- build:debug
artifacts:
when: always
paths:
- pcaps

test:external-ipv4-ping:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy ping gdb-batch

test:external-ipv4-udp:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy udp gdb-batch

test:external-ipv4-curl:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy curl gdb-batch

# FIXME: traffic volume mismatch
# test:external-ipv4-app:
# extends: .test-ext
# variables:
# TEST_PROFILES: type-tdf dispatch-trace proxy app-detection gdb-batch

test:external-ipv4-udp-pgw:
extends: .test-ext
variables:
TEST_PROFILES: type-pgw dispatch-trace proxy udp gdb-batch

test:external-ipv4-curl-pgw:
extends: .test-ext
variables:
TEST_PROFILES: type-pgw dispatch-trace proxy curl gdb-batch

# FIXME: traffic volume mismatch
# test:external-ipv4-app-pgw:
# extends: .test-ext
# variables:
# TEST_PROFILES: type-pgw dispatch-trace proxy app-detection gdb-batch

# TODO: match app
test:external-ipv4-curl-noproxy:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace curl gdb-batch

test:external-ipv6-udp:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy udp gdb-batch ipv6

test:external-proxy-crash:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy wrk gdb-batch

test:external-proxy-crash-pgw:
extends: .test-ext
variables:
TEST_PROFILES: type-pgw dispatch-trace proxy wrk gdb-batch

test:external-proxy-crash-netem:
extends: .test-ext
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy wrk netem gdb-batch
- test-out
reports:
junit: junit-output/*.xml

test:external-proxy-crash-netem-40k:
extends: .test-ext
test-e2e:debug:
extends: .test-e2e
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy wrk40k netem gdb-batch
TEST_ARCHIVE: debug/testfiles.tar.gz
TEST_TARGET: e2e-debug

test:session-deletion:
extends: .test-ext
test-e2e:release:
extends: .test-e2e
variables:
TEST_PROFILES: type-tdf dispatch-trace proxy repeat-session wrk gdb-batch
TEST_ARCHIVE: release/testfiles.tar.gz
TEST_TARGET: e2e-release
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax = docker/dockerfile:experimental
# the following is updated automatically by make update-build-image-tag
FROM quay.io/travelping/upg-build:3ba60fbb1e5584fac28ff8351b37c8e6 AS build-stage
FROM quay.io/travelping/upg-build:c0e8ed0e5ab1e1160f104c77fc4b5654 AS build-stage

ADD vpp /src/vpp
ADD upf /src/upf
Expand Down
15 changes: 14 additions & 1 deletion Dockerfile.build
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,13 @@ RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=private \
--mount=target=/var/cache/apt,type=cache,sharing=private \
apt-get update && \
apt-get dist-upgrade -yy && \
apt-get install -y software-properties-common && \
add-apt-repository ppa:longsleep/golang-backports && \
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
build-essential sudo git netbase curl ca-certificates && \
build-essential sudo git netbase curl ca-certificates \
golang-go iproute2 gdb tcpdump iputils-ping libpcap-dev \
dumb-init && \
curl -sSL "https://github.com/moby/buildkit/releases/download/${BUILDKIT_VERSION}/buildkit-${BUILDKIT_VERSION}.linux-amd64.tar.gz" | \
tar -xvz -C /usr/local bin/buildctl && \
echo "${BUILDCTL_SHA256} /usr/local/bin/buildctl" | sha256sum -c && \
Expand All @@ -35,3 +40,11 @@ RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=private \
make UNATTENDED=yes install-dep install-ext-dep && \
apt-get clean && \
rm -rf /src/vpp

ENV GOPATH /go

RUN go get github.com/onsi/ginkgo/ginkgo && \
mv /go/bin/ginkgo /usr/local/bin

CMD ["/bin/bash"]
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
2 changes: 1 addition & 1 deletion Dockerfile.devel
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax = docker/dockerfile:experimental
# the following is updated automatically by make update-build-image-tag
FROM quay.io/travelping/upg-build:3ba60fbb1e5584fac28ff8351b37c8e6 AS build-stage
FROM quay.io/travelping/upg-build:c0e8ed0e5ab1e1160f104c77fc4b5654 AS build-stage

ADD vpp /src/vpp
ADD upf /src/upf
Expand Down
14 changes: 12 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ ci-build: vpp
hack/ci-build.sh

ensure-build-image: vpp
bash -x hack/ensure-build-image.sh
hack/ensure-build-image.sh

update-build-image-tag: vpp
hack/update-build-image-tag.sh
Expand Down Expand Up @@ -46,5 +46,15 @@ retest-release:
hack/buildenv.sh make retest TEST=test_upf V=$(TEST_VERBOSITY) \
EXTERN_TESTS=../../upf/test

e2e-debug:
UPG_BUILDENV_PRIVILEGED=1 \
E2E_TARGET=debug \
hack/buildenv.sh ../hack/e2e.sh

e2e-release:
UPG_BUILDENV_PRIVILEGED=1 \
E2E_TARGET=release \
hack/buildenv.sh ../hack/e2e.sh

buildenv:
hack/buildenv.sh
UPG_BUILDENV_PRIVILEGED=1 hack/buildenv.sh
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,9 @@ The following `make` commands are supported:
* `make update-vpp` re-clones VPP into `vpp/` directory
* `make buildenv` runs an interactive shell inside the build
environment with UPG and VPP sources mounted into the container
* `make e2e-debug` and `make e2e-release` build UPG in debug and
release mode respectively and run E2E tests for it. For more information,
see [E2E test documentation](test/e2e/README.md)

If docker is used, one should set the following environment variable
to enable wrapping the internally run commands in a docker container:
Expand Down
20 changes: 18 additions & 2 deletions hack/buildenv.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,36 @@ set -o errtrace

cd "$(dirname "${BASH_SOURCE}")/.."

if [[ ! -e vpp/Makefile ]]; then
if [[ ! ${SKIP_VPP_SOURCE_CHECK:-} && ! -e vpp/Makefile ]]; then
echo >&2 "Please run 'make update-vpp'"
exit 1
fi

priv=
if [[ ${UPG_BUILDENV_PRIVILEGED:-} ]]; then
priv="--privileged"
fi

if [[ ! ${UPG_DOCKER_BUILDENV:-} ]]; then
cd vpp
exec "$@"
else
. hack/build-image-name.sh
docker run -it --rm --name vpp-build --shm-size 1024m \
${priv} \
-v $PWD:/src:delegated \
-v $PWD/vpp-out:/vpp-out \
-e LC_ALL=C.UTF-8 -e LANG=C.UTF-8 \
-e LC_ALL=C.UTF-8 \
-e LANG=C.UTF-8 \
-e E2E_RETEST="${E2E_RETEST:=}" \
-e E2E_PARALLEL="${E2E_PARALLEL:-}" \
-e E2E_PARALLEL_NODES="${E2E_PARALLEL_NODES:-}" \
-e E2E_FOCUS="${E2E_FOCUS:-}" \
-e E2E_TARGET="${E2E_TARGET:-}" \
-e E2E_ARTIFACTS_DIR="${E2E_ARTIFACTS_DIR:-}" \
-e E2E_JUNIT_DIR="${E2E_JUNIT_DIR:-}" \
-e E2E_QUICK="${E2E_QUICK:-}" \
-e E2E_FLAKE_ATTEMPTS="${E2E_FLAKE_ATTEMPTS:-}" \
-w /src/vpp \
"${build_image}" \
"$@"
Expand Down
76 changes: 76 additions & 0 deletions hack/e2e.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
set -o errtrace

cd "$(dirname "${BASH_SOURCE}")/.."

: "${E2E_RETEST:=}"
: "${E2E_PARALLEL:=}"
: "${E2E_PARALLEL_NODES:=10}"
: "${E2E_FOCUS:=}"
: "${E2E_TARGET:=debug}"
: "${E2E_ARTIFACTS_DIR:=}"
: "${E2E_JUNIT_DIR:=}"
: "${E2E_QUICK:=}"
: "${E2E_FLAKE_ATTEMPTS:=}"

if grep -q '^gtp ' /proc/modules; then
echo >&2 "* Using kernel GTP-U support for IPv4 PGW tests"
export UPG_TEST_GTPU_KERNEL=1
else
echo >&2 "* Kernel GTP-U support not available, using userspace GTP-U only"
fi

export UPG_TEST_QUICK="${E2E_QUICK}"

case ${E2E_TARGET} in
debug)
if [[ ! ${E2E_RETEST} ]]; then
make -C vpp build
fi
export VPP_PATH="${PWD}/vpp/build-root/install-vpp_debug-native/vpp/bin/vpp"
export VPP_PLUGIN_PATH="${PWD}/vpp/build-root/install-vpp_debug-native/vpp/lib/vpp_plugins"
export LD_LIBRARY_PATH="${PWD}/vpp/build-root/install-vpp_debug-native/vpp/lib"
;;
release)
if [[ ! ${E2E_RETEST} ]]; then
make -C vpp build-release
fi
export VPP_PATH="${PWD}/vpp/build-root/install-vpp-native/vpp/bin/vpp"
export VPP_PLUGIN_PATH="${PWD}/vpp/build-root/install-vpp-native/vpp/lib/vpp_plugins"
export LD_LIBRARY_PATH="${PWD}/vpp/build-root/install-vpp-native/vpp/lib"
;;
*)
echo >&2 "E2E_TARGET must be either debug or release"
;;
esac

cd test/e2e

ginkgo_args=(-trace -progress -reportPassed)

if [[ ${E2E_PARALLEL} ]]; then
ginkgo_args+=(-nodes "${E2E_PARALLEL_NODES}")
fi

if [[ ${E2E_FOCUS} ]]; then
ginkgo_args+=(-focus "${E2E_FOCUS}")
fi

if [[ ${E2E_FLAKE_ATTEMPTS} ]]; then
ginkgo_args+=(--flakeAttempts ${E2E_FLAKE_ATTEMPTS})
fi

ginkgo_args+=(--)

if [[ ${E2E_ARTIFACTS_DIR} ]]; then
ginkgo_args+=(-artifacts-dir "${E2E_ARTIFACTS_DIR}")
fi

if [[ ${E2E_JUNIT_DIR} ]]; then
ginkgo_args+=(-junit-output "${E2E_JUNIT_DIR}")
fi

ginkgo "${ginkgo_args[@]}"
Loading

0 comments on commit a125adf

Please sign in to comment.