Skip to content

Commit

Permalink
Add workflow for building the mvfst image for the quic interop running
Browse files Browse the repository at this point in the history
Summary:
This moves the mvfst interop files from https://github.com/lnicco/mvfst-qns

and adds a github workflow to build and publish the mvfst-interop image to Github packages instead of existing adhoc automation.

The new workflow should trigger once a week with new TagIt releases.

Tested the generated container on a private repo against the interop runner.

Reviewed By: lnicco

Differential Revision: D55076273

fbshipit-source-id: b6efc9a38c0f9613b51ef8c7241c2e28e92f81b8
  • Loading branch information
jbeshay authored and facebook-github-bot committed Mar 19, 2024
1 parent 89dbd59 commit 2dc105e
Show file tree
Hide file tree
Showing 4 changed files with 210 additions and 0 deletions.
47 changes: 47 additions & 0 deletions .github/workflows/publish_mvfst_interop.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Following instruction from https://docs.github.com/en/actions/publishing-packages/publishing-docker-images

name: Publish mvfst interop image

on:
push:
tags:
# Build a new image weekly with each TagIt release
- 'v20*'

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}/mvfst-interop

jobs:
build-and-push-image:
runs-on: ubuntu-latest

permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Log in to the Container registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Build and push Docker image
uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
with:
context: .
file: proxygen/httpserver/samples/hq/quic-interop/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
37 changes: 37 additions & 0 deletions proxygen/httpserver/samples/hq/quic-interop/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#
# Full build image
#
FROM martenseemann/quic-network-simulator-endpoint:latest

ENV TZ=America/Los_Angeles
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN apt-get update

RUN apt-get --yes --fix-missing update

# Get and build proxygen with HTTP/3 support
RUN apt-get install --yes wget net-tools iputils-ping tcpdump ethtool iperf git sudo cmake python3 libssl-dev m4 zlib1g-dev gcc g++
RUN mkdir proxygen
COPY . /proxygen
RUN cd proxygen && ./getdeps.sh --no-tests --allow-system-packages
RUN ldd /tmp/fbcode_builder_getdeps-ZproxygenZbuildZfbcode_builder-root/build/proxygen/proxygen/httpserver/hq | grep "=> /" | awk '{print $3}' > libs.txt
RUN tar cvf libs.tar --dereference --files-from=libs.txt

#
# Minimal image
#
FROM martenseemann/quic-network-simulator-endpoint:latest
# copy run script
COPY proxygen/httpserver/samples/hq/quic-interop/run_endpoint.sh .
RUN chmod +x run_endpoint.sh

# Copy HQ
COPY --from=0 /tmp/fbcode_builder_getdeps-ZproxygenZbuildZfbcode_builder-root/build/proxygen/proxygen/httpserver/hq /proxygen/_build/proxygen/bin/hq
# Copy shared libs
COPY --from=0 libs.tar /
RUN tar xvf libs.tar
RUN rm libs.tar
# Create the logs directory
RUN mkdir /logs

ENTRYPOINT [ "./run_endpoint.sh" ]
8 changes: 8 additions & 0 deletions proxygen/httpserver/samples/hq/quic-interop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
mvfst+proxygen Docker image for use with the
[quic-network-simulator](https://github.com/marten-seemann/quic-network-simulator/) and [quic-interop-runner](https://github.com/marten-seemann/quic-interop-runner)

Build your own by running this after checking out the repo:
```
docker build . -f proxygen/httpserver/samples/hq/quic-interop/Dockerfile
```
or download the latest image build from Github under the packages section of this repo.
118 changes: 118 additions & 0 deletions proxygen/httpserver/samples/hq/quic-interop/run_endpoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
#!/usr/bin/env bash
# Copyright (c) Meta Platforms, Inc. and affiliates.
# All rights reserved.
#
# This source code is licensed under the BSD-style license found in the
# LICENSE file in the root directory of this source tree.

# Extra debugging ?
set -x
set -o nounset

VERSION=1
HQ_CLI=/proxygen/_build/proxygen/bin/hq
PORT=443
LOGLEVEL=2

# Set up the routing needed for the simulation
/setup.sh

# Unless noted otherwise, test cases use HTTP/0.9 for file transfers.
PROTOCOL="hq-interop"
HTTPVERSION="0.9"

# Default enormous flow control.

CONN_FLOW_CONTROL="107374182"
STREAM_FLOW_CONTROL="107374182"
INVOCATIONS=$(echo "${REQUESTS}" | tr " " "\n" | awk -F '/' '{ print "/" $4 }' | paste -sd',')
EARLYDATA="false"
PSK_FILE="" # in memory psk
if [ -n "${TESTCASE}" ]; then
case "${TESTCASE}" in
"handshake") ;;
"multiconnect") ;;
"transfer")
STREAM_FLOW_CONTROL="262144"
CONN_FLOW_CONTROL="2621440"
;;
"retry")
exit 127
;;
"throughput")
LOGLEVEL=1
;;
"resumption")
INVOCATIONS=$(${INVOCATIONS//,/ })
PSK_FILE="/psk"
;;
"zerortt")
INVOCATIONS=$(${INVOCATIONS//,/ })
PSK_FILE="/psk"
EARLYDATA="true"
;;
"http3")
PROTOCOL="h3"
HTTPVERSION="1.1"
;;
*)
exit 127
;;
esac
fi

if [ "${ROLE}" == "client" ]; then
# Wait for the simulator to start up.
/wait-for-it.sh sim:57832 -s -t 10
echo "Starting QUIC client..."
if [ -n "${REQUESTS}" ]; then
REQS=("${REQUESTS}")
REQ=${REQS[0]}
SERVER=$(echo "$REQ" | cut -d'/' -f3 | cut -d':' -f1)

for INVOCATION in ${INVOCATIONS}; do

echo "requesting files '${INVOCATION}'"
${HQ_CLI} \
--mode=client \
--host="${SERVER}" \
--port=${PORT} \
--protocol=${PROTOCOL} \
--httpversion=${HTTPVERSION} \
--use_version=true \
--quic-version=${VERSION} \
--path="${INVOCATION}" \
--early_data=${EARLYDATA} \
--psk_file=${PSK_FILE} \
--conn_flow_control=${CONN_FLOW_CONTROL} \
--stream_flow_control=${STREAM_FLOW_CONTROL} \
--outdir=/downloads \
--logdir=/logs \
--qlogger_path=/logs \
--v=${LOGLEVEL} 2>&1 | tee /logs/client.log
done
# This is the best way to troubleshoot.
# Just uncomment the line below, run the test, then enter containers with
# docker exec -it [client|server|sim] /bin/bash
#/bin/bash
fi

elif [ "$ROLE" == "server" ]; then
echo "Running QUIC server on [::]:${PORT}"
${HQ_CLI} \
--mode=server \
--cert=/certs/cert.pem \
--key=/certs/priv.key \
--conn_flow_control=${CONN_FLOW_CONTROL} \
--stream_flow_control=${STREAM_FLOW_CONTROL} \
--port=${PORT} \
--httpversion=${HTTPVERSION} \
--h2port=${PORT} \
--static_root=/www \
--logdir=/logs \
--qlogger_path=/logs \
--host=:: \
--congestion=bbr \
--pacing=true \
--v=${LOGLEVEL} 2>&1 | tee /logs/server.log
fi

0 comments on commit 2dc105e

Please sign in to comment.