Skip to content

Commit

Permalink
JMH: Parameterize spark project version for JHM Benchmarks (apache#3946)
Browse files Browse the repository at this point in the history
Given that we have multiple Spark project versions in the codebase and
that users might want to run a particular Benchmark from a specific
Spark version, we should make the Spark project version a parameter of
the JMH Benchmark Action.
  • Loading branch information
nastra authored Jan 21, 2022
1 parent f8f7805 commit 6c7a114
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion .github/workflows/jmh-bechmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ on:
ref:
description: 'The branch name'
required: true
spark_version:
description: 'The spark project version to use, such as iceberg-spark-3.2'
default: 'iceberg-spark-3.2'
required: true
benchmarks:
description: 'A list of comma-separated double-quoted Benchmark names, such as "IcebergSourceFlatParquetDataReadBenchmark", "IcebergSourceFlatParquetDataFilterBenchmark"'
required: true
Expand Down Expand Up @@ -57,6 +61,7 @@ jobs:
echo "Repo: ${{ github.event.inputs.repo }}"
echo "Ref: ${{ github.event.inputs.ref }}"
echo "Benchmarks: ${{ needs.matrix.outputs.matrix }}"
echo "Spark Project Version: ${{ needs.matrix.outputs.spark_version }}"
echo "Found Benchmarks? ${{ needs.matrix.outputs.foundlabel }}"
run-benchmark:
Expand All @@ -82,7 +87,7 @@ jobs:
- run: echo -e "$(ip addr show eth0 | grep "inet\b" | awk '{print $2}' | cut -d/ -f1)\t$(hostname -f) $(hostname -s)" | sudo tee -a /etc/hosts

- name: Run Benchmark
run: ./gradlew :iceberg-spark:iceberg-spark3:jmh -PjmhIncludeRegex=${{ matrix.benchmark }} -PjmhOutputPath=benchmark/${{ matrix.benchmark }}.txt
run: ./gradlew :iceberg-spark:${{ github.event.inputs.spark_version }}:jmh -PjmhIncludeRegex=${{ matrix.benchmark }} -PjmhOutputPath=benchmark/${{ matrix.benchmark }}.txt

- uses: actions/upload-artifact@v2
if: ${{ always() }}
Expand Down

0 comments on commit 6c7a114

Please sign in to comment.