Codon benchmark suite

This folder contains a number of Codon benchmarks. Some are taken from the pyperformance suite while others are adaptations of applications we've encountered in the wild. Further, some of the benchmarks are identical in both Python and Codon, some are changed slightly to work with Codon's type system, and some use Codon-specific features like parallelism or GPU.

Some of the pyperformance benchmarks can be made (much) faster in Codon by using various Codon-specific features, but their adaptations here are virtually identical to the original implementations (mainly just the use of the pyperf module is removed).

Setup

The bench.sh script can be used to run all the benchmarks and output a CSV file with the results. The benchmark script looks at the following environment variables:

• EXE_PYTHON: Python command (default: python3)
• EXE_PYPY: PyPy command (default: pypy3)
• EXE_CPP: C++ compiler command (default: clang++; run with -std=c++17 -O3)
• EXE_CODON: Codon command (default: build/codon; run with -release)

Some benchmarks also require specific environment variables to be set for accessing data (details below).

Benchmarks

• chaos: Pyperformance's chaos benchmark.
• float: Pyperformance's float benchmark.
• go: Pyperformance's go benchmark.
• nbody: Pyperformance's nbody benchmark.
• spectral_norm: Pyperformance's spectral_norm benchmark.
• mandelbrot: Generates an image of the Mandelbrot set. Codon version uses GPU via one additional @par(gpu=True, collapse=2) line.
• set_partition: Calculates set partitions. Code taken from this Stack Overflow answer.
• sum: Computes sum of integers from 1 to 50000000 with a loop. Code taken from this article.
• taq: Performs volume peak detection on an NYSE TAQ file. Sample TAQ files can be downloaded and uncompressed from here (e.g. EQY_US_ALL_NBBO_20220705.gz). We recommend using the first 10M lines for benchmarking purposes. The TAQ file path should be passed to the benchmark script through the DATA_TAQ environment variable.
• binary_trees: Boehm's binary trees benchmark.
• fannkuch: See Performing Lisp analysis of the FANNKUCH benchmark by Kenneth R. Anderson and Duane Rettig. Benchmark involves generating permutations and repeatedly reversing elements of a list. Codon version is multithreaded with a dynamic schedule via one additional @par(schedule='dynamic') line.
• word_count: Counts occurences of words in a file using a dictionary. The file should be passed to the benchmark script through the DATA_WORD_COUNT environment variable.
• primes: Counts the number of prime numbers below a threshold. Codon version is multithreaded with a dynamic schedule via one additional @par(schedule='dynamic') line.