Tags: Beuth-Erdelt/Benchmark-Experiment-Host-Manager
Tags
V0.6.7 (#201) * Masterscript: ENVs for statefulsets * Masterscript: ENVs for statefulsets only adding * Masterscript: ENVs for sut-job * Masterscript: VolumeMounts optional in sut deployment * Masterscript: Volumes optional in sut deployment * Masterscript: service_name for loading scripts as argument * Masterscript: volumeClaimTemplates for statefulset reads storage requests * Masterscript: volumeClaimTemplates as a list * Masterscript: SUT's services can have component names different from selector * Masterscript: Catch more expections in get_host_diskspace_used_data() * Masterscript: storage_parameter in connection infos * Tool: Show worker volumes * Update README.md JOSS draft badge * Masterscript: Also remove worker storage after experiment * Masterscript: Label DBMS in job * Masterscript: Label DBMS in monitoring * Docs: CockroachDB tested successfully * Masterscript: Do not retry delete_pvc() if pvc cannot be found * Masterscript: Name of worker storage contains experiment code * Masterscript: Find worker pvc by labels * Masterscript: benchmarking_parameters per experiment and configuration * Masterscript: Wait 5s before (re)checking status of workers * Masterscript: Remove worker pv if not wanted * Masterscript: Loading waits 60 secs for all pods * Masterscript: Benchbase similar to ycsb * Masterscript: Benchbase convert results to df * Masterscript: Benchbase collect results into df * Masterscript: Benchbase collect results into df and set index * Masterscript: Benchbase uses name_format for results * Masterscript: Fix container to dashboard when copying to result component * Masterscript: Accept successful pods as completed * Masterscript: Benchbase all collect results into single df at end of benchmark * Masterscript: Benchbase no results for loading phase and benchmarker results per job only * Masterscript: YCSB all collect results into single df at end of benchmark * Masterscript: Debug messages about evaluation * Masterscript: BEXHOMA_CONNECTION set to connection for benchmarker, to configuration otherwise * Masterscript: HammerDB merge results into df * Masterscript: HammerDB merge results into dfm, differ between connection and configuration * Masterscript: BEXHOMA_CONNECTION test for benchbase * Masterscript: Benchbase merge results into dfm, differ between connection and configuration * Masterscript: Debug messages about evaluation for YCSB collected dfs * Masterscript: HammerDB extract pod name * Masterscript: YCSB extract pod name * Masterscript: YCSB dump more information * Masterscript: HammerDB extract pod name * Masterscript: YCSB extract pod name * Masterscript: YCSB evaluation improved * Masterscript: Benchbase evaluation improved * Masterscript: BEXHOMA meta data in job envs * Masterscript: HammerDB concat dbms infos * Masterscript: Fetch metrics for specific connection * Masterscript: HammerDB also keep config file for single connection * Masterscript: Benchbase requests schema file * Masterscript: All experiments keep config file for single connection * Masterscript: Allow all job ENVs to be overwritten * Masterscript: BEXHOMA_CLIENT set to number of benchmarker client * Masterscript: Fetch metrics for specific connection for all benchmarker * Masterscript: BEXHOMA_CLIENT set to number of benchmarker client, thus is 0 during loading * Masterscript: Show list of open benchmarks per configuration * Masterscript: NEVER rerun, only one connection in config for detached - all benchmarker collect all dbms in one connection file * Masterscript: Copy connection file for connection specified by name * Masterscript: Copy connection file for connection specified by name * Masterscript: fetch_metrics_loading() for connection * Masterscript: fetch_metrics_loading() for connection, run in dashboard pod * Masterscript: set connection file name * Masterscript: fetch_metrics_loading() after loading, dump results * Build script for Docker images * Python 3.11.5 instead of 3.12 because of bug in setuptools * Benchmarker: Less output * DBMS: YugyByteDB dummy deployment * Docs: YCSB at entry page * Docs: scaled-out drivers at entry page * Docs: TPC-C at entry page * Docs: scaled-out drivers at entry page * Docs: Example: Run a custom SQL workload * # Conflicts: # README.md # bexhoma/configurations.py # bexhoma/experiments.py * requirements: no nbconvert * requirements: python 3.11.15 * Docs: .readthedocs.yaml * requirements: no m2r2 * requirements: sphinx * Docs: Example: Run a custom SQL workload * Docs: Formatting * YCSB: scaling-factor-operations * YugabyteDB dummy less resources * fix: requirements.txt to reduce vulnerabilities * fix: requirements.txt to reduce vulnerabilities - Werkzeug>=3.0.1 * Require only Python 3.10.2 * v0.6.7 prerelease
V0.6.3 Unified benchmarking / evaluation components, refined loading … …and indexing components (#156) * Masterscript: Towards unifying collections of loading and benchmarking components * Masterscript: Unification of run_benchmarker_pod() * Evaluator: First sketch of implementation * Evaluator: Exit code * Masterscript: Currently, only benchmarking component fetches loading metrics * Evaluator: First sketch of implementation * Evaluator: Debugging * Masterscript: Unification of run_benchmarker_pod() * Masterscript: Unification of run_benchmarker_pod() - copy split connection files * Masterscript: Test benchbase result * Masterscript: get_workflow_list() to test results for completeness * Masterscript: Dump output of loading metrics pod * Masterscript: Reconstruct workflow need benchmarking df * Masterscript: Compare reconstructed workflow with benchmarker lists * Evaluator: HammerDB test result for formal correctness * Masterscript: use_distributed_datasource, default False * Masterscript: Loading metrics dumps debug infos * Masterscript: Monitoring set to 5s scraping interval * Masterscript: Monitoring set to 15s scraping interval * Masterscript: Monitoring scraping interval as parameter of experiment and configuration * HammerDB: Download results * Masterscript: More job labels to identify connection and time * Masterscript: More job labels to identify connection and time * Masterscript: Monitor stream first tests * Masterscript: Monitor stream for all benchmarking components * Masterscript: Monitoring transformed * Masterscript: Monitoring loading uses connection's specific config * Masterscript: Monitoring stream uses connection's specific config, show errors * Masterscript: Monitoring changes connection.config, overwrite it with correct data again * Masterscript: Remove old methods and set path more consistently * Masterscript: Upload connection.config * Masterscript: Docs * Masterscript: Loading time from time spans of pods * Masterscript: Loading time from time spans of pods - more debug output * Masterscript: Loading time from time spans of pods if exists, total time span otherwise * Masterscript: Also add timeGenerator as info * Masterscript: Remove old evaluation methods * Masterscript: Indexing reuses schema script methods * Masterscript: time_ingested at sut and pvc * Masterscript: No indexing per default * Masterscript: Also store time markers per pod and container of jobs (loading and benchmarking) * Masterscript: Use message queue for benchmarker * Masterscript: Also store time markers per pod and container of jobs (loading and benchmarking) * Masterscript: Also store time markers per pod and container of jobs (loading and benchmarking) in connection.config * Masterscript: Sketch storing index time, dbmsbenchmarker uses benchmarking_parameters * DBMSBenchmarker: Time synch 4 min in future * Masterscript: Copy configs to result folder before starting benchmarker component's job * Masterscript: Only delete job pods when job is completed * Masterscript: patch loading * Masterscript: store timeIngesting * Require: patch loading * Masterscript: patch loading * Masterscript: store timeIndex * Masterscript: set_pod_counter() to 0 before loading starts * Masterscript: read timeLoading after indexing (again) * Masterscript: store timeSchema * Masterscript: store all script_times * Masterscript: store all script_times as float * Masterscript: Log scripting times for debugging * Masterscript: Verify loading times * Masterscript: set_additional_labels() * Masterscript: set_additional_labels() SF for TPC-H
PreviousNext