Skip to content

Commit

Permalink
Move durations into tests directory. (conda#12411)
Browse files Browse the repository at this point in the history
  • Loading branch information
jezdez authored Mar 1, 2023
1 parent 584b160 commit 81beee7
Show file tree
Hide file tree
Showing 11 changed files with 16 additions and 14 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ jobs:
name: test-results-${{ github.sha }}-${{ runner.os }}-${{ matrix.default-channel }}-${{ matrix.python-version }}-${{ matrix.conda-subdir }}-${{ matrix.test-type }}-${{ matrix.test-group }}
path: |
.coverage
.test_durations_${OS}
tests\durations\${OS}.json
test-report.xml
retention-days: 1

Expand Down Expand Up @@ -215,7 +215,7 @@ jobs:
name: test-results-${{ github.sha }}-${{ runner.os }}-${{ matrix.default-channel }}-${{ matrix.python-version }}-${{ matrix.test-type }}-${{ matrix.test-group }}
path: |
.coverage
.test_durations_${OS}
tests/durations/${OS}.json
test-report.xml
retention-days: 1

Expand Down Expand Up @@ -384,7 +384,7 @@ jobs:
name: test-results-${{ github.sha }}-${{ runner.os }}-${{ matrix.default-channel }}-${{ matrix.python-version }}-${{ matrix.test-type }}-${{ matrix.test-group }}
path: |
.coverage
.test_durations_${OS}
tests/durations/${OS}.json
test-report.xml
retention-days: 1

Expand Down
2 changes: 1 addition & 1 deletion dev/linux/integration.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@ conda info
# put temporary files on same filesystem
export TMP=$HOME/pytesttmp
mkdir -p $TMP
python -m pytest --cov=conda --store-durations --durations-path=.test_durations_${OS} --splitting-algorithm=least_duration --basetemp=$TMP -m "integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
python -m pytest --cov=conda --store-durations --durations-path=tests/durations/${OS}.json --splitting-algorithm=least_duration --basetemp=$TMP -m "integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
python -m conda.common.io
2 changes: 1 addition & 1 deletion dev/linux/unit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ sudo rm -rf /opt/conda/pkgs/*-*-*
# put temporary files on same filesystem
export TMP=$HOME/pytesttmp
mkdir -p $TMP
python -m pytest --cov=conda --store-durations --durations-path=.test_durations_${OS} --splitting-algorithm=least_duration --basetemp=$TMP -m "not integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
python -m pytest --cov=conda --store-durations --durations-path=tests/durations/${OS}.json --splitting-algorithm=least_duration --basetemp=$TMP -m "not integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
2 changes: 1 addition & 1 deletion dev/macos/integration.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ TEST_GROUP="${TEST_GROUP:-1}"
eval "$(sudo python -m conda init bash --dev)"
conda info
conda-build tests/test-recipes/activate_deactivate_package
python -m pytest --cov=conda --store-durations --durations-path=.test_durations_${OS} --splitting-algorithm=least_duration -m "integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
python -m pytest --cov=conda --store-durations --durations-path=tests/durations/${OS}.json --splitting-algorithm=least_duration -m "integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
python -m conda.common.io
2 changes: 1 addition & 1 deletion dev/macos/unit.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ TEST_GROUP="${TEST_GROUP:-1}"

eval "$(sudo python -m conda init bash --dev)"
conda info
python -m pytest --cov=conda --store-durations --durations-path=.test_durations_${OS} --splitting-algorithm=least_duration -m "not integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
python -m pytest --cov=conda --store-durations --durations-path=tests/durations/${OS}.json --splitting-algorithm=least_duration -m "not integration" -v --splits ${TEST_SPLITS} --group=${TEST_GROUP}
2 changes: 1 addition & 1 deletion dev/windows/integration.bat
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ cd \conda_src || goto :error
CALL dev-init.bat || goto :error
CALL conda info || goto :error
CALL conda-build tests\test-recipes\activate_deactivate_package tests\test-recipes\pre_link_messages_package || goto :error
:: TODO --store-durations --durations-path=.test_durations_Windows
:: TODO --store-durations --durations-path=tests\durations\Windows.json
:: --splitting-algorithm=least_duration (causes tests to fail)
CALL pytest --cov=conda -m "integration" --basetemp=C:\tmp -v --splits=%TEST_SPLITS% --group=%TEST_GROUP% || goto :error
goto :EOF
Expand Down
2 changes: 1 addition & 1 deletion dev/windows/unit.bat
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ pushd %TEMP% || goto :error
cd \conda_src || goto :error
CALL dev-init.bat || goto :error
CALL conda info || goto :error
CALL pytest --cov=conda --store-durations --durations-path=.test_durations_Windows --splitting-algorithm=least_duration -m "not integration" -v --splits=%TEST_SPLITS% --group=%TEST_GROUP% || goto :error
CALL pytest --cov=conda --store-durations --durations-path=tests\durations\Windows.json --splitting-algorithm=least_duration -m "not integration" -v --splits=%TEST_SPLITS% --group=%TEST_GROUP% || goto :error
goto :EOF

:error
Expand Down
File renamed without changes.
File renamed without changes.
12 changes: 7 additions & 5 deletions tests/combine_durations.py → tests/durations/combine.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,25 @@
Script to combine test durations from all runs.
If the tests splits are looking uneven or the test suite has
siginificantly changed, update .test_durations in the root of the
siginificantly changed, update tests/durations/${OS}.json in the root of the
repository and pytest-split may work better.
`gh run list -b <interesting branch>`
`mkdir tests-data; cd tests-data` # will be filled with many artifacts
`gh run download <number of tests CI run>`
`python combine_durations.py`
`python tests/durations/combine.py`
Then copy `combined_durations.json`.
Then copy `combined.json`.
"""

import json
from pathlib import Path

count = 0
combined = {}
for path in Path(".").glob("*/.test_durations"):
this_dir = Path(__file__).parent

for path in this_dir.glob("*.json"):
data = json.loads(path.read_text())
for key in data:
if key in combined:
Expand All @@ -33,4 +35,4 @@

print(f"Read {count} .test_durations")

Path("combined_durations.json").write_text(json.dumps(combined, indent=4, sort_keys=True))
(this_dir / "combined.json").write_text(json.dumps(combined, indent=4, sort_keys=True))
File renamed without changes.

0 comments on commit 81beee7

Please sign in to comment.