Add committing to stateful test actions #466
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcherian
commented
Dec 12, 2024
dcherian
commented
Dec 12, 2024
| @@ -193,7 +193,7 @@ async def test_icechunk_can_read_old_repo(): | |||
| ] | |||
| assert sorted( | |||
| [p async for p in store.list_dir("group2/group3/group4/group5/inner")] | |||
| ) == ["c", "zarr.json"] | |||
| ) == ["zarr.json"] | |||
There was a problem hiding this comment.
this test never wrote a chunk, so c should not have been created.
There was a problem hiding this comment.
There is some weird behavior in here i think ... sometimes i need it sometimes i don't?
dcherian
force-pushed
the
update-stateful-test
branch
from
46f2427 to
40b559e
Compare
The "Unbounded" upper bound of the range query could return chunk coordinates for nodes that were not requested if there were arrays with NodeId > requested NodeId. This affected `list_prefix`, unclear if it affects anything else.
dcherian
force-pushed
the
update-stateful-test
branch
from
3c27994 to
e121a83
Compare
dcherian
commented
Dec 12, 2024
| None | ||
| } | ||
| self.last_key = Some(k.clone()); | ||
| Some((coord.clone(), payload.clone())) |
There was a problem hiding this comment.
With the take_while approach, we would return ChunkInfo here and unpack it up in verified_node_chunk_iterator. Worth it?
dcherian
commented
Dec 12, 2024
dcherian
commented
Dec 12, 2024
paraseba
approved these changes
Dec 13, 2024
dcherian
force-pushed
the
update-stateful-test
branch
from
3e93f55 to
a10e57f
Compare
5 tasks
dcherian
commented
Dec 18, 2024
This reverts commit 2c11287.
dcherian
force-pushed
the
update-stateful-test
branch
from
8470c6a to
cda8c36
Compare
dcherian
commented
Dec 18, 2024
dcherian
commented
Dec 18, 2024
dcherian
force-pushed
the
update-stateful-test
branch
from
3e66789 to
bd937b1
Compare
dcherian
commented
Dec 18, 2024
This reverts commit bd937b1.
dcherian
commented
Dec 20, 2024
dcherian
commented
Dec 20, 2024
| @@ -75,45 +75,39 @@ impl ChangeSet { | |||
| self.new_arrays.get(path) | |||
| } | |||
|
|
|||
| /// IMPORTANT: This method does not delete children. The caller | |||
There was a problem hiding this comment.
I think this is the cleanest solution. Session.delete_group already builds a list of nodes to delete. We could filter those to not be children of any groups added in this session (and may some other similarly complex predicate), or we can just not delete children ...
* main: Bring 'test_can_read_old' test back to life (#511) Join `RepositoryConfig` and `StoreConfig` (#507) Bump jinja2 in /docs in the pip group across 1 directory (#509) Python can now set checksums for virtual chunk refs (#508) Virtual chunk location inspection (#504) Canonicalize local fs object store (#501) Python library learns how to use virtual chunk containers (#502) [python] Add conflicted chunk report out on `RebaseFailedError` (#495) Refactor how we initialize and configure storage and repos (#497) Refine Ref APIs on Repository (#499)
paraseba
reviewed
Dec 27, 2024
paraseba
approved these changes
Dec 27, 2024
dcherian
added a commit
that referenced
this pull request
Jan 2, 2025
* main: (22 commits) Re-implement `list_dir` (#528) Fix rust library release action (#531) 0.1.0-alpha.8 release preparation (#530) Add docstrings (#529) Update dask_write.py to the new API (#527) Add support for Google Cloud Storage (#503) Fix s3 prefix issue (#525) Fix reset_branch op in stateful test (#521) Implement credentials refresh (#517) Add stateful testing for version control ops (#453) Create specific credential types for each object store (#512) Add committing to stateful test actions (#466) Bring 'test_can_read_old' test back to life (#511) Join `RepositoryConfig` and `StoreConfig` (#507) Bump jinja2 in /docs in the pip group across 1 directory (#509) Python can now set checksums for virtual chunk refs (#508) Virtual chunk location inspection (#504) Canonicalize local fs object store (#501) Python library learns how to use virtual chunk containers (#502) [python] Add conflicted chunk report out on `RebaseFailedError` (#495) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
xref #445
Fixes another bug found by hypothesis, if the same path is in the snapshot, then deleted, then recreated we list both old and new chunk paths