Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inconsistent lockfile for github + do not re-clone github based dependency #2865

Open
geoHeil opened this issue Jan 9, 2025 · 4 comments
Open
Labels
🐞 bug Something isn't working 🐍 pypi Issue related to PyPI dependencies

Comments

@geoHeil
Copy link

geoHeil commented Jan 9, 2025

Problem description

given

[project]
name = "foo"
version = "1.0.0"
description = "bar"
authors = [{ name = "baz", email = "bafoo" }]
requires-python = "==3.13"

[tool.pixi.project]
platforms = ["linux-64", "osx-arm64", "osx-64", "linux-aarch64"]
channels = ["conda-forge"]

[build-system]
build-backend = "hatchling.build"
requires = ["hatchling"]

[tool.pixi.environments]
layoutparser = { features = [
  "layoutparser",
], solve-group = "layoutparser",  no-default-feature = true }


[tool.pixi.feature.layoutparser.pypi-options]
no-build-isolation = ["detectron2"]

[tool.pixi.feature.layoutparser.dependencies]
python = "~=3.12.0"
pip = "*"
pytorch = { version = ">=2.5.0,<2.6", channel = "conda-forge" }
torchvision = { version = ">=0.20.0,<0.21", channel = "conda-forge" }
tesseract = "~=5.5.0"

[tool.pixi.feature.layoutparser.pypi-dependencies]
layoutparser = "==0.3.4"
pytesseract = "~=0.3.13"
detectron2 = { git = "https://github.com/facebookresearch/detectron2.git", tag = "v0.6" }

a first pixi shell -e layoutparser works as expected (everything is installed). However, a 2nd one is still re-downloading the git repository (which is already cloned). this makes this task very slow.

It can be remediated by: pixi shell -e layoutparser --frozen

However a:

pixi shell -e layoutparser --locked
Error:
  × lock-file not up-to-date with the project

Removing:

detectron2 = { git = "https://github.com/facebookresearch/detectron2.git", tag = "v0.6" }

fixes the inconsistency

@geoHeil geoHeil changed the title do not re-clone github based dependency inconsistent lockfile for github + do not re-clone github based dependency Jan 9, 2025
@geoHeil
Copy link
Author

geoHeil commented Jan 9, 2025

#2373 (comment) is a similar case (but different)

@ruben-arts
Copy link
Contributor

Thanks for the report, the problem is that tag and branch aren't "locked" versions, so we refetch to validate the state of the tag or branch. This takes time.

A workaround for now is the use of rev instead of tag.

What would you expect that pixi should do? We might be able to only refetch on pixi install or something a like?

@geoHeil
Copy link
Author

geoHeil commented Jan 10, 2025

I am OK with rev and understand tag takes time.

What is surprising for me though, that the lockfile is inconsistent.

With the same tag I would expect the lockfile to stay consistent

@ruben-arts
Copy link
Contributor

Right, this should be fixed. I believe @nichmor already had ideas for this.

@ruben-arts ruben-arts added 🐞 bug Something isn't working 🐍 pypi Issue related to PyPI dependencies labels Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working 🐍 pypi Issue related to PyPI dependencies
Projects
None yet
Development

No branches or pull requests

2 participants