Action for periodic refresh of pixi.lock file?

[traversaro]

Thanks a lot to all the contributors of setup-pixi action, is quite useful. I am interested in a action to complement its behavior. The logic of the action is not too complex so probably I could implement it independently, but I wanted to collect early feedback if anyone was interested.

In particular, I am super happy that now the CI works with a lock file, minimizing the possibility of environment regressions that make PRs fail for unrelated package upgrades. However, on the other hand as I typically maintain packages that are not always the leaf of a dependency tree, I do want to be aware if at some point a new version of a dependency make my package fail.

To solve this problem, I was thinking of creating an action that run periodically, and that does rm pixi.lock && pixi shell, to regenerate a fresh pixi.lock . Then it would open a PR to refresh the pixi.lock. On that PR, regular CI will run (I am not sure how to make CI run automatically on a PR create by an action, but that's another story) and if the CI is green, the PR with the new lock file will be merged. In this way, I would be notified if a new dependency broke my package, without loosing the advantages of lockfiles.

Anyone has any comment on this, for example an alternative workflow to achieve the same objective? I thought about adapting dependabot or similar tools, but that seems complex and their scope seems a bit different. What they do is to update the pinned version of dependencies in the pixi.toml, while in my case I would like to have as many dependencies in the pixi.toml set to = "*" (unless necessary, and I could discover this thanks to this action that periodically refresh the lockfile).

[pavelzw]

I am not sure how to make CI run automatically on a PR create by an action, but that's another story

try using SSH deploy keys in combination with with: ssh-key: ${{ secrets.SSH_PRIVATE_KEY }} in actions/checkout, then the PR won't be opened by the github user who cannot trigger actions but instead by the person who added the deploy key. (at least this worked for me in some situations)

[traversaro]

I am not sure how to make CI run automatically on a PR create by an action, but that's another story

try using SSH deploy keys in combination with with: ssh-key: ${{ secrets.SSH_PRIVATE_KEY }} in actions/checkout, then the PR won't be opened by the github user who cannot trigger actions but instead by the person who added the deploy key. (at least this worked for me in some situations)

Cool, thanks!

[pavelzw]

I think to do this properly, we would need a "lockfile diff tool" or pixi update (which supports json output that can tell you what exactly changed, xref #73 (comment)).

Ideally, imo, this behavior could be implemented in dependabot or renovate. I have heard there are some things happening on the renovate front with conda.

I am still a bit unsure what use cases such a tool should cover.

I think there are two main use cases:

Library

You develop a library and want to test against multiple versions (for example https://github.com/Quantco/polarify). There, the updater should come around occasionally and only update the upper bound if necessary. (you as maintainer should create necessary environments to properly test against all relevant versions).

Application

You develop an application and don't really care about supporting all versions of your dependency. There, the updater tool should come around occasionally and update both the upper bound and lower bound like dependabot for npm for example.

---

WDYT @0xbe7a @ruben-arts @wolfv @baszalmstra?

I'm also wondering how this is handled in languages with proper tooling such as rust.

I think #639 is also relevant for this issue.

[pavelzw]

@Hofer-Julian created a similar workflow in https://github.com/Deltares/iMOD-Documentation/blob/dd1c791944e1f513a8bd71a149b99e4a98e33a49/.github/workflows/pixi_auto_update.yml

[traversaro]

I think to do this properly, we would need a "lockfile diff tool" or pixi update (which supports json output that can tell you what exactly changed, xref prefix-dev/pixi#73 (comment)).

Why you think that is the case? If pixi.lock is unchanged, no PR will be opened as there is nothing to commit (that is already the default behavior of https://github.com/marketplace/actions/create-pull-request). By the way, thanks for linking #73, I remember something related but I could not find it.

I am still a bit unsure what use cases such a tool should cover.

It kind of covers the case in which you develop a library (so the first option), but you do not put upper bounds in the pixi.toml (almost only ="*"). I am not sure how niche is this user case is, but it is my use case at the moment. : ) If dependabot/renovate supported pixi.toml probably the need for having almost always ="*" would be reduced and hence the need of periodically refreshing pixi.lock without changing pixi.toml, but at the moment that is not the case.

@Hofer-Julian created a similar workflow in https://github.com/Deltares/iMOD-Documentation/blob/dd1c791944e1f513a8bd71a149b99e4a98e33a49/.github/workflows/pixi_auto_update.yml

That is exactly what I was looking for, thanks!

[Hofer-Julian]

I'm also wondering how this is handled in languages with proper tooling such as rust.

Most Rust projects that I know that have something like that use dependabot.
The supported update types are documented here: https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#specifying-dependencies-and-versions-to-ignore

[pavelzw]

Why you think that is the case?

At least the lockfile diff tool would be nice to be able to generate proper changelogs. Internally in our company, we have a tool to update lockfiles for conda which results in PR descriptions like this (see Quantco/multiregex#50):

It's not super pretty and maybe needs some formatting updates but if we decide to create something like this, I would like to have such changelogs in the PR descriptions.

[traversaro]

I see, thanks for the explanation.

[ruben-arts]

For Rust we often use cargo update and cargo upgrade. One updating the lockfile with current boundaries of the config file. And upgrade actually upgrading the boundaries as well. These are tools that I would love to see in pixi as well!

[ruben-arts]

The lockfile diff tool is also a good idea!

[traversaro]

For Rust we often use cargo update and cargo upgrade. One updating the lockfile with current boundaries of the config file. And upgrade actually upgrading the boundaries as well. These are tools that I would love to see in pixi as well!

As linked by @pavelzw, the issue #73 is kind of related to that.

[traversaro]

@Hofer-Julian created a similar workflow in https://github.com/Deltares/iMOD-Documentation/blob/dd1c791944e1f513a8bd71a149b99e4a98e33a49/.github/workflows/pixi_auto_update.yml

Just to have all aligned, at the moment by specific problem is solved by this. I think this may be a useful discussion, but anyhow I do not have need any specific change in setup-pixi in the short term. So, when you prefer feel free to close the issue, thanks!

[pavelzw]

IMO as of now, until pixi supports update and upgrade, I don't think we should act. @Hofer-Julian could think about making his solution a reusable workflow that other repos can use.
If pixi had all tools available (i.e. lockfile diff, update/upgrade command), we could think about creating an official reusable workflow for this in the prefix-dev github namespace. (At least until there is proper dependabot/renovate support for pixi.)

But I don't think that this should be happening in prefix-dev/setup-pixi. Maybe something like prefix-dev/update-pixi-lockfiles.

[Hofer-Julian]

@Hofer-Julian could think about making his solution a reusable workflow that other repos can use.

I didn't know about reusable workflows. Will see if I can find time to set that up.

[pavelzw]

Deltares/Ribasim#1210 for tracking

[pavelzw]

@ruben-arts can you transfer this issue into a prefix-dev/pixi discussion?

[traversaro]

Just an update on my side on this: after a bit of time, I started having doubts that continuously updating the pixi.lock unless necessary make sense for us. Instead, to reach the same goal (make sure that the software works with the latest version of the dependencies) I switched to have a scheduled job that runs the CI on a fresh pixi.lock, but then does not update it. The pixi.lock can be updated by the developers as necessary, but indipendently from CI. See robotology/whole-body-estimators#191 for more details.

[pavelzw]

I added an example on how to do this with prettier diffs, see #1456

[pavelzw]

Should be solved with https://pixi.sh/dev/advanced/updates_github_actions/

[traversaro]

Great, thanks a lot!