Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: allow dataset collections to be stored in data libraries #16171

Open
gregvonkuster opened this issue Jun 1, 2023 · 1 comment

Comments

@gregvonkuster
Copy link
Contributor

It would be very useful to be able to store dataset collections in data libraries. Currently, importing a dataset collection from a history to a data library will flatten the collection.

I have many workflows that expect dataset collections as inputs (in addition to samples being analyzed). These collection inputs for the workflow are typically static (i.e., the collection elements do not change for a particular workflow version), so importing the workflow inputs from the data library to a history for running the workflow would be streamlined if the collection inputs could be stored in the data library as a collection. Currently, users have to build the collection from the elements stored separately in the data library for each workflow run.

@mvdbeek
Copy link
Member

mvdbeek commented Jul 8, 2023

I personally (this is not the view of the team!) don’t think we should do this, unless the implementation is trivially easy and clean. I’d rather like libraries to act as a cache system, and use workflow default files for the usecase you're describing. I think that would result in more portable and better described workflows.

I'd also say that you could use the apply rules tool to build up the inputs from a flat collection as part of your workflow today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants