Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Implement an example using a neural-net based classifier to do Classifier CI Testing #43

Open
adam2392 opened this issue Sep 7, 2022 · 0 comments
Labels
conditional-independence Related to CI testing good first issue Good for newcomers

Comments

@adam2392
Copy link
Collaborator

adam2392 commented Sep 7, 2022

We are interested in a new user contribution of an example where they demonstrate how to use the ClassifierCITest with a scikit-learn classifier and a neural-net based classifier.

This would enable CI testing over high-dimensional objects, such as images and pixels for example. The example does not necessarily need to operate over images/pixels.

Ideally the pytorch model is already trained and fitted, and the model is just pulled from something like hugging face. We can introduce a test-time dependency on pytorch and hugging face. Then the model should be relatively small, so it can run in the continuous integration platform.

"Could we expand this method to work with a neural-net based classifier using Keras or Pytorch? It likely feels like overkill but it would be a first step to telling a story to the broader community that this repo is using cutting edge "deep" methods. We could also workshop an example with multidimensional variables later (e.g. pixels) though not in this PR.

Originally posted by @robertness in #28 (review)"

@adam2392 adam2392 added the good first issue Good for newcomers label Sep 7, 2022
@adam2392 adam2392 added the conditional-independence Related to CI testing label Feb 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conditional-independence Related to CI testing good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant