Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quality assurance idea: runtime quality checks #1245

Open
jayaddison opened this issue Sep 3, 2024 · 0 comments
Open

Quality assurance idea: runtime quality checks #1245

jayaddison opened this issue Sep 3, 2024 · 0 comments

Comments

@jayaddison
Copy link
Collaborator

The unit test suite is the primary component that we use to maintain the quality of the scrapers here; we use it to compare the output of scrapers on a sample, or a few samples, of test recipe HTML per recipe website.

However: that test coverage is extremely limited compared to all of the possible recipes that the code encounters when it is used in practice.

At a small performance penalty -- meaning that perhaps this feature should be opt-in -- we could add some runtime quality checks over the data that is returned when scraper fields are accessed. These checks could perhaps emit a warning-level message -- they certainly shouldn't block or break existing code -- to notify operators of a potential problem when scraping a recipe, and with instructions about how to report that as a bug here.

In particular it is #1238 that has made me think of this idea. In that PR, we replace static result values for a field (author) with dynamic retrieval that currently returns the same value -- and probably always should. That's the kind of situation where we could add a runtime check to ask "is the author value the expected static value?", and if not, emit a warning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant