First off, thank you for taking the time to contribute! I appreciate it 🤩
Anything you want to say is welcome - as long as you are considerate of other people and express yourself with dignity.
Please feel free to open an issue.
Prepare lists of user agent strings before testing
npm run prepare -- -f- Manual legit browsers list: fixtures/browsers.yml
- Manual known crawlers list: fixtures/crawlers.yml
- Downloaded resources end up in fixtures/downloads folder
- user-agents package is used to add randomly generated browser user agents
A user agent string may be legit even if it is included in one of the automated download lists in. To explicitly exclude it, add it to fixtures/browsers.yml.
A user agent string may be a bot even if it was generated by user-agents. To explicitly include it, add it to fixtures/crawlers.yml.