The project is moved to http://opendir.io Your help is welcome. Just sign in to opendir.io with github account and share cool github projects you know.
The list of tools, programming libraries and APIs used in web-scraping.
- Python
- PHP
- Ruby
- JavaScript
- Golang
- Feel free to add your favourite language. Use new_language_template.md as start point.
- Proxy Services
- Web Services
- Console tools
- Books
- lists - List of useful, silly and awesome lists curated on GitHub
- HeadlessBrowsers - a list of (almost) all headless web browsers in existence
Make this list better! Your contributions are always welcome! See contributing how-to
This list partially contains data from these sources:
- awesome-python by vinta / CC BY 4.0
- awesome-php by ziadoz
- awesome-ruby by markets
- ruby-nlp by diasks2
- awesome-javascript by sorrycc