Python Scrapy

Web Scraping with Scrapy

Start Project

1. Create a virtualenv

virtualenv -p python3 .venv

2. Activate virtualenv

source .venv/bin/activate

3. Install scrapy lib

pip install scrapy

1. List all spiders

scrapy list

2. Execute web scrapping

scrapy crawl quotes [opicional: -o output.json]

3. Shell test scrapy

scrapy shell 'http://exemplo.com'

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
tutorial		tutorial
.gitignore		.gitignore
README.md		README.md
__init__.py		__init__.py
scrapy.cfg		scrapy.cfg