python_crawler

Web crawling using Python is easy to follow because there are various libraries and its easiness to follow to achieve.

Here are three ways I have tried:

• 1. Using urlib
  • check crawler.py
• 2. Using scrapy
  • check crawler2.py
• 3. Using BeautifulSoup
  • check crawler3.py

Thing that I will crawl is going to be some reddit posts in reddit page in BlobkChain pages:

• Before doing this, please refer Here to aware of this information. Some websites have their specific rules of accessing their information.