Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 21, 2020

• Update requirements.txt

  pmyteh

  committed
  ba226f0

  Copy full SHA for ba226f0

• refetchcontrol: fix lack of debug information on state of debug flag(!)

  pmyteh

  committed
  6cdfd38

  Copy full SHA for 6cdfd38

• Add notes on S3 location overriding

  pmyteh

  committed
  df064e3

  Copy full SHA for df064e3

Commits on Apr 10, 2020

• Minor logging improvements

  pmyteh

  committed
  c9b265b

  Copy full SHA for c9b265b

Commits on Apr 9, 2020

• Tidy up .gitignored files

  pmyteh

  committed
  265c704

  Copy full SHA for 265c704

• .gitignore updates

  pmyteh

  committed
  04ed2ca

  Copy full SHA for 04ed2ca

Commits on Feb 5, 2020

• GuardianSpider: clean up bodytext

  pmyteh

  committed
  4e44bc5

  Copy full SHA for 4e44bc5

• Add new NewsSpecifedSpider class for fetching a fixed list of URLs rather than reading an RSS feed or sitemap

  Show description for 2b0820d
  pmyteh

  committed
  2b0820d

  Copy full SHA for 2b0820d

• Add articleid field to output

  Show description for 103e1e0
  pmyteh

  committed
  103e1e0

  Copy full SHA for 103e1e0

• Optimise readability extractor, and debug note if needed

  pmyteh

  committed
  cc4c896

  Copy full SHA for cc4c896

Commits on Jan 24, 2020

• Add BildSpider, docs

  pmyteh

  committed
  888ff0e

  Copy full SHA for 888ff0e

Commits on Jan 20, 2020

• Update several sitemap locations

  pmyteh

  committed
  e6ccc41

  Copy full SHA for e6ccc41

Commits on Jan 17, 2020

• LOG_LEVEL to 'INFO' and 'STRIPRAWPAGE_ENABLED' to True by default; update README.md

  pmyteh

  committed
  9f196ef

  Copy full SHA for 9f196ef

Commits on Jan 15, 2020

• README.md: Fix installation instructions, add overview

  pmyteh

  committed
  7a3d20e

  Copy full SHA for 7a3d20e

Commits on Jun 10, 2019

• settings: handle no-AWS case better. Closes #2

  pmyteh

  committed
  b8a16b7

  Copy full SHA for b8a16b7

Commits on May 30, 2019

• stack: update to scrapy:1.6-py37

  pmyteh

  committed
  3775eb9

  Copy full SHA for 3775eb9

Commits on May 27, 2019

• README: Notes on geolocation issues

  pmyteh

  committed
  49b4779

  Copy full SHA for 49b4779

• washingtonpost: update feed URL

  pmyteh

  committed
  398d6a1

  Copy full SHA for 398d6a1

• huffpouk: currently non-functional

  pmyteh

  committed
  0042f08

  Copy full SHA for 0042f08

• huffpode: removed spider, site closed

  pmyteh

  committed
  cd640b6

  Copy full SHA for cd640b6

Commits on May 26, 2019

• foxnews: update sitemap URL

  pmyteh

  committed
  bc19a46

  Copy full SHA for bc19a46

• buzzfeedall: updated feed URL

  pmyteh

  committed
  a12b0f7

  Copy full SHA for a12b0f7

• buzzfeednewsfr: removed; site has closed

  pmyteh

  committed
  f608d0a

  Copy full SHA for f608d0a

• settings: scrapy now seems to install spiders recursively

  pmyteh

  committed
  772d8df

  Copy full SHA for 772d8df

• bbc: fix code error

  pmyteh

  committed
  473aa10

  Copy full SHA for 473aa10

• businessinsiderus: new sitemap URL

  pmyteh

  committed
  ff9736e

  Copy full SHA for ff9736e

• businessinsiderde: new sitemap URL

  pmyteh

  committed
  d43a814

  Copy full SHA for d43a814

• businessinsideruk: removed, now no separate edition

  pmyteh

  committed
  e3ca962

  Copy full SHA for e3ca962

• bigstory.ap.com gone: AP crawler non-functional

  pmyteh

  committed
  bec1116

  Copy full SHA for bec1116

• Config tweaks

  pmyteh

  committed
  9c9366b

  Copy full SHA for 9c9366b

Commits on Aug 8, 2018

• RefetchControl: upgrade database messages to logger.info

  pmyteh

  committed
  2b7b1aa

  Copy full SHA for 2b7b1aa

Commits on Aug 7, 2018

• Add new Scrapinghub project

  pmyteh

  committed
  74c17ca

  Copy full SHA for 74c17ca

• Debug

  pmyteh

  committed
  1b7fb0c

  Copy full SHA for 1b7fb0c

• Update README.md (close issue #1)

  pmyteh

  committed
  a822687

  Copy full SHA for a822687

Commits on Jul 25, 2018

• Add 'originalurl' field to output, tracking redirects etc.

  pmyteh

  committed
  2d6d663

  Copy full SHA for 2d6d663

Pagination

Previous

Next