Commits

Branch selector

 pr/158

User selector

All users

Datepicker

All time

Commit History

Commits on Sep 28, 2016

• Support for per-host politeness

  Show description for 0f18d9a
  

  rzo1

  authored and

  Yasser Ganjisaffar

  committed
  0f18d9a

  Copy full SHA for 0f18d9a

Commits on Aug 10, 2016

• Merge pull request yasserg#151 from cmacdonald/PR-ContentType

  Show description for 096819e
  yasserg

  authored
  096819e

  Copy full SHA for 096819e

Commits on Aug 9, 2016

• dont throw NPE if Content-Type is missing

  

  Craig Macdonald

  authored and

  Craig Macdonald

  committed
  4bff407

  Copy full SHA for 4bff407

Commits on Apr 4, 2016

• Merge pull request yasserg#117 from rvenutolo/robotstxtuseragentlowercase

  Show description for 4a89f0a
  yasserg

  committed
  4a89f0a

  Copy full SHA for 4a89f0a

Commits on Apr 2, 2016

• Converted crawler user agent string to lowercase to address bug with non-lowercase values

  rvenutolo

  committed
  9076c81

  Copy full SHA for 9076c81

• Added unit test to demonstrate bug when crawler user agent is non-lowercase

  rvenutolo

  committed
  bb6bb78

  Copy full SHA for bb6bb78

Commits on Mar 21, 2016

• Merge pull request yasserg#115 from skalca/readme-typo-proxy-password

  Show description for 996f6d0
  yasserg

  committed
  996f6d0

  Copy full SHA for 996f6d0

• Fixed typo at proxy authentication

  skalca

  committed
  9566990

  Copy full SHA for 9566990

Commits on Jan 8, 2016

• Update README.md

  yasserg

  committed
  2932e72

  Copy full SHA for 2932e72

• Merge pull request yasserg#109 from natemago/master

  Show description for 16a7299
  yasserg

  committed
  16a7299

  Copy full SHA for 16a7299

• Attempt to use already closed Environment object when shutting down crawl controller explicitly

  Show description for 5bdcf4b
  natemago

  committed
  5bdcf4b

  Copy full SHA for 5bdcf4b

Commits on Dec 18, 2015

• Merge pull request yasserg#106 from rdelaoc/UrlRequestParameterOrdering

  Show description for fdb3baa
  yasserg

  committed
  fdb3baa

  Copy full SHA for fdb3baa

• The canonicalization of request URLs alphabetizes the parameters

  Show description for b52e54c
  

  rdelaoc

  committed
  b52e54c

  Copy full SHA for b52e54c

Commits on Dec 3, 2015

• [maven-release-plugin] prepare for next development iteration

  

  Yasser Ganjisaffar

  committed
  48ce714

  Copy full SHA for 48ce714

• [maven-release-plugin] prepare release crawler4j-4.2

  

  Yasser Ganjisaffar

  committed
  a8121e3

  Copy full SHA for a8121e3

Commits on Jul 21, 2015

• Merge pull request yasserg#70 from MadEgg/fetch-errors

  Show description for 810f705
  yasserg

  committed
  810f705

  Copy full SHA for 810f705

• onUnhandledException - fix duplicate logging issue

  EgbertW

  committed
  f9c82fe

  Copy full SHA for f9c82fe

Commits on Jul 20, 2015

• added a callback for unhandled exceptions during fetching

  EgbertW

  committed
  28ca8eb

  Copy full SHA for 28ca8eb

Commits on Jul 19, 2015

• Merge pull request yasserg#54 from MadEgg/binary-content

  Show description for bdbdc3e
  yasserg

  committed
  bdbdc3e

  Copy full SHA for bdbdc3e

• Merge pull request yasserg#55 from MadEgg/bug-html-metarefresh

  Show description for 373f09d
  yasserg

  committed
  373f09d

  Copy full SHA for 373f09d

• Merge pull request yasserg#65 from MadEgg/shutdown-empty

  Show description for 7ed6385
  yasserg

  committed
  7ed6385

  Copy full SHA for 7ed6385

Commits on Jun 30, 2015

• Merge pull request yasserg#75 from MadEgg/accept-2xx

  Show description for ce856f4
  yasserg

  committed
  ce856f4

  Copy full SHA for ce856f4

• 2xx status codes

  EgbertW

  committed
  f79a784

  Copy full SHA for f79a784

Commits on Jun 1, 2015

• Merge pull request yasserg#74 from albert0815/albert0815-close-entity

  Show description for de46d30
  yasserg

  committed
  de46d30

  Copy full SHA for de46d30

• Update PageFetcher.java

  albert0815

  committed
  4fd9c84

  Copy full SHA for 4fd9c84

• Update PageFetcher.java

  albert0815

  committed
  2adfeb3

  Copy full SHA for 2adfeb3

Commits on May 28, 2015

• Update README.md

  yasserg

  committed
  be02e29

  Copy full SHA for be02e29

• Update README.md

  yasserg

  committed
  3d4354a

  Copy full SHA for 3d4354a

• Add license notice

  yasserg

  committed
  64915c8

  Copy full SHA for 64915c8

• Update README.md

  yasserg

  committed
  7502a4a

  Copy full SHA for 7502a4a

• Add travis config

  yasserg

  committed
  e0422d7

  Copy full SHA for e0422d7

• Merge pull request yasserg#69 from MadEgg/fix-tld

  Show description for fd59541
  yasserg

  committed
  fd59541

  Copy full SHA for fd59541

Commits on May 26, 2015

• fix tldlist absolute path

  EgbertW

  committed
  5d555f3

  Copy full SHA for 5d555f3

• Added configuration parameter processBinaryContentInCrawling that

  Show description for 705cac0
  EgbertW

  committed
  705cac0

  Copy full SHA for 705cac0

• Fix bug where metaRefresh was used instead of metaLocation. Copy/paste

  Show description for 4f6b7db
  EgbertW

  committed
  4f6b7db

  Copy full SHA for 4f6b7db

Pagination

Previous

Next