Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Jan 15, 2016

• fix

  sjdirect

  committed
  513b1c0

  Copy full SHA for 513b1c0

• fix

  sjdirect

  committed
  2b374f9

  Copy full SHA for 2b374f9

• Fix to test

  sjdirect

  committed
  58369a0

  Copy full SHA for 58369a0

• Correcting issue with RateLimiter being a little under the expected wait time. Added 10 milli constructor param.

  sjdirect

  committed
  0805c0a

  Copy full SHA for 0805c0a

• Added CrawledPage.Elapsed

  sjdirect

  committed
  91af8cc

  Copy full SHA for 91af8cc

Commits on Jan 9, 2016

• same as last

  sjdirect

  committed
  02dfd5b

  Copy full SHA for 02dfd5b

• Broken build, appveyor must use an old compiler since string interpolation did not work

  sjdirect

  committed
  8e64960

  Copy full SHA for 8e64960

• Merge branch 'master' of github.com:sjdirect/abot

  sjdirect

  committed
  821d6e9

  Copy full SHA for 821d6e9

• updated some limitations with regard to concurrency and rate limiting

  sjdirect

  committed
  521d36f

  Copy full SHA for 521d36f

Commits on Dec 31, 2015

• Update README.md

  Show description for 0897eb7
  sjdirect

  committed
  0897eb7

  Copy full SHA for 0897eb7

Commits on Dec 6, 2015

• added new abot config value for x-robot-tag http header [skip ci]

  sjdirect

  committed
  0010c13

  Copy full SHA for 0010c13

Commits on Dec 5, 2015

• Added ability for abot to obey the http header X-Robots-Tag. fixes sjdirect#65

  sjdirect

  committed
  1b5feb3

  Copy full SHA for 1b5feb3

• Moved the cleanURLFun to the base class where it can clean on the full absolute link, fixes sjdirect#101

  sjdirect

  committed
  58eb440

  Copy full SHA for 58eb440

Commits on Dec 2, 2015

• changed log message for root redirection

  sjdirect

  committed
  b4c8b2f

  Copy full SHA for b4c8b2f

Commits on Nov 28, 2015

• [skip ci]

  sjdirect

  committed
  b180271

  Copy full SHA for b180271

• [skip ci]

  sjdirect

  committed
  59b8c6b

  Copy full SHA for 59b8c6b

• [skip ci]

  sjdirect

  committed
  3383156

  Copy full SHA for 3383156

• Update README.md

  sjdirect

  committed
  762fb95

  Copy full SHA for 762fb95

Commits on Nov 22, 2015

• Updated icon, project url and copyright of nuget package

  sjdirect

  committed
  6fa8337

  Copy full SHA for 6fa8337

Commits on Nov 12, 2015

• Making this message more prominent since processing stops on the page and unless you have the level to debug you will not know why.

  sjdirect

  committed
  dda9094

  Copy full SHA for dda9094

Commits on Nov 9, 2015

• Merge pull request sjdirect#97 from coveo/checkLinksOnce

  Show description for e6a70d5
  sjdirect

  committed
  e6a70d5

  Copy full SHA for e6a70d5

Commits on Nov 6, 2015

• Moved default impl instantiation to Scheduler

  sjdirect

  committed
  b086d64

  Copy full SHA for b086d64

• Made the CompactCrawledUrlRepository the default impl. The average url from several sources appears to be above 32 characters which would leave a smaller memory footprint.

  sjdirect

  committed
  ddb818c

  Copy full SHA for ddb818c

• Renamed

  sjdirect

  committed
  0d5147f

  Copy full SHA for 0d5147f

• Added HashedCrawledUrlRepository but its ot the default

  sjdirect

  committed
  5f5f2af

  Copy full SHA for 5f5f2af

Commits on Nov 4, 2015

• Added a check to see if everything below the root is "allowed" by using a the root + "/aaaaa". If that link is disallowed and the IsIgnoreRobotsDotTextIfRootDisallowedEnabled is set to true, it wil…

  Show description for 4524db0
  sjdirect

  committed
  4524db0

  Copy full SHA for 4524db0

Commits on Nov 2, 2015

• Moved back the Url repository in the scheduler and added two new methods

  mikegron

  committed
  a540895

  Copy full SHA for a540895

• Keeping the CrawledUrlRepo as member of the WebCrawler and adding a validation when scheduling Links to check each link only once.

  mikegron

  committed
  19a5cd2

  Copy full SHA for 19a5cd2

• Merge branch 'master' of github.com:sjdirect/abot

  sjdirect

  committed
  56b75d4

  Copy full SHA for 56b75d4

Commits on Oct 29, 2015

• Merge pull request sjdirect#95 from henrikrossen/patch-1

  Show description for 2bcd341
  sjdirect

  committed
  2bcd341

  Copy full SHA for 2bcd341

• Update README.md changed obsolete PageSizeInBytes

  henrikrossen

  committed
  8e31cce

  Copy full SHA for 8e31cce

Commits on Oct 16, 2015

• Merge pull request sjdirect#94 from poxet/master

  Show description for c5e17c6
  sjdirect

  committed
  c5e17c6

  Copy full SHA for c5e17c6

• Fixed issue with ProtocolViolationException for some sites.

  Show description for 4eb13f5
  poxet

  committed
  4eb13f5

  Copy full SHA for 4eb13f5

Commits on Sep 21, 2015

• Merge branch 'master' of github.com:sjdirect/abot

  sjdirect

  committed
  3bdff16

  Copy full SHA for 3bdff16

• Merge pull request sjdirect#90 from xavivars/fixing-ratelimiter-issues

  Show description for 06f91c5
  sjdirect

  committed
  06f91c5

  Copy full SHA for 06f91c5

Pagination

Previous

Next