Commits

Breadcrumbs

History for

onmaster

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 3, 2015

• Created the HttpWebResponseWrapper to abstract response creation - to allow a response to be created by a mean other than executing a web request.

  

  aleksandar.hummel

  committed
  e7540be

  Copy full SHA for e7540be

Commits on Apr 1, 2015

• Fixes sjdirect#68, "When you cancel the crawl job, the WebCrawler will dispose the _threadManager and its _resetEvent will also be displsed. But the ThreadManager class can't make sure that all thr…

  Show description for c983870
  sjdirect

  committed
  c983870

  Copy full SHA for c983870

Commits on Mar 31, 2015

• Added new IsIgnoreRobotsDotTextIfRootDisallowedEnabled config value which completely ignores the robots.txt file if it blocks the root. Ie.. act as if there is no robots.txt.

  sjdirect

  committed
  9bd3d7d

  Copy full SHA for 9bd3d7d

Commits on Mar 21, 2015

• [skip ci]

  sjdirect

  committed
  cf8f359

  Copy full SHA for cf8f359

Commits on Mar 17, 2015

• made all hyperlinkparser methods protected virtual

  sjdirect

  committed
  2853f8d

  Copy full SHA for 2853f8d

Commits on Feb 16, 2015

• Add canonical to links found in a page

  xavivars

  committed
  d450a13

  Copy full SHA for d450a13

Commits on Feb 14, 2015

• Minor refactoring to crawl delay [skip ci]

  sjdirect

  committed
  9f6bba1

  Copy full SHA for 9f6bba1

• removed unused line [skip ci]

  sjdirect

  committed
  374a54c

  Copy full SHA for 374a54c

Commits on Feb 13, 2015

• Removed Microsoft.Bcl.Async, dont need it yet.

  sjdirect

  committed
  3790b60

  Copy full SHA for 3790b60

• Slight refactoring of last checkin as well as adding checks to be sure abot doesn't get stuck in an infinite redirect

  sjdirect

  committed
  ce329e8

  Copy full SHA for ce329e8

Commits on Feb 11, 2015

• Adding support for not automatic redirects

  xavivars

  committed
  eb79e7d

  Copy full SHA for eb79e7d

Commits on Feb 5, 2015

• Merge branch 'master' of github.com:sjdirect/abot

  sjdirect

  committed
  695a72d

  Copy full SHA for 695a72d

• Raised the optimal bloom filter size to 2000001 items and the error rate to .1% to see if it addresses this issue...

  Show description for 7e6acdb
  sjdirect

  committed
  7e6acdb

  Copy full SHA for 7e6acdb

Commits on Feb 4, 2015

• updated all nuget packages

  sjdirect

  committed
  bc59f39

  Copy full SHA for bc59f39

Commits on Feb 3, 2015

• Fixes issue sjdirect#58 by handling both single and double quotes in the meta charset as well as handling a non closing meta tag.

  sjdirect

  committed
  f1f0153

  Copy full SHA for f1f0153

Commits on Feb 2, 2015

• updated all my nuget packages to latest versions. Changed out commoner.core to use the new nuget package that correctly utilizes the nuspec file.

  sjdirect

  committed
  014c3ff

  Copy full SHA for 014c3ff

• Removed submodules commoner and robots (both now have nuget packages) and split fiddler saz import functionality into commoner.core

  sjdirect

  committed
  4719c01

  Copy full SHA for 4719c01

• Removed assembly log4net configure call in abot.tests.unit to see if it runs better.

  sjdirect

  committed
  0ef49d1

  Copy full SHA for 0ef49d1

Commits on Jan 19, 2015

• removed some files

  sjdirect

  committed
  3d8ec23

  Copy full SHA for 3d8ec23

Commits on Jan 16, 2015

• Added AsyncWebCrawler with some stubbed work.

  sjdirect

  committed
  79bba99

  Copy full SHA for 79bba99

Commits on Jan 15, 2015

• Made all classes in Abot project Serializable so others may implement a pause/resume

  sjdirect

  committed
  db6bbce

  Copy full SHA for db6bbce

Commits on Jan 11, 2015

• Comment out all async code.

  sjdirect

  committed
  77d8d48

  Copy full SHA for 77d8d48

Commits on Jan 10, 2015

• Async impl now compiling but not tested

  sjdirect

  committed
  691d232

  Copy full SHA for 691d232

Commits on Jan 9, 2015

• Same as last checkin

  sjdirect

  committed
  c6cd041

  Copy full SHA for c6cd041

• Fixes sjdirect#55 by adding xml documentation for intellisense

  sjdirect

  committed
  48563d7

  Copy full SHA for 48563d7

Commits on Jan 5, 2015

• Starting to work on async PageRequester.MakeRequestAsync implementation. Commented out.

  sjdirect

  committed
  b07c9b1

  Copy full SHA for b07c9b1

Commits on Jan 1, 2015

• Removed items marked as obsolete

  sjdirect

  committed
  3e7fe4f

  Copy full SHA for 3e7fe4f

Commits on Dec 31, 2014

• Changed useragent to match ie11's useragent.

  sjdirect

  committed
  f209280

  Copy full SHA for f209280

• Merge branch 'master' of github.com:sjdirect/abot

  sjdirect

  committed
  ae768d2

  Copy full SHA for ae768d2

• Wrong check before invoking

  sjdirect

  committed
  2b8c598

  Copy full SHA for 2b8c598

Commits on Dec 29, 2014

• Added datetime stamps to allow performance calculations of before http request, after http request, before downloading page content and after downloading page content.

  sjdirect

  committed
  1264522

  Copy full SHA for 1264522

• Removed some project references with resharper, NOT Test.Unit and SiteSimulator since those caused issues.

  sjdirect

  committed
  5acc9bd

  Copy full SHA for 5acc9bd

Commits on Dec 28, 2014

• added todos

  sjdirect

  committed
  9c4ea12

  Copy full SHA for 9c4ea12

• Added basic retries, fixes issue sjdirect#47

  sjdirect

  committed
  a798626

  Copy full SHA for a798626

Commits on Dec 25, 2014

• Make all configs optional and have default values

  sjdirect

  committed
  ad47099

  Copy full SHA for ad47099

Pagination

Previous

Next