Commits

Branch selector

 master

User selector

All users

Datepicker

All time

Commit History

Commits on Apr 7, 2013

• Merge pull request theanti9#14 from miklagard/patch-1

  Show description for 3e3ede1
  theanti9

  committed
  3e3ede1

  Copy full SHA for 3e3ede1

Commits on Mar 18, 2013

• Update query.py

  Show description for f4e9cdd
  

  Cem YILDIZ

  committed
  f4e9cdd

  Copy full SHA for f4e9cdd

Commits on Feb 10, 2013

• Merge pull request theanti9#12 from EthanBlackburn/master

  Show description for 62d2dbc
  theanti9

  committed
  62d2dbc

  Copy full SHA for 62d2dbc

Commits on Jan 8, 2013

• fixed content processor

  Show description for 3d0dcc3
  

  Ethan Blackburn

  committed
  3d0dcc3

  Copy full SHA for 3d0dcc3

• Revert "fixed content_processor"

  Show description for 8169ee5
  

  Ethan Blackburn

  committed
  8169ee5

  Copy full SHA for 8169ee5

• fixed content_processor

  Show description for 8ea6f69
  

  Ethan Blackburn

  committed
  8ea6f69

  Copy full SHA for 8ea6f69

Commits on Dec 4, 2011

• Fixed merge conflict

  theanti9

  committed
  2d186f4

  Copy full SHA for 2d186f4

• Changes to the content processor

  theanti9

  committed
  38593b6

  Copy full SHA for 38593b6

Commits on Nov 27, 2011

• Merge pull request theanti9#8 from WilHall/master

  Show description for fafb8c1
  theanti9

  committed
  fafb8c1

  Copy full SHA for fafb8c1

• Added logging facility, ColorStreamHandler, and updated .gitignore

  WilHall

  committed
  d9ed584

  Copy full SHA for d9ed584

• Merge branch 'master' of https://github.com/theanti9/PyCrawler

  Show description for 5db8daf
  WilHall

  committed
  5db8daf

  Copy full SHA for 5db8daf

Commits on Nov 1, 2011

• Work on colors, Added option to rotate sqlite database files

  WilHall

  committed
  ad55cd8

  Copy full SHA for ad55cd8

• Fix the problem that caused it to leave thread pools open after starting and flooding the system with dead processes

  theanti9

  committed
  ca6d7a8

  Copy full SHA for ca6d7a8

• Added user agent header to requests

  theanti9

  committed
  3b7e262

  Copy full SHA for 3b7e262

• lots of fixes

  theanti9

  committed
  dacd344

  Copy full SHA for dacd344

• Created Printer class for color printing

  WilHall

  committed
  4425b4c

  Copy full SHA for 4425b4c

• Adding readme

  theanti9

  committed
  6fdbc75

  Copy full SHA for 6fdbc75

• Complete redo

  theanti9

  committed
  ad8c39f

  Copy full SHA for ad8c39f

Commits on Aug 19, 2011

• Changed the linkregex to the suggestion in gtoffoli's issue theanti9#7 temporarily while i restructure/rewrite the entire project

  theanti9

  committed
  c462210

  Copy full SHA for c462210

Commits on Aug 8, 2011

• Updated .gitignore

  theanti9

  committed
  a2abec1

  Copy full SHA for a2abec1

Commits on Jun 16, 2011

• Merge pull request theanti9#5 from sjparsons/status

  Show description for f24e79b
  theanti9

  committed
  f24e79b

  Copy full SHA for f24e79b

Commits on Jun 15, 2011

• Status Codes and URL limited

  Show description for ba0ecb9
  sjparsons

  committed
  ba0ecb9

  Copy full SHA for ba0ecb9

Commits on May 8, 2011

• Fixing issue 2, psyco in __main__

  theanti9

  committed
  82ccb6b

  Copy full SHA for 82ccb6b

Commits on Jan 9, 2011

• Fixed links regexp + newline at end of file

  mswietochowski

  committed
  7ce95c0

  Copy full SHA for 7ce95c0

Commits on Nov 23, 2010

• Error fix - couldn't join parse object in urljoin.

  theanti9

  committed
  e14a6d0

  Copy full SHA for e14a6d0

• Error fix - couldn't join parse object in urljoin.

  theanti9

  committed
  ee70c79

  Copy full SHA for ee70c79

Commits on Nov 7, 2010

• Fixed major glitch in url joining.

  theanti9

  committed
  6261c47

  Copy full SHA for 6261c47

Commits on Oct 11, 2010

• Updated readme

  theanti9

  committed
  9da2e16

  Copy full SHA for 9da2e16

• Added robots.txt compliance and some small fixes.

  theanti9

  committed
  3017da5

  Copy full SHA for 3017da5

• Added more comments. Added optional arg 'verbose' as 4th arg to specify if the urls should be printed as they are being crawled.

  theanti9

  committed
  f24055b

  Copy full SHA for f24055b

• Restructed the way the request is made very slightly to add a User-Agent

  theanti9

  committed
  848ce4a

  Copy full SHA for 848ce4a

Commits on May 2, 2010

• Changed syntax error

  theanti9

  committed
  7f8095b

  Copy full SHA for 7f8095b

Commits on May 1, 2010

• logic error debugging

  theanti9

  committed
  9a25e59

  Copy full SHA for 9a25e59

Commits on Apr 30, 2010

• Got rid of beautiful soup as it caused problems with any page that has any malformed html.fixed a few syntax errors

  theanti9

  committed
  2b3f0df

  Copy full SHA for 2b3f0df

• Restructured the crawler. works with the new tables, crawles to limited specified depth. keeps current and parent ids so the gui can display the nesting properly

  theanti9

  committed
  c07b5dd

  Copy full SHA for c07b5dd

Pagination

Previous

Next