MoaAB: Mother of All AD-BLOCKING

[xxcriticxx]

did you ran this list already?

https://adblock.mahakala.is/

[funilrys]

No but it's huge ...

[xxcriticxx]

@funilrys can you test this in travis with/without screen output ?

[funilrys]

It's almost impossible with the actual version of funceble because Travis have a hard coded timeout of 50 minutes but I'm currently working on a system to auto save/continue in case of interruption which could be great for this list and for the 1.6 Million domains of @mitchellkrogza which are waiting to be tested 😸

[mitchellkrogza]

Check the latest numbers at https://github.com/mitchellkrogza/Ultimate.Hosts.Blacklist/blob/master/README.md

[funilrys]

Aw sorry it's now 2.156 Millions 😹 😅 😹

[xxcriticxx]

this would probably crash my pi hole

[xxcriticxx]

@funilrys this big of job wont ping out your network card?

[mitchellkrogza]

@xxcriticxx @funilrys I added that one as an input source this morning on https://github.com/mitchellkrogza/Ultimate.Hosts.Blacklist

Went from

  Version: V1.2017.07.50
   Total Bad Hosts in hosts file: 2,155,768
   Total Bad Hosts in hosts.deny file: 64,020
   Total Bad Hosts and IP's in superhosts.deny file: 2,219,647

To:

    Version: V1.2017.07.51
    Total Bad Hosts in hosts file: 2,241,011
    Total Bad Hosts in hosts.deny file: 64,020
    Total Bad Hosts and IP's in superhosts.deny file: 2,304,890

And my hosts file is 💯 % guaranteed it has not one duplicate. That's a WHACK of domains and IP's now. 😬

[funilrys]

Awesome @mitchellkrogza 👍

@xxcriticxx Most WHOIS server will block your IP after 60 requests in an hour but that's why we use nslookup and soon the HTTP Code as analytic 😸

[mitchellkrogza]

I made some changes to that repo as it had grown into a massive 1.6 Gb in just a few days.

So now Travis is just committing modified files without creating tags and creating releases.

The hosts, hosts.deny and superhosts.deny files are now only provided in a zipped format.

Also added zipping and unzipping of template files into my build process and I truncate all combined input file lists before a commit.

MUCH smaller now 😁

Must be WARY of this with Travis and Git .... these releases tally up fast.
So pruning the repo occasionally is a necessity, busy with some final tests then I will do an aggressive prune on the repo and see what size it is after that.

[xxcriticxx]

@mitchellkrogza can you put copy on google drive or megadrive or something else?

[mitchellkrogza]

@xxcriticxx a copy of ??

[xxcriticxx]

copy of your host file

[mitchellkrogza]

@xxcriticxx why not just grab the zipped file? It's 1/4 the size. I could probably push the uncompressed files to a dropbox or google drive folder.

[xxcriticxx]

please provide links i hope my pi wont die

[mitchellkrogza]

Oh sorry, I see what you mean, you are pulling the raw hosts file. Ok I will leave it available in the repo both zipped and unzipped. New commit going out now.

[xxcriticxx]

yes raw links every sunday at midnight i think

[mitchellkrogza]

Raw files back in root of repo - https://github.com/mitchellkrogza/Ultimate.Hosts.Blacklist ..... zipped releases in their own folder 👍

[xxcriticxx]

there is 3 files what is what?

[mitchellkrogza]

@xxcriticxx read the README.md it explains what they are. You only really need the hosts file, but if you want you can also use the hosts.deny along with it.

hosts.deny > blocks incoming
hosts > blocks outgoing

superhosts.deny is just actually hosts.deny but everything combined into one massive file.

[mitchellkrogza]

@xxcriticxx I have once again removed the raw hosts files from the repo due to size issues but I have provided download links to the raw files in the README. The links to the raw downloads will always be up to date as this repo updates.

Download Raw hosts file

Download Raw hosts.deny file

Download Raw superhosts.deny file

[xxcriticxx]

@mitchellkrogza what's github size limit set to?

[mitchellkrogza]

@xxcriticxx They don't like files over 50mb in size. But the raw links I gave above are always up to date and my server is MUCH faster too. So it's a win win. When I have some more time I will look into Git Large File Storage - https://git-lfs.github.com and http://git.io/iEPt8g

[xxcriticxx]

@mitchellkrogza dont worry about i have to test first if your list doesn't cause to much problems in my house

Second list gets pulled once per week on Sunday only if it doesn't find new copy it keeps the old list

[mitchellkrogza]

@xxxcritic look forward to your feedback. Let me know how it works for you and over time I will perfect it into a seriously ultimate blacklist.

[xxcriticxx]

::: Getting hosts.ubuntu101.co.za list... done
:::   Status: Success (OK)
:::   List updated, transport successful!
::: 
::: Aggregating list of domains... done!
::: Formatting list of domains to remove comments.... done!
::: 3242283 domains being pulled in by gravity...
::: Removing duplicate domains.... done!
::: 2394006 unique domains trapped in the event horizon.

[funilrys]

Because it's now part of Ultimate Hosts Blacklist, closing.