Too many excludes are silently ignored

[szepeviktor]

Hello @allinurl !

I see you liked my excludes generator in #700 (comment)

For example only using Pingdom's IP-s goaccess silently ignores the end of the list
-e 5.172.196.188 -e 5.178.78.77 -e 23.22.2.46 -e 23.111.152.74 -e 27.122.14.7 -e 37.252.231.50 -e 43.225.198.122 -e 43.229.84.12 -e 46.20.45.18 -e 46.165.195.139 -e 50.16.153.186 -e 50.22.90.227 -e 50.23.94.74 -e 52.0.204.16 -e 52.24.42.103 -e 52.48.244.35 -e 52.52.34.158 -e 52.52.95.213 -e 52.52.118.192 -e 52.57.132.90 -e 52.59.46.112 -e 52.59.147.246 -e 52.62.12.49 -e 52.63.142.2 -e 52.63.164.147 -e 52.63.167.55 -e 52.73.209.122 -e 52.89.43.70 -e 52.197.31.124 -e 52.197.224.235 -e 52.198.25.184 -e 52.201.3.199 -e 52.209.34.226 -e 52.209.186.226 -e 52.210.232.124 -e 54.68.48.199 -e 54.70.202.58 -e 64.120.6.122 -e 64.237.49.203 -e 64.237.55.3 -e 67.228.213.178 -e 69.59.28.19 -e 70.32.40.2 -e 72.46.130.42 -e 72.46.140.106 -e 72.46.153.26 -e 76.72.167.90 -e 76.72.167.154 -e 76.72.172.208 -e 76.164.194.74 -e 82.103.136.16 -e 82.103.139.165 -e 83.170.113.210 -e 85.17.156.11 -e 85.17.156.76 -e 85.93.93.123 -e 85.93.93.124 -e 85.93.93.133 -e 89.163.146.247 -e 89.163.242.206 -e 94.247.174.83 -e 95.141.32.46 -e 95.211.198.87 -e 95.211.217.68 -e 96.31.66.245 -e 103.47.211.210 -e 108.62.115.226 -e 109.123.101.103 -e 159.8.146.132 -e 162.218.67.34 -e 168.1.92.58 -e 173.248.147.18 -e 173.254.206.242 -e 174.34.156.130 -e 174.34.162.242 -e 174.34.224.167 -e 175.45.132.20 -e 178.255.152.2 -e 178.255.153.2 -e 178.255.154.2 -e 178.255.155.2 -e 184.75.208.210 -e 184.75.209.18 -e 184.75.210.90 -e 184.75.210.226 -e 184.75.214.66 -e 185.39.146.214 -e 185.39.146.215 -e 185.70.76.23 -e 188.138.40.20 -e 199.87.228.66 -e 204.152.200.42 -e 207.244.80.239 -e 208.43.68.59 -e 208.64.28.194 -e 209.126.117.87 -e 209.126.120.29 -e 211.44.63.35
but not the option (-f) after excludes, so it is not a command line problem.

Is there a limit to the number of excludes?

I've trying with 901 IP-s/ranges.

[szepeviktor]

Ooops! It is 64 😢

goaccess/src/settings.h

Line 38 in 8dbfef2

#define MAX_IGNORE_IPS 64

Should I recompile goaccess to raise it?

[szepeviktor]

Could we get a warning on exceeding this limit?

[allinurl]

That's right, you certainly need to bump up the limit and recompile. Note that the size of the list is proportional to the run time, which could make this pretty slow though.

I like the idea of having a warning — currently excludes are being pushed under options.c.

[szepeviktor]

Thanks.

Note that the size of the list is proportional to the run time, which could make this pretty slow though.

Do you think 1024 is a big value?

[allinurl]

I think 1024 is not that bad, but I'd give it a shot and see how it performs for you from 64 to 1024. If you feel it's not bad, you could bump it up even more. However, using ranges as much as possible might be a better choice.

[szepeviktor]

Wooooorks! :)

[allinurl]

Per this commit, the debug warning has been added.

Closing this. Thanks!