ADBLK-12: Blocked Zones as list



Issue Information

Issue Type: Task
 
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: Adblock Lists (ADBLK)
Resolution: Done (2019-06-20 18:30:08)

Created: 2019-06-20 17:52:17
Time Spent Working


Description
https://dns.bentasker.co.uk consumes blockedzones.unbound.txt (implemented in ADBLK-8) in order to block entire zones. Currently it has to process the list upon receipt to extract the zones.

It'd be helpful if there were a documented list of blocked zones (i.e. not in unbound format) so that they could be used to build a regex to feed into things like ftldns.


Issue Links

Toggle State Changes

Activity



Repo: adblocklists
Commit: 23744b8a2f5819c8199879b58e92401c923ed4fa
Author: B Tasker <github@<Domain Hidden>>

Date: Thu Jun 20 18:04:59 2019 +0100
Commit Message: ADBLK-12 Generate a plaintext list of blocked zones - push into output file `blockedzones.txt`



Modified (-)(+)
-------
bin/update_addomains.sh




Webhook User-Agent

GitHub-Hookshot/81d186e


View Commit

I've just tested this by creating a script to translate it into a list of regex's for pihole
#!/bin/bash
#
# Pull down a list of blocked zones (implemented in ADBLK-12)
# then translate them into regex's to feed into Pihole
#
# This is better than blocking upstream, as a block in FTL will
# result in the adblock counter increasing
#
# Copyright (c) 2019 B Tasker
# Released under GNU GPL V3 - see https://www.gnu.org/licenses/gpl-3.0.txt
# 
#

# Create a temp file
OUT="$(mktemp)"
> $OUT


# Fetch the blocked zone file
for zone in `curl -s "https://www.bentasker.co.uk/adblock/blockedzones.txt"`
do
    regex=$(echo "$zone" | sed 's~\.~\\\.~g')
    echo "^.+\.$regex\$" >> $OUT
done

count=$(wc -l "$OUT" | cut -d\  -f1)

if [ $count -gt 0 ]
then

    # Check the files actually differ
    cd /etc/pihole
    diff -u "$OUT" regex.list > /dev/null
    if [ "$?" == "1" ]
    then
        # Files differ. Make a backup
        cp -n regex.list regex.list.old
        cat "$OUT" > regex.list

        # Tell Pihole to reload the regexes
        echo ">recompile-regex" | nc -q1 localhost 4711
    fi
fi

rm -f "$OUT"


I deliberately added some broken regex's to the file and triggered the recompile - pihole-FTL continues running and just complains about those regex's in /var/log/pihole-FTL.log (the other unbroken regex's carry on working), so this looks to be a fairly safe operation.
btasker changed status from 'Open' to 'Resolved'
btasker added 'Done' to resolution
btasker changed status from 'Resolved' to 'Closed'