Block ads easily using pfSense DNS Resolver

I have nothing against people who embed adverts into their web pages, hell, even I do it! But there are some websites where adverts are beyond a joke and they overpower the overall user experience, and we all know that’s just annoying!

This coincides with me being a user of pfSense for a good few years now, I recently took the plunge and ditched my EdgeRouter Lite at home for a Watchguard XC170 that I found on eBay for £24. Needless to say the WG runs the latest pfSense absolutely perfectly, it handles 4 VLAN’s, several LAN to LAN VPN’s and gets stuck into some serious caching. Whilst not essential I was looking to get even more functionality out of the red little beast, and the icing on the cake would ultimately be an automatic ad blocking system where I basically didn’t have to lift a finger.

I guess I should let you in on a secret… for a while now I’ve ran a private DNS server which intercepts known advertising domains and replaces the IP address leading to a blank GIF image, this works great but it’s yet another server to manage, so my thinking was to integrate the same functionality using pfSense, ultimately having everything hosted in one box.

The pfSense package manager has both pfBlockerNG and squidGuard available, I tried these but they felt over complicated for my needs, I wanted something light weight that I can set and forget, I didn’t want to go down the routes of a third party ad blocking plugin either, as well… you know what happened with AdBlock. Anywho, here’s where I got thinking.

Whilst this guide is specifically targeted to pfSense users, any system which runs the Unbound service will be able to work in the same way.

The Goal

  • I want to block all known advertising/shock site domains through the pfSense DNS resolver to create a cleaner browsing experience
  • The block should be done using DNS at router level, meaning it covers all present and future devices on the network and doesn’t rely on third-party plugins or complicated configurations for each individual device
  • The block list should update often from various sources and be downloaded regularly to pfSense without me having to do anything – true set and forget!
  • No complicated packages should need to be installed on the operating system, it should use software already built into pfSense (tools which are available with most Linux distro as standard, cURL and Crontab)

Installation

Installing doesn’t take long, in fact it took me less than a minute to get going – If you don’t want the script to automatically update then skip steps 1 and 2

  1. First off, head to System > Package Manager > Available Packages and install Cron – This package will be responsible for making the magic happen at set times
  2. Next, go to Services > Cron then click Add, set the schedule (I personally run this every hour, but you can change to your preference) and type the following in the Command box: curl –url http://www.nerdkey.co.uk/pfsense/resolver-ads.conf -o /tmp/resolver.conf > /tmp/resolver.log, then press Save – Curl downloads the latest resolver config file, then the second command reloads the DNS resolver service
  3. IMPORTANT! You must now copy the same command from above and run it in the Diagnostic > Command Prompt section, this will download the last file ready for the next step – If all goes well, you won’t see any output
  4. Head to Services > DNS Resolver option and select the Display Custom Options button, type the following into the Custom options box: include : /tmp/resolver.conf then press Save and apply the changes

Testing

To make sure everything has been setup properly, trying visiting a website known for over the top advertising, such as SpeedTest.net

I’m using a fresh install of Google Chrome with pfSense configured as above, check out the results below!

Troubleshooting

If you are still seeing advertisements, trying clearing your DNS cache. If that doesn’t work, make sure your DNS is pointing to your pfSense router, and make sure the DNS resolver is enabled.

Database

The latest database features 31, 731 known domains, sourced from various freely available lists including:

  • AdAway.org
  • EasyList.to – used by AdBlock
  • EricZhang.me – used to block Spotify ads
  • HostsFile.org
  • SomeoneWhoCares.org

Notes

  • Any known domains are redirected to an IP address of 51.255.30.129 where a blank GIF image is served via HTTP only, the most I do here is anonymously log requests to help with debugging
  • The remote config file is refreshed every 10 minutes
  • A debug log of the last download session can be found at /tmp/resolver.log

Future Plans

At the moment this is just a personal project that I thought others may find useful, who knows though, depending on the popularity, there may be scope for blocking categories of websites, such as social media, adult sites etc

Help

If you find any websites which aren’t blocked, find websites which don’t load correct, or have any other questions please email kieran@nerdkey.co.uk

Finally, this is essentially a DNS level block and nothing more, use it at your own risk, no warranty or guarantee implied.

Remove Adverts from All 4 Roku App

Disclaimer

This post is for educational purposes only, it briefly describes a technique for removing the adverts from Channel 4’s on demand service. I won’t be providing any working examples and won’t be held liable whatever the outcome if you try this, this was just setup as a test one afternoon and then destroyed shortly after. Do so at your own risk.

Why even bother?

Now I love TV but I always end up forgetting and then having to catch up later using on demand services via my NowTV box, some services are great – like the BBC iPlayer – where as others – 4OD or All 4 – lack basic features like being able to resume where you left off without having to sit through the ads again.

This got me thinking, is it possible to get around the ads? Picture this… you are watching an hour long programme on your Roku (or NowTV) box, you have 10 minutes to go and you have to nip out. You come back hoping to pick up where you left off.. but oh no, something happened and now you have to watch from the begining OR fast forward until you get to an ad break, watch the ads, then fast forward again… its not good right? This has happened to me many a time!

A quick Google suggested this is not possible, but that wasn’t good enough for me.

How did you get it to work?

It took a bit of nerdy know how, a decent router and a publicly accessible Linux box.

Decent router – I was using a NowTV (watered down Roku) box, these don’t have the option to manually specify the DNS server addresses so you have to set the DNS servers in my router

Linux box – I used a CentOS 7 box running BIND and Apache, BIND responded to the DNS requests aiming everything at the Apache server

The basic idea is to redirect any requests to ‘known advertiser servers’ to your own server which is returning a single pixel instead of the advertisers video, and it did work really well:

As you can see above the same programme has ads and one does not. This method also removes the ad cue points so you are literally just served with the entire video – cool, huh?

Notes

  • This was just a test, please don’t lecture me about the importance of advertising and the revenue it generates
  • I only tested it with the Roku app, although I think it would have worked for the Xbox app too
  • I guess the same tecnique could be used to create a ‘super’ ad blocker that works with more than just on demand services

Incoming search terms:

  • all4 hacked no adverts
  • all4 pay to skip advert app
  • c4 on demand adverts now tv