It seems like the Squid 3.x (at least for 3.1) parses acl rules a bit differently. I needed to add “all” to the http_access deny for it to work.
acl ads dstdom_regex "/etc/squid/ad_block.txt" http_access deny all ads
Instructions for initial setup can be found at https://calomel.org/squid_adservers.html.