Hi,
I'm currently facing a quite tricky problem. Here goes.
I have setup Squid as a transparent HTTP+HTTPS proxy in my local network. All web traffic gets handed over to Squid by an iptables script on the server. Here's the relevant section in /etc/squid/squid.conf:
--8<------------------------------------------------------------- # Ports du proxy http_port 3130 http_port 3128 intercept https_port 3129 intercept ssl-bump \ cert=/etc/squid/ssl_cert/amandine.sandbox.lan.pem \ generate-host-certificates=on dynamic_cert_mem_cache_size=4MB --8<-------------------------------------------------------------
And here's the corresponding section of my firewall script:
--8<------------------------------------------------------------- # Commandes IPT=/usr/sbin/iptables SYS=/usr/sbin/sysctl SERVICE=/usr/sbin/service
# Internet IFACE_INET=enp2s0
# Réseau local IFACE_LAN=virbr0 IFACE_LAN_IP=192.168.2.0/24
# Serveur SERVER_IP=192.168.2.1
...
# Squid $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3128 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3128 -j ACCEPT $IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \ --dport 80 -j REDIRECT --to-port 3128 $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3129 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3129 -j ACCEPT $IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \ --dport 443 -j REDIRECT --to-port 3129 $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3130 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3130 -j ACCEPT --8<-------------------------------------------------------------
This setup works nicely for the vast majority of web sites.
BUT: a handful of sites has some trouble with my local certificate. For example, I can't sync my local Github repo anymore. Or my local OwnCloud client spews back a warning message on every startup.
I asked on the Squid mailing list if there was a possibility to create an exception for a list of domains, so that these can simply bypass the proxy. The problem is, according to one of the developers, I have to tackle that problem earlier in the process, e. g. in the firewall setup.
So here's what I want to do, in plain words:
1. Redirect all HTTP traffic (port 80) to port 3128. So far so good.
2. Redirect all HTTPS traffic (port 443) to port 3129. Equally OK.
AND...
3. DO NOT REDIRECT traffic that goes to certain domains, like:
github.com credit-cooperatif.coop cloud.microlinux.fr squid-cache.org etc.
Ideally, these domains should be read from a simple text file.
Any idea how I could do that? I don't even know if this is theoretically possible.
Cheers,
Niki
Le 11/03/2018 à 11:01, Nicolas Kovacs a écrit :
So here's what I want to do, in plain words:
Redirect all HTTP traffic (port 80) to port 3128. So far so good.
Redirect all HTTPS traffic (port 443) to port 3129. Equally OK.
AND...
- DO NOT REDIRECT traffic that goes to certain domains, like:
github.com credit-cooperatif.coop cloud.microlinux.fr squid-cache.org etc.
I've experimented some more, and I have a partial success. Here, I'm redirecting all HTTPS traffic *except* the one that goes to my bank:
iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.credit-cooperatif.coop --dport 443 -j REDIRECT --to-port 3129
This works because my bank is hosted on a single IP. As soon as I replace that with a domain that's hosted on multiple IP's, I get this:
iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.google.com --dport 443 -j REDIRECT --to-port 3129
# firewall.sh iptables v1.4.21: ! not allowed with multiple source or destination IP addresses
So my question is: how can I write an iptables rule (or series of rules) that redirect all traffic to my proxy, *except* the one going to <list_of_domains> ?
Cheers,
Niki
Am 11.03.2018 um 11:53 schrieb Nicolas Kovacs info@microlinux.fr:
I've experimented some more, and I have a partial success. Here, I'm redirecting all HTTPS traffic *except* the one that goes to my bank:
iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.credit-cooperatif.coop --dport 443 -j REDIRECT --to-port 3129
This works because my bank is hosted on a single IP. As soon as I replace that with a domain that's hosted on multiple IP's, I get this:
iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.google.com --dport 443 -j REDIRECT --to-port 3129
May I ask, after all it doesn't work with google.com, right?
# firewall.sh iptables v1.4.21: ! not allowed with multiple source or destination IP addresses
So my question is: how can I write an iptables rule (or series of rules) that redirect all traffic to my proxy, *except* the one going to <list_of_domains> ?
It is not a good practice to place domain names into iptables rules. Define a custom table, place this table into your rule list (to stick at the right place) and feed that table with the resolved domain names. This can be altered while running in the case of changes (check resolving results periodically).
-- LF
Le 11/03/2018 à 13:09, Leon Fauster a écrit :
It is not a good practice to place domain names into iptables rules. Define a custom table, place this table into your rule list (to stick at the right place) and feed that table with the resolved domain names. This can be altered while running in the case of changes (check resolving results periodically).
I admit I've never worked with custom tables, so I don't know how to do this.
In the meantime, I found the following working solution.
# Exceptions EXCEPTIONS=$(egrep -v '(^#)|(^\s+$)' /usr/local/sbin/no-proxy.txt) for EXCEPTION in $EXCEPTIONS; do $IPT -A PREROUTING -t nat -i $IFACE_LAN -d $EXCEPTION -j ACCEPT done
# Squid $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3128 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3128 -j ACCEPT $IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \ --dport 80 -j REDIRECT --to-port 3128 $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3129 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3129 -j ACCEPT $IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \ --dport 443 -j REDIRECT --to-port 3129 $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3130 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3130 -j ACCEPT
And my no-proxy.txt file looks like this:
# Ne pas utiliser le proxy pour les domaines suivants # # Crédit Agricole www.credit-agricole.fr # Crédit Coopératif www.credit-cooperatif.coop # Github github.com # Microlinux microlinux.fr microlinux.eu # Squid squid-cache.org # Thunderbird start.thunderbird.net
Note that I can put either domain names or IP addresses in this file.
And it's only supposed to keep a list of a handful of URLs that don't play well with a transparent Squid for HTTPS.
Cheers,
Niki
Hi,
Another idea - but this gets complicated and with that, prone to faults - use a simple shell script to resolve the desired domains and keep their IPs in an ipset, then use the ipset in your firewall rules, this way you can keep your iptables rules static, your squid config static and simply add or remove IPs from the ipset.
-- Sent from the Delta quadrant using Borg technology!
Nux! www.nux.ro
----- Original Message -----
From: "Nicolas Kovacs" info@microlinux.fr To: "CentOS mailing list" centos@centos.org Sent: Sunday, 11 March, 2018 12:18:06 Subject: Re: [CentOS] Squid vs. iptables redirection: exception for certain domains ?
Le 11/03/2018 à 13:09, Leon Fauster a écrit :
It is not a good practice to place domain names into iptables rules. Define a custom table, place this table into your rule list (to stick at the right place) and feed that table with the resolved domain names. This can be altered while running in the case of changes (check resolving results periodically).
I admit I've never worked with custom tables, so I don't know how to do this.
In the meantime, I found the following working solution.
# Exceptions EXCEPTIONS=$(egrep -v '(^#)|(^\s+$)' /usr/local/sbin/no-proxy.txt) for EXCEPTION in $EXCEPTIONS; do $IPT -A PREROUTING -t nat -i $IFACE_LAN -d $EXCEPTION -j ACCEPT done
# Squid $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3128 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3128 -j ACCEPT $IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \ --dport 80 -j REDIRECT --to-port 3128 $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3129 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3129 -j ACCEPT $IPT -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d $SERVER_IP \ --dport 443 -j REDIRECT --to-port 3129 $IPT -A INPUT -p tcp -i $IFACE_LAN --dport 3130 -j ACCEPT $IPT -A INPUT -p udp -i $IFACE_LAN --dport 3130 -j ACCEPT
And my no-proxy.txt file looks like this:
# Ne pas utiliser le proxy pour les domaines suivants # # Crédit Agricole www.credit-agricole.fr # Crédit Coopératif www.credit-cooperatif.coop # Github github.com # Microlinux microlinux.fr microlinux.eu # Squid squid-cache.org # Thunderbird start.thunderbird.net
Note that I can put either domain names or IP addresses in this file.
And it's only supposed to keep a list of a handful of URLs that don't play well with a transparent Squid for HTTPS.
Cheers,
Niki
-- Microlinux - Solutions informatiques durables 7, place de l'église - 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info@microlinux.fr Tél. : 04 66 63 10 32 _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Le 12/03/2018 à 10:37, Nux! a écrit :
Another idea - but this gets complicated and with that, prone to faults - use a simple shell script to resolve the desired domains and keep their IPs in an ipset, then use the ipset in your firewall rules, this way you can keep your iptables rules static, your squid config static and simply add or remove IPs from the ipset.
Following a suggestion from Yuri Voinov on the Squid mailing list, I've found a better solution that works perfectly. I've added it to my blog here:
https://blog.microlinux.fr/squid-exceptions/#squid
Cheers,
Niki
On Sun, March 11, 2018 7:09 am, Leon Fauster wrote:
Am 11.03.2018 um 11:53 schrieb Nicolas Kovacs info@microlinux.fr:
I've experimented some more, and I have a partial success. Here, I'm redirecting all HTTPS traffic *except* the one that goes to my bank:
iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.credit-cooperatif.coop --dport 443 -j REDIRECT --to-port 3129
This works because my bank is hosted on a single IP. As soon as I replace that with a domain that's hosted on multiple IP's, I get this:
iptables -A PREROUTING -t nat -i $IFACE_LAN -p tcp ! -d www.google.com --dport 443 -j REDIRECT --to-port 3129
May I ask, after all it doesn't work with google.com, right?
I would also like to add: it is a bad practice IMHO to give preference to some particular search engine, unless it is single user personal machine. Many people prefer different search engines (duckduckgo.com just to mention one), some specifically avoid google.
Valeri
# firewall.sh iptables v1.4.21: ! not allowed with multiple source or destination IP addresses
So my question is: how can I write an iptables rule (or series of rules) that redirect all traffic to my proxy, *except* the one going to <list_of_domains> ?
It is not a good practice to place domain names into iptables rules. Define a custom table, place this table into your rule list (to stick at the right place) and feed that table with the resolved domain names. This can be altered while running in the case of changes (check resolving results periodically).
-- LF
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
-
Leon Fauster -
Nicolas Kovacs -
Nux! -
Valeri Galtsev