«

»

Aug 17

Configure Squid as HTTP and HTTPS Transparent Proxy

These days, it is really important to have proxy server to analyze web traffic of the organization. Among proxy servers, the Squid is very famous, because of it’s flexibility and easy of configuration. Squid can be operated at non-transparent and transparent mode which is going to discuss here. Main benefit of transparent mode is, clients are not aware that their requests are processed through the proxy. Simply there is no configuration at client side. So let’s look at how to configure Squid as HTTP and HTTPS Transparent Proxy

 

Notice

Before begin please adjust the ip and other configuration as per your requirement. Below values are used only for demonstration.

Internet –> etho
interface IP :- 192.168.2.39/24 Gateway:- 192.168.2.1

LAN –> eth1
interface IP :- 192.168.231.126/24  Gateway:- 0.0.0.0

 

If you have single interface no need to worry. you can create virtual interface which is act either LAN or Internet interface. This process has more steps to follow, so I thought to divide into 4 major section to make it more easy to understand.

(01) Install and Configure Squid

(02) Install bind DNS

(03) Configure iptables

(04) Configure Windows client.

So Let’s follow each section in depth.

(01) Install and Configure Squid

1) To analyses https traffic, following packages are required.

yum install openssl openssl-devel

2) Download and install latest Squid version

Download location :- http://www.squid-cache.org/Versions/

–squid run as squid user, and following parameters are mandatory.


./configure --with-openssl --enable-ssl-crtd --with-default-user=squid
make
make install

3) Initialize squid ssl_db directory

/usr/local/squid/libexec/ssl_crtd -c -s /var/lib/ssl_db
chown -R squid.squid /var/lib/ssl_db

Dynamically generated ssl certificates are stored at /var/lib/ssl_db directory

4) Comment or add following extra fields to squid.conf file.


http_port 3130
http_port 3128 intercept
https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

cert=/usr/local/squid/ssl_cert/myca.pem key=/usr/local/squid/ssl_cert/myca.pem

ssl_bump server-first all

sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db -M 4MB
sslcrtd_children 8 startup=1 idle=1

coredump_dir /usr/local/squid/var/cache/squid

5) create the certificate folder and generate the keys

mkdir /usr/local/squid/ssl_cert
chown -R squid.squid /usr/local/squid/ssl_cert
cd /usr/local/squid/ssl_cert

6) execute new certificate request

openssl req -new -newkey rsa:1024 -days 1365 -nodes -x509 -keyout myca.pem -out myca.pem

ex:-
Country Name (2 letter code) [XX]:lk
State or Province Name (full name) []:western
Locality Name (eg, city) [Default City]:colombo
Organization Name (eg, company) [Default Company Ltd]:it
Organizational Unit Name (eg, section) []:itdept
Common Name (eg, your name or your server's hostname) []:squidserver.local
Email Address []:admin@squidserver.local

7) Generate certificate for web browsers. later this der file (myca.der) needs to add into browser to avoid SSL Error.

openssl x509 -in myca.pem -outform DER -out myca.der

(02) Install bind DNS

1) install bind

yum install bind

2) Configure DNS

vim /etc/named.conf

 

3) Configure zone for squidserver.local

mkdir /var/named/squidserver.local
touch /var/named/squidserver.local/db.home
chown -R named.named /var/named/squidserver.local/db.home

4) Add following line to /var/named/squidserver.local/db.home

$ORIGIN squidserver.local.
$TTL 86400
@    IN    SOA    proxy.squidserver.local.    proxy.squidserver.local. (
2014032801 ; Serial
28800 ; Refresh
7200 ; Retry
604800 ; Expire
86400 ; Negative Cache TTL
)
@    IN    NS    proxy.squidserver.local.
proxy    IN    A    192.168.231.126

5) Start named

service named start

(03) Configure iptables

beware about the interface and ip address.alter those values according to your requirement. You can learn about iptables from here if you are novice.

1) Redirect HTTP and HTTPS traffic to squid

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp –dport 80 -j REDIRECT –to-ports 3128
iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp –dport 443 -j REDIRECT –to-ports 3129

2) Enable udp and tcp port 53 , tcp port 80,443,3128,3129 from inbound lan port.

ex:- Added to rule 5, it may be changed according to existing iptable rule
iptables -I INPUT 5 -p udp -m udp –dport 53 -j ACCEPT

iptables -I INPUT 5 -p tcp –dport 53 -m state –state NEW,ESTABLISHED  -j ACCEPT
iptables -I INPUT 5 -p tcp –dport 80 -j ACCEPT
iptables -I INPUT 5 -p tcp –dport 443 -j ACCEPT
iptables -I INPUT 5 -p tcp –dport 3128 -m state –state NEW,ESTABLISHED  -j ACCEPT
iptables -I INPUT 5 -p tcp –dport 3129 -m state –state NEW,ESTABLISHED  -j ACCEPT

3) So what happen to other traffic such as ftp, vpn. Let’s by pass those traffic.

Here assume squid does not handle those requests. Accept connection from inside  (eth1) and forward them to (eth0) internet
iptables -I FORWARD 1 -o eth0 -i eth1 -s 192.168.231.0/24 -m conntrack –ctstate NEW -j ACCEPT

We accept to forward all already established connection
 iptables -I FORWARD 2 -m conntrack –ctstate ESTABLISHED,RELATED -j ACCEPT

Masquerading (substitute the local source ip address to the public address)
iptables -A POSTROUTING -t nat -j MASQUERADE

4) enable packet forwarding for IPv4

edit /etc/sysctl.conf and add following
net.ipv4.ip_forward=1

(04) Configure Windows client.

1) Configure client Default gateway and DNS as 192.168.231.126 (LAN ip address)

2) Upload myca.der to web browser to avoid SSL error.

 

feels free to comment here if you have faced any issues 🙂

35 comments

Skip to comment form

  1. Mike Morgan

    Excellent article ! , It was very useful to me . Thx 🙂

  2. Ron

    Thanks ! this was help me a lot

  3. ShuMing

    Thanks! But can you teach 1-2 more clear?
    I don’t know where should type “./configure” ?
    Thx

    1. admin

      “./configure …” command should be typed inside the squid source folder which you have extracted . you can find more available option by typing “./configure –help”

  4. vijaymuddu

    What if i dont use/install dns server will it effect squid server

    1. admin

      No, it does not effect to squid server

  5. Sizomu

    HI, am I missing something? step 4 is to configure the clients, can you still call this transparent?

    1. admin

      Remember, here squid works transparent proxy for HTTPS traffic as well. if you need to capture the HTTPS traffic then you have to install certificate file to web browser otherwise browser consider HTTPS capturing as man in the middle attack!

      1. Sizomu

        I understand. but since you have to edit clients, that means its not really transparent (in my understanding “transparent/intercept” should require no manual settings, I understand https changes alot) this might not work with laptops that go in & out of your network. I am trying to get .pac files to work on my Ubuntu Server 14.04, Squid3, DHCP > sending the proxy settings to any IP request. basically setting proxy settings automatically in client browser. I am not sure if Https will then also accept the redirect. any experience with PAC scripts? Nice website btw, love the content! cheers

  6. Zohaib Ghafoor

    can i buy commercial certificates to install if yes then how and run without self signed certificate error in squid and don’t need browser to import der file?

    1. admin

      I did not try that yet. I hope it should be worked.
      You can try this https://letsencrypt.org which provides free SSL certificates.

    2. Anitha M

      Zohaib,

      I want to do the same. Did the public signed certificate is worked for you if its worked kindly guide me to achieve the same.

    3. Franceso G.

      No you cant.

      Since a valid “commercial” certificate must have a real common name eg:


      Common Name (eg, your name or your server’s hostname) []: proxy.yourcommecialdomain.com

      Wich will not match the browser url if you will use it for a transparent proxy.

      The only think will change is you will not need to install the certificate if you will NOT use transparent mode,
      eg: you browser will be forced to use https proxy “proxy.yourcommecialdomain.com”

  7. Anitha M

    Dear Team,

    Issue Description: I have install and configured squid3.3.4 in debain machine. To enable squid as HTTPS transparent proxy I have used public signed certificate(from Godaddy) and configured the same in squid.conf file. Once the configuration done I have tried to start the squid but while starting the squid I am getting the following error.
    Error: Squid Cache (Version 3.3.4): Terminated abnormally.
    CPU Usage: 0.020 seconds = 0.000 user + 0.020 sys
    Maximum Resident Size: 22416 KB
    Page faults with physical i/o: 3
    failed!
    Note: The public signed certificate (pem and file) has been converted from tomcat java keystore(certificate) file.
    Squid version: 3.3.4

    Kindly help me to solve this issue.

    Here is my squid.conf configuration.

    _______________________________________________________________
    # Squid.conf

    acl localhost src 127.0.0.1/32 ::1
    acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1
    acl ftp proto FTP

    acl localnet src 10.0.0.0/8
    acl localnet src 172.16.0.0/12
    acl localnet src 192.168.0.0/16
    acl connect method CONNECT

    acl blockfiles urlpath_regex “/etc/squid3/block.files.acl”

    http_access deny blockfiles

    acl SSL_ports port 443
    acl SSL_ports port 22
    acl SSL_ports port 21 8443
    acl SSL_ports port 8834 8100
    acl SSL_ports port 7004
    acl SSL_ports port 6667
    acl SSL_ports port 1863
    acl SSL_ports port 5050
    acl SSL_ports port 1863
    acl SSL_ports port 8001 8002 23 25 119 5100 80 1935
    acl Safe_ports port 80
    acl Safe_ports port 81
    acl Safe_ports port 21
    acl Safe_ports port 443
    acl Safe_ports port 70
    acl Safe_ports port 210
    acl Safe_ports port 1025-65535
    acl Safe_ports port 280
    acl Safe_ports port 488
    acl Safe_ports port 591
    acl Safe_ports port 8834
    acl Safe_ports port 777
    #acl YIM_ports port 5050
    #acl YIM_ports port 80
    #acl YIM_ports port 23
    acl CONNECT method CONNECT
    acl HTTPS method CONNECT

    follow_x_forwarded_for allow all

    #http_access allow manager localhost
    #http_access deny manager
    http_access deny !Safe_ports
    http_access deny CONNECT !SSL_ports
    http_access deny to_localhost
    http_access allow localhost
    http_access allow ftp

    #http_access allow ldapauth
    http_access allow localnet

    # Download Limit Size.
    # reply_body_max_size 5000 MB all
    reply_body_max_size 1024 MB

    # Proxy Port Configuration
    #http_port 3128

    http_port 3127
    http_port 3128 intercept
    https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid3/cert/squidtrans.pem key=/etc/squid3/cert/squidtrans.key
    ssl_bump server-first all

    sslcrtd_program /usr/lib/squid3/ssl_crtd -s /var/lib/ssl_db -M 4MB
    sslcrtd_children 8 startup=1 idle=1

    hierarchy_stoplist cgi-bin ?
    cache_mem 8 MB
    maximum_object_size_in_memory 50 KB

    # Cache Size Limit.
    # cache_dir ufs /var/spool/squid3 1000 16 256

    cache_dir ufs /var/spool/squid3 1024 16 256

    minimum_object_size 8000 KB
    maximum_object_size 500000 KB
    cache_swap_low 90
    cache_swap_high 95
    strip_query_terms off
    coredump_dir /var/spool/squid3

    refresh_pattern ^ftp: 1440 20% 10080
    refresh_pattern ^gopher: 1440 0% 1440
    refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
    refresh_pattern . 0 20% 4320
    cache_mgr anitha.m@paladion.net
    mail_program mail

    error_directory /usr/share/squid3/errors/en
    deny_info ERR_BLOCKED_FILES blockfiles

    # dns_nameservers 8.8.8.8

    memory_pools off
    http_access deny all
    #cache_access_log /backup/log/squid3/access.log
    #cache_log /backup/log/squid3/cache.log
    #cache_store_log /tmp/log/squid/store.log
    ___________________________________________________________________

    1. Squidblacklist (@Squidblacklist)

      Sir, Squid from the debian repositories does not have ssl flags enabled, you will not be using ssl with that.

  8. Franceso G.

    You can omit the whole DNS ( bind & co ) staff, just by using * as Common Name


    Common Name (eg, your name or your server’s hostname) []: *

  9. osmt

    Hi,

    Please i try to install the HTTPS,HTTP Squid proxy Solution on our entreprise network , 2 weeks ago but i succeed only with http intercepting of course in transparent mode , i use Centos 7 core the last stable version 3.3.8 .

    question 1 : in my case i use squid package from the Centos repositories , is this pose any problem with SSL-Bump using an intermediate CA.

    “./configure –with-openssl –enable-ssl-crtd –with-default-user=squid” —–> is squid installed with openssl and enable ssl-crtd in my case by : yum install squid ???

    question 2 : can i use firewalld in the place of iptables and which is the best for me ?

    1. Franceso G.

      Looks so.

      is yours squid.conf sslbumping ?

      eg:

      squid.conf
      #SSL STUFF
      https_port 3130 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/var/lib/ssl_cert/myca.pem key=/var/lib/ssl_cert/myca.pem

      ssl_bump server-first all

      sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
      sslcrtd_children 8 startup=1 idle=1

      #SSL

      # shell stuff to build “myca.pem”

      openssl req -new -newkey rsa:1024 -days 1365 -nodes -x509 -keyout /var/lib/ssl_cert/myca.pem -out /var/lib/ssl_cert/myca.pem -subj “/C=IT/ST=Network/L=COMPANY Appliance/O=COMPANY Appliance/OU=SSL Inspection/CN=*”

      openssl x509 -in /var/lib/ssl_cert/myca.pem-outform DER -out /var/lib/ssl_cert/for_the_clients.der

      1. osmt

        actually i add the #SSL STUFF in my self but this crashing squid ! in the default squid.conf there is no ssl configuration .

        please i need help to finish this project .i need block the https requests , youtube , and other sites bypass even the iptables Drop

      2. mmm

        Hi. Im having a problem with the config. Can you help me?
        Im using Squid 3.3.8 (not transparent) in Centos 7. All is working fine, but i want limit bandwith for youtube and facebook…
        when i put the config in squid for ssl-bump and try to go to a web… well all they say “the connection was refused” (or something like that… im not in the machine wright now).
        Can yo give me a hand?? im going crazy……

        (i dont speak english……… sorry for monkey talking)

  10. VR

    Hi. Does anyone knows a way to automate the distribution of the proxy certificate? I mean something like announcing it in the pac file? I know it still requires the user to accept the certificate. Thanks

  11. Squidblacklist (@Squidblacklist)

    This is not working in Centos7

    2016/10/18 23:54:45 kid1| helperOpenServers: Starting 1/8 ‘ssl_crtd’ processes
    (ssl_crtd): Cannot create /var/lib/ssl_db

    1. me

      make sure /var/lib/ssl_db is owned by user/gropu ==> squid:squid

  12. wrh webmaster

    What is this IP address 192.168.231.0?

  13. Mohammad Reza

    Thank you for article.

    I have done all steps, but when I start Squid, this error occurs:
    FATAL: No valid signing SSL certificate configured for HTTPS_port 0.0.0.0:3129

    what should I do?

  14. Muhammad Tahir Minhas

    Please can you explain step 3. where to write this command

    /usr/local/squid/libexec/ssl_crtd -c -s /var/lib/ssl_db
    chown -R squid.squid /var/lib/ssl_db

  15. Muhammad Tahir Minhas

    /usr/lib64/squid/ssl_crtd: Cannot create /var/lib/ssl_db

    Please tell me how to resolve this error

    1. ym

      You should pay attention to the directory permissions

  16. Muhammad Tahir Minhas

    https_port 3130 transparent ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

    cert=/usr/local/squid/ssl_cert/myca.pem key=/usr/local/squid/ssl_cert/myca.pem

    ssl_bump server-first all

    sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
    sslcrtd_children 8 startup=1 idle=1

    coredump_dir /usr/local/squid/var/cache/squid

    this is my configuration but service gives error please help me.

  17. Muhammad Tahir Minhas

    Squid give error on this. https_port not recognized in squid. please help me.

    https_port 3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

  18. Muhammad Tahir Minhas

    Admin where are you no reply for my queries

    1. admin

      So sorry, I did these squid config sometime ago. I hope your question would be answered by some Squid expert 🙂

  19. Manas

    I Followed all the steps but why it is not working

  20. Manas

    Please help me to setup transparent proxy .
    Normal http proxy is working.

Leave Your Thought Here