Proxy tuning fun

I really like the idea of a proxy cache for use with the various websites I access regularly, it seems to speed up load times considerably.  My OPNsense firewall/router offers Squid for the caching proxy capability.  I have played with it in the past and as much as I knew then it was doing what it should, but I was never certain of that.  Back then, when others besides myself were using the network I set up, I had enabled Squid to hopefully improve things for their frequent web access.  When they decided to find connectivity in other ways and left the network, I eventually removed the proxy cache.  After some time I enabled it again for myself and discovered it was one of only two things that might prevent success of the wonderful self-contained do-everything-itself firefox, the other being physical disconnection of the coax or Comcast connectivity failure itself.

This means of course that there have been a few times when the failure of the proxy has meant firefox also failed.  In the past, I simply turned on the proxy and set it for transparent.  More recently I setup the Proxy auto-config rules, and as before, as far as I knew everything worked and web site loading was improved.  As with any other new 'toy' or function, you go back to it to play some more, to tweak it and see if you can make it the most efficient.  I am not sure why, but if I set the SSL cache size too high, the proxy fails.  Luckily I knew that this was what I had adjusted most recently, so all I needed to do was attempt to return much of the configuration back to what I had when it was working.  Along with this was fighting with firefox as well to make it understand that it should not try to use a proxy.  Eventually I bumped into the issue with the SSL cache size being too large though why it causes any failure I do not know.

The largest I have discovered that still works is 768 (in mb) with the number of SSL workers set at 32 of a maximum of 32.  What I don't quite understand is that even with all of the proxy auto config settings described later in this post, firefox fails when I turn off 'transparent http proxy' while firefox itself has 'Auto-detect proxy settings for this network'  set in its network config.  I blame firefox for this failure, especially since everything else seems improved by the end of this blog post.

Since it seemed that the forward proxy configuration was now set about as optimally as possible, I looked over the other configurations.  I looked at the proxy auto-config rules and proxies and matches.  Since I noticed significant improvement, snappiness, after some adjustments, I am certain that what I thought was a proxy cache may have only been some re-routing of packets.  Below, the adjustment to the proxies is what seemed to instantly cause improvement.

With regard to the 'not internal to proxy' rule, what was missed was all types of proxies.  I am not sure which one of the three was entered previously but I suspect it was 'LAN proxy web' which seems reasonable, until thinking about what the rule is about.  The rule essentially indicates everything but the proxy itself, so all proxies should be listed.  Below are the definitions of each of the proxies.  I have the svn url excluded with a proxy definition because I wanted to be sure it was not obstructed, though I may change this later, it is further excluded by a match.

Although all of these proxy definitions existed, tuning to more appropriate proxy types as shown above likely also improved efficiency.  I assume I had them set as simply 'proxy' but as soon as I adjusted them I forgot what they had been, as is normal but at least I settled on what is likely optimal.  The definitions of each match follow below.

The only other tweaks I have played with are the cache itself, and I admit not fully understanding how the numbers of first or second level subdirectories affects efficiency.

You might wonder about my OPNsense firewall box, how I can set such things which may seem rather large.  The motherboard has onboard graphics which is only necessary for console access and so does not cause much heat.  Addon cards for networking only.  The cpu and memory and hard disk and various other information below.