Thursday, January 27, 2011

Speeding up the internet in FreeBSD?

Well, that was the initial idea having read this interesting post but, in all honesty, it was not the runaway success that I hoped for. Of course, the author of the post does say that it's really aimed at users with extremely low network speeds (he mentions 500 kbps) which is way, way below what I have.
Any way, I tried it out.
First I installed pdnsd from /usr/ports/dns/pdnsd and this proceeded without problems.
Now, I just used the default pdnsd.conf and, as it seemed to work, I didn't bother to make any changes to it, yet.
Just for completeness, I'll print the pdnsd.conf here:

// Sample pdnsd configuration file. Must be customized to obtain a working pdnsd setup!
// Read the pdnsd.conf(5) manpage for an explanation of the options.
// Add or remove '#' in front of options you want to disable or enable, respectively.
// Remove '/*' and '*/' to enable complete sections.

global {
# pid_file = /var/run/;
server_ip =; # Use eth0 here if you want to allow other
# machines on your network to query pdnsd.
status_ctl = on;
# paranoid=on; # This option reduces the chance of cache poisoning
# but may make pdnsd less efficient, unfortunately.
min_ttl=15m; # Retain cached entries at least 15 minutes.
max_ttl=1w; # One week.
timeout=10; # Global timeout option (10 seconds).

# The following section is most appropriate if you have a fixed connection to
# the Internet and an ISP which provides good DNS servers.
server {
label= "OpenDNS";
ip =; # Put your ISP's DNS-server address(es)
ip =;


uptest=if; # Test if the network interface is active.
interface=eth0; # The name of the interface to check.
interval=10m; # Check every 10 minutes.
purge_cache=off; # Keep stale cache entries in case the ISP's
# DNS servers go offline.

# The following section is more appropriate for dial-up connections.
# Read about how to use pdnsd-ctl for dynamic configuration in the documentation.
server {
label= "dialup";
file = "/etc/ppp/resolv.conf"; # Preferably do not use /etc/resolv.conf
interface = ppp0;
interval=10; # Check the interface every 10 seconds.

# The servers provided by OpenDNS are fast, but they do not reply with
# NXDOMAIN for non-existant domains, instead they supply you with an
# address of one of their search engines. They also lie about the addresses of
# of the search engines of google, microsoft and yahoo.
# If you do not like this behaviour the "reject" option may be useful.
server {
label = "opendns";
ip =,;
reject =, # You may need to add additional address ranges, # here if the addresses of their search engines; # change.
reject_policy = fail; # If you do not provide any alternative server
# sections, like the following root-server
# example, "negate" may be more appropriate here.
timeout = 4;
uptest = ping; # Test availability using ICMP echo requests.
ping_timeout = 100; # ping test will time out after 10 seconds.
interval = 15m; # Test every 15 minutes.
preset = off;

# This section is meant for resolving from root servers.
server {
label = "root-servers";
root_server = discover; # Query the name servers listed below
# to obtain a full list of root servers.
randomize_servers = on; # Give every root server an equal chance
# of being queried.
ip =, # This list will be expanded to the full; # list on start up.
timeout = 5;
uptest = query; # Test availability using empty DNS queries.
interval = 30m; # Test every half hour.
ping_timeout = 300; # Test should time out after 30 seconds.
purge_cache = off;
exclude = .localdomain;
policy = included;
preset = off;

source {

include {file="/etc/pdnsd.include";} # Read additional definitions from /etc/pdnsd.include.

rr {

neg {;
types=domain; # This will also block, etc.

neg {; # Badly behaved server you don't want to connect to.

Next I created a very simple script, made it executable and started it at boot by placing it in the System>>Preferences>>Startup Applications.
The script is
#! /bin/sh
sudo pdnsd --daemon --config-file /usr/local/etc/pdnsd.conf

Then I added as the ONLY IP to /etc/resolv.conf and rebooted.
After startup, I ran dig and got a query time of 16 ms. Not bad, but the second time I got the following:
$ dig

; <<>> DiG 9.6.-ESV-R3 <<>>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 62192
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

; IN A

;; Query time: 0 msec
;; WHEN: Thu Jan 27 23:57:14 2011
;; MSG SIZE rcvd: 28

which shows a query time of 0 ms. Wow, a saving of a whole 16 ms!!
Yes, I know that it doesn't sound like much but in practice pages on the browser open much faster (once they have been opened at least once before, of course).
So, I believe this was a success and I'm interested to try it on some of my other computers too.

OK, next I tried squid as a caching proxy to help get pages to render faster as a lot of the page content (logos and the like) should be cached from previous openings.
Again, details are provided here but I actually followed this guide more closely.
Whereas the pdnsd install and configuration couldn't have been simpler, I had quite some problems with getting squid to work.
The first problem was getting a squid.conf that was right for me and this is what I came up with after much trial and error:
http_port 3128
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
cache_mem 8 MB
maximum_object_size 50960 KB
maximum_object_size_in_memory 16 KB
cache_dir diskd /squidcache/squid/cache 10000 16 256
access_log /var/log/squid/access.log squid
cache_log /var/log/squid/cache.log
cache_store_log /var/log/squid/store.log
pid_filename /usr/local/etc/squid/
hosts_file /etc/hosts
auth_param basic children 5
auth_param basic realm Squid proxy-caching web server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 10080
acl all src
acl manager proto cache_object
acl localhost src
acl to_localhost dst
acl SSL_ports port 443 563
acl Safe_ports port 80 # http
acl Safe_ports port 8080 #also http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl blacklist dstdomain "/usr/local/etc/squid/blacklist.txt"
http_access deny blacklist
http_access allow manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
#change below to what matches your LAN IP address space
acl our_networks src
http_access allow our_networks
http_access allow localhost
http_access deny all
http_reply_access allow all
icp_access allow all
cache_effective_user squid
cachemgr_passwd secret all
coredump_dir /squidcache/coredump

Much more detail on what I did to get squid running and allowing the browsers to accept the proxy server for operation is given in this forum post.
To get squid to start on boot, I simply addedd

to /etc/rc.conf.
So changing from the proxy hostname plus port number to immediately allowed surfing in both browsers (Firefox and Midori).
Did it make any difference to browsing speed? In reality, not that you'd notice.
Additionally I find squid (as I've set it up) to be somewhat unstable and it seems to crash with some frequency.
Perhaps this is something I can overcome with some more refinement in the configuration.

No comments:

Post a Comment