proactive redundancy


after the fiasco earlier this week, i’ve been taking steps to minimize the impact if were to go down. it’s still a large spof (single-point-of-failure), but i’m reasonably certain that at least the irc net will remain up and functional in the event of another outage.

the first thing that i set up was a handful of additional ircd nodes: see the wiki for a full list. is on my personal vps, and is hosted on the bsd vps that i set up for

i added the ipv4 addresses for these machines, along with the ip for as A records for, creating a dns round-robin. host will return all four. requesting the dns record will return any one of them, rotating them in a semi-random fashion. this means that when connecting to on 6697 for irc, you might end up on any of {your,team,bsd,slash}

this creates the additional problem that visiting the site will end up at any of those 4 machines in much the same way. for the moment, the site is deployed on all of the boxes, making site setup issues hard to debug. the solution to this problem is to use a subdomain as the roundrobin host, as other networks like freenode do (see host for the list of servers).

i’m not sure how to make any of the other services more resilient. it’s something that i have been and will continue to research moving forward.

the other main step that i have taken to prevent the same issue from happening again was to configure the firewall to drop outgoing requests to the subnets as defined in rfc 1918.

i’d like to consider at least this risk to be mitigated.

thanks for reading,


update: the round robin host is now, which resolves the site issues that we were having, due to the duplicated deployments.