Load balancing UDP on EC2

I had been tasked with finding a way to load balance UDP on AWS. Normally we would use ipvsadm in at layer 4 DR but because of the limitations in ec2 this is not possible so another method was needed.
So this is a short guide on how to get a simple UDP load balancer running. We will be using pen, it is described as a simple tcp load balancer and has been around for a while.

UPDATE: Our current version of ENTERPRISE AWS for EC2 now fully supports UDP load balancing in high performance layer 4 NAT mode.
Continue reading

Open Source Windows service for reporting server load back to HAProxy (load balancer feedback agent).

In general when you are load balancing a cluster you can evenly spread the connections through the cluster and you get pretty consistent and even load balancing. However with some applications such as RDS (Microsoft Terminal Servers), you can get very high load from just a  few users doing heavy work. The solution to this is to use some kind of server load feedback agent. We’ve had one of these for a while in our product but now with a lot of help from Simon Horman we’ve managed to integrate the functionality into the main branch (well soon anyway) of HAproxy. We thought it would be a good idea to open source the previous work on Ldirectord/LVS, make it compatible with HAProxy, and release our Windows service code as GPL.

Continue reading

Look why can’t you just tell me how many people are connected to the load balancer?

I must confess , at certain times it has looked like open warfare would break out between the support team and development team at Loadbalancer.org over the last few months.
One reason being that a major change in v7.5 is that all connection information is now displayed as a graph (rather than a static number). The support team was convinced that customers just wanted a number, a single number , a golden number… The development team was convinced that such a number was illogical irrelevant and meaningless……

SysOverview

It will be very interesting to see what the customers think!

Continue reading

Upstart Script for Collectd

We have been doing some internal testing with Collectd.

“collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you’re at the right place, too ;).” (taken from their website)

Anyway we are quite impressed with it but during the configuration I couldn’t locate an upstart script for it so I made my own.

Continue reading

Load balancing Windows Terminal Server – HAProxy and RDP Cookies or Microsoft Connection Broker

When you have users depending on Windows Terminal Services for their main desktop, it’s a good idea to have more than one Terminal Server. RDP, however, is not an easy protocol to load balance; sessions are long-lived and need to be persistent to a particular server, and users may connect from different source addresses during one session.

The current development version of HAProxy has made an important step forward in making this possible. Thanks to work by Exceliance, it now supports RDP Cookies, offering a solution to the persistence problem.

Continue reading

Transparent proxy of SSL traffic using Pound to HAProxy backend patch and howto

OK so I’ve previously blogged about how to get TPROXY and HAProxy working nicely together. But what if you want to terminate SSL traffic on the load balancer in order to use HaProxy to insert cookies in the standard HTTP stream to the backend servers?

Many thanks to Krisztián Ivancsó  for working on the TPROXY patch for Pound for us, we can finally do this!

Continue reading

How to upgrade VMware tools on ClusterLoad ESX or Loadbalancer.org VA

UPDATED November 2014 – Most versions of the appliance come with VMware tools pre-installed by default. These tools enable functions such as controlled OS shutdown, controlled OS restart etc. When installing or updating the tools it’s also possible to update various kernel modules and drivers, but this is not recommended by Loadbalancer.org as it can potentially cause issues for the load balancer. Important drivers will be updated as and when necessary via the online or offline software update feature available in the WUI on the appliance.

Continue reading

How to Setup Subversion on Ubuntu

For our software development we use Subversion – a powerful free tool for version control of files. Any file types are supported, even binary files. To be able to work with Subversion you have to setup a Subversion server. Then you can access the server from Subversion clients.

This blog entry shows one way to setup Subversion server on a Linux machine.

Continue reading

Configure HAProxy with TPROXY kernel for full transparent proxy

Standard Kernel builds don’t support TPROXY ( 2.6.28 does now!).
For example if you use HaProxy as the load balancer then all of the backend servers see the traffic coming from the IP address of the load balancer. TPROXY allows you to make sure the backend servers see the true client IP address in the logs.

Ps. An easier alternative is inserting the clients ip in the x-forwarded-for header (option forwardfor).

For TPROXY to work you need three things:

1) TPROXY compiled into the linux kernel
2) TPROXY / Socket compiled into netfilter / iptables (due in v1.4.3?)
3) HaProxy compiled with the USE_LINUX_TPROXY option

The TPROXY patch for Linux Kernel 2.6.25.11 is here:
http://www.balabit.com/downloads/files/tproxy/tproxy-kernel-2.6.25-20080519-165031-1211208631.tar.bz2

The following is a guide how to install on Centos 5.1:
Continue reading

CentOS 5 and the Dell R200

A few months ago on my personal blog I wrote about the difficulty of installing CentOS on the Dell R200, owing to the SATA DVD drive used in the unit. The R200 is our unit of choice for our ClusterScale “Pegasys” product, so installing Linux distros on this server is a regular occurrence for us. Fortunately, we got hold of a Redhat driver image from Dell and we managed to load it on using a USB key. This is a simple case of typing linux dd at the installation command prompt, but note that you must copy the .img file onto the USB disk, rather than the files within the image itself.

Originally the problem was with CentOS 5.0, but as far as I know it’s still a problem with versions 5.1 and 5.2. Ubuntu installations do not seem to pose the same problem, so presumably they bundle SATA DVD drivers with the distro. I still can’t find this particular driver on the Dell site, so seems like a good idea to host it here too.

LVS Local node patch for Linux 2.6.25, Centos 5 kernel build how-to

Standard Kernel builds of LVS (Linux Vitual Server) don’t have the ability to load balance traffic that is from the local node.
For example if you terminated some SSL traffic using stunnel or pound on the load balancer you then wouldn’t be able to forward that traffic to a backend real server through LVS.

First many thanks to Siim Põder for helping to port Carlos Lozano’s patch from 2.4 -> 2.6
In order to run an SSL reverse proxy on the same node that is running LVS
i.e.

External client —> pound:443 –> Local:443 —> IPVS:80 —> RealServer

The patch for Linux Kernel 2.6.25 is here:
http://downloads.loadbalancer.org/lvs/patches/ip_vs_locallvs.patch

The following is a guide how to install on Centos 5.1:
Continue reading