To follow up to Aarons blog on HAProxy emails alerts using logwatch I was looking into different ways to achieve the same results.
Now the ideal way to monitor the health of the real servers is to to have a dedicated monitoring system in place such as Nagios( It even has a HAProxy plugin). However this is not always an option for some so they require the loadbalancer to send an alert. So I investigated some different options.
I had been tasked with finding a way to load balance UDP on AWS. Normally we would use ipvsadm in at layer 4 DR but because of the limitations in ec2 this is not possible so another method was needed.
So this is a short guide on how to get a simple UDP load balancer running. We will be using pen, it is described as a simple tcp load balancer and has been around for a while.
In general when you are load balancing a cluster you can evenly spread the connections through the cluster and you get pretty consistent and even load balancing. However with some applications such as RDS (Microsoft Terminal Servers), you can get very high load from just a few users doing heavy work. The solution to this is to use some kind of server load feedback agent. We’ve had one of these for a while in our product but now with a lot of help from Simon Horman we’ve managed to integrate the functionality into the main branch (well soon anyway) of HAproxy. We thought it would be a good idea to open source the previous work on Ldirectord/LVS, make it compatible with HAProxy, and release our Windows service code as GPL.
I must confess , at certain times it has looked like open warfare would break out between the support team and development team at Loadbalancer.org over the last few months.
One reason being that a major change in v7.5 is that all connection information is now displayed as a graph (rather than a static number). The support team was convinced that customers just wanted a number, a single number , a golden number… The development team was convinced that such a number was illogical irrelevant and meaningless……
It will be very interesting to see what the customers think!
In this guide I show a very simple solution to get HAProxy email alerts configured using Logwatch. While the first part is aimed at users of our V7 appliance I think anyone wanting to get email alerts for HAProxy will also find this a good example.
First from the WUI :
1. Set the external relay(Smart Host) under Edit Configuration > Physical – Advanced Configuration.
Transparent mode with HAProxy allows you to see the IP Address of the clients computer while still having a high availability service using HAProxy.
This posting shows how to setup a blank virgin installation of Centos 6.3 64bit minimum installation.
We have been doing some internal testing with Collectd.
“collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you’re at the right place, too ;).” (taken from their website)
Anyway we are quite impressed with it but during the configuration I couldn’t locate an upstart script for it so I made my own.
When you have users depending on Windows Terminal Services for their main desktop, it’s a good idea to have more than one Terminal Server. RDP, however, is not an easy protocol to load balance; sessions are long-lived and need to be persistent to a particular server, and users may connect from different source addresses during one session.
The current development version of HAProxy has made an important step forward in making this possible. Thanks to work by Exceliance, it now supports RDP Cookies, offering a solution to the persistence problem.
OK so I’ve previously blogged about how to get TPROXY and HAProxy working nicely together. But what if you want to terminate SSL traffic on the load balancer in order to use HaProxy to insert cookies in the standard HTTP stream to the backend servers?
Many thanks to Krisztián Ivancsó for working on the TPROXY patch for Pound for us, we can finally do this!
UPDATED November 2014 – Most versions of the appliance come with VMware tools pre-installed by default. These tools enable functions such as controlled OS shutdown, controlled OS restart etc. When installing or updating the tools it’s also possible to update various kernel modules and drivers, but this is not recommended by Loadbalancer.org as it can potentially cause issues for the load balancer. Important drivers will be updated as and when necessary via the online or offline software update feature available in the WUI on the appliance.
For our software development we use Subversion – a powerful free tool for version control of files. Any file types are supported, even binary files. To be able to work with Subversion you have to setup a Subversion server. Then you can access the server from Subversion clients.
This blog entry shows one way to setup Subversion server on a Linux machine.
Standard Kernel builds don’t support TPROXY ( 2.6.28 does now!).
For example if you use HaProxy as the load balancer then all of the backend servers see the traffic coming from the IP address of the load balancer. TPROXY allows you to make sure the backend servers see the true client IP address in the logs.
Ps. An easier alternative is inserting the clients ip in the x-forwarded-for header (option forwardfor).
For TPROXY to work you need three things:
1) TPROXY compiled into the linux kernel
2) TPROXY / Socket compiled into netfilter / iptables (due in v1.4.3?)
3) HaProxy compiled with the USE_LINUX_TPROXY option
The TPROXY patch for Linux Kernel 18.104.22.168 is here:
The following is a guide how to install on Centos 5.1:
A few months ago on my personal blog I wrote about the difficulty of installing CentOS on the Dell R200, owing to the SATA DVD drive used in the unit. The R200 is our unit of choice for our ClusterScale “Pegasys” product, so installing Linux distros on this server is a regular occurrence for us. Fortunately, we got hold of a Redhat driver image from Dell and we managed to load it on using a USB key. This is a simple case of typing
linux dd at the installation command prompt, but note that you must copy the .img file onto the USB disk, rather than the files within the image itself.
Originally the problem was with CentOS 5.0, but as far as I know it’s still a problem with versions 5.1 and 5.2. Ubuntu installations do not seem to pose the same problem, so presumably they bundle SATA DVD drivers with the distro. I still can’t find this particular driver on the Dell site, so seems like a good idea to host it here too.
Standard Kernel builds of LVS (Linux Vitual Server) don’t have the ability to load balance traffic that is from the local node.
For example if you terminated some SSL traffic using stunnel or pound on the load balancer you then wouldn’t be able to forward that traffic to a backend real server through LVS.
First many thanks to Siim Põder for helping to port Carlos Lozano’s patch from 2.4 -> 2.6
In order to run an SSL reverse proxy on the same node that is running LVS
External client —> pound:443 –> Local:443 —> IPVS:80 —> RealServer
The patch for Linux Kernel 2.6.25 is here:
The following is a guide how to install on Centos 5.1: