To follow up to Aarons blog on HAProxy emails alerts using logwatch I was looking into different ways to achieve the same results.
Now the ideal way to monitor the health of the real servers is to to have a dedicated monitoring system in place such as Nagios( It even has a HAProxy plugin). However this is not always an option for some so they require the loadbalancer to send an alert. So I investigated some different options.
I had been tasked with finding a way to load balance UDP on AWS. Normally we would use ipvsadm in at layer 4 DR but because of the limitations in ec2 this is not possible so another method was needed.
So this is a short guide on how to get a simple UDP load balancer running. We will be using pen, it is described as a simple tcp load balancer and has been around for a while.
In general when you are load balancing a cluster you can evenly spread the connections through the cluster and you get pretty consistent and even load balancing. However with some applications such as RDS (Microsoft Terminal Servers), you can get very high load from just a few users doing heavy work. The solution to this is to use some kind of server load feedback agent. We’ve had one of these for a while in our product but now with a lot of help from Simon Horman we’ve managed to integrate the functionality into the main branch (well soon anyway) of HAproxy. We thought it would be a good idea to open source the previous work on Ldirectord/LVS, make it compatible with HAProxy, and release our Windows service code as GPL.
I must confess , at certain times it has looked like open warfare would break out between the support team and development team at Loadbalancer.org over the last few months.
One reason being that a major change in v7.5 is that all connection information is now displayed as a graph (rather than a static number). The support team was convinced that customers just wanted a number, a single number , a golden number… The development team was convinced that such a number was illogical irrelevant and meaningless……
It will be very interesting to see what the customers think!
In this guide I show a very simple solution to get HAProxy email alerts configured using Logwatch. While the first part is aimed at users of our V7 appliance I think anyone wanting to get email alerts for HAProxy will also find this a good example.
First from the WUI :
1. Set the external relay(Smart Host) under Edit Configuration > Physical – Advanced Configuration.
Transparent mode with HAProxy allows you to see the IP Address of the clients computer while still having a high availability service using HAProxy.
This posting shows how to setup a blank virgin installation of Centos 6.3 64bit minimum installation.
We have been doing some internal testing with Collectd.
“collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Or if you just want pretty graphs of your private server and are fed up with some homegrown solution you’re at the right place, too ;).” (taken from their website)
Anyway we are quite impressed with it but during the configuration I couldn’t locate an upstart script for it so I made my own.
When you have users depending on Windows Terminal Services for their main desktop, it’s a good idea to have more than one Terminal Server. RDP, however, is not an easy protocol to load balance; sessions are long-lived and need to be persistent to a particular server, and users may connect from different source addresses during one session.
The current development version of HAProxy has made an important step forward in making this possible. Thanks to work by Exceliance, it now supports RDP Cookies, offering a solution to the persistence problem.
OK so I’ve previously blogged about how to get TPROXY and HAProxy working nicely together. But what if you want to terminate SSL traffic on the load balancer in order to use HaProxy to insert cookies in the standard HTTP stream to the backend servers?
Many thanks to Krisztián Ivancsó for working on the TPROXY patch for Pound for us, we can finally do this!
From v7.5.2 VMware tools are pre-installed on the appliance which enables basic console control functions such as power on/off etc. However, the running version of the various kernel modules is still controlled by us at build time to ensure that only stable tested versions are deployed. If however the tools are later upgraded, these modules will be over-written. Therefore we do NOT recommend that the tools are updated.
For our software development we use Subversion – a powerful free tool for version control of files. Any file types are supported, even binary files. To be able to work with Subversion you have to setup a Subversion server. Then you can access the server from Subversion clients.
This blog entry shows one way to setup Subversion server on a Linux machine.
Standard Kernel builds don’t support TPROXY ( 2.6.28 does now!).
For example if you use HaProxy as the load balancer then all of the backend servers see the traffic coming from the IP address of the load balancer. TPROXY allows you to make sure the backend servers see the true client IP address in the logs.
Ps. An easier alternative is inserting the clients ip in the x-forwarded-for header (option forwardfor).
For TPROXY to work you need three things:
1) TPROXY compiled into the linux kernel
2) TPROXY / Socket compiled into netfilter / iptables (due in v1.4.3?)
3) HaProxy compiled with the USE_LINUX_TPROXY option
The TPROXY patch for Linux Kernel 188.8.131.52 is here:
The following is a guide how to install on Centos 5.1:
A few months ago on my personal blog I wrote about the difficulty of installing CentOS on the Dell R200, owing to the SATA DVD drive used in the unit. The R200 is our unit of choice for our ClusterScale “Pegasys” product, so installing Linux distros on this server is a regular occurrence for us. Fortunately, we got hold of a Redhat driver image from Dell and we managed to load it on using a USB key. This is a simple case of typing
linux dd at the installation command prompt, but note that you must copy the .img file onto the USB disk, rather than the files within the image itself.
Originally the problem was with CentOS 5.0, but as far as I know it’s still a problem with versions 5.1 and 5.2. Ubuntu installations do not seem to pose the same problem, so presumably they bundle SATA DVD drivers with the distro. I still can’t find this particular driver on the Dell site, so seems like a good idea to host it here too.
I’ve been toying with various flavors of laptops to use on my travels for a while (I’m not really a road warrior but my wife just can’t stop booking holidays).
Most of the time I get by with my trusty Blackberry for emails but sometimes you need a real computer… So I’ve always had to take a clunky laptop, normally dual boot Windows XP / Ubuntu (only need the XP as my current 3G card is somewhat flaky on Linux.).
Standard Kernel builds of LVS (Linux Vitual Server) don’t have the ability to load balance traffic that is from the local node.
For example if you terminated some SSL traffic using stunnel or pound on the load balancer you then wouldn’t be able to forward that traffic to a backend real server through LVS.
First many thanks to Siim Põder for helping to port Carlos Lozano’s patch from 2.4 -> 2.6
In order to run an SSL reverse proxy on the same node that is running LVS
External client —> pound:443 –> Local:443 —> IPVS:80 —> RealServer
The patch for Linux Kernel 2.6.25 is here:
The following is a guide how to install on Centos 5.1:
I’m going on holiday next week to a villa in Spain with the extended family. I’ll take my Blackberry & laptop with Ubuntu and a 3G card because I’m one of those sad people who like to stay in touch with the office… But I’d also like to use the laptop for my 6 year old son to watch DVDs on the plane, now thats all very well but the DVD drains the battery pretty quick and the reliability is pretty awful. So I thought no problem I’ll just rip the DVDs into ogg movie files.
Now I haven’t used any DVDs in Hardy 64 bit (only 32bit which worked fine) but I instantly hit a problem, after being automatically prompted to install the illegal gstreamer library (nice feature that auto prompt) Totem refused to play ball with the DVD:
“An error occurred Could not read from resource”
Open Source applications are even better in many ways, but that is another story ….
We started using an online web site chat service a while back, as it’s an invaluable sales and support tool for us. When we did a quick reckon of the market we quickly came across a problem. It was that old chestnut platform compatibility, we use a mixture of Macs, Linux (mainly Ubuntu) & a few sales guys on Windows.