introduction
Use ipvsadm to instruct IPVS to forward requests from the Internet to ifw01 and onto the server cluster.
I built the LIC (Larg's Internet Cluster) to use a VS/NAT (Virtual Server via NAT) technique. There are two other ways of doing it: VS/TUN (Virtual Server via IP Tunneling) and VS/DR (Virtual Server via Direct Routing). See http://www.linuxvirtualserver.org/how.html for more information.
Part of this procedure makes the rules permanent. This is only useful at this stage of the build process. This configuration becomes redundant after I add the ldirectord application to ifw01. ldirectord will then manage the IPVS table: it adds and deletes rules when it feels like it. "Whatever, loser", it says, "your mum works in MacDonalds for the minimum wage".
The server cluster here is only ics01. Not much of a cluster. That's like a crowd of one. The other web servers get added later.
The firewall hosts ifw01 and ifw02 each contain an IPVS (IP Virtual Server) . They also contain synchronising daemons which monitor customer connections, so if one host dies the other can take over without customers noticing. This provides HA (High Availability).
| LIC topology: ipvsadm on ifw01 |
|---|
pre-flight checks
Check the current state of things before you start messing about. I get a test page using lynx, a web browser. You can also check a plaintext web server using wget
- Open a CLI on the server host ifw01.
- Get a page from the web server. This checks that the web server is running,
the test page exists and the routing works between this firewall and the web server.
issalarg@ifw01:~$ lynx -dump http://192.168.3.2/infrastructure/host1 ics01 issalarg@ifw01:~$
- Get a page.
issalarg@ifw01:~$ lynx -dump http://200.0.0.10/infrastructure/host1 ifw01 issalarg@ifw01:~$
- Open a CLI on the server host xcl01.
- Request a page.
issalarg@xcl01:~$ lynx -dump http://200.0.0.10/infrastructure/host1 ifw01 issalarg@xcl01:~$
- Close the CLI.
install ipvsadm
- Open a CLI on the server host.
- Use su to switch to the root account.
- install
ifw01:~# apt-get install ipvsadm Reading package lists... Done Building dependency tree... Done Suggested packages: heartbeat keepalived ldirectord The following NEW packages will be installed ipvsadm 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 47.5kB of archives. After unpacking 184kB of additional disk space will be used. WARNING: The following packages cannot be authenticated! ipvsadm Install these packages without verification [y/N]? - Close the CLI.
add rules
- Open a CLI on the server host ifw01.
- Use su to switch to the root account.
- Create rules temporarily.
ifw01:~# ipvsadm -A -t 200.0.0.10:80 -p 600 ifw01:~# ipvsadm -a -t 200.0.0.10:80 -r 192.168.3.2 -m -w 100 ifw01:~#
Those options mean- -A Add a virtual service.
- -t
- -p persistent service. A client's requests all get sent to the same server. The 600 is a timeout. If the client doesn't do anything for 600 seconds, then pick any server for their next request.
- -r real server that a request is forwarded to.
- -m Use masquerading. NAT (Network Address Translation), in other words.
- -w weight. IPVS assumes the server with the biggest number can handle the most requests, the server with the smallest number can handle the least requests, and the server with 0 weight can't handle any requests.
check your work
Look, listen and look again.
- List the rules.
ifw01:~# ipvsadm -l IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP ifw01.local:www wlc persistent 600 -> 192.168.3.2:www Masq 100 0 1 ifw01:~#
- Request a page.
issalarg@ifw01:~$ lynx -dump http://200.0.0.10/infrastructure/host1 ifw01 issalarg@ifw01:~$
It's the same! Why was it not forwarded to ics01? Because this only works from outside ifw01. - Open a CLI on the server host xcl01.
- Request a page.
issalarg@xcl01:~$ lynx -dump http://200.0.0.10/infrastructure/host1 ics01 issalarg@xcl01:~$
- Close the CLI.
make the ipvsadm rules permanent
This is simpler than making the iptables rules permanent.
- Dump the rules.
ifw01:~# ipvsadm -S -n -A -t 200.0.0.10:80 -s wlc -p 600 -a -t 200.0.0.10:80 -r 192.168.3.2:80 -m -w 100 ifw01:~#
- Back up the ipvsadm configuration file.
- Add to the rules file.
ifw01:~# cd /etc ifw01:/etc# vi ipvsadm.rules # emtpy rules file for ipvsadm -A -t 200.0.0.10:80 -s rr -a -t 200.0.0.10:80 -r 192.168.3.2:80 -m -w 1
But wait! The options have changed! If you spotted this, have a cookie. The new options mean- -s rr Tell the scheduler to use Robin Robin. Distribute requests equally amongst the available real servers.
- Close the CLI.
ifw01:~# cp /etc/ipvsadm.rules /home/issalarg/ ifw01:~#
Argh! I've made a mistake!
Don't worry. Be happy.
- Wipe out the rules.
ifw01:~# ipvsadm -C ifw01:~# - Check they are gone.
ifw01:~# ipvsadm -l IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn ifw01:~# - Start again.
troubleshoot
web server down
I forgot to switch on ics01.
ifw01:~# lynx -dump http://192.168.3.2/infrastructure/host1 Looking up 192.168.3.2 Making HTTP connection to 192.168.3.2 Alert!: Unable to connect to remote host. lynx: Can't access startfile http://192.168.3.2/infrastructure/host1 ifw01:~#
inactive connection
Look at the InActConn column. It has a 1 in it.
ifw01:~# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP ifw01.local:www wlc persistent 600
-> 192.168.3.2:www Masq 100 0 1
ifw01:~#
missing port
I missed the port 80 bit from my first line.
ifw01:~# ipvsadm -A -t 200.0.0.10 -p 600 ifw01:~# ipvsadm -a -t 200.0.0.10:80 -r 192.168.3.2 -m -w 100 Service not defined ifw01:~#
