07.13.07
Posted in OpenSourceSoftware, Virtualization, CentOS at 3:46 pm by Edward Bjarte Fjellskål
I have been running CentOS 5 with Xen virtualization since around mid April (I started to play with Redhat 5 beta and its Xen in Desember 2006). CentOS 5 ships with Xen-3.0.3 and with RedHat’s virt-manager, which is a very simple but OK tool to manage Xen guests/domUs. The only 1337 about virt-manager at this point, in my opinion, is the vnc integration.
A quick way to get up and running (if you are connected to internet), is to set up dhcpd on Dom0 and export an kickstart.cfg file by httpd etc. That is, if you dont have a dhcp on you network. If you do have a dhcp-server on your network, you should drop the dhcp part.
# yum install dhcp httpd
# cp /root/anaconda-ks.cfg /var/www/html/xen.cfg
# vi /var/www/html/xen.cfg
Change the permision of the file and remove etc. cdrom (install media) and %packages that you dont need. Driveorder should be xvda –driveorder=xvda.
Also add url –url http://your-centos-mirror.something.com/centos/5/os/{ARCH}. http://www.centos.org/modules/tinycontent/index.php?id=13 has a list of mirrors
# vi /etc/dhcpd.conf # setup you own network:
ddns-update-style interim;
ignore client-updates;
subnet 10.10.10.0 netmask 255.255.255.0 {
# — default gateway
option routers 10.10.10.1;
option subnet-mask 255.255.255.0;
option domain-name “v.gamelinux.org”;
option domain-name-servers 10.10.10.2;
# we want control over DomU’s adress
host node1 {
next-server node1.gamelinux.org;
hardware ethernet 00:16:3e:3e:c2:fd,;
fixed-address 10.10.10.101;
}
host node2 {
next-server node2.gamelinux.org;
hardware ethernet 00:16:3e:4a:6f:87;
fixed-address 10.10.10.102;
}
}
# service dhcpd start
# service httpd start
For some reason I like to have my DomU’s on a logical volume. I do not recommend this yet, cuz I lack testing, but here is how I end up doing it:
# lvcreate -L 10G -n LogVolN1 VolGroup00
# lvcreate -L 10G -n LogVolN2 VolGroup00
My setup, needs me to tail /var/log/messages to get the MAC address of the nodes (when you start the node installation), then edit the dhcp.conf and restart dhcpd. You could just setup a dhcp-pool.
Then you just fire up virt-manager, enter system name (node1 etc), choose your flavor of virtualization, and point to a centos mirror (http://mirror.hh.se/centos/5.0/os/i386/ ect.). The kickstart URL, should be the ip/host of you Dom0 (http://dom0.v.gamelinux.org/xen1.cfg etc.). Then choose normal disk partition : and use the lvm we just created (/dev/VolGroup00/LogVolN1). Choose an amount of memory and cpus, and the start the installation.
How much cpu and memory you give your guests are all up to you!
Permalink
07.03.07
Posted in OpenSourceSoftware at 1:38 pm by Edward Bjarte Fjellskål
Hi,
This time I have been playing with Apache 2.2.4 and Tomcat 5.0.30. This is a powerful setup, which can give you a fairly enterpri$e web solution 
. Technical platforms I have been working on is Ubuntu Dapper (using Heartbeat2) and CentOS 5 (with Redhat cluster-manager, to make it even more enterpri$e 
).
The main goal is to have a redundant installation with high availability, capable of scaling, so that your java powered website will be able to meet tomorrows demand of serving.
First of all, I like to grab the source of tomcat and apache, to get the latest and greatest?!? Head over to Apache to for fill your needs. When I worked with this, tomcat 5.0 was in release branche 5.0.30-beta and httpd was in 2.2.4. This article will probably be relevant to other versions of tomcat and httpd.
My article will be based on the following setup in mind:
You have a java powered website, using a database to store data and a file system to store multimedia files etc.
I will assume that you have two backends, for redundancy of course , to serve NFS/GFS like file system and a database of your choice (Probably connected to a SAN and maybe you use DRBD?).
I will concentrate on the frontends.
Depending on your funds, I will recommend two machines in front for running the Apache httpd service, two machines in the middle running tomcat, and two machines running as backends. For even more ways to put more money into it, add two redundant cache servers up front (running varnish of course). That should make you ready for traffic!
The more simple setup, which goes along way to, is to have apache tomcat and httpd running on the same hardware. I will have this in mind when I’m writing this, and for advanced users, this should be a no-brainer to scale up.
I assume I have a NFS like file system mounted under /shared/ with webapplications under /shared/webapps/ and my bogus webapp under /shared/webapps/gamelinux/. I will not write about Redhat Cluster-Manager, nor about heartbeat, as I belive that the web has plenty of stuff about it, and google is your friend here
So you have downloaded apache httpd 2.2.4 (or newer)
I compile apache like this:
./configure --prefix=/usr/local/apache-2.2.4 --enable-disk-cache --enable-mem-cache --enable-cache --enable-proxy-balancer --enable-proxy --enable-proxy-connect --enable-proxy-http --enable-rewrite --enable-expires --enable-headers --enable-http --enable-so --enable-proxy-ajp --enable-info --with-mpm=worker
I enable caching in apache, dough it is very simple, and I dont use it daily, it can give you some nice benefits and it can be a nice swiss-army-knife! (”cache everything for ever - run for an hour or wget most of you site - take down tomcat, mysql etc.” or maybe you are getting slashdoted?) For my setup I use mpm=worker, which works best (memory and crash issues) for me. The mod_proxy_balancer with ajp support is the coolest thing here. We want httpd and tomcat to talk over ajp, since it is a more efficient way of communicating than using the http protocol.
(For a more state-of-the-art, high-performance HTTP accelerator, head over to the Varnish website)
The important part of the apache config, that makes it talk nice to tomcat, is something like this:
...
ProxyTimeout 30
ProxyPass / balancer://prodcluster/ stickysession=JSESSIONID nofailover=On
ProxyPassReverseCookieDomain balancer://prodcluster/ www.gamelinux.org
ProxyPassReverseCookiePath / /
<Proxy balancer://prodcluster/>
BalancerMember ajp://10.10.10.1:8009 route=tomcathost1
BalancerMember ajp://10.10.10.2:8009 route=tomcathost2
</Proxy>
…
Other things that could be nice to have in your httpd config:
...
ProxyPass /balancer-manager !
<Location /balancer-manager>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from 10.10.10.1
</Location>
…
DocumentRoot /shared/webapps/gamelinux
<Directory "/shared/webapps/gamelinux">
Options Indexes FollowSymLinks
AllowOverride None
Order allow,deny
Allow from all
</Directory>
# Serve static content from document root with httpd
RewriteCond %{REQUEST_URI} !^(.*.jsp)$
RewriteRule ^/(.*) %{DOCUMENT_ROOT}/$1 [L]
There are alot more of details that I could give you about my setup, but for simplicity I have presented the essentials. In a large setup, you should always stress test and read documentation to find and enhance your weak points.
For tomcat, I install it also under /usr/local/tomcat-5.0.30/.
I usually write my own custom startup scripts, but I don’t want to go into details here… The important part, is the server.xml in the tomcat config directory. For my basic setup, you need the ajp-connector, jvmroute(which should be different for each tomcat server) and in-memory session replication.
I give you a brief summary of the parts here:
…
<Connector className="org.apache.coyote.tomcat5.CoyoteConnector"
port="8009" minProcessors="100" maxProcessors="2000"
enableLookups="false" acceptCount="1000" debug="0"
connectionTimeout="90000" useURIValidationHack="false"
protocolHandlerClassName="org.apache.jk.server.JkCoyoteHandler"/>
…
<Engine name="Standalone" defaultHost="localhost" debug="0" jvmRoute="tomcathost1">
…
<Cluster className="org.apache.catalina.cluster.tcp.SimpleTcpCluster" managerClassName="org.apache.catalina.cluster.session.DeltaManager" expireSessionsOnShutdown="false" useDirtyFlag="true" name="gamelinux">
<Membership className="org.apache.catalina.cluster.mcast.McastService" mcastAddr="228.0.0.4" mcastPort="45564" mcastFrequency="500" mcastDropTime="3000"/>
<Receiver className="org.apache.catalina.cluster.tcp.ReplicationListener" tcpListenAddress="auto" tcpListenPort="4001" tcpSelectorTimeout="100" tcpThreadCount="6"/>
<Sender className="org.apache.catalina.cluster.tcp.ReplicationTransmitter"replicationMode="pooled"/>
<Valve className="org.apache.catalina.cluster.tcp.ReplicationValve" filter=".*\.png;.*\.swf;.*\.jpeg;.*\.bmp;.*\.gif;.*\.flv;.*\.js;.*\.jpg;.*\.htm;.*\.html;.*\.txt;"/>
</Cluster>
…
You also need to add <distributable /> to your web.xml in your /shared/webapps/gamelinux/WEB-INF/web.xml file.
So…
When you got tomcat and apache up and running, you should check out http://your.httpd.host/balancer-manager/ to see that both tomcats are in state OK. You should then check to see if JSESSIONS survives takeing down and up one tomcat server at the time. A simple code snippet to check that could be:
In /shared/webapps/gamelinux/index.jsp
<%@ page language="java" %>
<html>
<body>
<h1><font color="blue">Test page served by Tomcat5</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
Hopefully you now have a state-of-the-art apache httpd and tomcat setup. You should configure RHCM or heartbeat to fail over an IP address along with httpd. I also recommend to setup database pooling in tomcat, which would make tomcat more failsafe if the backend has problems.
Here is a simple apache cache config as a bonus. You should play around with this abit, and you might get to like it…:
CacheEnable mem /
MCacheSize 4096
CacheEnable disk /
CacheRoot /var/cache/apache224/gamelinux
CacheDirLevels 5
CacheDirLength 3
CacheDefaultExpire 300
CacheIgnoreNoLastMod On
ProxyVia Full
Permalink
