Archive for: ‘February 3, 2013’

Load balancing with Apache Karaf Cellar, and mod_proxy_balancer

February 3, 2013 Posted by jbonofre

Thanks to Cellar, you can deploy your applications, CXF services, Camel routes, … on several Karaf nodes.

When you use Cellar with web applications, or CXF/HTTP endpoints, a “classic” need is to load balance the HTTP requests on the Karaf nodes.

You have different ways to do that:
– using Camel Load Balancer EIP: it’s an interesting EIP, working with any kind of endpoints. However, it requires to have a Karaf running the Load Balancer routes, so it’s not always possible depending of the user security policy (for instance, putting it in DMZ or so)
– using hardware appliances like F5, Juniper, Cisco: it’s a very good solution, “classic” solution in network teams. However, it requires expensive hardwares, not easy to buy and setup for test or “small” solution.
– using Apache httpd with mod_proxy_balancer: it’s the solution that I’m going to detail. It’s a very stable solution, powerful and easy to setup. And it costs nothing 😉

For instance, you have three Karaf nodes, exposing the following services and the hostname:
– http://192.168.134.3:8040/services
– http://192.168.134.4:8040/services
– http://192.168.134.5:8040/services

We want to load balance those three nodes.

On a dedicated server (it could be installed on one hosting Karaf), we just install Apache httpd:


# on Debian/Ubuntu system
aptitude install apache2


# on RHEL/CentOS/Fedora system
yum install httpd
# enable network connect on httpd
/usr/sbin/setsebool -P httpd_can_network_connect 1

Apache httpd comes with mod_proxy, mod_proxy_http, and mod_proxy_balancer modules. Just check if those modules are loaded in the main httpd.conf.

You can now create a new configuration for your load balancer (directly in the main httpd.conf or by creating a conf file in etc/httpd/conf.d):


<Proxy balancer://mycluster>
  BalancerMember http://192.168.134.3:8040
  BalancerMember http://192.168.134.4:8040
  BalancerMember http://192.168.134.5:8040
</Proxy>
ProxyPass /services balancer://mycluster

The load balancer will proxy the /services requests to the different Karaf nodes.

By default, the mod_proxy_balancer module uses a byrequests algorithm: all nodes will receive the same number of requests.
You can switch to bytraffic (using the lbmethod=bytraffic in the proxy configuration): in that case, all nodes will receive the same amount of traffic (by KB).

The mod_proxy_balancer module is able to support session “affinity” if your application needs it.
When a request is proxied to some back-end, then all following requests from the same user should be proxied to the same back-end.
For instance, you can use the cookie in header to define the session affinity:


Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
<Proxy balancer://mycluster>
  BalancerMember http://192.168.134.3:8040 route=1
  BalancerMember http://192.168.134.4:8040 route=2
ProxySet stickysession=ROUTEID
</Proxy>
ProxyPass /myapp balancer://mycluster

The mod_proxy_balancer module also provide a web manager allowing you to see if your Karaf nodes are up or not, the number of requests received by each node, and the current lbmethod in use.

To enable this balancer manager, you just have to add a dedicated handler:


<Location /balancer-manager>
  SetHandler balancer-manager
  Order allow,deny
  Allow from all
</Location>

Point your browser to http://host/balancer-manager and you will see the manager page.

You can find more information about mod_proxy_balancer here: http://httpd.apache.org/docs/2.2/mod/mod_proxy_balancer.html.

Apache httpd with mod_proxy_balancer is an easy and good HTTP load balancer solution in front of Karaf and Cellar.

Multiple HTTP connectors in Apache Karaf

February 3, 2013 Posted by jbonofre

Installing the http feature in Karaf leverages Pax Web to embed a Jetty webcontainer.

By default, Karaf create a Jetty connector on the 8181 http port (and 8443 for https). You can change this port number by providing etc/org.ops4j.pax.web.cfg file.

But, you can also create new connector in the embedded Jetty.

You may see several advantages for multiple connectors:

  • you can isolate a set of applications, CXF services, Camel routes on a dedicated port number
  • you can setup a different configuration for each connector. For instance, you can create two SSL connectors, each with a different keystore, truststore, …

You can find etc/jetty.xml configuration file where you can create custom Jetty configuration.

NB: if you want to have both etc/org.ops4j.pax.web.cfg and etc/jetty.xmll, don’t forget to reference jetty.xml in org.ops4j.pax.web.cfg using the org.ops4j.pax.web.config.file property pointing to the jetty.xml, for instance:


# in etc/org.ops4j.pax.web.cfg
org.ops4j.pax.web.config.file=${karaf.home}/etc/jetty.xml

To configure a new connector, you can add a addConnector call in this configuration. For instance, we can create a new connector on 9191 http port number (and 9443 https port number):


  <Call name="addConnector">
    <Arg>
      <New class="org.eclipse.jetty.server.nio.SelectChannelConnector">
        <Set name="host">0.0.0.0</Set>
        <Set name="port">9191</Set>
        <Set name="maxIdleTime">300000</Set>
        <Set name="Acceptors">1</Set>
        <Set name="statsOn">false</Set>
        <Set name="confidentialPort">9443</Set>
        <Set name="name">myConnector</Set>
      </New>
    </Arg>
  </Call>

Now, Karaf will listen on 8181 and 9191 (for http), 8443 and 9443 (for https).

You can also define a connector dedicated to https with dedicated configuration for this connection, especially keystore, truststore, and client authentication:


  <Call name="addConnector">
    <Arg>
      <New class="org.eclipse.jetty.server.ssl.SslSelectChannelConnector">
        <Set name="port">9443</Set>
        <Set name="maxIdleTime">30000</Set>
        <Set name="keystore">./etc/keystore</Set>
        <Set name="password">password</Set>
        <Set name="keyPassword">password</Set>
      </New>
    </Arg>
  </Call>

By default, the web application will be bind on all connectors. If you want that your web application use a specific connector, you have to define it in the MANIFEST using the following properties:


Web-Connectors: myConnector
Web-VirtualHosts: localhost

If you use CXF services or Camel routes, if you use a connetor hostname and port number in the endpoint, it will use the corresponding connector.

For instance, the following CXF endpoint of a Camel route will use myConnector:


...
  <cxf:cxfEndpoint id="cxfEndpoint" address="http://localhost:9191/services/myservice" wsdlUrl="..."/>
...

Karaf allows you a fine grained Jetty configuration. Karaf becomes a real complete WebContainer, with custom configuration on several connectors. It’s especially interesting for SSL connector where each connector can have a dedicated keystore and truststore, and client authentication configuration.