Jean-Baptiste Onofré's Blog

Posts Tagged: ‘karaf’

Apache Karaf 2011 Millesime

January 2, 2012 Posted by jbonofre

We are preparing the new Karaf “Cuvée” 2012, with the main new release Karaf 3.0.

Karaf 2011 is a good millésime with the following activity:

New sub-projects: 3 (Cellar, Cave, WebConsole)
Number of commits: 1,851
Number of releases: 11
Number of messages on the forums/mailing lists: 3119
Number of direct downloads: 44,829
Number of tickets open/resolved: 782/577
Karaf active team: 12

A good millésime for sure.

Now, it’s time to focus on the 2012 vendange

Thanks for the Karaf users and the wonderful Karaf team

Coming in Karaf 3.0: new KAR service, command, management

December 27, 2011 Posted by jbonofre

We are working hard on Karaf 3.0. This major new version of Karaf will bring a lot of new features and enhancements.

KAR in Karaf 2.2.4

As you may know, Karaf 2.2.x already provides “basic” KAR support.

A KAR (KARaf archive) is a zip file which gathers a features XML and all bundles and configuration files dependencies. It means that a KAR is an atomic artifact which can be deployed without an Internet connection (as all is shipped in the KAR file itself).

In Karaf 2.2.x, the KAR support was:

a KAR deployer: you can copy a KAR file into the deploy folder, and KARAF will uncompress the KAR file, and install all features containing in the shipped features XML.
a create-kar goal in the features-maven-plugin: this goal reads a features XML and creates a KAR file, ready to be deployed.

So, all KAR logics were in the KAR deployer.

It means that you are not able to deploy a KAR from a remote URL, etc.

What’s coming in Karaf 3.0

The KAR service

In Karaf 3.0, the KAR core bundle provides a org.apache.karaf.kar.KarService OSGi service.

This service allows you to list the installed KAR files, deploy a KAR file from an URL, uninstall a KAR file.

Now the KAR service use a data/kar directory to store the KAR file and uncompress the KAR directly in the system repository (the local-repo directory is no more needed and created).

The KAR shell commands

Now, you have a complete control on the KAR files (including remotely using a SSH connection).

You can list, install, and uninstall KAR files using the kar:* commands:

kar:list displays the installed KAR files
kar:install installs a KAR file from a given URL. The URL could be a “classic” file: or http:, but, as for all stuff in Karaf, mvn: (for instance mvn:net.nanthrax/test/1.0-SNAPSHOT/kar)
kar:uninstall uninstalls a KAR file (identified by the name as displayed by kar:list). This command doesn’t alter the system repository by default, but you have an option -c (–cleanup) to cleanup the system repository.

The KAR MBean

You can also manipulate KAR files using JMX via a new MBean: org.apache.karaf:type=kar,name=root.

This MBean provides quite the same actions that you can do using the commands:

getKars()
install(url)
uninstall(name)
uninstall(name, clean)

In progress

It’s still a work in progress. I will add a bunch of unit tests and new features around the KAR files.

Read full article |Leave a comment
Tags: kar, karaf
Categories: Apache, Apache Karaf

Overview on Apache Karaf, Pax Web, and Camel archetypes

December 19, 2011 Posted by jbonofre

In my previous blog post, I introduced the Karaf Maven plugins.
The Karaf Maven plugins are really helpful, starting from an existing POM.

If you can write this POM by hand (it’s my favorite way ), we also provide several archetypes which create and pre-configure a Maven project for you.

Karaf Archetypes

The next Karaf release (2.2.5) provides a set of new archetypes:

Assembly Archetype

The karaf-assembly-archetype create a Maven project which create a custom Karaf distribution.

It allows you to create your own Karaf distribution. The project downloads a Karaf standard distribution (in tar.gz and zip formats), unpack it, and create a new distribution.

The easiest way to use it is to use the interactive mode:

mvn archetype:generate -DarchetypeGroupId=org.apache.karaf.archetypes -DarchetypeArtifactId=karaf-assembly-archetype -DarchetypeVersion=2.2.5-SNAPSHOT

Bundle Archetype

If basically a bundle is a jar file with some special statement in the MANIFEST, the easiest way to create a bundle is to use the Felix maven-bundle-plugin.

Karaf provides an archetype which prepare a Maven project and provide a bundle Activator (a special callback class when the bundle start/stop).

To generate this project, simply type:

mvn archetype:generate -DarchetypeGroupId=org.apache.karaf.archetypes -DarchetypeArtifactId=karaf-bundle-archetype -DarchetypeVersion=2.2.5-SNAPSHOT

NB: I think that this archetype should be in Felix (as the same level as maven-bundle-plugin), I will propose a donation to the Felix community.

Blueprint Archetype

Blueprint is IoC approach applied to OSGi (it comes from Spring DM in fact). It allows you to avoid to write bundle Activator, ServiceTracker, etc.

Karaf provides a karaf-blueprint-archetype which prepare a Maven project including the maven-bundle-plugin, and a blueprint descriptor with a sample of OSGi service definition:

mvn archetype:generate -DarchetypeGroupId=org.apache.karaf.archetypes -DarchetypeArtifactId=karaf-blueprint-archetype -DarchetypeVersion=2.2.5-SNAPSHOT

NB: I think that this archetype should be in Aries (as it’s the blueprint implementation used in Karaf), I will propose a donation to the Aries community.

Feature Archetype

In Karaf, a feature is an application descriptor, defining all bundles, configurations/configuration files, others features. This features descriptor could be generated by hand (it’s my favorite way, and I guess the recommended one), we also provide a archetype which prepare a project to generate features file regarding the POM dependencies:

mvn archetype:generate -DarchetypeGroupId=org.apache.karaf.archetypes -DarchetypeArtifactId=karaf-feature-archetype -DarchetypeVersion=2.2.5-SNAPSHOT

Kar Archetype

We saw in my previous post a new goal of the features-maven-plugin to create a kar file starting from a features XML. A KAR file is a zip file containing the features XML and all its dependencies.

We also provide an archetype to prepare a Maven project containing a features XML and generate a KAR file:

mvn archetype:generate -DarchetypeGroupId=org.apache.karaf.archetypes -DarchetypeArtifactId=karaf-kar-archetype -DarchetypeVersion=2.2.5-SNAPSHOT

Pax-Web Archetypes

We also add several archetypes in Pax Web to deal with the web bundle. Charles talk about wab-gwt archetype in his last blog entry (http://cmoulliard.blogspot.com/2011/12/run-google-web-toolkit-2-project-on.html), but I added also two others.

Web Bundle Archetype

The Web Bundle Archetype create a “special” bundle containing web resources and statements. It creates a Maven project with webapp resources and WebApp Context in the POM:

mvn archetype:generate -DarchetypeGroupId=org.ops4j.pax.web.archetypes -DarchetypeArtifactId=wab-archetype -DarchetypeVersion=1.1.2-SNAPSHOT

War Archetype

We also provide an archetype to create a Maven project which generate a standard war (that you can deploy “outside” of OSGi) but including OSGi statements in the MANIFEST (which allows you to use OSGi values):

mvn archetype:generate -DarchetypeGroupId=org.ops4j.pax.web.archetypes -DarchetypeArtifactId=war-archetype -DarchetypeVersion=1.1.2-SNAPSHOT

Camel Archetypes

Camel also provides a set of very useful archetypes, especially:

camel-archetype-blueprint to generate a Maven project with a blueprint XML in which you can define your routes
camel-archetype-component providing a template to create your own Camel component
camel-archetype-dataformat providing a template to create your own Camel data format
camel-archetype-java providing a template with a class in which you can define your routes
camel-archetype-spring to generate a Maven project with a Spring XML in which you can define your routes

For instance, to create a Camel blueprint Maven project, simply type:

mvn archetype:generate -DarchetypeGroupId=org.apache.camel.archetypes -DarchetypeArtifactId=camel-archetype-blueprint -DarchetypeVersion=2.9-SNAPSHOT

Read full article |Leave a comment
Tags: camel, karaf, pax-web
Categories: Apache, Apache Camel, Apache Karaf

Do you know the Apache Karaf Maven plugins ?

December 12, 2011 Posted by jbonofre

Apache Karaf is not only an OSGi container, it also provides a set of Maven plugins for tooling.

In the next Apache Karaf 2.2.5 release, you will find two Maven plugins:

cmdhelp-maven-plugin generates documentation (in DocBook or Scalate format) for Karaf commands
features-maven-plugin provides a set of goals to manipulate Karaf features

In this blog post, I will cover the features-maven-plugin as I think it’s the most interesting for your life with Karaf.

Generate a features XML

If I prefer to handle and create the features XML by hand, the features:generate-features-file goal could do it for you.

It takes the dependencies of your project (described in the POM), and create the features XML.

For instance, in the following example, a features XML file will be generated containing the commons-lang bundle:

<xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">


  <modelVersion>4.0.0</modelVersion>
  <groupId>net.nanthrax</groupId>

  <artifactId>test-features</artifactId>

  <version>1.0-SNAPSHOT</version>

  <packaging>pom</packaging>
  <dependencies>

    <dependency>

      <groupId>org.apache.servicemix.bundles</groupId>

      <artifactId>org.apache.servicemix.bundles.commons-lang</artifactId>

      <version>2.4_4</version>

    </dependency>

  </dependencies>

You can find the features XML in the target directory.

Copy features resources in a local directory

The features:add-features-to-repo goal reads a set of features and copy features resources into a target folder. Especially, it allows you to prepare a custom distribution, allowing this distribution to avoid Internet connection to resolve features resources.

The following example will read a features XML and populate the target/system directory with the feature A, B, and C resources (bundles and configuration files):


  <modelVersion>4.0.0</modelVersion>
  <groupId>net.nanthrax</groupId>

  <artifactId>test-features</artifactId>

  <version>1.0-SNAPSHOT</version>

  <packaging>pom</packaging>

<build> <plugins> <plugin> <groupId>org.apache.karaf.tooling</groupId> <artifactId>features-maven-plugin</artifactId> <version>2.2.5-SNAPSHOT</version> <executions> <execution> <id>add-features-to-repo</id> <goals> <goal>features-add-to-repo</goal> </goals> <configuration> <descriptors> <descriptor>file:${project.basedir}/src/main/resources/features.xml</descriptor> </descriptors> <features> <feature>A</feature> <feature>B/2.0-SNAPSHOT</feature> </features> <repository>target/system</repository> </configuration> </execution> </executions> </plugin> </plugins> </build> </project>

NB: with Karaf 2.2.5, you can define a feature with just its name, or with name/version. It allows to target explicitly with which feature populate the target repository directory.

Create a KAR file

A KAR file (Karaf ARchive) is a zip file which package a features XML with all dependencies (bundles and configuration files).

It allows you to deploy (just by copying the kar file in the Karaf deploy folder) an atomic archive shipping all required resources (so no Internet connection is required).

The features:create-kar goal create a kar file starting from a given features XML.

The following example create a kar file (in the target folder) starting from a src/main/resources/features.xml:


  <modelVersion>4.0.0</modelVersion>
  <groupId>net.nanthrax</groupId>

  <artifactId>test-features</artifactId>

  <version>1.0-SNAPSHOT</version>

  <packaging>pom</packaging>

<build> <plugins> <plugin> <groupId>org.apache.karaf.tooling</groupId> <artifactId>features-maven-plugin</artifactId> <version>2.2.5-SNAPSHOT</version> <executions> <execution> <id>create-kar</id> <goals> <goal>create-kar</goal> </goals> <configuration> <featuresFile>${project.basedir}/src/main/resources/features.xml</featuresFile> </configuration> </execution> </executions> </plugin> </plugins> </build> </project>

Karaf 3.0 changes

In Karaf 3.0.0, you will find only one Maven plugin: karaf-maven-plugin, gathering the two “old” one.

The KAR support is also extended and give more place to KAR archives (used to construct distributions now).

Read full article |Leave a comment
Tags: karaf
Categories: Apache, Apache Karaf

Apache Karaf Cellar and DOSGi

November 29, 2011 Posted by jbonofre

Next version of Apache Karaf Cellar will include a DOSGi (Distributed OSGi) implementation.

There are several existing DOSGi implementations: in Apache CXF (powered by the CXF stack), in Fuse Fabric, etc.

The purpose of the Cellar one is to leverage the Cellar existing (Hazelcast instance, distributed map, etc), and to be very easy to use.

Karaf Cellar DOSGi installation

Cellar DOSGi is part of the main cellar feature. To install it:

karaf@root> features:addurl mvn:org.apache.karaf.cellar/apache-karaf-cellar/3.0.0-SNAPSHOT/xml/features karaf@root> features:install cellar

Distributed services

You can note a new command available in Cellar:

karaf@root> cluster:list-services

It displays the list of services “cluster aware”. It means a service that could be used remotely from another node.

Code sample

To illustrate the Cellar DOSGi usage, we will use two bundles:

the provider bundle is installed on node A and “expose” a distributed service
the client bundle is installed on node B and will use the provider service

Provider bundle

The provider bundle expose an OSGi service, flagged as a distributed service.

The service is very simple, it’s just an echo service.

Here’s the interface describing the service:

package org.apache.karaf.cellar.sample;


public interface EchoService {
  String process(String message);

}

and the corresponding implementation:

package org.apache.karaf.cellar.sample;


public class EchoServiceImpl implements EchoService {
&nbps; public String process(String message) {

    return "Processed by distributed service: " + message;

  }

}

Up to now, nothing special, nothing related to DOSGi or Cellar.

To expose the service in Karaf, we create a blueprint descriptor:

<?xml version="1.0" encoding="UTF-8"?> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">


  <bean id="echoService" class="org.apache.karaf.cellar.sample.EchoServiceImpl"/>
  <service ref="echoService" interface="org.apache.karaf.cellar.sample.EchoService">

    <service-properties>

      <entry key="service.exported.interfaces" value="*"/>

    </service-properties>

  </service>

</blueprint>

We can note that the only “special” part is that we added the service.exported.interfaces property to the echoService.

It’s just a flag to define the interface/service to define as distributed (it means accessible from remote node).

We didn’t change the code of the service itself, just added this property. It means that it’s really easy to turn an existing service as a distributed service.

Client bundle

The client bundle will get a reference to the echoService. In fact, the reference will be a kind of proxy to the service implementation located remotely, on another node.

The client is really simple, it indefinitely iterates to use the clusterService:

package org.apache.karaf.cellar.sample.client;


public class ServiceClient {
  private EchoService echoService;
  public void setEchoService(EchoService echoService) {

    this.echoService = echoService;

  }
  public EchoService getEchoService() {

    return this.echoService;

  }
  public void process() throws Exception {

    int count = 0;

    while (true) {

      System.out.println(echoService.process("Call " + count));

      Thread.sleep(5000);

      count++;

    }

  }

}

We inject the echoService using Blueprint:

&lt?xml version="1.0" encoding="UTF-8"?> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0">


  <reference id="echoService" interface="org.apache.karaf.cellar.sample.EchoService"/>
  <bean id="serviceClient" class="org.apache.karaf.cellar.sample.client.ServiceClient" init-method="process">

&nbps;   <property name="echoService" ref="echoService"/>

  </bean>

</blueprint>

It’s done. The serviceClient will use the echoService. If a “local” echoService exists, the OSGi framework will bind the reference to this service, else Cellar will look for a distributed service (on all node) exporting the EchoService interface and bind a proxy to the distributed service.

Read full article |Leave a comment
Tags: karaf, servicemix
Categories: Apache, Apache Karaf, Apache ServiceMix

Apache Karaf moved to OSGi r4.3

November 3, 2011 Posted by jbonofre

Apache Karaf trunk (future 3.0 release) now fully supports OSGi 4.3 release by running Apache Felix framework 4.0.1 and Eclipse Equinox 3.7.1.

If this update is just a support update, it gives us the opportunity to see what we can leverage from the OSGi r4 early draft.

What’s in preparation in OSGi r4 ?

I will not cover the whole OSGi 4 specification. I will just spot some features that will be “key” features in Karaf.

New OBR specification (RFC-0112)

Currently, there are a number of available solutions to downloading and installing bundles:

current OBR stores the bundles and allows users to discover bundles
P2 provisioning platform used in Eclipse
Maven repositories as used in Karaf
Nimble Repositories which are an extension of OBR to deal with active state dependencies

The idea is to gather all these concepts into a new OBR specification.

The new OBR will be able to:

handle bundle fragments dependencies, provide management tooling
can use a resolver strategy allowing to work with Felix OBR, Sigil or Nimble
a normalized and new OBR XML schema, supporting bundle execution environment, etc.

Karaf Cave will be the perfect place to leverage new OBR specification implementation. It will allow Karaf to deploy bundles provided by P2, OBR, Nimble, etc.

Karaf Cave will also extend the OBR specification in order to directly handle Karaf features.

ACE will also use this new OBR specification for the ACE repository implementation.

Subsystems (RFC-0152)

Most of container defines a concept to gather bundles together and provide complete applications.

Depending of the container, we have different wording:

in Karaf, we name it features
in Spring DM, the name is applications
in Aries, it’s applications too
in Eclipse, it’s features or installable units

If the core concept is the same, the implementations are very different.

The Subsystems specification aim to use an unique way to define the concept of applications.

Subsystems leverages others OSGi technologies:

OBR, we already discussed about that in the first section. It’s the repository used to store all bundles required to be deploy to form an application.
framework hooks (RFC-0138) to isolate or group subsystems within a single OSGi framework.
deployment admin defines a file format to ship and deploy applications.
application admin provides the concept of an application in OSGi.

Subsystems could become a key feature in Karaf 3.0, as it could handle Karaf features and KAR.

We implemented in features concepts defined in this specification. For instance, in a Karaf features, we can use an OBR resolver.

More over, subsystems provides new features missing in the current Karaf features, especially a complete lifecycle, better subsystems dependencies.

Regarding the new OBR and subsystems specifications, we can have a consistent way to deploy applications and bundles.

OSGi for Enterprise (RFC-0164 and RFC-0146)

RFC-0146 covers Java EE JCA for OSGi. It will provides support for resource adapters, packaged as a RAR. The resources can be integrate with OSGi services.

On the other hand, RFC-0164 deals with Blueprint Declarative Transaction. It means that we can define the transaction on a bean methods by declaration in the blueprint descriptor:

<bean ...> <tx:transaction method="count*" value="Required"/> <tx:transaction method="count*Row" value="RequiresNew"/> </bean>

These two specifications are a step forward to a Karaf EE distribution, by including such features and providing others.

OSGi JMX (RFC-0169)

Lot of comments and bugs have been raised regarding the OSGi-JMX specification.

This specification aim to implement most of suggestions and fix bugs.

Read full article |Leave a comment
Tags: karaf, osgi
Categories: Apache, Apache Karaf

What’s new in Karaf 2.2.4 ?

October 17, 2011 Posted by jbonofre

Apache Karaf 2.2.4 has been released this week end.

You can take a look on the release notes.

More than a bug fixes release, this version includes several new features and enhancements.

Command aliases

Previously, the osgi:* commands gathered different kind of usage: osgi:start, osgi:stop, etc are related to bundles, osgi:shutdown is related to container itself, and osgi:ls is OSGi service related.

To be clearer for the users, new aliases have been introduced.

NB: on Karaf trunk (future 3.0.0 release), the commands have been fully renamed (not aliased).

system:*

The system:* commands are related to the Karaf container itself:

system:shutdown is equivalent to osgi:shutdown and shutdown the Karaf container
system:start-level is equivalent to osgi:start-level and defines the default start-level to consider bundles as system bundles

services:*

The services:* commands are related to OSGi services:

services:list is equivalent to ls and lists the OSGi services available

bundles:*

The bundles:* commands are related to OSGi bundles:

bundles:list is equivalent to osgi:list and lists the OSGi bundles
bundles:install is equivalent to osgi:install and installs OSGi bundles
bundles:uninstall is equivalent to osgi:uninstall and uninstalls OSGi bundles
bundles:start is equivalent to osgi:start and starts OSGi bundles
bundles:stop is equivalent to osgi:stop and stops OSGi bundles
bundles:header is equivalent to osgi:headers and displays OSGi bundles headers
bundles:info is equivalent to osgi:info and displays bundle.info file
bundles:refresh is equivalent to osgi:refresh and refresh OSGi bundles
bundles:resolve is equivalent to osgi:resolve and resolves OSGi bundles
bundles:restart is equivalent to osgi:restart and restarts OSGi bundles
bundles:start-level is equivalent to osgi:start-level and defines the OSGi bundles start level
bundles:update is equivalent to osgi:update and updates OSGi bundles

New MBeans

To be able to monitor and administrate with your own tool (Nagios, Apache Kalumet, your own JMX client), Karaf now provides a set of new MBeans.

Bundles MBean

The bundles MBean (org.apache.karaf:type=bundles) allows you to view and change the bundles. As with bundles:* shell commands, you can select bundles using bundle ID (for instance 20), bundle ID range (for instance 21-25 means bundle ID 21, 22, 23, 24, 25), bundle name and version including support of regex (for instance mybunde, mybundle/version, myb*).

list() operation gives your a tabular view about all bundles deployed, including the ID, the location, and the current status of the bundle
getStartLevel() and setStartLevel() allows you to get/set the start level of bundles
refresh() allows to refresh bundles
update() allows to update bundles
resolve() allows to resolve bundles
restart() allows to restart bundles
install() installs bundles from an URL. The URL could be a “classic” one (file: or http:), a Maven URL (mvn:), an OBR URL (obr:)
start() starts bundles
stop() stops bundles
uninstall() uninstalls bundles

Config MBean

The config MBean (org.apache.karaf:type=config) allows you to view and change the OSGi configuration (ConfigAdmin layer). As with config:* shell commands, the changes are flushed into the /etc configuration file.

list() gives you all configuration PID present in the Karaf container
create() creates a new configuration PID
delete() deletes a configuration PID
proplist() list properties for a given configuration PID
propdel() deletes a property from a given configuration PID
propappend() appends content to a property in the given configuration PID
propset() set/add a property in the given configuration PID

Dev MBean

The dev MBean (org.apache.karaf:type=dev) allows you view the OSGi framework in use and set the framework options:

framework() gives the current OSGi framework in use (Felix or Equinox)
frameworkOptions() allows you to define OSGi framework options (debug mode, framework to use)
restart() allows you to restart the Karaf container

Log MBean

The log MBean (org.apache.karaf:type=log) allows you to view/change the log level currently used (globally or for a given logger):

set() sets the log level to use (for the root logger, or for a given logger)
get() gives the log level in use (for the root logger, or for a given logger)

OBR MBean

The OBR MBean (org.apache.karaf:type=obr) allows you to manipulate the OBR client:

listUrls() list the OBR server URL (containing the repository.xml file) register in the Karaf OBR
addUrl() adds a new OBR server in the OBR client
removeUrl() removes an OBR server from the OBR client
refreshUrl() updates the OBR information in the OBR client
list() lists the bundles available via the OBR client
deploy() deploys a bundle (and its dependency bundles) from the OBR

Packages MBean

The packages MBean (org.apache.karaf:type=packages) allows you to get information about exported and imported packages (and the related bundles):

exportedPackages lists the exported package (globally or for a given bundle)
importedPackages lists the imported package (globally or for a given bundle)

Services MBean

The services MBean (org.apache.karaf:type=services) allows you to view the OSGi services available:

list() lists the OSGi services available (globally, or only one in use, or provided by a given bundle)

System MBean

The system MBean (org.apache.karaf:type=system) allows you to manipulate the Karaf container itself:

shutdown() shutdown the Karaf container

Web MBean

The web MBean (org.apache.karaf:type=web) allows you to see the web bundles (web application) available:

list() gives you a tabular view of all web application deployed in Karaf (including the bundle and the context path)

Better KAR support

A KAR (Karaf ARchive) artifact is a zip file containing a feature XML descriptor and related bundles jar.

In previous Karaf version, we had issue with the Kar support. In Karaf 2.2.4:

the features Maven plugin provides a goal to create a KAR. Simply use a the create-kar goal with a feature.xml and you will have a ready to deploy KAR in the target directory
the KAR deployer was not able to correctly uncompress the KAR zip entries, it’s now fixed
the KAR deployer now automatically starts all features containing in a KAR file. It means that simply by dropping the KAR file in the deploy folder, Karaf will uncompress the KAR file, correctly creating a Karaf local repository for the KAR, and start all features containing in the KAR shipped feature XML

Spring features

To avoid Spring versions mismatch in others projects (such as Apache Camel, CXF, or ServiceMix), Karaf now provides a large set of Spring features:

spring
spring-web
spring-aspects
spring-dm
spring-dm-web
spring-instrument
spring-jdbc
spring-jms
spring-struts
spring-test
spring-orm
spring-oxm
spring-tx
spring-web
spring-web-portlet

It allows others projects to directly use these features (with a version range), and so, be less coupled to a Spring version.

Others

Some others minors things have been added:

the dev:watch command now supports multiple bundle as argument
the wrapper service provides a lib for Linux 64 bits platform
the J2SE-1.6 and J2SE-1.7 execution environment have been added
obr:deploy shell command now supports a -s option to automatically start bundles
Karaf JMX service could be configured to use SSL now
system:shutdown shell command displays the instance name (for confirmation)

Read full article |Leave a comment
Tags: apache, karaf
Categories: Apache, Apache Karaf

Apache Karaf Cave preview

August 25, 2011 Posted by jbonofre

During the Karaf birthday concall, the Karaf team launched the idea of implementing an easy to use OBR (OSGi Bundle Repository) server, and extending it to provide a Karaf Features Repository (KFR).

Welcome to Apache Karaf Cave

Cave is an Apache Karaf sub-project.

The core OBR is a service (RepositoryAdmin service) that can automatically install a bundle, with its deployment dependencies, from a repository.

Cave is a work in progress in the Karaf Sandbox SVN. I think it’s time to provide an initial overview about what we can do with Cave.

Cave already provides the following features:
- Storage: Cave includes a storage backend to host the artifacts. The default storage is a filesystem. We designed the Cave backend to be plug and play. This means that you can implement your own storage backend. For instance, I hope to provide implementations to store the bundles into a database, a LDAP server, or use directly a Maven repository manager as Apache Archiva.
- OBR Metadata Generation: Cave automatically looks for valid OSGi bundles and generates the OBR metadata.
- OBR Service Registration: Cave allows you to directly register a Cave Repository into the OBR RepositoryAdmin service.
- Populate Repository: of course, we can upload a single artifact in a Cave Repository, but you can also grab resources from a remote repository (for instance a Maven repository) via HTTP.
- Proxy Repository: Cave is also able to generate OBR metadata locally, referencing resources present on a remote repository.
- HTTP Wrapper Service: Cave exposes OBR metadata and bundles in an embedded HTTP server, allowing a OBR client to remotely access the Cave Server resources.
- REST Service: Cave provides a REST service layer, to let you administer the Cave Server using a REST client.
- Administration: Cave provides a set of MBeans, allowing you to complete administration of the Cave Server using a JMX client.
- Client Proxy: Cave also provides a Cave Client. It’s an OBR RepositoryAdmin service implementation which delegates the method calls to a remote Cave Server. This means that Karaf will see a local OBR RepositoryAdmin service which is actually using a remote Cave Server.

Checkout and build Cave

The Cave sources are on the Karaf sandbox subversion. You can checkout with svn:

svn co http://svn.apache.org/repos/asf/karaf/cave/trunk cave

To be able to build Cave, you have to use JDK 1.6 and Maven 3.0.3. Simply do, in the directory where you have checkout Cave:

mvn clean install

You should see something like:

[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Karaf :: Cave .............................. SUCCESS [1.221s] [INFO] Apache Karaf :: Cave :: Server .................... SUCCESS [0.101s] [INFO] Apache Karaf :: Cave :: Server :: API ............. SUCCESS [2.779s] [INFO] Apache Karaf :: Cave :: Server :: Storage ......... SUCCESS [4.287s] [INFO] Apache Karaf :: Cave :: Server :: Management ...... SUCCESS [0.879s] [INFO] Apache Karaf :: Cave :: Server :: Command ......... SUCCESS [1.372s] [INFO] Apache Karaf :: Cave :: Server :: HTTP ............ SUCCESS [0.873s] [INFO] Apache Karaf :: Cave :: Assembly .................. SUCCESS [7.400s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 19.558s [INFO] Finished at: Thu Aug 25 19:01:33 CEST 2011 [INFO] Final Memory: 46M/531M [INFO] ------------------------------------------------------------------------

Install Cave

To install Cave Server in a running Karaf instance, you have to register the Cave features descriptor:

karaf@root>> features:addurl mvn:org.apache.karaf.cave/apache-karaf-cave/3.0.0-SNAPSHOT/xml/features

You should be able to see the cave-service feature:

karaf@root> features:list|grep -i cave [uninstalled] [3.0.0-SNAPSHOT ] cave-server repo-0

To install and start Cave, simply install the cave-server feature:

karaf@root> features:install cave-server

NB: the installation of the cave-server feature will install others features, such as obr, http, war, and cxf. It could require several minutes depending of your Internet connection speed.

You can see the Cave bundles installed:

karaf@root> la|grep -i cave [ 134] [Active ] [ ] [ ] [ 60] Apache Karaf :: Cave :: Server :: API (3.0.0.SNAPSHOT) [ 135] [Active ] [Created ] [ ] [ 60] Apache Karaf :: Cave :: Server :: Storage (3.0.0.SNAPSHOT) [ 136] [Active ] [Created ] [ ] [ 60] Apache Karaf :: Cave :: Server :: Management (3.0.0.SNAPSHOT) [ 137] [Active ] [Created ] [ ] [ 60] Apache Karaf :: Cave :: Server :: Command (3.0.0.SNAPSHOT) [ 138] [Active ] [ ] [ ] [ 60] Apache Karaf :: Cave :: Server :: HTTP (3.0.0.SNAPSHOT)

and the Cave commands are now available:

karaf@root> cave: cave:create-repository cave:destroy-repository cave:list-repositories cave:populate-repository cave:proxy-repository cave:register-repository cave:scan-repository cave:upload-artifact

Cave Repositories

There is no limit in number of Cave Repositories you can create.

A Cave Repository is a container for:
- the OSGi bundles (jar file resources)
- the OBR metadata (repository.xml descriptor)

By default, with the filesystem storage, Cave uses the KARAF_BASE/cave directory to store the repositories.

You can change this storage location in the etc/org.apache.karaf.cave.server.storage.cfg configuration file:

# default value storage.location=cave # custom storage location #storage.location=/path/to/your/storage/folder

Create a Cave Repository

The cave:create-repository command create a Cave Repository:

karaf@root> cave:create-repository cave-repo

A Cave Repository is identified by a name.

Cave creates the repository storage in the global storage location. In our example, we can see a cave-repo directory in the default storage location:

shell$ ls cave/ cave-repo

You can use an existing directory (which can already contain artifacts) using the -l or –location option:

karaf@root< cave:create-repository -l /home/jbonofre/.m2/repository m2

By default, Cave creates the OBR metadata. If you don’t want to create the OBR metadata at repository creation time, you have to use the -nu or –no-update option:

karaf@root> cave:create-repository -nu -l /home/jbonofre/.m2/repository m2

A Cave Repository is directly register in the OBR service. If you don’t want this, you can use -nr or –no-register option:
You can also directly register the Cave Repository in the OBR service as soon as it’s created. To do this, use the -r or –register option.

List of Cave Repositories

You can list the Cave Repositories:

karaf@root> cave:list-repositories Name Location [cave-repo] [/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo] [m2] [/home/jbonofre/.m2/repository]

Remove and destroy a Cave Repository

You can remove a Cave Repository using the cave:remove-repository command:

karaf@root> cave:remove-repository cave-repo

This command only removes the Cave Repository from the repositories registry. It doesn’t physically delete the OBR metadata or the artifacts. It means that you can create the repository using the existing location.

If you want to destroy the Cave Repository including the artifacts and the storage directory, you have to use:

karaf@root> cave:destroy-repository cave-repo

Generate OBR metadata

At any time, you can generate/update the OBR metadata using:

karaf@root> cave:update-repository cave-repo

Cave will scan the repository storage, looking for OSGi bundles, and generate (or update) the OBR metadata.

Register Cave Repository

Once your Cave Repository contains OBR metadata, you can directly register the repository into the OBR service:

karaf@root> cave:register-repository cave-repo

As you will see later in this article, you are now ready to use OBR commands.

We will also see that your repository is remotely available using the Cave HTTP Wrapper Service or the Cave Client.

Populate a Cave Repository

Upload a single artifact

The first way to populate your Cave Repository is by uploading a single artifact:

karaf@root> cave:upload-artifact cave-repo file:/home/jbonofre/.m2/repository/org/apache/servicemix/bundles/org.apache.servicemix.bundles.asm/3.3_2/org.apache.servicemix.bundles.asm-3.3_2.jar karaf@root> cave:upload-artifact cave-repo http://svn.apache.org/repos/asf/servicemix/m2-repo/org/apache/qpid/qpid-broker/0.8.0/qpid-broker-0.8.0.jar

As you can see, you can use file: or http: URL. But you can also use Maven URL (mvn:groupId/artifactId/version):

karaf@root> cave:upload-artifact cave-repo mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.ant/1.7.0_5

Populate from an external repository

If you have a bunch of artifacts to upload, it’s not very efficient to use the cave:upload-artifact command.

The cave:populate-repository command allows you to upload a set of artifacts from an “external” repository:

karaf@root> cave:populate-repository cave-repo file:/home/jbonofre/.m2/repository

In this example, Cave will browse the file:/home/jbonofre/.m2/repository location, looking for OSGi bundles, and will copy the artifacts in your Cave Repository storage location.

Cave supports file: but also http: URL. It means that Cave is able to browse a remote repository.

For instance, you can populate your Cave Repository with all Ant ServiceMix bundles available on the Maven Central Repository:

karaf@root> cave:populate-repository cave-repo http://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.ant/

You can also populate your Cave Repository using the whole Maven Central Repository:

karaf@root> cave:populate-repository cave-repo http://repo1.maven.org/maven2

*WARNING*: the Maven Central Repository is really huge and it will require very very long time and space on the hard drive. It’s just for demonstration purpose, not really usable in the real life

By default, Cave updates the OBR metadata at population time. If you don’t want it, you can use the -nu or –no-update option:

karaf@root> cave:populate-repository -nu cave-repo http://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles/

Once your have populated your Cave Repository, you don’t need an Internet connection anymore as the bundles are present in the Cave Repository storage.

Proxy a Repository

A great advantage of Cave Repository population is that the bundles are locally present on the Cave Server (in the Cave Repository storage location).

But, you may also prefer to have the bundles on a remote repository (like the Maven Central Repository) and let Cave only handles the OBR metadata.

In that case, you can use the cave:proxy-repository command. The bundles stay on the “external” repository, the Cave Repository only stores the corresponding OBR metadata for the remote artifacts:

karaf@root> cave:proxy-repository cave-repo http://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.commons-lang/

*NB*: in this situation, the Cave Repository handles only the OBR metadata, it doesn’t monitor the remote repository for new artifacts or removed artifacts. It means that, if the remote repository changes (for instance, new artifacts are available), you have to re-execute the cave:proxy-repository command to update the OBR metadata.

*NB*: a best practice is to create a Cave Repository dedicated to a remote repository proxy.

OBR commands

When you have register your Cave Repository in the OBR service (using the cave:register-repository command), you can see it using the obr:list command:

karaf@root> obr:listurl file:/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo/repository.xml

and the bundles present in your Cave Repository are available in the OBR service:

karaf@root> obr:list [...] slf4j.log4j12 - slf4j-log4j12 (1.6.1) slf4j.simple - slf4j-simple (1.6.1) org.springframework.aop - Spring AOP (3.0.5.RELEASE) org.springframework.asm - Spring ASM (3.0.5.RELEASE) org.springframework.beans - Spring Beans (3.0.5.RELEASE) org.springframework.context - Spring Context (3.0.5.RELEASE) org.springframework.core - Spring Core (3.0.5.RELEASE) org.springframework.expression - Spring Expression Language (3.0.5.RELEASE) org.springframework.web - Spring Web (3.0.5.RELEASE) org.springframework.osgi.extensions.annotations - spring-osgi-annotation (1.2.1) org.springframework.osgi.core - spring-osgi-core (1.2.1) org.springframework.osgi.extender - spring-osgi-extender (1.2.1) org.springframework.osgi.io - spring-osgi-io (1.2.1) stax2-api - Stax2 API (3.1.1) stax2-api - Stax2 API (3.0.2) woodstox-core-asl - Woodstox XML-processor (4.1.1) org.apache.ws.xmlschema.core - XmlSchema Core (2.0.0)

You can also get detailed information about a bundle:

karaf@root> obr:info slf4j.api,1.6.1 --------- slf4j-api --------- id: slf4j.api/1.6.1 description: The slf4j API symbolicname: slf4j.api presentationname: slf4j-api uri: file:/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo/slf4j-api-1.6.1.jar size: 25496 version: 1.6.1 Requires: package:(&(package=org.slf4j.impl)(version>=1.6.0)) ee:(|(ee=J2SE-1.3)) Capabilities: bundle:{manifestversion=2, symbolicname=slf4j.api, presentationname=slf4j-api, version=1.6.1} package:{package=org.slf4j, version=1.6.1} package:{package=org.slf4j.spi, version=1.6.1} package:{package=org.slf4j.helpers, version=1.6.1}

*NB*: in Karaf, the OBR entry format is symbolicname,version

You have detailed information about an OSGi bundle, especially the bundle requirements and capabilities.

The OBR service (client) is able to resolve the dependencies between bundles, depending the requirements and capabilities of each bundle.

For instance, we have the following commons-pool bundle details:

karaf@root> obr:info org.apache.servicemix.bundles.commons-dbcp -------------------------------------------- Apache ServiceMix :: Bundles :: commons-dbcp -------------------------------------------- id: org.apache.servicemix.bundles.commons-dbcp/1.4.0.1 description: This OSGi bundle wraps commons-dbcp 1.4 jar file. documentation: http://www.apache.org/ symbolicname: org.apache.servicemix.bundles.commons-dbcp presentationname: Apache ServiceMix :: Bundles :: commons-dbcp license: http://www.apache.org/licenses/LICENSE-2.0.txt uri: file:/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo/org.apache.servicemix.bundles.commons-dbcp-1.4.0.1.jar size: 159721 version: 1.4.0.1 Requires: package:(&(package=javax.naming)) package:(&(package=javax.naming.spi)) package:(&(package=javax.sql)) package:(&(package=javax.transaction)) package:(&(package=javax.transaction.xa)) package:(&(package=org.apache.commons.pool)(version>=1.3.0)(!(version>=2.0.0))) package:(&(package=org.apache.commons.pool.impl)(version>=1.3.0)(!(version>=2.0.0))) package:(&(package=org.xml.sax)) package:(&(package=org.xml.sax.helpers)) Capabilities: bundle:{manifestversion=2, symbolicname=org.apache.servicemix.bundles.commons-dbcp, presentationname=Apache ServiceMix :: Bundles :: commons-dbcp, version=1.4.0.1} package:{package=org.apache.commons.dbcp.cpdsadapter, uses:=org.apache.commons.dbcp,javax.naming,javax.sql,org.apache.commons.pool.impl,org.apache.commons.pool,javax.naming.spi, version=1.4.0} package:{package=org.apache.commons.dbcp, uses:=org.apache.commons.pool.impl,org.apache.commons.pool,javax.sql,javax.naming,javax.naming.spi,org.apache.commons.jocl,org.xml.sax, version=1.4.0} package:{package=org.apache.commons.dbcp.managed, uses:=org.apache.commons.dbcp,javax.sql,org.apache.commons.pool.impl,javax.transaction,org.apache.commons.pool,javax.transaction.xa, version=1.4.0} package:{package=org.apache.commons.dbcp.datasources, uses:=javax.sql,org.apache.commons.pool,javax.naming,org.apache.commons.dbcp,javax.naming.spi,org.apache.commons.pool.impl, version=1.4.0} package:{package=org.apache.commons.jocl, uses:=org.xml.sax.helpers,org.xml.sax, version=1.4.0}

We can see that commons-dbcp requires org.apache.commons.pool package (in version range 1.3.0 and 2.0.0).

If we take a look on commons-pool bundle details:

karaf@root> obr:info org.apache.servicemix.bundles.commons-pool -------------------------------------------- Apache ServiceMix :: Bundles :: commons-pool -------------------------------------------- id: org.apache.servicemix.bundles.commons-pool/1.5.4.3 description: This OSGi bundle wraps commons-pool 1.5.4 jar file. documentation: http://www.apache.org/ symbolicname: org.apache.servicemix.bundles.commons-pool presentationname: Apache ServiceMix :: Bundles :: commons-pool license: http://www.apache.org/licenses/LICENSE-2.0.txt uri: file:/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo/org.apache.servicemix.bundles.commons-pool-1.5.4.3.jar size: 97332 version: 1.5.4.3 Capabilities: bundle:{manifestversion=2, symbolicname=org.apache.servicemix.bundles.commons-pool, presentationname=Apache ServiceMix :: Bundles :: commons-pool, version=1.5.4.3} package:{package=org.apache.commons.pool.impl, uses:=org.apache.commons.pool, version=1.5.4} package:{package=org.apache.commons.pool, version=1.5.4}

This bundle provides package org.apache.commons.pool capability.

It means that if we deploy the commons-dbcp bundle, the OBR should also deploy the commons-pool bundle:

karaf@root> obr:deploy org.apache.servicemix.bundles.commons-dbcp Target resource(s): ------------------- Apache ServiceMix :: Bundles :: commons-dbcp (1.4.0.1)


Required resource(s):

---------------------

   Apache ServiceMix :: Bundles :: commons-pool (1.5.4.3)

Deploying...done.

Done: the OBR has resolved that commons-pool has capabilities corresponding to the commons-dbcp requirements and so installed the commons-pool bundle in the same time of commons-dbcp.

*NB*: in the obr:deploy command, if we don’t explicitly mention the bundle version, the OBR will take the higher version available for the bundle symbolic name.

*NB*: the obr:deploy command doesn’t start the bundle, it only installs. I will enhance this command to add a -s option to start the bundles.

Cave HTTP Wrapper Service

When you install the Cave Service, it automatically starts an HTTP wrapper service.

This service allows you to access the OBR metadata and bundle artifacts via HTTP.

OBR metadata

For instance, you have the following Cave Repositories:

karaf@root> cave:list-repositories Name Location [cave-repo] [/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo] [m2] [/home/jbonofre/.m2/repository]

You can access the OBR metadata using the following URL in your favorite browser:


http://localhost:8181/cave/m2-repository.xml

*NB*: the 8181 port number is the default one used by the Karaf HTTP service.

To access the OBR metadata, the URL format is:


http://[cave_server_hostname]:[http_service_port]/cave/[cave_repository_name]-repository.xml

It means that you can register the Cave Repositories on a remote Karaf instance.

In the remote Karaf instance, you just have to install the obr feature and register the Cave HTTP Wrapper repository.xml URL:

karaf@other> features:install obr karaf@other> obr:addurl http://cave_server:8181/cave/cave-repo-repository.xml

OSGi bundles access

Cave HTTP Wrapper Service also exposes the bundles via HTTP.

For instance, if you have registered the cave-repo Cave Repository in the OBR service using:

karaf@localhost> cave:register-repository cave-repo

you have the following bundles available in the OBR service:

karaf@localhost> obr:list org.apache.servicemix.bundles.commons-dbcp - Apache ServiceMix :: Bundles :: commons-dbcp (1.4.0.1) org.apache.servicemix.bundles.commons-pool - Apache ServiceMix :: Bundles :: commons-pool (1.5.4.3)

If we take a look on the commons-dbcp bundle details:

karaf@localhost> obr:info org.apache.servicemix.bundles.commons-dbcp -------------------------------------------- Apache ServiceMix :: Bundles :: commons-dbcp -------------------------------------------- id: org.apache.servicemix.bundles.commons-dbcp/1.4.0.1 description: This OSGi bundle wraps commons-dbcp 1.4 jar file. documentation: http://www.apache.org/ symbolicname: org.apache.servicemix.bundles.commons-dbcp presentationname: Apache ServiceMix :: Bundles :: commons-dbcp license: http://www.apache.org/licenses/LICENSE-2.0.txt uri: file:/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo/org.apache.servicemix.bundles.commons-dbcp-1.4.0.1.jar size: 159721 version: 1.4.0.1 Requires: package:(&(package=javax.naming)) package:(&(package=javax.naming.spi)) package:(&(package=javax.sql)) package:(&(package=javax.transaction)) package:(&(package=javax.transaction.xa)) package:(&(package=org.apache.commons.pool)(version>=1.3.0)(!(version>=2.0.0))) package:(&(package=org.apache.commons.pool.impl)(version>=1.3.0)(!(version>=2.0.0))) package:(&(package=org.xml.sax)) package:(&(package=org.xml.sax.helpers)) Capabilities: bundle:{manifestversion=2, symbolicname=org.apache.servicemix.bundles.commons-dbcp, presentationname=Apache ServiceMix :: Bundles :: commons-dbcp, version=1.4.0.1} package:{package=org.apache.commons.dbcp.cpdsadapter, uses:=org.apache.commons.dbcp,javax.naming,javax.sql,org.apache.commons.pool.impl,org.apache.commons.pool,javax.naming.spi, version=1.4.0} package:{package=org.apache.commons.dbcp, uses:=org.apache.commons.pool.impl,org.apache.commons.pool,javax.sql,javax.naming,javax.naming.spi,org.apache.commons.jocl,org.xml.sax, version=1.4.0} package:{package=org.apache.commons.dbcp.managed, uses:=org.apache.commons.dbcp,javax.sql,org.apache.commons.pool.impl,javax.transaction,org.apache.commons.pool,javax.transaction.xa, version=1.4.0} package:{package=org.apache.commons.dbcp.datasources, uses:=javax.sql,org.apache.commons.pool,javax.naming,org.apache.commons.dbcp,javax.naming.spi,org.apache.commons.pool.impl, version=1.4.0} package:{package=org.apache.commons.jocl, uses:=org.xml.sax.helpers,org.xml.sax, version=1.4.0}

we can see that the URI is file:/home/jbonofre/apache-karaf-2.2.2/cave/cave-repo/org.apache.servicemix.bundles.commons-dbcp-1.4.0.1.jar.

The Cave HTTP Wrapper Service also exposes the bundle on:


http://localhost:8181/cave/org.apache.servicemix.bundles.commons-dbcp-1.4.0.1.jar

Cave is able to handle bundle URI relatively to the repository one.

If means that, if you register the cave-repo Cave Repository on a remote Karaf instance using the HTTP service:

karaf@remote> features:install obr karaf@remote> obr:addurl http://cave_server:8181/cave/cave-repo-repository.xml

you can take a look on the commons-dbcp bundle details:

karaf@remote> obr:info org.apache.servicemix.bundles.commons-dbcp -------------------------------------------- Apache ServiceMix :: Bundles :: commons-dbcp -------------------------------------------- id: org.apache.servicemix.bundles.commons-dbcp/1.4.0.1 description: This OSGi bundle wraps commons-dbcp 1.4 jar file. documentation: http://www.apache.org/ symbolicname: org.apache.servicemix.bundles.commons-dbcp presentationname: Apache ServiceMix :: Bundles :: commons-dbcp license: http://www.apache.org/licenses/LICENSE-2.0.txt uri: http://cave_server:8181/cave/org.apache.servicemix.bundles.commons-dbcp-1.4.0.1.jar size: 159721 version: 1.4.0.1 Requires: package:(&(package=javax.naming)) package:(&(package=javax.naming.spi)) package:(&(package=javax.sql)) package:(&(package=javax.transaction)) package:(&(package=javax.transaction.xa)) package:(&(package=org.apache.commons.pool)(version>=1.3.0)(!(version>=2.0.0))) package:(&(package=org.apache.commons.pool.impl)(version>=1.3.0)(!(version>=2.0.0))) package:(&(package=org.xml.sax)) package:(&(package=org.xml.sax.helpers)) Capabilities: bundle:{manifestversion=2, symbolicname=org.apache.servicemix.bundles.commons-dbcp, presentationname=Apache ServiceMix :: Bundles :: commons-dbcp, version=1.4.0.1} package:{package=org.apache.commons.dbcp.cpdsadapter, uses:=org.apache.commons.dbcp,javax.naming,javax.sql,org.apache.commons.pool.impl,org.apache.commons.pool,javax.naming.spi, version=1.4.0} package:{package=org.apache.commons.dbcp, uses:=org.apache.commons.pool.impl,org.apache.commons.pool,javax.sql,javax.naming,javax.naming.spi,org.apache.commons.jocl,org.xml.sax, version=1.4.0} package:{package=org.apache.commons.dbcp.managed, uses:=org.apache.commons.dbcp,javax.sql,org.apache.commons.pool.impl,javax.transaction,org.apache.commons.pool,javax.transaction.xa, version=1.4.0} package:{package=org.apache.commons.dbcp.datasources, uses:=javax.sql,org.apache.commons.pool,javax.naming,org.apache.commons.dbcp,javax.naming.spi,org.apache.commons.pool.impl, version=1.4.0} package:{package=org.apache.commons.jocl, uses:=org.xml.sax.helpers,org.xml.sax, version=1.4.0}

we can see that the URI is now http://cave_server:8181/cave/org.apache.servicemix.bundles.commons-dbcp-1.4.0.1.jar.

We can use directly obr:deploy command as previously:

karaf@root> obr:deploy org.apache.servicemix.bundles.commons-dbcp Target resource(s): ------------------- Apache ServiceMix :: Bundles :: commons-dbcp (1.4.0.1)


Required resource(s):

---------------------

   Apache ServiceMix :: Bundles :: commons-pool (1.5.4.3)

Deploying...done.

It’s completely transparent for the Karaf instance.

Cave REST Service

Cave Server also provides a REST service API, allowing you to handle the Cave Repositories (create, remove, destroy, etc).

By default, the REST Service is bound on http://localhost:8181/services/cave.

I will write a dedicated blog about that.

*NB*: Cave REST Service uses CXF JAX-RS implementation.

Administrate Cave Server

When you install Cave Server, it provides a set of MBeans. These MBeans allow you to monitore and administer the Cave Server.

It means that you can manipulate the Cave Server using a simple JMX client, like jconsole.

I wil write a dedicated blog about that.

Cave Client

The Cave Client is an implementation of the OBR RepositoryAdmin service which proxy all method calls to a remote Cave Server.

For the Karaf instance, the OBR service looks local, but in fact, each method call is forwarded/proxied to a remote Cave Server.

The Cave Client is still a work in progress and not yet available in this preview. However, you can already use a remote Cave Server using the Cave HTTP Wrapper Service as seen above.

TODO

I hope that this first Cave preview answer to some of your questions.

The current Cave TODO list is:
- finalize the REST service
- finalize the JMX MBeans
- finalize the Cave Client (OBR RepositoryAdmin proxy)
- add the Karaf Features Repository (KFR) support, both for the server and the client
- add a plug for the Karaf Web-Console

Feel free to post any comments, remarks, or questions, it’s exactly the purpose of this blog

Read full article |Leave a comment
Tags: karaf, obr
Categories: Apache, Apache Karaf

Use Camel, CXF and Karaf to implement batches

August 23, 2011 Posted by jbonofre

Introduction

Apache Camel has not be designed to be used for implementing batch tasks.

For instance, if your Camel route has a consumer endpoint polling files in a directory, Camel will periodically and indefinitely monitor the folder and poll any new incoming files.
It’s not a batch behavior: in batch mode, we want to run the file polling on demand, at a certain time, launched by a batch scheduler like ControlM, $Universe or Tivoli Worksheet Scheduler.

However, there are several interesting points to use Camel for batch implementation. First, Camel provides a large set of components. A lot of batches read/write files, read from a JMS queues, write into JMS queues, etc. Usage of Camel components in a batch way is really valuable.
Second, Camel uses a DSL to describe the process executed by the routes. Especially, it supports “human readable” DSL like Spring XML or Blueprint XML. It means that it’s easy to review what the batch is doing, eventually change an endpoint definition, etc. Most of the time, batches are “black box”: you run it, and you only get a status code to know if it’s OK or not. With Camel, you have a look on the batch process.
Third, Camel is a highly plug and play framework. It means that it’s easy to replace an endpoint by another one. For instance, if your batch polls files in a folder currently, it’s very easy to change this to poll messages from a JMS queue. You don’t really have to re-implement the whole batch.

More over, tools like Talend ESB Studio provide an IDE to create and design your Camel routes.

In this article, we are going to see how to use Camel in a “batch way”.

Design

In fact, we are going to have two Camel routes:
- the first one is called “control”. This route will “expose” a REST service to start the batch. A bean in this route will be responsible to start the “batch” route.
- the second one is called “batch”. It’s the core implementation of our batch. It’s a “standard” route, but at the end, we have a processor that “stop” the route (to avoid to have the route up indefinitely). This route is not auto started as it will be controller by the first one.

It means that a simple HTTP client (like a browser or REST client) will start the batch, on-demand. Most of enterprise batch schedulers ship a component to make HTTP requests.

POM

Our batch will be packaged as an OSGi bundle. It will allow us to deploy the batch in an Apache Karaf OSGi container:

<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">


  <modelVersion>4.0.0</modelVersion>
  <groupId>net.nanthrax.examples</groupId>

  <artifactId>camel-batch</artifactId>

  <version>1.0-SNAPSHOT</version>

  <packaging>bundle</packaging>
  <properties>

    <camel.version>2.8.0</camel.version>

    <cxf.version>2.4.1</cxf.version>

  </properties>
  <dependencies>

    <dependency>

      <groupId>org.apache.camel</groupId>

      <artifactId>camel-core</artifactId>

      <version>${camel.version}</version>

    </dependency>

    <dependency>

      <groupId>org.apache.camel</groupId>

      <artifactId>camel-spring</artifactId>

      <version>${camel.version}</version>

  &nbsp </dependency>

    <dependency>

      <groupId>org.apache.camel</groupId>

      <artifactId>camel-cxf</artifactId>

      <version>${camel.version}</version>

    </dependency>

    <dependency>

      <groupId>org.apache.cxf</groupId>

      <artifactId>cxf-rt-frontend-jaxrs</artifactId>

      <version>${cxf.version}</version>

    </dependency>

    &lt,dependency>

      <groupId>org.apache.cxf</groupId>

      <artifactId>cxf-rt-transports-http</artifactId>

      <version>${cxf.version}</version>

    </dependency>

    <dependency>

      <groupId>org.apache.cxf</groupId>

      <artifactId>cxf-rt-transports-http-jetty</artifactId>

      <version>${cxf.version}</version>

    </dependency>

  </dependencies>
  <build>

    <plugins>

      <plugin>

        <groupId>org.apache.felix</groupId>

        <artifactId>maven-bundle-plugin</artifactId>

        <version>2.3.4</version>

        <extensions>true</extensions>

        <configuration>

          <instructions>

            <Bundle-SymbolicName>${project.artifactId}</Bundle-SymbolicName>

            <Require-Bundle>org.apache.cxf.bundle,org.apache.camel.camel-cxf,org.springframework.beans</Require-Bundle>

          </instructions>

        </configuration>

      </plugin>

    </plugins>

  </build>

</project>

In this POM, we can see:
- the packaging is an OSGi bundle. That’s why we use the Apache Felix maven-bundle-plugin. We name the bundle with the project artifactId, and we define Camel and CXF bundles as dependencies (Require-Bundle).
- in the dependency sets, we define the Camel components that we use (camel-core, camel-spring to use the Camel Spring XML DSL, and camel-cxf to use the CXF JAX-RS implementation) and the CXF dependency to be able to create a JAX-RS server.

Control route

The first Camel route is the “control” one. This route will bind a JAX-RS server, listening HTTP requests (consumer) and will start the “batch” route on-demand.

The route definition will be located in the META-INF/spring/routes.xml folder of our bundle. We use the Camel Spring XML DSL in this file:

<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.org/schema/cxf" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd http://camel.apache.org/schema/cxf http://camel.apache.org/schema/cxf/camel-cxf.xsd ">


  <cxf:rsServer id="rsServer" address="http://localhost:9090/batch"

                  serviceClass="net.nanthrax.examples.camel.batch.impl.ControllerService"/&t;
  <camelContext xmlns="http://camel.apache.org/schema/spring">

    <route id="control">

      <from uri="cxfrs:bean:rsServer"/>

      <to uri="log:net.nanthrax.examples.camel.batch"/>

      <to uri="controllerBean"/>

    </route>

  </camelContext>
  <bean id="controllerBean" class="net.nanthrax.examples.camel.batch.impl.ControllerBean">

    <property name="routeId" value="batch"/>

  </bean>

</beans>

We use the Camel CXF to create the JAX-RS server (using <cxf:rsServer/gt;). This JAX-RS server will listen on the local machine on the 9090 port, and the context path is /batch.
To “describe” the REST service behavior, we define the ControllerService class in the serviceClass attribute.
The ControllerService class is just an “empty container”. The purpose is just to “describe” the REST service, not to process it:

package net.nanthrax.examples.camel.batch.impl;


import javax.ws.rs.GET;

import javax.ws.rs.Path;

import javax.ws.rs.Produces;
/**

 * REST service implementation of the Camel batch service.

 */

@Path("/")

public class ControllerService {
  @GET

  @Path("/start")

  @Produces("text/plain")

  public String startRoute() throws Exception {

    // nothing to do, it's just a wrapper

    return null;

  }

}

We can see the JAX-RS annotations:
- the ControllerService REST Path is /, it means directly bound to the JAX-RS server context path.
- the startRoute() method will accept GET HTTP method, on the context path /start and it will produce pure text (text/plain).

The process itself will be performed in the controllerBean:

package net.nanthrax.examples.camel.batch.impl;


import org.apache.camel.CamelContext;

import org.apache.camel.Handler;
/**

 * Camel controller bean involved in the starting routed

 */

public class ControllerBean {
  private String routeId;
  public String getRouteId() {

    return this.routeId;

  }
  public void setRouteId(String routeId) {

    this.routeId = routeId;

  }
  @Handler

  public String startRoute(CamelContext camelContext) throws Exception {

    camelContext.startRoute(routeId);

    return "Batch " + routeId + " started.";

  }

}

We inject the Camel route ID of the batch route: “batch”. The CamelContext is automatically injected by Camel. This bean is quite simple, as it only starts the “batch” route.

Batch route

This route contains the “batch logic”. You can use any kind of routes, components, Enterprise Integration Patterns, etc provided by Camel. The only specific parts are:
- the autoStartup attribute set to false, to avoid to start the route automatically at context bootstrap
- the final processor which stop the route after processing.

We gather the two routes in the same META-INF/spring/routes.xml file:


  <cxf:rsServer id="rsServer" address="http://localhost:9090/batch"

  serviceClass="net.nanthrax.examples.camel.batch.impl.ControllerService"/>
  <camelContext xmlns="http://camel.apache.org/schema/spring">

    <route id="control">

      <from uri="cxfrs:bean:rsServer"/>

      <to uri="log:net.nanthrax.examples.camel.batch"/>

      <to uri="controllerBean"/>

    </route>

    <route id="batch" autoStartup="false">

      <from uri="file:/tmp"/>

      <to uri="file:output"/>

      <process ref="stopProcessor"/>

    </route>

  </camelContext>
  <bean id="controllerBean" class="net.nanthrax.examples.camel.batch.impl.ControllerBean">

    <property name="routeId" value="batch"/>

  </bean>
  <bean id="stopProcessor" class="net.nanthrax.examples.camel.batch.impl.StopProcessor">

    <property name="routeId" value="batch"/>

  </bean>

</beans>

In this example, the batch polls files in the /tmp folder, and copies into the output folder.

The StopProcessor is Camel processor (aka, it implements the Camel Processor interface). It stops the route after processing the incoming message (we inject the “batch” route ID using Spring):

package net.nanthrax.examples.camel.batch.impl;


import org.apache.camel.CamelContext;

import org.apache.camel.Exchange;

import org.apache.camel.Processor;
/**

 * A Camel processor which stop routes.

 */

public class StopProcessor implements Processor {
  private String routeId;
  public String getRouteId() {

    return this.routeId;

  }
  public void setRouteId(String routeId) {

    this.routeId = routeId;

  }
  public void process(Exchange exchange) throws Exception {

    CamelContext camelContext = exchange.getContext();

    // remove myself from the in flight registry so we can stop this route without trouble

    camelContext.getInflightRepository().remove(exchange);

    // stop the route

    camelContext.stopRoute(routeId);

  }

}

Deployment and execution

Now, we can build our OSGi bundle, simply using:

mvn clean install

In a fresh Apache Karaf instance, we have first to install the CXF and Camel features:

karaf@root> features:addurl mvn:org.apache.cxf.karaf/apache-cxf/2.4.1/xml/features karaf@root> features:install cxf karaf@root> features:addurl mvn:org.apache.camel.karaf/apache-camel/2.8.0/xml/features karaf@root> features:install camel-spring karaf@root> features:install camel-cxf

Now, we can install our bundle:

karaf@root> osgi:install -s mvn:net.nanthrax.examples/camel-batch/1.0-SNAPSHOT

Our bundle appears as “created”:

karaf@root> la|grep -i batch [ 134] [Active ] [ ] [Started] [ 60] camel-batch (1.0.0.SNAPSHOT)

Using a simple browser, we can access to http://localhost:9090/batch/start. The route is started (as a batch) and we can see in the browser:

Batch batch started.

Conclusion

Even if the first Camel route purpose is to be up and running all the time, we can use it in a more “batch” way. It allows developers to use the large set of Camel components, and be able to use all Enterprise Integration Patterns. For instance, the batch needs to copy a file, and after send an e-mail and a message into a JMS queue, it’s very easy using a recipient list. You have to send to a target endpoint depending of the content of the message, no problem using a Content Based Router.

You can run such kind of batches in Talend ESB. It’s an interesting addition to the Talend Data Integration products (ETL jobs, MDM, DQ, etc).

Read full article |Leave a comment
Tags: apache, camel, karaf
Categories: Apache, Apache Camel, Apache CXF, Apache Karaf

Apache Karaf Cellar 2.2.2 release

August 8, 2011 Posted by jbonofre

What’s new

Quite one month ago, we released Karaf Cellar 2.2.1, the first “official” release of the Karaf clustering sub-project.

This new Karaf Cellar 2.2.2 release includes bug fixes, especially one bug was a blocker as it was not possible to install Cellar on a Karaf instance running on Equinox OSGi framework.

But, it’s not just a bug fix release, we merge two features from the Cellar trunk.

Bundle synchronization

In Karaf Cellar 2.2.1, we were able to synchronize features (including features repositories) and configuration between Karaf Cellar instances. It means that you can install a feature on one node (cluster:features-install group feature), the feature will be install on each Karaf note.

Karaf Cellar 2.2.2 includes the same behavior for pure OSGi bundle. You can install a bundle on one node, the bundle will be installed on each other nodes on the same cluster group.

karaf@root> osgi:install mybundle

mybundle will be installed on all nodes in the same cluster group.

It’s a first step, as we have for features and config, we will add specific command to manipulate bundle, something like:

karaf@root> cluster:install-bundle group mybundle

Cloud support

Cellar relies on Hazelcast in order to discover cluster nodes. This can happen either by using multicast or by unicast (specifying the ip address of each node).
Unfortunately multicast is not allowed in most IaaS providers and specifying the all the ip addresses is not very flexible, since in most cases they are not known in advance.

Cellar solves this problem using a cloud discovery service powered by jclouds.

Cloud discovery service

Most cloud providers among other provide cloud storage. Cellar uses the cloud storage via jclouds, in order to put there the ip addresses of each node so that Hazelcast can found them.
This approach is also called blackboard and in other words is the process where each nodes registers itself in a common storage, so that other nodes know its existence.

Installing Cellar cloud discovery service

To install the cloud discovery service simply the appropriate jclouds provider and then install cellar-cloud feature. For the rest of this manual I will use amazon s3 as an example, but it applies to any provider supported by jclouds.

karaf@root> features:install jclouds-aws-s3 karaf@root> features:install cellar-cloud

Once the feature is installed, it requires you to create a configuration that contains credentials and type of the cloud storage (aka blobstore).
To do that add a configuration file under etc with the name org.apache.karaf.cellar.cloud- .cfg and put there the following information:

provider=aws-s3 (this varries according to the blobstore provider)
identity=”the identity of the blobstore account”
credential=”the credential/password of the blobstore account)”
container=”the name of the bucket”
validity=”the amount of time an entry is considered valid, after that time the entry is removed”

After creating the file the service will check for new nodes. If new nodes are found the Hazelcast instance configuration is updated and the instance is restarted.

Read full article |Leave a comment
Tags: cellar, karaf
Categories: Apache, Apache Karaf

February 2012
M	T	W	T	F	S	S
« Jan
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29