ECF remote OSGi DS Declarative Services example

In this post we will give a fair example of the workings of the OSGi DS Declarative Services remoted with ECF.

For this example we will be using the Eclipse IDE for Java with the ECF Remote Services Target Components  plug-in, the Equinox framework and ZooKeeper as a service discovery provider.

Suppose we have a service: HelloService, its interface: IHello and a client: HelloConsumer who declares a dependency on the service by pointing to its interface using the DS method. He would also like to be able to remotely access it.

When a component declares it provides a service, the OSGi framework stores that information in its internal registry.

Download here the example's source code. You should be able to import everything inside Eclipse as-is. Inside you will find:

• Interface - IHello.java:

package it.eng.test.remote.ds.hello;
public interface IHello {
    public String greet(String to);
}

• Implementation - HelloService.java:

package it.eng.test.remote.ds.helloservice;
import it.eng.test.remote.ds.hello.IHello;
public class HelloService implements IHello{
    public String greet(String to){
        return "Greetings "+to;
    } 
}

As we can see, the service offers a single, simple method which returns a String.

• Client - HelloConsumer.java:

package it.eng.test.remote.ds.helloconsumer;
import it.eng.test.remote.ds.hello.IHello;
public class HelloConsumer { 
    public void bindHello(IHello proxy) {
        System.out.println("Got proxy IHello="+proxy);     
        System.out.println(proxy.greet(proxy.toString()));
    } 
}

The consumer declares the dependency on the service by requiring its interfaces and has a single method, bindHello, which prints something on standard output as soon as the required remote service becomes available to the consumer.

The framework knows that HelloConsumer needs that particular services since he declared it in his XML configuration which must be put under the OSGI-INF/ folder:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" name="it.eng.test.remote.ds.helloconsumer">
   <implementation class="it.eng.test.remote.ds.helloconsumer.HelloConsumer"/>
   <reference bind="bindHello" cardinality="0..n" interface="it.eng.test.remote.ds.hello.IHello" name="IHello" policy="dynamic"/>
</scr:component>

Here the bundle is saying that the class it.eng.test.remote.ds.helloconsumer.HelloConsumer dynamically and optionally depends on one of the possible it.eng.test.remote.ds.hello.IHello interface implementations and that when one of them becomes available inside the framework, its bindHello method should be called.

The dynamically policy attribute means that the bundle is able to work properly even when the services are dynamically switched; the alternative would be static, which would require the DS to deactivate the component and create a new instance each time the required service changes.

The dependency is optional since the cardinality was set to 0..N, meaning the component can be started even if that particular dependency is not currently available and it requires the DS to invoke the associated method multiple times, one for each service instance currently available in the registry. 1..1 would mean instead that the dependency is mandatory. Another alternative would be 0..1.

Furthermore, the name of the bind* method can be changed as pleased as long as it reflects the one written in the java code.

On its side, the service must declare its own XML configuration, listing the interface implementations offered, the classes implementing them, and a series of parameters used to make the service remotable.

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" name="it.eng.test.remote.ds.helloservice">
   <implementation class="it.eng.test.remote.ds.helloservice.HelloService"/>
   <property name="service.exported.interfaces" type="String" value="*"/>
   <property name="service.exported.configs" type="String" value="ecf.generic.server"/>
   <property name="ecf.exported.containerfactoryargs" type="String" value="ecftcp://192.168.23.28:6666/hello"/>
   <service>
      <provide interface="it.eng.test.remote.ds.hello.IHello"/>
   </service>
</scr:component>

This line indicates that all methods described in the service interface are to be made remotable:

   <property name="service.exported.interfaces" type="String" value="*"/>

This value is linked to the OSGi framework implementation we used (Equinox + ECF) and indicates that the service should be exposed as a Web Service:

   <property name="service.exported.configs" type="String" value="ecf.generic.server"/>

located here:

   <property name="ecf.exported.containerfactoryargs" type="String" value="ecftcp://192.168.23.28:6666/hello"/>

If we do not set the previous property, the service will be published by ZooKeeper as remotable anyway but it will use a random port, the machine hostname instead of its IP and a auto-generated folder name. Currently there is no way to have it use the IP in any case.

As you can see by looking at the source code, nowhere is written where to publish or locate a service (assume we did not set the last of the above properties). However, since magic happens only in movies, there must be some place where to define those properties, else nothing will work.

In our setup, using ZooKeeper allows us to configure the service discovery protocol in multiple ways:

• standalone: each framework's ZooKeeper instance must know the exact IP of the others in order to communicate with it. If a new client appears, there is no way to make it join the network without having to reconfigure and restart every instance;


• centralized: a central ZooKeeper instance acts as a server with which other instances register. When a new client appears it must know the central ZooKeeper's IP in order to koin the network. All services are shared through the one and only central instance, if it goes down, the entire network crashes;


• replicated: a group of servers is configured to communicate, perform leader elections and manage the service discovery routine. The group servers IPs must still be known by every peer, but this allows for more flexibility.

we chose the standalone one which means we configure the Java VM at launch time with the needed information.

Before being able to export our applications from Eclipse or run them within it, we shall follow additional steps to effectively create Eclipse Applications from our code.

Lars Vogel wrote a good tutorial about Eclipse RCP Applications. We can follow it, but keep in mind we are not exactly going to deploy an Eclipse application, instead we will be using those mechanics to successfully export our OSGi bundles along with all necessary dependencies and configuration.

We must thus add a class implementing the IApplication interface in both service and client along an XML configuration file which shall go under the project's root folder.

Service - HelloApp.java:

package it.eng.test.remote.ds.helloservice;
import org.eclipse.equinox.app.IApplication;
import org.eclipse.equinox.app.IApplicationContext;
public class HelloApp implements IApplication {
    boolean done = false;
    Object appLock = new Object();
    public Object start(IApplicationContext context) throws Exception {
        // We just wait...everything is done by DS and HelloComponent
        synchronized (appLock) {
            while (!done) {
                try {
                    appLock.wait();
                } catch (InterruptedException e) {
                    // do nothing
                }
            }
        }
        return IApplication.EXIT_OK;
    }
    public void stop() {
        synchronized (appLock) {
            done = true;
            appLock.notifyAll();
        }
    }
}

with this XML configuration:

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.7"?>
<plugin>
   <extension
         id="HelloServiceTest"
         point="org.eclipse.core.runtime.applications">      <application
            cardinality="*"
            thread="any"
            visible="true">
         <run               class="it.eng.test.remote.ds.helloservice.HelloApp">
         </run>
      </application>
   </extension>
</plugin>

Client - HelloCApp.java:

package it.eng.test.remote.ds.helloconsumer;
import org.eclipse.equinox.app.IApplication;
import org.eclipse.equinox.app.IApplicationContext;
public class HelloCApp implements IApplication {
    boolean done = false;
    Object appLock = new Object();
    public Object start(IApplicationContext context) throws Exception {
        // We just wait...everything is done by DS and HelloComponent
        synchronized (appLock) {
            while (!done) {
                try {
                    appLock.wait();
                } catch (InterruptedException e) {
                    // do nothing
                }
            }
        }
        return IApplication.EXIT_OK;
    }
    public void stop() {
        synchronized (appLock) {
            done = true;
            appLock.notifyAll();
        }
    }
}

with this XML configuration :

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.4"?>
<plugin>
   <extension
         id="HelloConsumerTest"
         point="org.eclipse.core.runtime.applications">
      <application
            cardinality="*"
            thread="any"
            visible="true">
         <run               class="it.eng.test.remote.ds.helloconsumer.HelloCApp">
         </run>
      </application>
   </extension>
</plugin>

We have hence defined for both service and client two methods: start and stop which, guess what, are called when the application is started or stopped.

In the plugin.xml configuration we say that our applications shall be visible for the user; for example, some applications may provide features to other applications but nothing directly to the user, in this case the application should not be revealed to the user to start it individually. They may run in any thread instead of only the main one and that an unlimited number of instances can be active at the same time.

Now, before successfully launch them, we must configure ZooKeeper to properly do its job. We can do this by changing the application launch parameters to include all necessary dependencies (including org.apache.hadoop.zookeeper and org.eclipse.ecf.provider.zookeeper) and the following Java VM arguments inside the Run configuration:

-Dzoodiscovery.dataDir=bla -Dzoodiscovery.flavor=zoodiscovery.flavor.standalone=192.168.23.28:3030;clientPort=3031

for the service, and:

-Dzoodiscovery.autoStart=true; -Dzoodiscovery.flavor=zoodiscovery.flavor.standalone=192.168.23.28:3031;clientPort=3030

for the client.

With that we are saying that the service's ZooKeeper component should use the bla/ directory to store its temporary files and publish the service in a standalone configuration at the specified IP:PORT using clientPort for incoming connections; the client's ZooKeeper component on its side should instead automatically scan the network for other ZooKeeper instances at startup time connecting to the one at the specified address in order to share services between the two involved frameworks (in our case they are on the same host but cannot be inside the same framework - how would configure ZooKeeper then?).

Eclipse allows us to create from the previously generated Run configurations the Product configurations for our applications and store them in particular XML files called product files, usually stored under the product/ folder. These files also include the OSGi framework's required plug-ins in order for every component to work:

Service - helloservicezookeeper.product

<?xml version="1.0" encoding="UTF-8"?>
<?pde version="3.5"?>
<product application="it.eng.test.remote.ds.helloservice.HelloServiceTest" useFeatures="false" includeLaunchers="true">
   <configIni use="default">
   </configIni>
   <launcherArgs>
      <programArgs>-consoleLog -console</programArgs>
      <vmArgs>-Declipse.ignoreApp=true -Dosgi.noShutdown=true -Dzoodiscovery.dataDir=bla
-Dzoodiscovery.flavor=zoodiscovery.flavor.standalone=192.168.23.28:3030;clientPort=3031</vmArgs>
      <vmArgsMac>-XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts</vmArgsMac>
   </launcherArgs>
   <launcher>
      <solaris/>
      <win useIco="false">
         <bmp/>
      </win>
   </launcher>
   <vm>
   </vm>
   <plugins>
      <plugin id="it.eng.test.remote.ds.hello"/>
      <plugin id="it.eng.test.remote.ds.helloservice"/>
      <plugin id="javax.transaction" fragment="true"/>
      <plugin id="org.apache.hadoop.zookeeper"/>
      <plugin id="org.apache.log4j"/>
      <plugin id="org.eclipse.core.contenttype"/>
      <plugin id="org.eclipse.core.jobs"/>
      <plugin id="org.eclipse.core.runtime"/>
      <plugin id="org.eclipse.core.runtime.compatibility.registry" fragment="true"/>
      <plugin id="org.eclipse.ecf"/>
      <plugin id="org.eclipse.ecf.discovery"/>
      <plugin id="org.eclipse.ecf.identity"/>
      <plugin id="org.eclipse.ecf.osgi.services.distribution"/>
      <plugin id="org.eclipse.ecf.osgi.services.remoteserviceadmin"/>
      <plugin id="org.eclipse.ecf.osgi.services.remoteserviceadmin.proxy"/>
      <plugin id="org.eclipse.ecf.provider"/>
      <plugin id="org.eclipse.ecf.provider.remoteservice"/>
      <plugin id="org.eclipse.ecf.provider.zookeeper"/>
      <plugin id="org.eclipse.ecf.remoteservice"/>
      <plugin id="org.eclipse.ecf.sharedobject"/>
      <plugin id="org.eclipse.ecf.ssl" fragment="true"/>
      <plugin id="org.eclipse.equinox.app"/>
      <plugin id="org.eclipse.equinox.common"/>
      <plugin id="org.eclipse.equinox.concurrent"/>
      <plugin id="org.eclipse.equinox.ds"/>
      <plugin id="org.eclipse.equinox.preferences"/>
      <plugin id="org.eclipse.equinox.registry"/>
      <plugin id="org.eclipse.equinox.util"/>
      <plugin id="org.eclipse.equinox.weaving.hook" fragment="true"/>
      <plugin id="org.eclipse.osgi"/>
      <plugin id="org.eclipse.osgi.services"/>
      <plugin id="org.eclipse.osgi.services.remoteserviceadmin"/>
      <plugin id="org.eclipse.persistence.jpa.equinox.weaving" fragment="true"/>
   </plugins>
</product>

Client - helloconsumerzookeeper.product

<?xml version="1.0" encoding="UTF-8"?>
<?pde version="3.5"?>
<product application="it.eng.test.remote.ds.helloconsumer.HelloConsumerTest" useFeatures="false" includeLaunchers="true">
   <configIni use="default">
   </configIni>
   <launcherArgs>
      <programArgs>-consoleLog -console -clean</programArgs>
      <vmArgs>-Declipse.ignoreApp=true -Dosgi.noShutdown=true -Dzoodiscovery.autoStart=true; -Dzoodiscovery.flavor=zoodiscovery.flavor.standalone=192.168.23.28:3031;clientPort=3030</vmArgs>
      <vmArgsMac>-XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts</vmArgsMac>
   </launcherArgs>
   <launcher>
      <solaris/>
      <win useIco="false">
         <bmp/>
      </win>
   </launcher>
   <vm>
   </vm>
   <plugins>
      <plugin id="it.eng.test.remote.ds.hello"/>
      <plugin id="it.eng.test.remote.ds.helloconsumer"/>
      <plugin id="javax.transaction" fragment="true"/>
      <plugin id="org.apache.hadoop.zookeeper"/>
      <plugin id="org.apache.log4j"/>
      <plugin id="org.eclipse.core.contenttype"/>
      <plugin id="org.eclipse.core.jobs"/>
      <plugin id="org.eclipse.core.runtime"/>
      <plugin id="org.eclipse.core.runtime.compatibility.registry" fragment="true"/>
      <plugin id="org.eclipse.ecf"/>
      <plugin id="org.eclipse.ecf.discovery"/>
      <plugin id="org.eclipse.ecf.identity"/>
      <plugin id="org.eclipse.ecf.osgi.services.distribution"/>
      <plugin id="org.eclipse.ecf.osgi.services.remoteserviceadmin"/>
      <plugin id="org.eclipse.ecf.osgi.services.remoteserviceadmin.proxy"/>
      <plugin id="org.eclipse.ecf.provider"/>
      <plugin id="org.eclipse.ecf.provider.remoteservice"/>
      <plugin id="org.eclipse.ecf.provider.zookeeper"/>
      <plugin id="org.eclipse.ecf.remoteservice"/>
      <plugin id="org.eclipse.ecf.sharedobject"/>
      <plugin id="org.eclipse.ecf.ssl" fragment="true"/>
      <plugin id="org.eclipse.equinox.app"/>
      <plugin id="org.eclipse.equinox.common"/>
      <plugin id="org.eclipse.equinox.concurrent"/>
      <plugin id="org.eclipse.equinox.ds"/>
      <plugin id="org.eclipse.equinox.preferences"/>
      <plugin id="org.eclipse.equinox.registry"/>
      <plugin id="org.eclipse.equinox.util"/>
      <plugin id="org.eclipse.equinox.weaving.hook" fragment="true"/>
      <plugin id="org.eclipse.osgi"/>
      <plugin id="org.eclipse.osgi.services"/>
     <plugin id="org.eclipse.osgi.services.remoteserviceadmin"/>
      <plugin id="org.eclipse.persistence.jpa.equinox.weaving" fragment="true"/>
   </plugins>
</product>

After exporting the applications as Eclipse products we are left with two folders with an executable file in them. Do not even bother to try to run it as id will not work, instead open a terminal and browse to the exported_application_folder/plugins folder; here you will find all the required OSGi plug-ins for your application. You can run the service by typing:

java -Dzoodiscovery.dataDir=bla -Dzoodiscovery.flavor=zoodiscovery.flavor.standalone=192.168.23.28:3030;clientPort=3031 -jar org.eclipse.osgi.jar -configuration ../configuration -console

and the client with:

java -Dzoodiscovery.autoStart=true; -Dzoodiscovery.flavor=zoodiscovery.flavor.standalone=192.168.23.28:3031;clientPort=3030 -jar org.eclipse.osgi.jar -configuration ../configuration -console

Now if you start your bundles inside their respective frameworks, you should see, along ZooKeeper's infos, "Got proxy IHello=OSGi_PROXY_NAME" and "Greetings PROXY_NAME". You can download the already exported and ready-to-go service and client applications here.