Camel development series part 5

Hello, and welcome back to the Camel development series and to part 5. Its been a couple of weeks since I wrote and this has been mainly due to lack of time but hopefully I can get back to track.

In this series we are going to look at a classic case of content based routing. This means, based on some criteria the input data should be transformed/routed in one way and on another criteria transformed/routed in another way.

We will build up on our previous work but this will be in a new project. You can find all the source code here https://github.com/SoucianceEqdamRashti/Integration/tree/master/CamelContentbasedRouter .

Problem

Let’s say we have a file placed in some folder. Based on the file name we want to either map it to XML or to JSON. The input format is basic CSV.

  1. If the file name contains the string ”xml” then the data should be mapped to XML.
  2. If the file name contains the string ”json” then the data should be mapped to JSON.

Our end result should produce an XML or a JSON based on the file name.

Solution

We will introduce a few new concepts there. One of the most powerful ideas in the Camel world is the use of Predicates. You can read more about it here http://camel.apache.org/predicate.html . Predicates allow you to define complicated conditions upon which to based your logic and choices. We will use them here to make choices. We will also look at how to send and use bean transformation to work together with JAXB. Finally we will also look at how to read the file name and how to actually map to XML from CSV.

Main RouteBuilder class

Let us first look at the main RouteBuilder class to get an idea how the main flow looks like.

package org.souciance.integration.contentbasedrouter;

import org.apache.camel.LoggingLevel;
import org.apache.camel.Predicate;
import org.apache.camel.PropertyInject;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.converter.jaxb.JaxbDataFormat;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;
import java.lang.String;

public class MainRouteBuilder extends RouteBuilder {
@PropertyInject("{{input.folder}}")
private String inputFolder;
@PropertyInject("{{output.folder}}")
private String outputFolder;
@PropertyInject("{{input.filename}}")
private String inputFileName;
@PropertyInject("{{output.filename}}")
private String outputFileName;

@Override
public void configure() throws Exception {

Predicate toXML = header("CamelFileName").contains(String.valueOf("xml"));
Predicate toJSON = header("CamelFileName").contains(String.valueOf("json"));

BindyCsvDataFormat bindy = new BindyCsvDataFormat(Person.class);
JaxbDataFormat jaxbDataFormat = new JaxbDataFormat(Persons.class.getPackage().getName());
jaxbDataFormat.setPrettyPrint(true);
from("file://" + inputFolder+"?include=.*.csv").routeId("ContentbasedRouter")
.log(LoggingLevel.INFO, "body","body: ${body}")
.unmarshal(bindy)
.choice()
.when(toXML)
.log(LoggingLevel.INFO, "Data will be mapped to xml structure")
.bean(Transform.class)
.marshal(jaxbDataFormat)
.log(LoggingLevel.INFO, "Data will be mapped to xml structure: ${body}")
.to("file://"+outputFolder+"/?fileName=output.xml")
.when(toJSON)
.log(LoggingLevel.INFO, "Data will be mapped to json structure.")
.unmarshal(bindy)
.to("PersonMapperToJson")
.log(LoggingLevel.INFO, "PersonMapperToJson", "Mapped Pojo to json: ${body}")
.to("file://"+outputFolder+"/?fileName="+"output.json")
.endChoice()
.end();

}
}

The main idea should be readable even if you don’t know all the details. We are polling a directory. If a file with extension .csv is placed in the directory we look at the file name. Here is where predicates come in handy. Let’s look at it in more detail:

Predicate toXML = header("CamelFileName").contains(String.valueOf("xml"));
Predicate toJSON = header("CamelFileName").contains(String.valueOf("json"));

We have defined two predicates, and each one of them looks at the header named CamelFileName. This is a standard Camel header where the name of the file is placed. In the rule we check if the value xml or json is found. These are examples of simple predicates, you can build much more complicated rules and also combine them with property files to make dynamic rules.

Next let’s look at the new idea of DataFormat and especially the jaxb data format.

JaxbDataFormat jaxbDataFormat = new JaxbDataFormat(Persons.class.getPackage().getName());
jaxbDataFormat.setPrettyPrint(true);

DataFormats are what allows Camel to recognize formats and transform or otherwise known as marshal/unmarshal data. We need two types of data formats in our case. One is the one we have introduced earlier which is the Bindy format to parse CSV into POJO. The second seen above is the JAXB dataformat to transform the list of POJOs to XML. For more on data formats read here https://camel.apache.org/data-format.html .

The second new part and where all the interesting logic happens is here:

.choice()
.when(toXML)
.log(LoggingLevel.INFO, "Data will be mapped to xml structure")
.bean(Transform.class)
.marshal(jaxbDataFormat)
.log(LoggingLevel.INFO, "Data will be mapped to xml structure: ${body}")
.to("file://"+outputFolder+"/?fileName=output.xml")

You can see that we use the .choice() tag to start our content based routing. We use the .when(toXML) to use the predicate defined earlier. Essentially this means in plain English, when the predicate toXML holds true enter this block. After the log statement you notice the .bean() tag and this is an interesting tag. It allows us to send the exchange to the bean and do whatever we want with it and then continue. In our cause we are going to do some JAXB logic in the transform bean.
Then finally we have the marshal statement which refers to the jaxb data format. The last two are simple log and writing to file statements.

Let’s look at our Transform class

Transform Bean

Our transform bean looks as follows:

package org.souciance.integration.contentbasedrouter;

import java.util.List;
import org.apache.camel.Exchange;
import org.apache.camel.Message;

public class Transform {

public void process(Exchange exchange) throws Exception {
Message msg = exchange.getIn();

@SuppressWarnings("unchecked")
List personList = (List) msg.getBody();

ObjectFactory objectFactory = new ObjectFactory();

Persons persons = objectFactory.createPersons();

for (Person aPerson : personList) {
persons.setPerson(aPerson);

exchange.getIn().setBody(persons);
}

}

}

The code is essentially receiving a list of Person POJOs which were created when marshalling to the bindy format. Then an instance of ObjectFactory is created and we are adding each POJO from the list to the ArrayList in Persons class. Finally we set the exchange to contain the Persons list.

The remaining parts are the annotated classes and the ObjectFactory class. So let’s look at them.

ObjectFactory

The object factory class and the annotated classes were inspired by the post found here http://www.developpez.net/forums/d1216465/java/serveurs-conteneurs-java-ee/autres/camel-convertir-csv-xml-bindy-jaxb/ . Although it is in French, the gist of the idea should be comprehensible.

Here is our class:

//
// This file was generated by the JavaTM Architecture for XML Binding(JAXB) Reference Implementation, v2.2.8-b130911.1802
// See <a href="http://java.sun.com/xml/jaxb">http://java.sun.com/xml/jaxb</a>
// Any modifications to this file will be lost upon recompilation of the source schema.
// Generated on: 2016.03.05 at 06:27:39 PM CET
//

package org.souciance.integration.contentbasedrouter;

import javax.xml.bind.annotation.XmlRegistry;

/**
* This object contains factory methods for each
* Java content interface and Java element interface
* generated in the org.souciance.integration.contentbasedrouter.jaxb package.
*

An ObjectFactory allows you to programatically
* construct new instances of the Java representation
* for XML content. The Java representation of XML
* content can consist of schema derived interfaces
* and classes representing the binding of schema
* type definitions, element declarations and model
* groups. Factory methods for each of these are
* provided in this class.
*
*/
@XmlRegistry
public class ObjectFactory {

/**
* Create a new ObjectFactory that can be used to create new instances of schema derived classes for package: org.souciance.integration.contentbasedrouter.jaxb
*
*/
public ObjectFactory() {
}

/**
* Create an instance of {@link Persons }
*
*/
public Persons createPersons() {
return new Persons();
}

/**
* Create an instance of {@link Persons.Person }
*
*/
public Person createPerson() {
return new Person();
}

}

Annotated classes

We need annotated classes for bindy and jaxb processing. For completeness sake here they are.

Persons class

//
// This file was generated by the JavaTM Architecture for XML Binding(JAXB) Reference Implementation, v2.2.8-b130911.1802
// See <a href="http://java.sun.com/xml/jaxb">http://java.sun.com/xml/jaxb</a>
// Any modifications to this file will be lost upon recompilation of the source schema.
// Generated on: 2016.03.05 at 06:27:39 PM CET
//

package org.souciance.integration.contentbasedrouter;

import java.util.ArrayList;
import java.util.List;
import javax.xml.bind.annotation.XmlAccessType;
import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import javax.xml.bind.annotation.XmlType;

/**
*

Java class for anonymous complex type.
*
*

The following schema fragment specifies the expected content contained within this class.
*
*
<pre> * &lt;complexType&gt;
 *   &lt;complexContent&gt;
 *     &lt;restriction base="{http://www.w3.org/2001/XMLSchema}anyType"&gt;
 *       &lt;sequence&gt;
 *         &lt;element name="person" maxOccurs="unbounded" minOccurs="0"&gt;
 *           &lt;complexType&gt;
 *             &lt;complexContent&gt;
 *               &lt;restriction base="{http://www.w3.org/2001/XMLSchema}anyType"&gt;
 *                 &lt;sequence&gt;
 *                   &lt;element name="firstname" type="{http://www.w3.org/2001/XMLSchema}string"/&gt;
 *                   &lt;element name="lastname" type="{http://www.w3.org/2001/XMLSchema}string"/&gt;
 *                   &lt;element name="identity" type="{http://www.w3.org/2001/XMLSchema}string"/&gt;
 *                   &lt;element name="country" type="{http://www.w3.org/2001/XMLSchema}string"/&gt;
 *                   &lt;element name="phone" type="{http://www.w3.org/2001/XMLSchema}string"/&gt;
 *                 &lt;/sequence&gt;
 *               &lt;/restriction&gt;
 *             &lt;/complexContent&gt;
 *           &lt;/complexType&gt;
 *         &lt;/element&gt;
 *       &lt;/sequence&gt;
 *     &lt;/restriction&gt;
 *   &lt;/complexContent&gt;
 * &lt;/complexType&gt;
 *</pre>
*
*
*/
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name = "", propOrder = {
"person"
})
@XmlRootElement(name = "persons")
public class Persons {
@XmlElement(required = false, name = "person")
protected List person;

public List getPerson() {
if (person == null) {
person = new ArrayList();
}
return this.person;
}
public void setPerson(Person person){
this.getPerson().add(person);
}

}

Person class

package org.souciance.integration.contentbasedrouter;

import javax.xml.bind.annotation.XmlAccessorType;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlType;
import javax.xml.bind.annotation.XmlAccessType;
import org.apache.camel.dataformat.bindy.annotation.CsvRecord;
import org.apache.camel.dataformat.bindy.annotation.DataField;

@CsvRecord(separator = ",")

@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(name="person")
public class Person {

@XmlElement(name = "identity")
@DataField(pos=1)
private int identity;

@XmlElement(name = "firstname")
@DataField(pos=2)
private String firstname;

@XmlElement(name = "lastname")
@DataField(pos=3)
private String lastname;

@XmlElement(name = "phonenumber")
@DataField(pos=4)
private int phone;

@XmlElement(name = "country")
@DataField(pos=5)
private String country;

public int getIdentity() {
return identity;
}
public void setIdentity(int identity) {
this.identity = identity;
}
public String getFirstname() {
return firstname;
}
public void setFirstname(String firstname) {
this.firstname = firstname;
}
public String getLastname() {
return lastname;
}
public void setLastname(String lastname) {
this.lastname = lastname;
}
public int getPhone() {
return phone;
}
public void setPhone(int phone) {
this.phone = phone;
}
public String getCountry() {
return country;
}
public void setCountry(String country) {
this.country = country;
}

}

Summary

In this project we have looked at a common problem, how to transform and route data in different ways based on some condition. We defined predicates to define our condition rules, we used the choice and when tags to check our predicates, we use bean transformation to connect our jaxb processing and then we marshalled  the list of objects to XML using jaxb marshalling.

The input data looked like this:
12345,souciance,rashti,012458478,Sweden
12345,souciance,rashti,012458478,Sweden

The filename was csvtoxml.csv

The final output is:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<persons>
    <person>
        <identity>12345</identity>
        <firstname>souciance</firstname>
        <lastname>rashti</lastname>
        <phonenumber>12458478</phonenumber>
        <country>Sweden</country>
    </person>
    <person>
        <identity>12345</identity>
        <firstname>souciance</firstname>
        <lastname>rashti</lastname>
        <phonenumber>12458478</phonenumber>
        <country>Sweden</country>
    </person>
</persons>

It works similarly for the second when block for the JSON which is exactly as the previous series.

Annonser