Camel development series – Part 3

Hello again and welcome to the third part of the Camel development series.

In part 2 we touched upon a very basic mapping scenario, namely mapping a single row of CSV data to a simple json structure. It touched upon some basic Camel ideas such as combing blueprint and the java dsl, injecting beans, using processors and the jackson json API.

In this part we will create a new project and handle multiple rows of CSV data and map it to a json array structure. You can find all the source code and test data from here https://github.com/SoucianceEqdamRashti/Integration

Input and Output

12345,souciance,rashti,012458478,Sweden
12346,souciance,eqdam rashti,012458478,Sweden
12347,souciance,eqdam,012458478,Sweden
12348,Moeed,eqdam rashti,012458478,Sweden

This above is our input. As you can see these are simple fields and each row is delimited by a control line feed and each field is delimited by a comma. Our desired output is:

{
	"ListOfRows": [{
		"firstname": "souciance",
		"lastname": "rashti",
		"phone": 12458478,
		"country": "Sweden"
	},
	{
		"firstname": "souciance",
		"lastname": "eqdam rashti",
		"phone": 12458478,
		"country": "Sweden"
	},
	{
		"firstname": "souciance",
		"lastname": "eqdam",
		"phone": 12458478,
		"country": "Sweden"
	},
	{
		"firstname": "Moeed",
		"lastname": "eqdam rashti",
		"phone": 12458478,
		"country": "Sweden"
	}]
}

Our output contains a JSON array called ListOfRows which contains the data from every row in the CSV file. Let’s go through the code to see how we can acheive this.

Blueprint code

The blueprint.xml file is a copy of the one found in part 2. It looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"        xsi:schemaLocation="        http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd        http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">

  <bean id="CsvToJsonRouteBuilder" class="org.souciance.integration.csvstojson.CsvToJsonRouteBuilder">
  </bean>
  <bean id="IdentityToJson" class="org.souciance.integration.csvstojson.IdentityToJson">
  </bean>

  <camelContext xmlns="http://camel.apache.org/schema/blueprint">
    <routeBuilder ref="CsvToJsonRouteBuilder" />    

  </camelContext>

</blueprint>

The only thing that has changed is the package names. I should point out one thing. I am probably the worst person in creating good names for classes, variables and packages and have to constantly refactor. So, don’t pay too much attention to the naming 😉

RouteBuilder

Again the RouteBuilder class is pretty much a copy of the one in part 2. It looks like this:

package org.souciance.integration.csvstojson;

import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.dataformat.bindy.csv.BindyCsvDataFormat;

/**
 * A bean which we use in the route
 */
public class CsvToJsonRouteBuilder extends RouteBuilder {

	@Override
	public void configure() throws Exception {
		// TODO Auto-generated method stub
		BindyCsvDataFormat bindy = new BindyCsvDataFormat(org.souciance.integration.csvstojson.Identity.class);		

		from("file:C:/test/?fileName=input.csv")
		.unmarshal(bindy)
		.log("${body}")
		.to("IdentityToJson")
		.to("file:C:/test/?fileName=output.json")
		.log("done!")
		.end();

	}
}

What we are doing is to define our bindy dataformat as CSV. We let it know to use the Identity POJO as our model class to parse the CSV rows. You can see that we are doing the unmarshal directly after the ”from” statement. Since this file contains multiple rows the important part is to understand that the exchange now contains a list. This means that we are dealing with a list of objects rather than the objects themselves. Here we can approach it in different ways.

  1. We can use the Splitter EIP to split the file and produce a file for each row. But this is not what we want.
  2. We can split and aggregate at the end – possible but a bit too complicated for our scenario.
  3. We send the entire list to our mapping class and deal with the list there. This is our approach.

Identity class – our model object

Just for reference our model class for the bindy dataformat looks like this:

package org.souciance.integration.csvstojson;

import org.apache.camel.dataformat.bindy.annotation.CsvRecord;
import org.apache.camel.dataformat.bindy.annotation.DataField;

@CsvRecord(separator = ",")
public class Identity {

	@DataField(pos=1)
	private int identity;
	@DataField(pos=2)
	private String firstname;
	@DataField(pos=3)
	private String lastname;
	@DataField(pos=4)
	private int phone;
	@DataField(pos=5)
	private String country;
	public int getIdentity() {
		return identity;
	}
	public void setIdentity(int identity) {
		this.identity = identity;
	}
	public String getFirstname() {
		return firstname;
	}
	public void setFirstname(String firstname) {
		this.firstname = firstname;
	}
	public String getLastname() {
		return lastname;
	}
	public void setLastname(String lastname) {
		this.lastname = lastname;
	}
	public int getPhone() {
		return phone;
	}
	public void setPhone(int phone) {
		this.phone = phone;
	}
	public String getCountry() {
		return country;
	}
	public void setCountry(String country) {
		this.country = country;
	}

}

As you can see we have not changed anything there.

IdentityToJson class – Where we do all the json magic

In this class we do all the json conversion. It looks as follows:

package org.souciance.integration.csvstojson;

import java.io.ByteArrayOutputStream;
import java.util.List;

import org.apache.camel.Exchange;
import org.apache.camel.Processor;

import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;

public class IdentityToJson implements Processor {

	@Override
	public void process(Exchange exchange) throws Exception {
		// TODO Auto-generated method stub

		/*initialize Jackson
		 *we need to create an outer object, an inner array and node objects for individual array elements
		 */
		JsonNodeFactory factory = new JsonNodeFactory(false);
		JsonFactory jsonFactory = new JsonFactory();
		ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
		JsonGenerator generator = jsonFactory.createGenerator(outputStream);
		ObjectMapper mapper = new ObjectMapper();
		ObjectNode listOfPersons = mapper.createObjectNode();
		ArrayNode persons = factory.arrayNode();

		/*get the current row and add it to the json array item by
		*traversing the incoming List of Identity objects
		*person is the json object containing each row
		*persons is the array that contains multiple person items
		*/
		if (exchange.getIn().getBody() instanceof java.util.List) {
			@SuppressWarnings("unchecked")
			List<Identity> listOfIdentities = ((List<Identity>)exchange.getIn().getBody());
			for (Identity identity : listOfIdentities ) {
				ObjectNode person = factory.objectNode();
				person.put("firstname", identity.getFirstname());
				person.put("lastname", identity.getLastname());
				person.put("phone", identity.getPhone());
				person.put("country", identity.getCountry());
				persons.add(person);
			}
		}
		//finally we create the entire json structure by adding the array to the root object ListOfRows
		listOfPersons.putArray("ListOfRows").addAll(persons);

		//write the json string to the exchange
		mapper.writeTree(generator,  listOfPersons);
		String json = new String(outputStream.toString());
		exchange.getIn().setBody(json);

	}

}

There are several changes made here.

  1. In the json initialization block you can see that we have added not only normal json objects but also array nodes. Essentially listOfPersons contains Persons which contains mulitple persons.
  2. The crucial part is the second part where we check if the object in the exchange is an instaneof List – that is if it contains a list. If it does, we iterate and extract the values and add it the person node. At the end of the iteration the person node is added to persons array.
  3. Once the iteration is done, we add the array to the listOfPersons object using the putArray and addAll jackson methods.

Summary

In essence this is one approach to map Csv to JSON format and customising your json structure. There are I am sure better ways or other approaches. If you know of others please do post them 😉 Stay tuned for the next series.

 

Annonser

Kommentera

Fyll i dina uppgifter nedan eller klicka på en ikon för att logga in:

WordPress.com Logo

Du kommenterar med ditt WordPress.com-konto. Logga ut / Ändra )

Twitter-bild

Du kommenterar med ditt Twitter-konto. Logga ut / Ändra )

Facebook-foto

Du kommenterar med ditt Facebook-konto. Logga ut / Ändra )

Google+ photo

Du kommenterar med ditt Google+-konto. Logga ut / Ändra )

Ansluter till %s