Camel development series part 8

In this series I will simply touch upon a few issues I encountered during coding that you might find useful. For reference this is on Camel version 2.17.0.

PollEnrich will not move file after completion

So I had a route where in the middle of the route I had to enrich it with a file, and after consuming the file, and the route finished it should move it to the a specific folder with a timestamp. The route looked like this:

 

from(....)
  ...
  ...
 .pollEnrich().simple("file:" + location + "?fileName=${header.File}&charset=iso-8859-1&" +
            "move=../archive/${file:name.noext}-${date:now:yyyyMMddHHmm}.${file:ext}")

We are using simple because we want pollEnrich to evaulate dynamic endpoints from headers. This can be found in the documentation. However, what actually happened was that the pollEnrich created the archive folder and the completely ignore the file expression language and just created a folder with the timestamp. No matter how hard I tried to change it still produced an output similar to this. My intention was to create a folder called archive and in there save the files with a timestamp attached to their name.

The problem was that the ”?move=” part doesn’t evaluate to anything meaningful at the point it’s evaluated. The ”move=” expressions need to be evaluated after the file has been picked up, but they are being evaluated before the file is read.

So my workaround was to replace the pollEnrich with the following:

 ConsumerTemplate consumer = exchange.getContext().createConsumerTemplate();
    String fileName = exchange.getIn().getHeader("File", String.class);
    String fileUri = "file://" + location + "?fileName=" + fileName + "&charset=iso-8859-1&move=archive/${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}&moveFailed=failed/${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}";
byte[] fileBody = consumer.receiveBody(fileUri,5000, byte[].class);

The above code resides in a processor. Basically I had to revert to a ConsumerTemplate instead. This works fine, it was just confusing that the pollEnrich didn’t work with dynamic properties.

Building predicates from exchange properties

Exchange properties are very useful when you want to pass properties from one exchange to another. Predicates are very powerful to build rich expressions in order to make to complex statements. These two can then be combined into something as simple as:

    Predicate success = exchangeProperty("Status").isEqualTo("true");
Predicate failure = exchangeProperty("Status").isEqualTo("false");
.......
.........
from(..)
..
..
.choice()
.when(success)
.otherwise()
.when(failure)
..

The code statement is easier to read and you have replaced a complex statement with a simpler one that can be used in the dsl. This is something really cool about Camel.

Splitting large files

Suppose you have a big file that you want to read. Maybe you have millions of rows of CSV data and you want to split them by a certain number or simply one by one.Since each row should be handled the same, you also want things to be done in parallel.

The most performance optimal way can be expressed like this:

from("file:...")
.split().tokenize(System.lineSeparator(), 1000).streaming()
//use threading to handle the different split chunks in parallel
.threads(20, 50)
.to("someProcessor");

So, what we have done is to split based on a token. In this case it is by the operating system new line token e.g. in windows and in linux world. We also group them such that each split shall contain 1000 rows. Then we do streaming so that lazy parsing is done to save memory and finally we add threads so things can be done in parallel.

Specify exchange pattern and delay

The last two simple tips are for when you want to specify the exchange pattern directly as an endpoint and for introducing a simple delay in ms.

//specify the exhange pattern
.inOnly("seda:outputQueue")
//introduce a delay of 100 ms
 .delay(100)

As you can see we can write .inOnly or .inOut and then provide the endpoint. This can be good if you want to do some processing but don’t want to wait for a response, instead want to send an immediate reply to caller.

The delay is simple, simply add the value and you have your delay!

Annonser