Blog

Correctly Using Camels AdviceWith in Unit Tests

ApacheCon Denver

Ready to take your data from A to B through any terrain 🙂

We care a lot about the stuff that goes around Solr and Elasticsearch in our clients infrastructure. One area that seems to always be being reinvented for-better-or-worse is the data ETL/data ingest path from data source X to the search engine. One tool we’ve enjoyed using for basic ETL these days is Apache Camel. Camel is an extremely feature-rich Java data integration framework for wiring up just about anything to anything else. And by anything I mean anything: file system, databases, HTTP, search engines, twitter, IRC, etc.

One area I initially struggled with with Camel was exactly how to test my code. Lets say I have defined a simple Camel route like this:

from("file:inbox").unmarshall(csv)  // parse as CSV.split() 
// now we're operating on individual CSV lines   
.bean("customTransformation")  
// do some random operation on the CSV line   
.to("solr://localhost:8983/solr/collection1/update")

Great! Now if youve gotten into Camel testing, you may know there’s something called “AdviceWith”. What is this interesting sounding thing? Well I think its a way of saying “take these routes and muck with them” – stub out this, intercept that and dont forward, etc. Exactly the kind of slicing and dicing Id like to do in my unit tests!

I definitely recommend reading up on the docs, but heres the real step-by-step built around where you’re probably going to get stuck (cause its where I got stuck!) getting AdviceWith to work for your tests.

1. Use CamelTestSupport

Ok most importantly, we need to actually define a test that uses CamelTestSupport. CamelTestSupport automatically creates and starts our camel context for us.

 public class ItGoesToSolrTest extends CamelTestSupport {    ... }

(My usual disclaimers about my tendancy to write non-compiling code apply here 🙂 )

2. Specify the route builder we’re testing

In our test, we need to tell CamelTestSupport where it can access its routes:

@Overrideprotected RouteBuilder createRouteBuilder() {    return new MyProductionRouteBuilder();}

3. Specify any beans we’d like to register

It’s probably the case that youre using Java beans with Camel. If you’re using the bean integration and referring to beans by name in your camel routes, you’ll need to register those names with an instance of your class.

@Overrideprotected Context createJndiContext() throws Exception {   
  JndiContext context = new JndiContext();   
  context.bind("customTransformation", new CustomTransformation());   
  return context;
}

4. Monkey with our production routes using AdviceWith

Second we need to actually use the AdviceWithRouteBuilder before each test:

@Beforepublic void mockEndpoints() throws Exception {
  AdviceWithRouteBuilder mockSolr = new AdviceWithRouteBuilder() {  
     @Override   
     public void configure() throws Exception {  
         // mock the for testing      
         interceptSendToEndpoint("solr://localhost:8983/solr/collection1/update")       
         .skipSendToOriginalEndpoint()         
         .to("mock:catchSolrMessages");     
         } 
     })    
  context.getRouteDefinition(1).   
  .adviceWith(context, mockSolr); 
  }

Theres a couple things to notice here:

  1. In configure we simply snag an endpoint (in this case Solr) and then we have complete freedom to do whatever we want. In this case, were rewiring it to a mock endpoint we can use for testing.
  2. Notice how we get a route definition by index (in this case 1) to snag the route were testing and that we’d like to monkey with. This is how Ive seen it in most Camel examples, and it’s hard to guess how Camel is going to assign some index to your route. A better way would be to give our route definition a name:
    from(“file:inbox”)
    .routeId(“csvToSolrRoute”)
    .unmarshall(csv) // parse as CSV

then we can refer to this name when retrieving our route:

 context.getRouteDefinition("csvToSolrRoute").   
 .adviceWith(context, mockSolr);

5. Tell CamelTestSupport you want to manually start/stop Camel

One problem you will run into with the normal tutorials is that CamelTestSupport may start routes before your mocks have taken hold. Thus your mocked routes won’t be part of what CamelTestSupport has actually started. You’ll be pulling your hair out wondering why Camel insists on attempting to forward documents to an actual Solr instance and not your test endpoint.

To take matters into your own hands, luckily CamelTestSupport comes to the rescue with a simple method you need to override to communicate your intent to manually start/stop the camel context:

@Override public boolean isUseAdviceWith() {
    return true;
}

Then in your test, youll need to be sure to do

@Testpublic void foo() { 
   context.start();  
   // tests!   
   context.stop();
   }

6. Write a test!

Now you’re equipped to try out a real test!

@Test public void testWithRealFile() {  
   MockEndpoint mockSolr = getMockEndpoint("mock:catchSolrMessages");
   File testCsv = getTestfile();
   context.start();
   mockSolr.expectedMessageCount(1);
   FileUtils.copyFile(testCsv, "inbox");
   mockSolr.assertIsSatisfied();
   context.stop(); 
}

And that’s just scratching the surface of Camel’s testing capabilities. Check out the Camel docs for information on stimulating endpoints directly with the ProducerTemplate thus letting you avoid using real files – and all kinds of goodies.

Anyway, hopefully my experiences with AdviceWith can help you get it up and running in your tests!

If you’d love to utilize Solr or Elasticsearch for search and analytics, but can’t figure out how to integrate them with your data infrastructure – contact us! Maybe there’s a camel recipe we could cook up for you that could do just the trick.