Storm persistence

Now that we know Storm and its internals very well, let's wire in persistence to Storm. Well we have done all the computation and code, so now it's very important to store the computed results or intermediate references into a database or some persistence store. You have the choice of writing your own JDBC bolts or you can actually use the implementation provided for using Storm persistence.

Let's start with writing our own JDBC persistence first. Once we have touched upon the nitty gritty, then we can look at and appreciate what Storm provides for. Let's say we are setting up software system at toll gates that could monitor the emission rates of vehicles and track the details of the vehicles that are running at emissions beyond the prescribed limit.

Here, all the vehicle details and their emissions are captured in the file that is being read by the file reader spout. The spout then reads the record and feeds it into the topology where it is consumed by the parser bolt, which...