Apache Kafka on Heroku
Apache Kafka is an open source message streaming platform that allows you to publish, subscribe, stream, and process millions of data in real time. Since IoT Event Data is in millions, the Apache Kafka platform can be used to process and transform streaming data. It is also a distributed system and is designed to be fault tolerant. Salesforce IoT Cloud (Thunder) is built using Apache Kafka.
Kafka API
The streams of data are divided into topics in Kafka. It is run as a cluster on one or more servers, and has four core APIs, as shown in the following table:
API | Functionality |
Producer API | This allows other applications to publish record streams to Kafka topics. |
Consumer API | This allows applications to subscribe to the topics and process the record stream. |
Streams API | This consumes input streams for one or more topics and translates to output stream to one or more output topics. |
Connector API | This allows Kafka topics to connect to data systems or applications. Use the connector API... |
