Understanding the Trident API
Trident API supports five broad categories of operations:
Operations for manipulations of partitioning local data without network transfer
Operations related to the repartitioning of the stream (involves the transfer of stream data over the network)
Data aggregation over the stream (this operation do the network transfer as a part of operation)
Grouping over a field in the stream
Merge and join
Local partition manipulation operation
As the name suggests, these operations are locally operative over the batch on each node and no network traffic is involved for it. The following functions fall under this category.
Functions
This operation takes single input value and emits zero or more tuples as the output
The output of these function operations is appended to the end of the original tuple and emitted to the stream
In cases where the function is such that no output tuple is emitted, the framework filters the input tuple too, while in other cases the input tuple is duplicated...
