Application development and data in the emerging world of stream processing

06/07/2016 - 09:30 to 10:30

Session abstract: 

For a long time, a substantial portion of data processing that companies did ran as big batch jobs -- CSV files dumped out of databases, log files collected at the end of the day etc. But businesses operate in real-time and the software they run is catching up. Rather than processing data only at the end of the day, why not react to it continuously as the data arrives? This is the emerging world of stream processing. But stream processing only becomes possible when the fundamental data capture is done in a streaming fashion; after all, you can’t process a daily batch of CSV dumps as a stream. This shift towards stream processing has driven the popularity of Apache Kafka. Making all the organization's data is available centrally as free-flowing streams enables a company's business logic to be represented as stream processing operations. Essentially, applications are stream processors in this new world of stream processing.
In her keynote, she will explain how the fundamental nature of application development will change as stream processing goes mainstream.