KerA is a distributed message broker inspired by Apache Kafka developed by INRIA (Rennes) and University of Luxembourg. KerA focuses on load balancing, fault tolerance and fast networking using InfiniBand. Its implementation builds upon the RAMCloud system. KerA itself does not provide stream analytics but can be integrated with different stream processing system, e.g. Apache Flink. So far the integration is done using Apache Plasma allowing data exchange using shared memory between a KerA process and a Flink instance running on the same machine. The main objective of this thesis is to replace the current log-structured memory implementation in KerA by RocksDB. The integration interface in KerA for this thesis is the virtual log abstraction which will provide the remote replication for RocksDB. The vision (beyond this thesis) is to support different storage solutions depending on the workload characteristics, e.g. FASTER from Microsoft is an alternative to RocksDB. Another aspect is the potential tighter integration of KerA and backend stream systems. For example Flink is also supporting RocksDB for storing states of stateful operators which will make it natural to later share state directly between KerA and Flink using RocksDB (replacing the Apache Plasma interface which still implies copying data between two connected systems if data needs to be persistent. |