TU Darmstadt / ULB / TUbiblio

Operator as a Service: Stateful Serverless Complex Event Processing

Luthra, Manisha ; Hennig, Sebastian ; Razavi, Kamran ; Wang, Lin ; Koldehofe, Boris (2020):
Operator as a Service: Stateful Serverless Complex Event Processing.
In: Proceedings of the 2020 IEEE International Conferences on Big Data,
IEEE, 9th Workshop on Scalable Cloud Data Management, virtual Conference, 10.-13.12.2020, [Conference or Workshop Item]

Abstract

Complex Event Processing (CEP) is a powerful paradigm for scalable data management that is employed in many real-world scenarios such as detecting credit card fraud in banks. The so-called complex events are expressed using a specification language that is typically implemented and executed on a specific runtime system. While the tight coupling of these two components has been regarded as the key for supporting CEP at high performance, such dependencies pose several inherent challenges as follows. (1) Application development atop a CEP system requires extensive knowledge of how the runtime system operates, which is typically highly complex in nature. (2) The specification language dependence requires the need of domain experts and further restricts and steepens the learning curve for application developers. In this paper, we propose CEPLESS, a scalable data management system that decouples the specification from the runtime system by building on the principles of serverless computing. CEPLESS provides “operator as a service” and offers flexibility by enabling the development of CEP application in any specification language while abstracting away the complexity of the CEP runtime system. As part of CEPLESS, we designed and evaluated novel mechanisms for in-memory processing and batching that enable the stateful processing of CEP operators even under high rates of ingested events. Our evaluation demonstrates that CEPLESS can be easily integrated into existing CEP systems like Apache Flink while attaining similar throughput under high scale of events (up to 100K events per second) and dynamic operator update in ˜238 ms.

Item Type: Conference or Workshop Item
Erschienen: 2020
Creators: Luthra, Manisha ; Hennig, Sebastian ; Razavi, Kamran ; Wang, Lin ; Koldehofe, Boris
Title: Operator as a Service: Stateful Serverless Complex Event Processing
Language: English
Abstract:

Complex Event Processing (CEP) is a powerful paradigm for scalable data management that is employed in many real-world scenarios such as detecting credit card fraud in banks. The so-called complex events are expressed using a specification language that is typically implemented and executed on a specific runtime system. While the tight coupling of these two components has been regarded as the key for supporting CEP at high performance, such dependencies pose several inherent challenges as follows. (1) Application development atop a CEP system requires extensive knowledge of how the runtime system operates, which is typically highly complex in nature. (2) The specification language dependence requires the need of domain experts and further restricts and steepens the learning curve for application developers. In this paper, we propose CEPLESS, a scalable data management system that decouples the specification from the runtime system by building on the principles of serverless computing. CEPLESS provides “operator as a service” and offers flexibility by enabling the development of CEP application in any specification language while abstracting away the complexity of the CEP runtime system. As part of CEPLESS, we designed and evaluated novel mechanisms for in-memory processing and batching that enable the stateful processing of CEP operators even under high rates of ingested events. Our evaluation demonstrates that CEPLESS can be easily integrated into existing CEP systems like Apache Flink while attaining similar throughput under high scale of events (up to 100K events per second) and dynamic operator update in ˜238 ms.

Title of Book: Proceedings of the 2020 IEEE International Conferences on Big Data
Publisher: IEEE
Uncontrolled Keywords: C2, C7
Divisions: 18 Department of Electrical Engineering and Information Technology
18 Department of Electrical Engineering and Information Technology > Institute of Computer Engineering
18 Department of Electrical Engineering and Information Technology > Institute of Computer Engineering > Multimedia Communications
20 Department of Computer Science
20 Department of Computer Science > Telecooperation
DFG-Collaborative Research Centres (incl. Transregio)
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres > CRC 1053: MAKI – Multi-Mechanisms Adaptation for the Future Internet
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres > CRC 1053: MAKI – Multi-Mechanisms Adaptation for the Future Internet > C: Communication Mechanisms
DFG-Collaborative Research Centres (incl. Transregio) > Collaborative Research Centres > CRC 1053: MAKI – Multi-Mechanisms Adaptation for the Future Internet > C: Communication Mechanisms > Subproject C2: Information-centred perspective
Event Title: 9th Workshop on Scalable Cloud Data Management
Event Location: virtual Conference
Event Dates: 10.-13.12.2020
Date Deposited: 12 Feb 2021 07:45
Corresponding Links:
Export:
Suche nach Titel in: TUfind oder in Google
Send an inquiry Send an inquiry

Options (only for editors)
Show editorial Details Show editorial Details