Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ververica/airlines-cep-demo
Demo application to illustrate Complex Event Processing (CEP) with Apache Flink on Ververica
https://github.com/ververica/airlines-cep-demo
Last synced: about 1 month ago
JSON representation
Demo application to illustrate Complex Event Processing (CEP) with Apache Flink on Ververica
- Host: GitHub
- URL: https://github.com/ververica/airlines-cep-demo
- Owner: ververica
- Created: 2024-10-09T17:00:45.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-11-25T19:31:14.000Z (about 2 months ago)
- Last Synced: 2024-11-25T20:27:59.194Z (about 2 months ago)
- Language: Java
- Homepage:
- Size: 354 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
Awesome Lists containing this project
README
Complex Event Processing (CEP) Demo
-----------------------------------
This is a demo application, to illustrate **Complex Event Processing (CEP) with Apache Flink on Ververica.**
You can also check [this blog post](https://www.ververica.com/blog/real-time-insights-for-airlines-with-complex-event-processing).
### Background
Airline operations are complex and time-sensitive. Monitoring and detecting critical events, such as delays or security threats, are essential to maintaining efficient operations and ensuring passenger safety.
The goal is to create a **real-time streaming application using Apache Flink** that:
- ✅ Detects flight delays correlated with adverse weather conditions.
- ✅ Monitors and escalates security incidents (e.g., unattended luggage).
- ✅ Analyzes aircraft turnaround times to optimize ground operations.
With Apache Flink’s CEP library, we define the following patterns:
### Weather-Related Flight Delays
This pattern detects delays associated with severe weather. We define it to look for a sequence where a `DELAY` event is followed by a `WEATHER` event within a short time frame.
```java
Pattern weatherPattern = Pattern.begin("weatherAlert")
.where(new IterativeCondition() {
@Override
public boolean filter(FlightEvent event, Context ctx) throws Exception {
return event.getEventType().equals("WEATHER") && event.getDetails().contains("storm");
}
});
```
### Security Incident Escalation
This pattern monitors security alerts that escalate, such as a sequence where an initial `SECURITY` event is followed by further details indicating a heightened threat (e.g., suspicious activities or threat investigations).
```java
Pattern.begin("securityAlert")
.where(new IterativeCondition() {
@Override
public boolean filter(FlightEvent event, Context ctx) throws Exception {
return event.getEventType().equals("SECURITY");
}
});
```
### Turnaround Efficiency
This pattern tracks if an aircraft departs within a 45-minute window after landing. This helps evaluate how efficiently the airline handles aircraft turnaround.
```java
Pattern turnaroundPattern = Pattern.begin("arrival")
.where(new IterativeCondition() {
@Override
public boolean filter(FlightEvent event, Context ctx) throws Exception {
return event.getEventType().equals("LANDING");
}
})
.next("departure")
.where(new IterativeCondition() {
@Override
public boolean filter(FlightEvent event, Context ctx) throws Exception {
FlightEvent arrivalEvent = ctx.getEventsForPattern("arrival").iterator().next();
return event.getEventType().equals("TAKEOFF") &&
event.getFlightId().equals(arrivalEvent.getFlightId()) &&
event.getAirportCode().equals(arrivalEvent.getAirportCode()) &&
(event.getTimestamp() - arrivalEvent.getTimestamp()) <= 45 * 60 * 1000; // 45 minutes
}
});
```
When these patterns are detected, Flink generates alerts using the **PatternSelectFunction**
```java
securityPatternStream.select((PatternSelectFunction) pattern -> {
FlightEvent securityEvent = pattern.get("securityAlert").get(0);
return "Security Alert: Incident detected at " + securityEvent.getAirportCode() + " for Flight " + securityEvent.getFlightId() +
" - " + securityEvent.getDetails();
}).name("SecurityPattern").uid("SecurityPattern");
```
### Deployment
Run `mvn clean package` to create a jar file
On the `Artifacts` tab upload the generated jar file
Then navigate to the `Deployments` tab, click `new deployment` and put the required fields.
Finally click `start` and after a while you will see your job running.
### Sample Output
```shell
Turnaround Efficiency Achieved: Flight EK201 landed at DXB and took off within 45 minutes.
Security Alert: Incident detected at LAX for Flight EK201 - Suspicious activity
Weather Alert: Severe weather condition at DXB - Heavy storm warning
Security Alert: Incident detected at LAX for Flight EK201 - Bomb threat investigation
Security Alert: Incident detected at DXB for Flight EK205 - Suspicious activity
Weather Alert: Severe weather condition at LHR - Heavy storm warning
```