Complex Event Processing

Drools Complex Event Processing Example
(Email author for the Code).

1. CEP Overview
1.CEP Definitions and Terms
2.Use Case Example Architecture
3.Architecture of our Event(s)
4.CEP Patterns & Rules
5.Temporal Reasoning / Complexity of Rules
6.Implemented Rules
7.Allen’s (5 of 13) Temporal Operator Diagrams
8.Example Output ( & code overview), App Demo.
9.Forward Chaining Inference Engine
10.Backward Chaining Inference Engine
11.Comparison: Forward vs Backward Chaining

2. Complex Event Processing
Dr. David Luckham's book "The Power of Events“ - 2002. 9.ForwardF Cwhw Iaa1:aUcT9 I9OEWcfEL0UcT9 i
Event Stream [ needle(s) in haystack(s) ]
Time Zero Time Infinity
e1, e2, e3, e4, e5, e6, e7, e8 ,e9, e10, e11, e12, e13, e14, e15, e16, e17, e18, e19, e20, e21, e22, e23, ….
Definitions:
Events – “something of interest that happens”: (1) AccessAttempt, (2) DataAccessAttempt.
Event Streams – “massive streams of data” (some important, most not).
Temporal Reasoning – “reasoning over time, factoring in time during reasoning”. The
computer must have a way to understand time and reason while incorporating it.
Complex Event Processing – “the task of processing multiple events with the goal of
identifying the meaningful events (over time) within the event cloud (stream).”
* Detection of complex patterns of many events,
* Event Correlation, Abstraction, and Hierarchies,
* Relationships between events: (causality, membership, timing, etc…)
Common Examples:
* Stock Broker Application: Ticker Monitor
* Our Example: Military ID Monitor System

3. Example Use Case:
ID Events Processing
Events: AccessAttempts and DataAccessAttempts
OFFICE
CAFE
BARRACKS
•No multiple logins permitted.
•The only area where access to restricted data
is permitted.
•No logins from unknown origins permitted.
•No repeat logon failures permitted.
•No multiple logins permitted.
•Accessing restricted data not permitted here
•No repeat logon failures permitted.
•No multiple logins permitted.
•Accessing restricted data not permitted here.
•No repeat logon failures permitted.
UNKNOWN
•No logins permitted from here at all.
•No data access of any kind permitted
from here.
•Alert if multiple access failures emanate
from here.
CEP: TRACKING SYSTEM

4. Architecture of Events: (Java Objects)
public class AccessAttempt {
private String employeeId;
private String location;
private String computerId;
private String action;
private String result;
……….
} // End Class LoginAttempt
public class DataAccessAttempt {
private String dataClassification;
private String computerId;
private String employeeId;
…….
} // End Class DataAccessAttempt
Correlated Attributes
•employeeId
•computerId
private static final String LOGIN = "login";
private static final String LOGOUT = "logout";
private static final String SUCCESSFUL = "successful";
private static final String FAILURE = "failure";
private static final String TOPSECRET = "topsecret";
private static final String SECRET = "secret";
private static final String CLASSIFIED = "classified";
private static final String UNCLASSIFIED = "unclassified";
private static final String OFFICE = "office";
private static final String CAFE = "cafe";
private static final String BARRACKS = "barracks";
private static final String UNKNOWN = "unknown";

5. Complex Patterns – Our Example:
Important, but not meaningful alone:
Login / Logout – [normal, paired, properly sequenced, singleton]
Data Access – [unclassified – anywhere known]
Data Access: [classified, secret, top secret – with in the office, proper procedure]
Important and Meaningful: – Trigger Alerts
Data Access: [classified, secret, top secret – in any public area, or from unknown places]
Login – from any unknown place.
Multiple Logins of same ID from different places
Repeat Login Failures (X failures over N minutes)
Complex Event Processing
Events: AccessAttempts and DataAccessAttempts

6. Expert Systems Rule
Forward Chaining Inference Engine
Complexity of Rules which reason using time (Temporal)
Example:
“Multiple Logins”
[t0] [Event A – logIN ID: “234”, “Office”] [t+10m] [Event B – logIN ID: “234”, “Cafe’”] [t+30m] [Event C – logOUT ID: “234”, “Office”]
Notes:
Event A has to happened before Event B, but Event C cannot happen before Event B.
Key Concepts:
Temporal Operators: “not”, “before”, “after”… Accumulate : (reasoning over collections of data)
Drools implements all 13 operators defined by Allen and also their logical complement (negation).
• [ALLEN81] Allen, J.F..An Interval-based Representation of Temporal Knowledge. 1981.
• [ALLEN83] Allen, J.F..Maintaining knowledge about temporal intervals. 1983.
Temporal Operators:
(1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met
By, (10) Overlaps, (11) Overlapped By, (12) Starts, (13) Started By.

7. Implemented Rules: Arch Types
Patterns which raise alerts:
1.Concurrent logins (one ID can’t be in 2 places at the same time)
2.Multiple failed attempts during a specific time period for any location
3.Successful logins from any unknown locations
4.Any data access from any unknown location
5.Restricted data access from non-secure area (public areas)
RULE EXAMPLE:
rule "ALERT: Access of TopSecret Data from unkown Location"
when
$loginUnknown : AccessAttempt( location == "unknown", action == "login", $eid : employeeId, $cid : computerId, result == "successful") from entry-point "idsystem"
$dataAccess : DataAccessAttempt( this after $loginUnknown, dataClassification == "topsecret", employeeId == $eid, computerId == $cid ) from entry-point
"idsystem"
then
System.out.println("ALERT: TopSecret Data Breach -> [ " + $dataAccess + " ]");
end

8. Allen: Temporal Operator Diagrams
Temporal Operators:
(1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11)
Overlapped By, (12) Starts, (13) Started By.
Current vs Correlated Events: [Current: “Event of Focus” / Correlated: “Event that is related to”]
Event
A
Event
B
Current Event:
Correlated Event:
AFTER
Finishes
Starts
Event
B
Event
A
Current Event:
Correlated Event:
BEFORE
Finishes
Starts
DURING
Current Event:
Correlated Event:
Starts
Event
B
Finishes
Event
A
Starts
Event
A
Finishes
Time Window: Mins, Hrs
Event
B
Time 0 Time 00)
Time Window: Mins, Hrs

9. Allen: Temporal Operator Diagrams
Event
B
OVERLAPS
Current Event:
Correlated Event:
Starts
Event
B
Finishes
Event
A
Starts
Event
A
Finishes
Temporal Operators:
(1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11)
Overlapped By, (12) Starts, (13) Started By.
Current vs Correlated Events: [Current: “Event of Focus” / Correlated: “Event that is related to”]
Time Window: Mins, Hrs
Event
B
INCLUDES
Current Event:
Correlated Event:
Starts
Event
B
Finishes
Event
A
Starts
Event
A
Finishes
Time Window: Mins, Hrs
Time 0 Time 00)

10. CEP Sample Output
Dec 08, 2014 12:54:13 PM com.military.idcard.IDCardEventExample main
INFO: *********************************>>>> Drools CEP ID Card Example
ALERT: Restricted Data Breach -> [ DataAccessAttempt
[dataClassification=secret, computerId=barracks3333, employeeId=3333] ]
ALERT: Restricted Data Breach -> [ DataAccessAttempt
[dataClassification=classified, computerId=cafe2222, employeeId=2222] ]
ALERT: Restricted Data Breach -> [ DataAccessAttempt
[dataClassification=topsecret, computerId=unknown, employeeId=4444] ]
ALERT: Unknown Location Login -> [ Unknown Location ] : AccessAttempt
[employeeId=4444, location=unknown, computerId=unknown, action=login,
result=successful]
ALERT: TopSecret Data Breach -> [ DataAccessAttempt
[dataClassification=topsecret, computerId=unknown, employeeId=4444] ]
ALERT: Multiple Logins: (Successful) -> [ 1st: Barracks and 2nd: cafe ] - ID: 2023
ALERT: Multiple Logins: (Successful) -> [ 1st: Cafe and 2nd: barracks ] - ID: 1023
ALERT: Multiple Logins: (Successful) -> [ 1st: Office and 2nd: cafe ] - ID: 4023
ALERT: Multiple Logins: (Successful) -> [ 1st: Office and 2nd: cafe ] - ID: 1123
ALERT: Multiple Logins: (Failures) -> CAFE 6
ALERT: Multiple Logins: (Failures) -> BARRACKS 5
ALERT: Breach Attempts - Multiple Failures: (UNKNOWN) 5
*** DONE ***

11. Forward Chaining Inference Engine

12. Backward Chaining Inference Engine

13. Forward vs Backward Chaining
•Backward chaining (a la Prolog) is more like finding what initial conditions form a path to
your goal. At a very basic level it is a backward search from your goal to find
conditions that will fulfill it.
•Backward chaining is used for interrogative applications (finding items that fulfill certain
criteria) - one commercial example of a backward chaining application might be finding
which insurance policies are covered by a particular reinsurance contract.
•Forward chaining (a la Drools) matches conditions and then generates inferences
from those conditions. These conditions can in turn match other rules. Basically, this
takes a set of initial conditions and then draws all inferences it can from those conditions.
•STUDY OF DIFFERENCE BETWEEN FORWARD AND BACKWARD REASONING
http://www.ijetae.com/files/Volume2Issue10/IJETAE_1012_48.pdf
•Forward chaining means applying rules in a forward direction: recursively applying the
rules over data to generate more data (and applying the rules over that data... I have a
member of po:Person... it must also be a member of foaf:Person... and so it must be
a foaf:Agent and dc:Agent... and so...)
•Backward chaining means applying rules in a backwards manner: taking a goal (e.g., a
query) and recursively working backwards to find more data that can satisfy the goal (I'm
looking for foaf:Agents... I should also look for dc:Agents and foaf:Persons
and po:Persons...)