Flanders Make Tutorial 2024-Harmony in Complexity_Blended Reasoning in Agent-based Digital Twins for Multi-robot Systems.pptx

A novel approach for blended adaptive reasoning in Digital Twins.

1. Harmony in Complexity: Blended Reasoning in Agent-based Digital Twins for Multi-robot Systems Hussein Marah and Moharram Challenger AnSyMo - Antwerp Systems and software Modelling MICSS - Modelling Intelligent Complex Software and Systems 1

2. 2 Harmony in Complexity!!!

3. 3 Contents

4. 4 Contents

5. https://www.digitaltwinconsortium.org/initiatives/the-definition-of-a-digital-twin/ 5 Digital Twin Definition

6. 6 “Digital twin (DT) mainly acts as a virtual exemplification of a real-world entity, system, or process via multi-physical and logical models, allowing the capture and synchronization of its functions and attributes” [1]. Digital Twin – Definition [1] Marah, Hussein, and Moharram Challenger. "Adaptive hybrid reasoning for agent-based digital twins of distributed multi-robot systems." SIMULATION (2024): 00375497231226436. Photo credit: © Siemens 2024

7. 7 Digital Twins Applications

8. 8 Challenges of Digital Twins Marah, H., & Challenger, M. (2023). MADTwin : a framework for multi-agent digital twin development: smart warehouse case study. Annals of Mathematics and Artificial Intelligence, 1-31.

9. 9 The challenge lies in effectively performing multi-level reasoning and decision-making processes across different levels of the system by utilizing its digital twin. Problem domain

10. 10 Motivation Handling c omplexity Adaptability and flexibility. Autonomy and decentralization. Interoperability and Integration. Knowledge representation. Multi- level reasoning and real-time decision-making.

11. Centralized and Monolithic Decentralized and Distributed https://www.plattform-i40.de/IP/Navigation/EN/Industrie40/WhatIsIndustrie40/what-is-industrie40.html 11 Standard System Structures

12. 12 Contents

13. Marah, H., & Challenger, M. (2023). MADTwin : a framework for multi-agent digital twin development: smart warehouse case study. Annals of Mathematics and Artificial Intelligence, 1-31. 13 Agent-based Paradigm Ontology-enabled Agents Real-time Event Processing The Proposed Solution 1 2 3

14. 14 Contents

15. [Wooldridge and Jennings, 1995]: “An encapsulated computer system that is situated in some environment, and that is capable of flexible and autonomous action in that environment in order to meet its design objectives.” [Russel and Norvig, 1995]: “As something autonomous that perceives and acts in an environment, being its choice depends on its own experience rather than on knowledge of the environment.” [Ferber, 1995] “A physical or virtual entity which: (1) is capable of acting in an environment, (2) can communicate directly with other agents, (3) has autonomous behaviour , (4) has only a partial representation of this environment, (5) may be able to reproduce itself, and (6) possesses skills and can offer services.” 15 Agent Definition

16. Autonomous component that represents physical or logical element in the system, capable to act and achieve its goals and to interact and cooperate with other agents when it doesn’t possess enough knowledge and skills to reach its objectives. Source: Russel and Norvig1995 Agent Actions (e.g., pick up) Percepts (e.g., pressure value) Knowledge Decisions What is an Agent? 16

17. 17 Agents’ Characteristics Autonomy and reactivity: capability to react and make their own decisions. Cooperation and social capabilities: ability to cooperate and communicate. Proactivity: able to take the initiative. Optional properties: mobility, learning..

18. The power of agents is to have group of agents that work together to solve complex problems beyond the capabilities of a single agent . MAS is a society of agents that are organized in a certain environment and capable of interacting and cooperating to achieve their individual objectives and the objectives of the overall system. Derived from Distributed AI. 18 Multi-Agent System (MAS)

19. Example of MAS working in practice Blas, Héctor Sánchez San, et al. "A multi-agent system for data fusion techniques applied to the internet of things enabling physical rehabilitation monitoring." Applied Sciences 11.1 (2020): 331. 19

20. MAS Vs. Traditional systems 20 Flexibility and Adaptability . Heterogeneity modeling. Decentralization and Parallelism . Scalability . Robustness and Fault Tolerance .

21. 21 Contents

22. Ontology – Definition https://www.ontotext.com 22

23. Ontology – Function https://www.ontotext.com 23

24. Ontology Integration Workflow Data integration Knowledge representation Reasoning & inferencing 24

25. Ontology – Decision Making Process 25 Ontology Knowledge Insights Decisions Results : represents :provides :supports : delivers

26. 26 Contents

27. Event processing such as Complex Event Processing (CEP) is a technology of tracking and analyzing streams of information (events) from various sources in real-time to identify and detect meaningful patterns, and correlations. 27 The Proposed Solution: Event Processing Photo credit: © hazelcast.com

28. Recap 28 R epresent system’s entities as distributed autonomous agents with their capabilities and behaviors. C apture and model complex relationships, semantics, and the domain knowledge. Utilized in event correlation and pattern recognition which enables rapid response to critical events.

29. System Architecture 29

30. Deployment Example: Blended Reasoning-Enabled Agent-based Digital Twins 30

31. 31 Contents

32. Case Study: D istributed M ulti-robot S ystem Photo credit: © talkinglogistics.com Features: Move autonomously. Find shortest-path. Detect collisions. Reason about different situations. 32

33. Agents Deployment 33

34. Ontology Design Partial Ontology Representation for the Warehouse Case Study 34

35. Ontology Representation and Instantiation Entity Property Value hasValue rdf:type Robot : R2 Robot rdfs:subclassOf Position Velocity 24 [20332, 21362] Robot Robot : R1 Velocity 28 rdf:type MovementTask CB rdfs:subclassOf Instantiation Ontology MovementTask BA Position [19372, 18302] hasProperty Relationship hasObject hasSubject hasTask Robot_Information Task : T1 Task : T2 isAssigned rdfs:subclassOf hasRelationship 35

36. Event Processing Implementation 36 Identify the applicable patterns. Determine the target events. Define the pattern and create query statement.

37. 37 Contents

38. Collision Detection Scenario 38

39. Collision Detection Scenario R1 D E C Target: C Speed: 10 Route: EC Battery Level: 75% Target: C Speed: 10 Route: DC Battery Level: 65% I’m in DE, but I will go to C soon! Where are you, Bro ? What is your target? 39 R2

40. Collision Detection Scenario 40 1. Physical Agents 2. Digital Agents (Digital Twins) are defined

41. Collision Detection Scenario: Demo 41

42. Collision Detection Scenario ∀t [ RobotEvent ( movementTask.toVertex ='C', t) → (∃t' (t ≤ t' < t + 60) ∧ RobotEvent ( movementTask.toVertex ='C', t'))] 42

43. Collision Detection Scenario 43

44. SPARQL query (ETA) 44

45. 45 Contents

46. Power Management Scenario 46

47. Power Management Scenario: Demo 47

48. Summary & Conclusion This presentation has discussed enabling agent-based DTs with blended reasoning approach. By leveraging agents to model the system entities, ontologies to represent the domain knowledge, and complex event processing for pattern recognition, the reasoning and decision-making is performed in multi-levels: Agents level. Events level. Knowledge graph level. 48

49. 49 Thanks for you attention! [email protected] Contact me via LinkedIn: