LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method

Research paper presentation at the BPM'2024 International Conference.
The paper studies how to use LLMs to assist business users in analyzing and redesigning business processes to optimize waiting time. The study compares two methods to prompt an LLM to recommend redesign options to reduce waiting times: (1) a baseline (“zero-shot”) method involving a minimalistic prompt; and (2) an enhanced method where the prompt includes descriptions of redesign patterns that may lead to redesigned processes with lower waiting times. To compare these methods, we conduct a user evaluation that combines semi-structured interviews with a survey involving process analysts. The analysts compare the recommended redesigns in terms of desirable properties of recommendation systems, such as relevance, usefulness, and diversity of the recommended redesigns. The results suggest that the enhanced prompting method yields more relevant and actionable redesign options. In contrast, the baseline produces high-level recommendations more suited for managerial decision-making.
Full paper available at: https://link.springer.com/chapter/10.1007/978-3-031-70396-6_27

1. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Katsiaryna Lashkevich, Fredrik Milani, Maksym Avramenko and Marlon Dumas BPM 2024 1

2. Why am I waiting? LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Submit paper 4d 3h 20m Review paper 1d 2h 15m Why? 43d 2h 28m Images: Flaticon.com 2

3. Why am I waiting? LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Submit paper 4d 3h 20m Review paper 1d 2h 15m Why? 43d 2h 28m Batching Resource contention Prioritization Resource unavailability Extraneous factors Images: Flaticon.com 3

4. Submit paper → Review paper 43d 2h 28m LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Why am I waiting? Submit paper 4d 3h 20m Review paper 1d 2h 15m Why? 43d 2h 28m 4

5. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Solution 5

6. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 6

7. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 7

8. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 8

9. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 9

10. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 10

11. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 11

12. Prompt template Descriptive analysis Prescriptive analysis Assistant-level instructions role and goal of the LLM definitions (e.g., waiting time causes), descriptions of all data inputs Redesign patterns Message-level instructions preferred data format instructions for analyzing and presenting improvement opportunities (what aspects must be included in the response) Message-level instructions [User prompt] LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 12

13. Prompt template Descriptive analysis Prescriptive analysis Assistant-level instructions role and goal of the LLM definitions (e.g., waiting time causes), descriptions of all data inputs Redesign patterns structured dataset containing 60 redesign patterns aimed at temporal improvement Message-level instructions preferred data format instructions for analyzing and presenting improvement opportunities (what aspects must be included in the response) Message-level instructions instructions for how the recommendations on redesign patterns should be provided (what aspects must be included in the response) [User prompt] LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 13

14. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Redesign patterns included in the dataset Redesign patterns Quantity Arrival time incentives, Capture data at source, Case buffering, Case reassignment, Case-based work, Contact reduction, Customer integration, Customer scheduling, Customer teams, Data composition, Data elimination, Data standardization, Empower, Exception, Extra resources, Fixed assignment, Flexible assignment, Follow-up, Fragment automation, Fragment elimination, IT system integration, IT system interfacing, IT system upgrade, Integral technology, Inventory buffering, Order types (Case types), Parallelism, Periodic action, Process automation, Process decomposition, Process generalization, Process standardization, Resequencing, Resource scheduling, Role independence, Specialist, Task automation, Task composition (combination), Task decomposition, Task elimination, Triage. 41 patterns Buffer information, Establish standardized interfaces, Information buffering, Order assignment, Resource centralization, Trusted party. 6 patterns Automate for environmental impact, Department-based assignment, Experience-based task assignment, Expertise-based task assignment, First-contact problem resolution, Performance-based task assignment, Role-based task assignment, Task delegation, Teamwork-based assignment, Workload-based task assignment. 10 patterns Batch strategy optimization, Prioritization strategy optimization, Resource schedule optimization. 3 patterns Total 60 patterns 14

15. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Excerpt from the dataset of redesign patterns 15

16. Prompting method LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 16

17. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Conversational interface workflow 17

18. Application on a procure-to-pay log User: Where is the highest waiting time in this process on average? Analyze it. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method 18

19. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Application on a procure-to-pay log User: How to reduce waiting time in this transition? 19

20. Evaluation: Descriptive analysis LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Computation evaluation Mean accuracy = 4.3 (out of 5.0) 20 350 questions 4 different event logs LLM as a judge Compared LLM’s answers vs ground truth on a scale

21. Evaluation: Descriptive analysis LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method User evaluation 21 12 participants 6.5 years on average experience in process analysis 4.5 years on average experience in process mining A survey to ask about usefulness/sufficiency/satisfaction

22. Response B (without redesign patterns) Response E (with redesign patterns) LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Evaluation: Prescriptive analysis 22

23. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Evaluation: Prescriptive analysis User evaluation - survey results 23

24. Conclusion LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Prompting method to fine-tune an LLM for analyzing waiting time causes and recommending targeted redesign options to waiting times. The core of this method is a structured dataset of redesign patterns. While the baseline is useful for high-level suggestions, the prompt-enhanced LLM offers more relevant and actionable redesigns that are particularly valuable for process analysts. Future work: repositories of validated prompts to facilitate process optimization across other KPIs, such as cost reduction. 24

25. LLM-Assisted Optimization of Waiting Time in Business Processes: A Prompting Method Katsiaryna Lashkevich, Fredrik Milani, Maksym Avramenko and Marlon Dumas BPM 2024 Thank you! 25