Lecture 2 - Reflex Agents and State space search.pptx

Reflex Agents

1. AI 202: Trends & Techniques in Artificial Intelligence Lecture 2 – Reflex Agents and State Space Representation Instructor: Dr. Hashim Ali Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi [Spring 2024]

2. A Quick overview of last lecture (Short) history of AI Current state of AI  brain vs computers Expectations from this course What can AI do? Applications of AI

3. Designing Rational Agents An agent is an entity that perceives and acts . A rational agent selects actions that maximize its (expected) utility . Characteristics of the percepts, environment, and action space dictate techniques for selecting rational actions This course is about: General AI techniques for a variety of problem types Learning to recognize when and how a new problem can be solved with an existing technique Agent ? Sensors Actuators Environment Percepts Actions

4. Pac-Man as an Agent Agent ? Sensors Actuators Environment Percepts Actions Pac-Man is a registered trademark of Namco-Bandai Games, used here for educational purposes

5. Agent Agent: a computer program or system that is designed to perceive its environment , make decisions and take actions to achieve a specific goal or set of goals . The agent operates autonomously, meaning it is not directly controlled by a human operator.

6. Agents Reactive Agents: respond to immediate stimuli from their environment and take actions based on those stimuli. Proactive agents: take initiative and plan ahead to achieve their goals. The environment in which an agent operates can also be fixed or dynamic. Fixed environments : static set of rules that do not change Dynamic environments: constantly changing and require agents to adapt to new situations.

7. Types of rational agents Simple reflex agents Table lookup approach; needs fully-observable environment. Model-based reflex agents Adds state information to handle partially observable environments. Goal-based reflex agents Adds concept of goals to augment knowledge to help choose best actions. Utility-based reflex agents Adds utility to decide “good” and “bad” with conflicting goals. Learning-based reflex agents Adds ability to learn situations that affect performance; decides how to change things to improve.

8. Simple reflex agents Agent Environment Percepts Actions Sensors Actuators What action should I do now ? What the world is like now ? Condition -action rules

9. Model-based reflex agents Agent Environment Percepts Actions Sensors Actuators What action should I do now ? What the world is like now ? Condition -action rules State How the world evolves What my actions do?

10. Goal-based reflex agents Agent Environment Percepts Actions Sensors Actuators What action should I do now ? What the world is like now ? Goals State How the world evolves What my actions do? What it will be like if I do Action A

11. Utility-based reflex agents Agent Environment Percepts Actions Sensors Actuators What action should I do now ? What the world is like now ? Utility State How the world evolves What my actions do? What it will be like if I do Action A How happy will I be in such a state ?

12. Learning agents Agent Environment Percepts Actions Sensors Actuators Performance element Critic Learning element Problem generator Performance standard Experiments Learning goals Feedback Changes Knowledge

13. AI 202: Trends & Techniques in Artificial Intelligence Lecture 2 – State space search Ghulam Ishaq Khan Institute of Engineering Sciences and Technology, Topi

14. This week Agents that Plan Ahead Search Problems Uninformed Search Methods Depth-First Search Breadth-First Search Uniform-Cost Search

15. Agents that Plan

16. Reflex Agents Reflex agents: Choose action based on current percept (and maybe memory) May have memory or a model of the world’s current state Do not consider the future consequences of their actions Consider how the world IS Can a reflex agent be rational? Example: Vacuum cleaner moving towards nearest dirt.

17. Reflex Agents

18. Reflex Agents

19. Planning Agents Planning agents: Ask “what if” Decisions based on (hypothesized) consequences of actions Must have a model of how the world evolves in response to actions Must formulate a goal (test) Consider how the world WOULD BE Optimal vs. complete planning Planning vs. replanning

20. Reflex Agents

21. Reflex Agents

22. Search Problems

23. Search Problems A search problem consists of: A state space A successor function (with actions, costs) A start state And a goal test A solution is a sequence of actions (a plan) which transforms the start state to a goal state “N”, 1.0 “E”, 1.0

24. Search Problems Are Models

25. What’s in a State Space? Problem: Pathing States: ( x,y ) location Actions: North South East West Successor: update location only Goal test: is ( x,y )=END Problem: Eat-N-Items States: {( x,y ), # of items} Actions: North South East West Successor: update location and possibly # of eaten items Goal test: # of eaten items = N The world state includes every last detail of the environment A search state keeps only the details needed for planning (abstraction) Problem: Eat-All-Dots States: {( x,y ), list of dots as bools} Actions: North South East West Successor: update location and possibly list of dots Goal test: dots all false

26. State Space Sizes? World state: Agent positions: 120 Food count: 30 Ghost positions: 12 Agent facing: NSEW How many World states? 120x(2 30 )x(12 2 )x4 States for pathing ? 120 States for eat-all-dots? 120x(2 30 )

27. Quiz: Safe Passage Problem: eat all dots while keeping the ghosts perma -scared What does the state space have to specify? (agent position, list of dots, list of power pellets, remaining scared time)

28. Examples – Travelling in Romania Formulate the search problem? (States, initial state, goal state, successor function(actions, path cost)).

29. Examples – Travelling Salesperson Suppose a salesperson has 5 cities to visit, and then must return home. Find the shortest path for the salesperson to travel, visiting each city and then return to the starting city.

30. Examples – 8 Queens (Chess) Problem Suppose you want to put 8 Queens on the chess board such that no two queens attack each other. What is a possible solution?

31. Examples – Spam email classifier States: settings of the parameters in our model Initial state: random parameter settings Actions: moving in parameter space Goal test: optimal accuracy on the training data Path Cost: time taken to find optimal parameters (Note: this is an optimization problem – many machine learning problems can be cast as optimization)

32. Examples – 8 Puzzle game states? initial state? actions? goal test? path cost? Initial state Final state

33. Examples – 8 Puzzle game states? locations of tiles initial state? given actions? move blank left, right, up, down goal test? goal state (given) path cost? 1 per move Initial state Final state

34. State space of 8 Puzzle game

35. Examples – Water jug problem You have a 4-gallon and a 3-gallon water jug. You have a faucet with an unlimited amount of water You need to get exactly 2 gallons in 4-gallon jug

36. Puzzle solving as search problems – Water Jug State representation: (x, y) x: Contents of four gallon y: Contents of three gallon Start state : (0, 0) Goal state (2, n) Operators Fill 3-gallon from faucet , fill 4-gallon from faucet Fill 3-gallon from 4-gallon , fill 4-gallon from 3-gallon Empty 3-gallon into 4-gallon, empty 4-gallon into 3-gallon Dump 3-gallon down drain , dump 4-gallon down drain

37. Production rules – Water jug 1 ( x,y )  (4,y) if x < 4 2 ( x,y )  (x,3) if y < 3 3 ( x,y )  (x – d,y ) if x > 0 4 ( x,y )  ( x,y – d) if x > 0 5 ( x,y )  (0,y) if x > 0 6 ( x,y )  (x,0) if y > 0 7 ( x,y )  (4,y – (4 – x)) if x + y ≥ 4 and y > 0 Fill the 4-gallon jug Fill the 3-gallon jug Pour some water out of the 4-gallon jug Pour some water out of the 3-gallon jug Empty the 4-gallon jug on the ground Empty the 3-gallon jug on the ground Pour water from the 3-gallon jug into the 4-gallon jug until the 4-gallon jug is full

38. Production rules – Water jug ( ctd ) 8 ( x,y )  (x – (3 – y),3) if x + y ≥ 3 and x > 0 9 ( x,y )  (x + y, 0) if x + y ≤ 4 and y > 0 10 ( x,y )  (0, x + y) if x + y ≤ 3 and x > 0 Pour water from the 4-gallon jug into the 3- gallon jug until the 3-gallon jug is full Pour all the water from the 3-gallon jug into the 4-gallon jug Pour all the water from the 4-gallon jug into the 3-gallon jug

39. One solution to the water jug problem Gallons in 4-gallon jug Gallons in 3-gallon jug Rule applied 2 3 9 3 2 3 3 7 4 2 5 2 9 2

40. References & Acknowledgements Partially adapted from lecture slides from Stanford University, UCIrvine , and UC Berkeley. Some videos taken from UC Berkeley website. Contents from George F. Luger, AI: Structures and strategies for complex problem solving, 6 th Ed.