Unit2: Agents and Environment

Artificial Intelligence agents and Environment

1. Unit 2: Agents and Environment LH 7
Presented By : Tekendra Nath Yogi
[email protected]
College Of Applied Business And Technology

2. Contd…
•Contents:
–2.1Agent,Rationalagent,andIntelligentAgent
–2.2Relationshipbetweenagentsandenvironments
–2.3Environmentsanditsproperties(Fullyobservablevs.partiallyobservable,
singleagentvs.multi-agent,deterministicvs.stochastic,episodicvs.
sequential,staticvs.dynamic,discretevs.continuous,knownvs.unknown)
–2.4Agentstructures
•2.4.1Simplereflexagents
•2.4.2Model-basedreflexagents
•2.4.3Goal-basedagents
•2.4.4Utility-basedagents
•2.4.5Learningagents
–2.5Performanceevaluationofagents:PEASdescription
212/2/2018 By: Tekendra Nath Yogi

3. Agent, Rational agent, intelligent agent
•Agent:
–Anagentisjustsomethingthatacts.
•RationalAgent:
–ARationalAgentisonethatactssoastoachievethebestoutcomeor,
whenthereisuncertainty,thebestexpectedoutcome.
•I.e.,onethatbehavesaswellaspossible.
–Howwellanagentcanbehavedependsonthenatureofthe
environment;someenvironmentsaremoredifficultthanothers.
312/2/2018 By: Tekendra Nath Yogi

4. Contd…
•Intelligentagent:
–ASuccessfulsystemcanbecalledintelligentagent.
–Fundamentalfacultiesofintelligenceare:Acting,Sensing,Understanding
,Reasoning,Learning
–Inordertoactintelligentagentmustsense.Blindactionsisnotcharacterizationof
intelligence.Understandingisessentialtointerpretthesensoryperceptsanddecide
onanaction.
–Therefore,Intelligentagent:Mustact,Mustsense,Mustbeautonomous,Mustberational.
–Note:intelligentagentmeansitdoesthingsbasedonreasoning,whilerationalagent
meansitdoesthebestaction(orreaction)foragivensituation.
–However,Throughoutthiscoursewewillusethetermagent,rationalagentand
intelligenceagentsynonymously.
412/2/2018 By: Tekendra Nath Yogi

5. Basic terminology
•Percept:Refer to the agent's perceptual inputs at any given instant.
•percept sequence:
–An agent's percept sequence is the complete history of everything the agent has
ever perceived.
–In general, an agent's choice of action at any given instant can depend on the entire
percept sequence observed to date.
•Agent Function:
–The agent function is mathematical concept that maps percept sequence to
actions(agent‟s behavior).
f : P* A
•Agent Program: The agent program is a concrete implementation of agent function
,running within some physical architecture to produce f.
512/2/2018 By: Tekendra Nath Yogi

6. What do you mean, sensors, percepts effectors and actions?
•ForHumans
–Sensors:
•Eyes(vision),ears(hearing),skin(touch),tongue(gestation),nose
(olfaction).
–Percepts:
•Atthelowestlevel–electricalsignalsfromthesesensors
•Afterpreprocessing–objectsinthevisualfield,auditorystreams.
–Effectors:
•limbs,digits,eyes,tongue,…..
–Actions:
•liftafinger,turnleft,walk,run,carryanobject,…
612/2/2018 By: Tekendra Nath Yogi

7. Agents and Environments
•Anagentisjustsomethingthatacts.
•Toactanagentperceivesitsenvironmentviasensorsandactsrationallyuponthat
environmentwithitseffectors(actuators).
•Thissimpleideaisillustratedinthefollowingfigure:
Fig: Agents interact with environments through sensors and actuators
Presented By: Tekendra Nath Yogi 7

8. Contd…
•ExamplesofAgent:
–Ahumanagent
•haseyes,ears,andotherorgansforsensorsandhands,legs,mouth,and
otherbodypartsforactuators.
–Aroboticagent:
•mighthavecamerasandinfraredrangefindersforsensorsandvarious
motorsforactuators.
–Asoftwareagent:
•receiveskeystrokes,filecontents,andnetworkpacketsassensoryinputs
andactsontheenvironmentbydisplayingonthescreen,writingfiles,and
sendingnetworkpackets.
812/2/2018 By: Tekendra Nath Yogi

9. Contd…
•Propertiesoftheagent:
–Anagentisjustsomethingthatact.Ofcourse,allcomputerprograms
dosomething,butcomputeragentareexpectedtodomore:
•Operateautonomouslyi.e.,canworkontheirown.
•Perceiveandreacttotheirenvironment.
•Pro-active(i.e.,shouldbegoaloriented)
•capableoftakingonanother„sgoal.
•Theyarepersistentoveraprolongedtimeperiod.And
•Adapttochangei.e.,Theyshouldhaveabilitytolearn.
912/2/2018 By: Tekendra Nath Yogi

10. Contd..
The vacuum-cleaner world: Example of Agent
•Toillustratetheintelligentagent,averysimpleexample-thevacuum-cleanerworld
isusedasshowninFigurebelow:
•Thisworldissosimplethatwecandescribe
everythingthathappens;it'salsoamade-up
world,sowecaninventmanyvariations.
•Thisparticularworldhasjusttwolocations:squaresAandB.
–I.e.Environment:squareAandB
•Thevacuumagentperceiveswhichsquareitisinandwhetherthereisdirtinthe
square.
–i.e.,Percepts:[locationandcontent] E.g.[A,Dirty]
•Itcanchoosetomoveleft,moveright,suckupthedirt,ordonothing.
–i.e.,Actions:left,right,suck,andno-op
Presented By: Tekendra Nath Yogi 10

11. Contd..
The vacuum-cleaner world: Example of Agent
•One very simple agent function is the following:
–if the current square is dirty, then suck, otherwise move to the other square.
•A partial tabulation of agent function is shown in Table below:
•A simple agent program for this agent function is given in the next
slide.
Presented By: Tekendra Nath Yogi 11

12. Contd..
The vacuum-cleaner world: Example of Agent
•FunctionVacuumAgent[location,Status]returnsanaction
–Ifstatus=DirtythenReturnsuck
–Elseiflocation=AthenReturnRight
–Elseiflocation=BthenreturnLeft
Presented By: Tekendra Nath Yogi 12

13. Good Behavior: The concept of rationality
•Arationalagentisonethatdoestherightthingi.e.,onethatbehavesas
wellaspossible.
•Rightthingistheonethatwillcausetheagenttobemostsuccessful.
–Forexample:Whenanagentisinanenvironment,itgeneratesasequenceof
actionsaccordingtotheperceptsitreceives.Thissequenceofactionscauses
theenvironmenttogothroughasequenceofstates.Ifthesequenceis
desirable,thentheagenthasperformedwell.
•Thisnotionofdesirabilityiscapturedbyaperformanceevaluation.
Presented By: Tekendra Nath Yogi 13

14. Contd.. Performance Measures
•Evaluates any given sequence of environment states and determine the success of
the agent.
•But, It is not easy task to choose the performance measure of an agent. Because the
performance measure doesn‟t depends on the task and agent but it depends on the
circumstances.
•Therefore, It is better to design Performance measure according to what is wanted in
the environment instead of how the agents should behave.
–E.g., The possible performance measure of a vacuum-cleaner agent could be amount of
dirt cleaned up, amount of time taken, amount of electricity consumed, amount of noise
generated, etc.
–But if the performance measure for automated vacuum cleaner is ―The amount of dirt
cleaned within a certain time. Then a rational agent can maximize this performance by
cleaning up the dirt , then dumping it all on the floor, then cleaning it up again , and so
on.
–So, “How clean the floor is” is better choice for performance measure of vacuum cleaner.
Presented By: Tekendra Nath Yogi
14

15. Contd…
•WhatisRationalityatanygiventimedependsonfourthings:
–Theperformancemeasurethatdefinethecriterionofsuccess.
–Theagent‟spriorknowledgeoftheenvironment
–Theactionsthattheagentcanperform
–Theagent‟sperceptsequencetodate.
Presented By: Tekendra Nath Yogi 15

16. Omniscience Versus rationality
•Anomniscientagentknowstheactualoutcomeofitsactionsandcanact
accordingly;butomniscienceisimpossibleinreality.
•Rationalityisnotthesameasperfection.Rationalitymaximizesexpected
performance,whileperfectionmaximizesactualperformance.
–ForExample:IamwalkingalongtheRingroadonedayandIseeanold
friendacrosstheroad.ThereisnotrafficnearbyandI'mnototherwise
engaged,so,beingrational,Istarttocrosstheroad.Meanwhile,at33,000
feet,acargodoorfallsoffapassingairliner,andbeforeImakeittothe
othersideoftheroadIamflattened.
Presented By: Tekendra Nath Yogi 16

17. Environments
•The first step to design a rational agent is the specification of its task
environment.
•Task environments are essentially the „problems‟ to which rational agents are
the „solutions‟.
•Generally task environments are specified by using the following four
parameter
–Performance
–Environment
–Actuators
–Sensors
•Therefore, task environment is also called PEAS description of the
environment.
Presented By: Tekendra Nath Yogi 17

18. Contd…
•Example:PEASdescriptionofthetaskenvironmentforanFullyautomatedtaxi
–Performance: The following can be the possible measures for the performance of
automated taxi:
•Getting to the correct destination(destination).
•Minimizing the fuel consumption and wear and tear(damage that naturally occurs
as a result of aging), and Minimizing the trip time or cost( Profit)
•Minimizing the violations of traffic laws and disturbances to other
drivers(legality)
•Maximizing the safety and passenger comfort ( Safety and comfort)
–Environment:This is the driving environment that the taxi will face
•Streets/freeways, other traffic, pedestrians, weather(rain , snow,..etc), police cars,
etc.
1812/2/2018 By: Tekendra Nath Yogi

19. Contd…
•Actuators:TheActuatorsforanautomatedtaxiincludethoseavailabletoa
humandriver:
–Steering,accelerator,brake,horn,speaker/display,…
•Sensors:Thebasicsensorsforthetaxiincludes:
–Oneormorecontrollablevideocamerastoseetheroad
–Infraredandsonarsensorstodetectthedistancestoothercarsandobstacles
–Toavoidspeedingtickets,thetaxishouldhaveaspeedometers
–Tocontrolthevehicleonthecurveitshouldhaveanaccelerometer
–Todeterminethemechanicalstateofthevehicle,itshouldhaveenginesensors.
–ItshouldhaveGPSsothatitdoesn‟tgetlost.
–Itshouldhavekeyboardormicrophoneforthepassengertorequestthedestination.
1912/2/2018 By: Tekendra Nath Yogi

20. Properties of environment (classes of environment)
•Followingarethedimensionsalongwhichenvironmentcanbecategorized:
–Fullyobservableversuspartiallyobservable
–Singleagentversusmulti-agent
–Deterministicversusstochastic
–Episodicversussequential
–Staticversusdynamic
–Discreteversuscontinuous
–Knownversusunknown.
2012/2/2018 By: Tekendra Nath Yogi

21. Contd…
•Fullyobservableversuspartiallyobservable:
–Ifanagent'ssensorsgiveitaccesstothecompletestateofthe
environmentateachpointintime,thenwesaythatthetask
environmentisfullyobservable.
•Forexample:chessplaying.
–Anenvironmentmightbepartiallyobservablebecauseofnoisyand
inaccuratesensors.
•Forexample:
–avacuumagentwithonlyalocaldirtsensorcannottellwhetherthere
isdirtinothersquares.
2112/2/2018 By: Tekendra Nath Yogi

22. Contd…
•Singleagentvs.multi-agent:
•Example:anagentsolvingacrosswordpuzzlebyitselfisclearlyina
single-agentenvironment,whereasanagentplayingchessisinatwo-agent
environment.
•Multi-agentenvironmentcanbe:
–Competitive:Forexample,inchess,theopponententityBistryingto
maximizeitsperformancemeasure,which,bytherulesofchess,minimizes
agentA'sperformancemeasure.Thus,chessisacompetitivemulti-agent
environment.
–Cooperative:Inthetaxi-drivingenvironment,ontheotherhand,avoiding
collisionsmaximizestheperformancemeasureofallagents,soitisapartially
cooperativemulti-agentenvironment.
2212/2/2018 By: Tekendra Nath Yogi

23. Contd…
•Deterministicvs.stochastic:
–Ifthenextstateoftheenvironmentiscompletelydeterminedbythe
currentstateandtheactionexecutedbytheagent,thenwesaythe
environmentisdeterministic;otherwise,itisstochastic.
–ThesimplevacuumworldisdeterministicwhereastheTaxidrivingis
clearlystochasticinthissense,becauseonecanneverpredictthe
behavioroftrafficexactly;moreover,one'stiresblowoutandone's
engineseizesupwithoutwarning.
2312/2/2018 By: Tekendra Nath Yogi

24. Contd…
•Episodicversussequential:
–Inepisodicenvironments,thechoiceofactionineachepisodedepends
onlyontheepisodeitselfi.e.,thenextepisodedoesnotdependonthe
actionstakeninpreviousepisodes.
•Forexampleanagentthathastospotdefectivepartsonanassemblyline
baseseachdecisiononthecurrentpart,regardlessofpreviousdecisions;
moreover,thecurrentdecisiondoesn'taffectwhetherthenextpartis
defective.
–Insequentialenvironments,ontheotherhand,thecurrentdecision
couldaffectallfuturedecisions.
•Forexample:Chessandtaxidrivingaresequential
2412/2/2018 By: Tekendra Nath Yogi

25. Contd…
•Staticvs.dynamic:
–Iftheenvironmentcanchangewhileanagentisdeliberating,thenthe
environmentisdynamicforthatagent;otherwise,itisstatic.
–Staticenvironmentsareeasytodealwithbecausetheagentneednot
keeplookingattheworldwhileitisdecidingonanaction,norneedit
worryaboutthepassageoftime.
•ForExample:Crosswordpuzzlesarestatic.
–Dynamicenvironments,ontheotherhand,arecontinuouslyaskingthe
agentwhatitwantstodo;ifithasn'tdecidedyet,thatcountsasdeciding
todonothing.
•Forexample:Taxidrivingisdynamicbecausetheothercarsandthetaxi
itselfkeepmovingwhilethedrivingalgorithmdithersaboutwhattodonext.
2512/2/2018 By: Tekendra Nath Yogi

26. Contd…
•Discretevs.continuous:
–Thediscrete/continuousdistinctioncanbeappliedtothestateofthe
environment,tothewaytimeishandled,andtotheperceptsand
actionsoftheagent.
•Forexample,adiscrete-stateenvironmentsuchasachessgame
hasafinitenumberofdistinctstates.Chessalsohasadiscreteset
ofperceptsandactions.
•ExampleofcontinuousstateenvironmentincludesTaxidriving:
thespeedandlocationofthetaxisweepthrougharangeof
continuousvaluesanddososmoothlyovertime.
2612/2/2018 By: Tekendra Nath Yogi

27. Contd…
•Knownversusunknown:
–Thisdistinctionrefersnottotheenvironmentitselfbuttotheagent‟s
stateofknowledgeabouttheenvironment.
–Inaknownenvironment,theoutcomesforallactionsaregiven.
–Obviously,iftheenvironmentisunknown,theagentwillhavetolearn
howitworksinordertomakegooddecisions.
2712/2/2018 By: Tekendra Nath Yogi

28. The structure of the agents
•Agent‟s structure can be viewed as:
–Agent = Architecture + Agent Program
•Architecture = the machinery that an agent executes on.
•Agent Program = an implementation of an agent function.
Presented By: Tekendra Nath Yogi 28

29. Contd…
•Agentsaregroupedintofiveclassesbasedontheirdegreeof
perceivedintelligenceandcapability:
–simplereflexagents
–model-basedreflexagents
–goal-basedagents
–utility-basedagents
–learningagents
2912/2/2018 By: Tekendra Nath Yogi

30. Contd…
•Simplereflexagents:
–Simplereflexagentsactonlyonthebasisofthecurrentpercept,
ignoringtherestofthepercepthistory.
•Forexample:vacuumcleaneragent.
–Firstofallthesimplereflexagentperceivestheperceptsfromthe
environmentandtheagentinterpretinputtogenerateanabstractstate
descriptionofthecurrentstatefromthepercept.
–Thisgeneratedstatedescriptionisthenmatchedagainstthecondition
partoftherulesintheruleset.
–Thenitactaccordingtoafirstrulewhoseconditionmatchesthecurrent
state,asdefinedbythepercept.
3012/2/2018 By: Tekendra Nath Yogi

31. Contd…
•The following figure shows the structure of the simple reflex agent
Fig: simple reflex agent
Presented By: Tekendra Nath Yogi 31

32. Contd…
•Characteristics:
–simple,butverylimitedintelligence.
–Thesimplereflexagentworkonlyiftheenvironmentisfully
observable.Evenalittlebitofunobservabilitycancauseserious
trouble.
–Lackinghistory,easilygetstuckininfiniteloops
3212/2/2018 By: Tekendra Nath Yogi

33. Contd…
•ModelBasedReflexagent:
•Maintainainternalstatetokeeptrackofpartofworlditcannotsee
now.
•Internalstateisbasedonpercepthistoryandkeepstwokindsof
knowledge:
•how the world evolves independently of the agent
•How the agent's own actions affect the world
•Thenitcombinescurrentperceptwiththeoldinternalstatetogenerate
theupdateddescriptionofthecurrentstate.
•Itthenchoosesanactioninthesamewayasreflexagent.
3312/2/2018 By: Tekendra Nath Yogi

34. Contd…
•Thefollowingfigureshowsthestructureofthemodelbasedreflexagent
Fig:Modelbasedreflexagent
3412/2/2018 By: Tekendra Nath Yogi

35. Contd…
•GoalBasedagent:
–Goal-basedagentsfurtherexpandonthecapabilitiesofthemodel-
basedagents,byusing"goal"information.
–Goalinformationdescribessituationsthataredesirable.Thisallowsthe
agentawaytochooseamongmultiplepossibilities,selectingtheone
whichreachesagoalstate.
–Itismoreflexiblebecausetheknowledgethatsupportsitsdecisionsis
representedexplicitlyandcanbemodified.
3512/2/2018 By: Tekendra Nath Yogi

36. Contd…
•Thefollowingfigureshowsthestructureofthegoalbasedagent
Fig:Goalbaseagent
3612/2/2018 By: Tekendra Nath Yogi

37. Contd…
•Utilitybasedagent:
–Goal-basedagentsonlydistinguishbetweengoalstatesandnon-goal
states.
–Itispossibletodefineameasureofhowdesirableaparticularstateis.
Thismeasurecanbeobtainedthroughtheuseofautility
functionwhichmapsastatetoameasureoftheutilityofthestate.
–Amoregeneralperformance(forexample,speedandsafety)measure
shouldallowacomparisonofdifferentworldstatesaccordingto
exactlyhowhappytheywouldmaketheagent.Thetermutilitycanbe
usedtodescribehow"happy"theagentis.
3712/2/2018 By: Tekendra Nath Yogi

38. Contd…
•Thefollowingfigureshowsthestructureoftheutilitybasedagent
Fig:utilitybaseagent
3812/2/2018 By: Tekendra Nath Yogi

39. Contd…
•Learningagents:Alearningagentcanbedividedintofour
conceptualcomponents:
–"learningelement",whichisresponsibleformakingimprovements
–"performanceelement“(entireagent),whichisresponsibleforselecting
externalactions.i.e.,ittakesinperceptsanddecidesonactions.
–Thelearningelementusesfeedbackfromthe"critic"onhowtheagentis
doinganddetermineshowtheperformanceelementshouldbemodifiedto
dobetterinthefuture.
–Thelastcomponentofthelearningagentisthe"problemgenerator".Itis
responsibleforsuggestingactionsthatwillleadtonewandinformative
experiences.
3912/2/2018 By: Tekendra Nath Yogi

40. Contd…
•followingfigureshowsthestructureoftheLearningagent
Fig:Learningbaseagent
4012/2/2018 By: Tekendra Nath Yogi

41. Applications of the agents
•Intelligentagentsareappliedasautomatedonlineassistants,wherethey
functiontoperceivetheneedsofcustomersinordertoperform
individualizedcustomerservice.
•Suchanagentmaybasicallyconsistofadialogsystem,aswellanexpert
systemtoprovidespecificexpertisetotheuser.
•Theycanalsobeusedtooptimizecoordinationofhumangroupsonline.
4112/2/2018 By: Tekendra Nath Yogi

42. Homework
•Defineinyourownwordsthefollowingterms:agent,agentfunction,agent
program,rationality,autonomy,reflexagent,model-basedagent,goal-
basedagent,utility-basedagent,learningagent.
•Boththeperformancemeasureandtheutilityfunctionmeasurehowwell
anagentisdoing.Explainthedifferencebetweenthetwo
•Whatisthedifferencesbetweenagentfunctionsandagentprograms.
•Whatanagentcomprisesof?
•Whatarethevarioustaskenvironments?
12/2/2018 By:TekendraNathYogi 42

43. Thank You !
43By: Tekendra Nath Yogi12/2/2018