Anders Dinsen: Embrace Epic Failures and Make Testing Great Again

ConTEST NYC 2017

1. @andersdinsen

2. @andersdinsen

3. @andersdinsenKnight Capital Group’sEpic
Failure
•In August 2012, Knight Capital Group lost US$ 460
million
•Knight wasknownfor theirefficient high-frequency
tradingalgorithms
•However, a new tradingalgorithmput in production
leadto bankruptcyafteronly45 minutesin production
•Testingplayeda part as the accident happened because
a component used for testing was deployed
•The component simulated the market by generating
random market data for penny-stocks

4. @andersdinsen
Idea vs. reality at Knight

5. @andersdinsen
EpicFailure
•Epic comes from Greek epos meaning word. Its
source is PIE root *wekw-"to speak". Sanskrit: vakti
"speaks, says". Old Prussian wackis"cry".
•Failure comes from Old French "falir":"be lacking,
miss, not succeed; run out, come to an end; err,
make a mistake; be dying; let down, disappoint“
Source: ETYMONLINE.COM

6. @andersdinsen
Is testingfailingas a craft?
•Agile has been with us for
a long time and we have
adjusted
•But we see new types of
organizations emerging
•Self-management is
becoming the key to
success
•But individualization isn't
healthy to testing -or to
testers
•Our identities as testers
are threatened
•At the same time
automation is booming
•The complexity of
systems is growing
•We lack tools &
techniques to deal with
all this complexity

7. @andersdinsenA museum dedicatedto
failure

8. @andersdinsen

9. @andersdinsen

10. @andersdinsen
Is thisthe New Reality?
Failure
FailureFailure
Failure
Future?

11. @andersdinsenSomeFailuresareeasily
Understood
•Failures where I am
obviously the root cause
of what happens.
•There is no-one else to
blame.
•E.g.: I blow a tyreon my
car because I keep going
when the tyreis trapped
against the curb
•Failures where it was not
“thinking me” that
caused failure, but it’s
still pretty obvious what
happened
•E.g.: A glass slipped out
of my hand and crashed
•E.g.: under-maintained
MTA stock again delays
my train home -and I fail
to getting home in time
to my kids' school event

12. @andersdinsenSomeFailuresareHarder to
‘Get’
•They are understandable in hindsight only
•But the image is blurred and confusing
•"How could I have missed that?" we may ask
•But the picture is only clear after the failure
happened
•And different narratives always exists
•These types of failures often "emerge" from the
system
•A few of these failures may in fact lead to great
ideas much later

13. @andersdinsen
Black Swans
•Like the ‘Hard to Get’ failures, this type of failure is
random or uncaused
•But it has game-changing potential
•Libanesewriter and New Yorker Nassim Talebcalls
them ‘Black Swans’
•Knight Capital Group’s bankruptcy seems a good
example
•The EFI Scandal is obviously the same
•But the game changing event can still lead to
something new and better

14. @andersdinsenFailurechallengesour
human values
The True
The Just
The Beautiful
The Good
Freedom

15. @andersdinsenA Value-basedTypologyof
Failure
True Failures
Unfair FailuresUglyFailures
Black Swans
Success?
Innovation?

16. @andersdinsen

17. @andersdinsenA (hypothetical) typologyof ”gracial
failure”
True Failures
Just FailuresBeautifulFailures
Good Failures
InnovationFixes

18. @andersdinsen
How canwefailbetterin
today’scomplex
contexts?

19. @andersdinsenGetyourteam ”back in school”
with somebasic rules
•Be curious together
•Ask questions to
understand and learn –
not only to find answers
•Don’t make decisions
•People don’t have to
agree
•Don’t hurry

20. @andersdinsen
Brain storming
•The guywasdrunk
•Oh that’sstupid!
•He couldbebreakinga law
•Did the brakesnot workon
thatmotorbike?
•Wholooks afterthe
motorbike?
•It’skind of interestingthat
therewasa video camera
ready
•Wasthisdone onlyfor
entertainment?
•Whyaretheretworamps?
•Rampmakersshouldreally
put a disclaimernote on
the instructionsin order
not to getfined
•A general rule: Never ride
withouta helmet!
•Nice fall, by the way–he
shouldgeta job as a
stuntman
•Thinkthingsthrough–
always!

21. @andersdinsen
IntroducingCynefin

22. @andersdinsen
A CynefinProcedure
1.Walkthroughdifferentnarratives aboutit
•Rememberthatthereis not ”onetrue story” but youwant
to capturedifferentstories
2.Ask peopleto captureideasaboutthe narratives
whilelistening:
•Whathappened
•Possiblecauses
•Factors thatcontributedto makingit worse
•Purelyrandomevents thatcontributed

23. @andersdinsen
Organizethe ideas
1.Find the simplest, truestor most obviousideaand placeit in the
lowerright corner
2.Find the ideathatrepresentsthe ideathatrepresentsthe most unfair
thingaboutfailure, the thingwheresomethingoutsidewasmost
clearlyto blame, and placeit in the upper right corner
3.Now find the ideathatmost representsthe most uglyand surprising
thingaboutthe failure, the thingno-onehad anticipated, yetin
hindsightseemslogical, and placeit in the upper leftcorner
4.Finally, pickthe ideathatrepresentsthe most randomthingabout
the failure, the onethatcouldnot have beenpredictedas it wasand
still is completelyoutsideof anyone’scontrol, and placeit in the
lowerleftcorner.
5.Now, placethe rest of the post-it’srelative to the fouryoujust
placed. Imagine theyhave rubber-bands tiedto themand placethem
accordingly

24. @andersdinsenApplythisto yourTesting
Strategy
•Testingfor true failurescan(probably) be
automated
•Unfair failuresarea littlemore complicatedas we
needto go and talk to otherpeople
•Uglyfailuresrequireexplorativetestingand great
observation skills
•Randomfailuresareverydifficultto test for, but be
prepared

25. @andersdinsen
Tips aboutCommunication
•Most peoplehireexpertslike usto makethings
simple
•Let’snot overcomplicatethings
•True Failures: Pickanybestpractice
•Unfair failures: Talk to peopleand influencethem
•Uglyfailures: Advocatesomeresearch and
explorationof the problem
•Black Swans: Informmanagement aboutwhatwe
see–it’stheirjob to deal with these

26. @andersdinsen
A personalattitude to
failure

27. @andersdinsen
Personal Failures
•Failure is always with us
•Our brains and bodies remember failures far better
than success
•But the risk of failure can be a heavy burden on us
•And failure itself is one of the things that make people
want to escape a problem, burn bridges, and move on
•As testers, we’re in a way immune to failure
•But let’s be empathic about failures happening in our
projects
•And not forget that in projects we succeed together,
and we fail togehter

28. @andersdinsenKeep thinkingpositively:
E.g. look for perfects
@jlottosen

29. @andersdinsenBuilding Greatness… while
Failing
A ModernStoic’sApproach to Failure:
•Fix the things youcan fix
•Influence othersto fix what they
can fix
•Acceptthings that can’t be fixed
(now)
•Learn to knowthe difference