Estimation is dead - Tbilisi, by John Coleman 26 April 2024 final.pdf

In this Agile International Summit opening keynote talk in Tbilisi, Gerogia, John Coleman explored relative sizing (story points, t-shirt size, time reference), right sizing, and dynamic design to cost. It's an of the InfoQ paper "sizing and forecasting in Scrum" except this talk was not focused on only Scrum.

1. Estimation is dead?
John Coleman
@JohnColemanIRL
https://linktr.ee/johncolemanxagility
https://www.infoq.com/articles/sizing-
forecasting-scrum/

2. Complex Work
Where More
than Expertise
is Needed
Handling it requires:
•Fresh thinking
•Experimentation
•Openness to “I wonder if” old solutions to
old problems will work on the new one.
•Trust
•Frequent adjustment

5. Sizing caveats
People who do the work do the sizing, no one else!
If we don’t ”clean up up the kitchen” as a habit, work will take longer than before
The most popular sizing techniques are either based on data or educated guesses

6. Sizing options we’ll talk about today
Relative estimation (time, story points, t-shirt)
Counting the number of valuable items delivered (throughput)
Rightsizing
#Noestimates
Dynamic Design To Cost

7. Sizing is devalued by
•Not having caveats associated with the start date,
e.g., nine weeks from the date we start
•Not recognizing the amount of work in progress
and the progress of that work
•The severity of impediments
•Not ordering items higher up the backlog according
to delivery risk
•A sub-optimal approach to handling dependencies
•Confusing outputs with outcomes
•Not engaging in discovery activities when the risk
of not harvesting potential value is high
•Delusions of accuracy and pursuing more accuracy

9. Story points
Potential upsides
Useful to avoid bringing “elephants” into WIP
Could be used to limit work in progress
Easy to pick reference items from the past
Simple to do
Developers like the conversation it triggers
Often paired with t-shirt sizing or wall estimation
Potential downsides
Creator regrets story points
Only for the team
Story point inflation
BS story points
Often paired with planning poker (time consuming)

10. T-shirt sizes
Potential upsides
Useful to avoid bringing “elephants” into Work In
Progress
Could be used to limit work in progress
Easy to pick reference items from the past
Developers like the conversation it triggers
Simple to do
Requires very little detail
Potential downsides
Converted to numbers quite often, numbers that get
used to forecast when work might be done

11. Time reference
Potential downsides
Requires suitable reference items from the past
Prone to abuse be people with a focus on people
utilization
Unsuitable for probabilistic forecasting
Potential upsides
Speaks in the customers language
Easy to pick reference items from the past
Waiting time is included in our memory of how long it
takes
Simple to do

13. Rightsizing
Potential upsides
Simple
Less “analysis paralysis”
Supports recurring probabilistic forecasting
Potential downsides
Items right-sized just in time or in product backlog
refinement
Misunderstood that all right sized items must be of
equal size
Disconnect in Kanban community about use of item
split rate to support probabilistic forecasting
If most days a team has no throughput, probabilistic
forecasting will have low quality

14. Wall / table estimation
Potential upsides
Useful to avoid bringing “elephants” into Work In Progress
Could be used to limit work in progress
Easy to pick reference items from the past
Developers like the conversation it triggers
Simple and quick to do; requires very little detail
Guesstimate for potential value sized as well as effort typically,
priming ordering for value divided by size
Potential downsides
Converted to numbers quite often, numbers that get
used to forecast when work might be done
Often one and done –should be revisited regularly

16. Variable quality with sizing an item
Factors for how long things take
The batch size –the level of effort actually needed
Waiting
time
…
Sizing for the level of effort considers
Complexity of the work
Riskiness of the work
Whether we did something similar before
Perception of skill levels required to complete the work and
availability of those skills
Availability of tools and skills using those tools
Dependencies

17. 100% Resource utilization = 0% flow
(Henrik Kniberg)
Image courtesy of LeSS.works

19. Throughput

20. Guesstimating / counting the number/
range of items to deliver a goal
Potential upsides
•Suitable inputs for probabilistic forecasting
•Can be used across teams
Potential downsides
•People prefer relative sizing, and almost “cannot let go”
•Misunderstood that all items need to be of equal size
•Different product backlog item types
•Prone to the use of averages

21. Monte Carlo simulations
model a future based on
data and assumptions

22. Dynamic
Design to
Cost

23. Make the requirements less dumb
–Elon Musk

24. It appears that once (delivery leaders) know how to
get benefits right, they know how to get everything
right.
Bent Flyvbjerg – author of How Big Things Get Done

25. Keys to Success in
“How Big Things Get Done”
Look for similar initiatives in your
organization, across the world,
and assess the problems they had
and how long they took.
Aim to “Lego-ize” rather than
customize – you’re not so special!

29. Planguage Example
Unambiguous
•TAG NPS
•GIST Improve Net Promoter Scoreto what is
deemed good by NPS experts
•AMBITION Segment leading NPS in the region.
•STAKEHOLDER Product Manager
•CONSTRAINT {current moment, end-to-end-
customer-satisfaction-with-product}
•DEFINED
https://en.wikipedia.org/wiki/Net_promoter_score
•AUTHORITY Market Insights team
Clarity
•SCALE Net Promoter Score
•METER high to low NPS range previous 180 days
•NOW 5
•TOLERABLE >0
•GOAL 20
•STRETCH 30
•WISH 50
•PAST -30
•TREND -30 |<-| to +10 |-->| last 18 months

31. Credibility of costs or impact
1.Wild guess, no credibility
2.We know it has been done somewhere
3.We have one measurement somewhere
4.There are several estimates in the estimated range
5.Several measurements relevant for the use case (e.g. qualifier
values)
6.Several relevant measurements obtained using a reliable
method
7.Have used the same design / solution / experiment option
previously in your organization
8.Reliable in-house measurements of same design / solution
option / experiment option
9.In-house measurements of design / same design / solution
option /experiment option correlate to external sources
10.Have previously used this design / same design / solution
option / experiment option on this initiative and measured it
11.Have solid, contract-guaranteed, long-term experience of this
design or solution/experiment option on this initiative

32. •Plan - What can you deliver this week?
•Do it
•Study and Learn
•Act / Adapt / Adjust

33. Key take aways
Avoid story points, counting non-valuable
product backlog items, counting unDone
work as Done, use of averages
Avoid
Consider historical reference items but
beware ofaccidental complicationConsider
Try probabilistic forecasting based on
counting valuable product backlog items to
Done
Try
Try #NoEstimates and “rolling wave
forecasts” of valuable product backlog
items to Done
Try
For complex work, promotemanaging
expectations about uncertainty over
managing expectations about dates
Dynamic Design to Cost
Promote

34. Question
Time

35. Thank you
John Coleman
@JohnColemanIRL
https://linktr.ee/johncolemanxagility
https://www.infoq.com/articles/sizing-forecasting-
scrum/