Estimation Maturity with NESMA Slides.pdf

In this webinar, the emphasis is on the software project maturity, and how organizations can improve their project success by becoming more mature in their processes.

1. Nesma webinar: Estimation Maturity
July 2024

2. www.nesma.org

3. Nesma - Software Measurement Standards and Improvement
Nesma is a non-profit international software measurement organization, founded in 1989, focused on:
▪Governing the Nesma standard for functional size measurement.
▪Promoting measurement and metrics based on functional size, which is the best proxy for business value.
▪Spreading knowledge aboutsoftware measurementandsoftware metrics.
▪Act asaBody of Knowledgefor the industry regarding the use of software metrics in all business areas.
▪Remainindependent, objective, andnot-for-profit.
▪Research the applicability of software metrics inall business areas.
▪Connect relevant organizationsin the industry that are expert in the areaswhere software measurement and
metrics are important.
▪Producerelevantguidelines, reports and other information productsthat are useful for the software industry.
▪Produce a platform where experts can discuss issues, they experience with software measurement and metrics
or where theycanexchange ideas and/or knowledge.
Nesma is Gold Partner of International Software Benchmarking Standards Group , partner in the International
Cost Estimation and Analysis Association (ICEAA) Software Special Interest Group and partner with the China
Software Process Improvement group.

4. © 2024 Galorath Inc.
Estimation Maturity

5. © 2024 Galorath Inc.
Agenda
DEFINITION
Explain Estimation Maturity and define Key Concepts
MOTIVATION
Why should we care?
THE FRAMEWORK
Presenting an example of a cost estimation maturity framework
SUCCESS STORY
An example of all of this applied in real life

6. Estimation Maturity
Defining Estimation
Maturity
•An indication of the degree to which an organization
has adopted estimation industry best practices to
continuously improve its estimation process through
consistent measurement and analysis
•Key Terms:
•Industry Best Practices
•Continuous Improvement
•Measurement and Analysis
•Estimation Maturity is measured/rated against a
framework or model (Similar to SEI Capability Model
Maturity Integration a.k.a CMMi)
6
© 2024 Galorath Inc.

7. Estimation Maturity
Estimation Industry Best Practices
•Developed through experience and
research
•Collaborative knowledge from multiple
sources
•Professional (organizations)
•Academic
•Literature
•Can be customized to fit a specific
organization and its environment
7
Best
Practices
Estimation
Methods
Sizing
Techniques
Tracking &
Control
Estimation
Lifecycle
Estimation
Processes
Confidence
Levels
Risk
Assessment
Historical Data
Collection
Measurement
& Analysis
Benchmarking
© 2024 Galorath Inc.

8. Estimation Maturity 8
Continuous Improvement
•Based on the traditional Deming Cycle
(Plan-Do-Check-Act)
•Ongoing effort to improve a product,
service or process
•Lessons learned and data are gathered
at the end of the process are fed back
for improvement and calibration
•The objective is to constantly learn and
adjust from experience
© 2024 Galorath Inc.

9. Estimation Maturity
Measurement &
Analysis
•Measure and collect metrics
(Effort, Cost, Schedule, Size,
Maintenance, Defects)
•Analyze the data to identify
trends and correlations
•Identify gaps and conduct
continuous improvement
9

10. Estimation Maturity
Why is Estimation Maturity Important?
•Estimation Maturity is synonymous with:
•Adoption of Industry Best Practices – What has brought good results
to others
•Formal sizing techniques and estimation methods (Function Points,
parametric models, historical data collection)
•Consistent and scalable processes
•Mature organizations are able to deliver better estimates:
•More reliable cost and schedule estimates
•Based on Industry Data - Justifiable
•Refined using historical data – Reflects the organization trend
13
A good estimate is a key condition for project success. Good
estimates are achieved via a mature estimation process
© 2024 Galorath Inc.

11. © 2024 Galorath Inc.
Galorath Cost Estimation
Maturity Framework

12. © 2024 Galorath Inc.
Estimation Maturity Model
15

13. Estimation Maturity
Level 1: Informal – No Process
•Key Characteristic:
•No estimation process at all
•Or Estimation performed in an inconsistent manner
•Home-grown methods such as spreadsheets and Ad-Hoc processes
•Inconsistent and hardly repeatable
16
GUESSING is the most widely used estimation method
Level 1
Primitive estimation
or no estimation at all
Manual effort
estimation without a
process
Ad-Hoc Processes/
Spreadsheets
© 2024 Galorath Inc.

14. Estimation Maturity
Estimating without a Process
•Estimate are a wild GUESS done by developers or Project Managers
•Inaccurate estimates reduce project success
•Poor Implementations
•Customer Dissatisfaction
•Lack of credible baseline
•Project is likely to overrun cost and/or schedule
•Organizations suffer from cancelled/failed projects, resulting in significant
economical impacts
•Poor estimates/plans are a root cause of project risk
17© 2024 Galorath Inc.

15. Estimation Maturity
Basic Estimation Tribes
•Napkins – Ad-Hoc, hero driven estimators… past successes are legendary… napkin
scribbling taken as gospel
•Guts – Estimates based on feelings/experiences. Trusted regardless of actual results
•Home-Grown Spreadsheets – Former Napkins and Guts translating tribal
knowledge into spreadsheets
•Bestows mathematical accuracy and empirical integrity on home-grown estimation
algorithms
•Tribal estimation knowledge can and does work – But it comes with high risk and cost
•Rarely repeatable and sporadically consistent
•Heroic energy is kept in reserve and used to mitigate risk
•Knowledge is never institutionalized and never leaves Desktop PC files
•Knowledge is gone when heroes retire or leave
18© 2024 Galorath Inc.

16. Estimation Maturity
© 2024 Galorath Inc.
Level 2 – Introduction to Formal
•Key Characteristic: First steps in adopting a formal sizing technique
•Simple CERs(Cost Estimation Relationships) and primitive use of
parametric models
•Processes are Informal and not institutionalized
20
Estimation bias mitigation begins at this level by introducing
a formal sizing approach
Level 2
Intro to Formal
Sizing (e.g.
Function Points)
Sizing applied
inconsistently
and/or
incorrectly
Simple CERs
(Size x
Productivity)
Basic
Measurement
Informal
Processes

17. Estimation Maturity
© 2024 Galorath Inc.
Sizing Metrics
•Many Sizing techniques are available nowadays.
•Organizations must choose based on goals and project nature
21
Software Type in this column
Is best characterized by this sizing technique(s)
SLOCsFunction
Points
Use Cases SNAP
Traditional Information TechnologyX X X X
Embedded/Algorithmic Processing X X X X
Code Generator X X X
COTS Integration X X X
Non-line based X X X
Cloud X X X X
Use ranges to best quantify size early

18. Estimation Maturity
© 2024 Galorath Inc.
Level 3 – Implemented
•Key Characteristic: Processes are formalized and standardized => Consistency
and Repeatability
•Basic Measurement and Analysis of Estimates vs Actuals
•Formal Sizing + Parametric Estimation = Key for better estimates
23
Estimation Bias mitigation is solid in this level with the
combination of Formal Sizing and Robust Parametric Estimation
Level 3
Formal Sizing (e.g.
Function Points)
Robust Parametric
Estimation
Basic Estimate vs.
Actuals Analysis
Measurement
& Analysis
Formal & Standardized
Processes

19. Estimation Maturity
© 2024 Galorath Inc.
Estimation Processes are Defined
•Define and Document the
estimation processes
•Workflows
•Checklists
•Roles and Responsibilities
•Entry and Exit Criteria
•Include interactions with
other areas of the
organization
•Train and Inform the Users
•Ensure processes are
followed (Estimation Quality
Assurance)
24
Example of an Estimation Process flow in BPM Notation.
Image courtesy of Galorath Inc. © 2018. All Rights Reserved.

20. Estimation Maturity
© 2024 Galorath Inc.
Learn and Improve from Metrics
•Show actual data, trends and
correlations
•Benchmark estimates
against the Data
•Use as “Sanity Check” to
validate the estimate
•Measurement and Analysis is
an ongoing process
25
Image courtesy of Galorath Inc. © 2024. All Rights Reserved.

21. Estimation Maturity
© 2024 Galorath Inc.
Estimation Lifecycle
When do we build estimates?
•Traditional Estimate “Tollgates”
•During Feasibility
•At Concept
•After Requirements
•After Design
•At the end (Post Mortem)
260
200
400
600
800
1000
1200
1400
1600
FeasibilityConcept RequirementsDesign-Code-
Test
Delivery
Effort
Phase
Estimate Convergence
Most
Least
Likely
Estimation is not a one-time
task. The estimate must be
revised throughout the
lifecycle as more (and better)
information becomes
available

22. Estimation Maturity
© 2024 Galorath Inc.
Estimation is not a one-time thing!
27

23. Estimation Maturity
© 2024 Galorath Inc.
Estimation Lifecycle
When do we build estimates (Agile)?
•Agile Estimate Tollgates
•Backlog Planning
•Before each release
•Before each sprint
28
Agile estimates are no
different. The Development
Method is part of the
solution, not the problem
Image courtesy of Galorath Inc. © 2018. All Rights Reserved.

24. Estimation Maturity
© 2024 Galorath Inc.
Level 4 – Refined
•Key Characteristic: Estimation Process improved via Lessons Learned and Data
Collection
•Estimation Processes and tools are defined throughout the organization (i.e.
Standardized and Institutionalized)
•Rigorous Measurement and Analysis
32
Level 4
Formal Sizing
Formal &
Standardized
Processes
Robust Parametric
Estimation
Rigorous
Measurement &
Analysis
Project Tracking
& Control
Process
Improvement via
Lessons Learned

25. Estimation Maturity
© 2024 Galorath Inc.
Lessons Learned Reviews
•Capture lessons at the end of an estimate and at the end of the project
•Document missing or incomplete information
•Capture risks, issues and problems addressed
•Document key decisions made during the estimation process – Capture
both, the Good and the Bad
•Document dynamics that occurred during the estimation:
•Interactions of the estimation team
•Interfaces with stakeholders
•Trade-offs made to address issues
33
Capture Lessons Learned ASAP while memories are still “Fresh”.
Every software project is an opportunity to improve the estimation
process

26. Estimation Maturity
© 2024 Galorath Inc.
Measurement & Analysis
•Collect Metrics (Effort, Cost, Duration, Size, Defects)
•Compare your estimates against the metrics (trend lines,
correlations)
•Identify gaps and adjust the estimation process and tools
accordingly
34
Your
estimate
Industry/
Historical
Trendline
Industry/
Historical
Data
Image courtesy of Galorath Inc. © 2018. All Rights Reserved.

27. Estimation Maturity
© 2024 Galorath Inc.
Using Data to compare against the Industry
•Measuring and Collecting data will also allow you to “see
where you are” compared to the Industry
•Understand the size of the applications and software portfolio
•How much am I spending in Development and Maintenance
compared to the Industry?
•How good is my productivity when compared to the Industry?
•Am I being Cost Efficient?
•How is the quality of my software (defect density) compared to
the Industry?
•Mature organizations need to be able to answer these types of
questions
36
Improving performance is vital for survival.
But… Performance can only be improved once
it can be measured

28. Estimation Maturity
© 2024 Galorath Inc.
Level 5 – Continuously Improved
•Quantitative Targets are established based on the strategic goals of the
organization
•Continuous Process Improvement is oriented towards these targets
•Detailed performance measures are collected and analyzed
•Total Cost of Ownership Estimates are used for Strategic Business Decisions
38
Level 5
Formal &
Standardized
Processes
Robust
Parametric
Estimation
Rigorous
Measurement &
Analysis
Project
Tracking &
Control
Continuous
Process
Improvement
Business
Strategy
Support

29. Estimation Maturity
© 2024 Galorath Inc.
Total Cost of Ownership (TCO) Costs
•Software Development
•Software Maintenance/Ongoing Support
•IT Infrastructure & Services (this can represent
60+% of TCO)
39

30. Estimation Maturity
© 2024 Galorath Inc.
Making Business Decisions via TCO and ROI
Quantify Benefits
•Savings
•Additional Revenue
Estimate TCO
•Development
•Maintenance
•Infrastructure
•Operations
Determine if
Project is Worth
•Sufficient ROI
•Aligned with Strategy
•Net Present Value
40

31. © 2024 Galorath Inc.
Success Story

32. © 2024 Galorath Inc.
43
•ABC contacted Galorath with a requirement
to enhance their cost estimation processes
for Software and IT projects (January 2022)
•Galorath conducted an Estimation Maturity
Assessment and provided a series of
recommendations
•The main objective was to equip ABC
with a mature cost estimation process in line
with industry best practices and GAO
recommendations.
Background
•Industry Leaders in Cost Estimation
•40+ years of experience
•Customers around the globe in multiple
industries
•Predictive cost analytics based on real industry
data
•Passionate about success

33. © 2024 Galorath Inc.
Estimation Maturity Assessment
44

34. © 2024 Galorath Inc.
Weaknesses
•No Data Collection
Process/Templates
•Many existing processes are
AD-HOC
•No Formal Sizing
•No estimate review process
•No Cost Estimation tools
•Absence of a Historical Data
Repository
•No Lessons Learned and
Feedback loop
45
Strengths
•Project Scope and business
requirements clearly identified
•Adherence to key PMBOK
practices
•Risk Management process in
place
•Project Plans are used (MS
Project and ServiceNow PPM)
•Estimates are updated
throughout lifecycle
Estimation Maturity Assessment Results for ABC

35. © 2024 Galorath Inc.
46
Main Components for Level 3:
1.Deploy a Cost Estimation Center of
Excellence (CECoE)
2.Establish consistent Estimation
Processes based on best practices
3.Formal Sizing. Galorath
recommends International Function
Points Users Group (IFPUG)
4.Historical Data Repository
(Estimates vs Actuals)
5.Formal Training
6.Continuous Improvement and
Calibration using lessons learned
The Journey to Estimation Maturity Level 3

36. © 2024 Galorath Inc.
Cost Estimation Process Improvement
47
We are
currently here!
The Roadmap to get there
Assess
Current State
Define Goal
Self-Assessment
Perform GAP
Analysis
Maturity
Objectives/Goals
Develop
Improvement Plan
What needs to
be improved
(processes,
tools, training)
Execute the
Plan
Capture
Metrics/Data
Cost
Estimation
Process
Process
Improvement
Plan

37. © 2024 Galorath Inc.
Estimation Process Improvement Plan
(EPIP)
48
Documents results
and findings from
the Estimation
Maturity
Assessment
Identifies the gaps
(areas that need
improvement)
Provides a
roadmap for
implementation

38. © 2024 Galorath Inc.
Cost Estimation Center of Excellence (CECoE)
Why?
CENTRALIZED KNOWLEDGE REPOSITORY
STANDARD ESTIMATION PRACTICES AND METHODS
ESTIMATION ACCURACY & PERFORMANCE MONITORING
HISTORICAL DATA MANAGEMENT
ESTIMATE VALIDATION AND APPROVAL

39. © 2024 Galorath Inc.
50
Estimation Process Workflow – Mapping to GAO Process
Step 1: Define Purpose
Step 2: Develop Plan
Step 3: Define the Program
Step 4: Estimate Structure (WBS)
Step 5: Ground Rules and AssumptionsStep 6: Obtain the Data
Step 7: Develop Point Estimate
Step 8: Sensitivity Analysis
Step 9: Risk & Uncertainty Analysis
Step 10: Document the Estimate
Step 11: Review and Approve
Step 12: Control and Update Estimate

40. New Nesma Guide: Easy Functional Sizing (EFS)
▪Especially for Agile working teams
(but others can use it as well!)
▪Only Logical Files, External input and
External outputs are measured!
▪Logical File: 7 FP
▪External input: 4 FP
▪External output: 5 FP
▪Positioned at the heart of the
Organization!
▪Efs.nesma.org

41. Elevate Your Expertise in Software Measurement & Estimation
Join the Global Community of Nesma Professionals!
•Access a network of experts:
•Learn from experienced practitioners.
•Collaborate on projects and initiatives.
•Gain valuable resources:
•Free Research publications, white papers, case studies.
•Free Standardized measurement methods and guidelines (FPA standard).
•High discounts on ISBSG data!
•Boost your career:
•Professional development opportunities & certifications.
•Increased industry recognition and credibility.
•Access and contribute to the ICEAA Software SIG.
•Expand your network:
•Connect with potential employers and clients on our free annual physical
networking event!
•Contribute to shaping the future of the field.
•Stay informed:
•Gain insights into industry trends and challenge.
•Sign up now: Membership (fee: 125 EUR per Year).
Join us June 6!!
De Soesterduinen, NL

42. ISBSG data and discounts for Nesma members
Nesma members get discount on ISBSG data subscriptions.
▪Developments & Enhancements: 11800 data points of new application developments and releases.
▪Maintenance & Support: 1921 data points of application Maintenance & Support.

▪Corporate Subscription: All (updates of) the Development & Enhancements data + all the Maintenance & Support data
▪Data Subscription: Subscription to only D&E or Only the M&S data.
▪Productivity Query Tool: High-level analysis tool on a subset of the D&E data.
▪All ISBSG reports are free for Nesma members.
Check for more information here:
https://nesma.org/publications/isbsg-data-and-reports/isbsg-project-data-subscription/ ISBSG offer Non-MemberNesma MemberDiscount%
ISBSG data subscription 1.830€ 1.525€ 305€ 17%
Corporate subscription 4.575€ 3.500€ 1.075€ 23%
PDQ tool 120€ -€ 120€

43. Thank you for attending this webinar!
▪
Haroldvanheeringen
Become a Nesma member now! Annual fee is only € 125
https://nesma.org/members/registration-form/
Connect with the Nesma community, including
free downloads of all digital products +
free access to a physical member meeting/network event!
large discounts on ISBSG data!
Nesma: http:// www.nesma.org
[email protected]