DevOps and Testing slides at DASA Connect

My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.

1. DevOps and Testing
Kari Kakkonen, Dragons Out Oy
https://www.linkedin.com/in/karikakkonen/
Rik Marselis, Sogeti
https://www.linkedin.com/in/rik-marselis/
DASA Connect 30.5.2024
1© CopyrightDragons Out Trainings 2024

2. ROLES
•Dragons Out Oy, Director of Training, Trainer
and Coach
•Children’s and testing author at Dragons Out
Oy
•TMMi, Board of Directors
•Treasurer of Finnish Software Testing Board
(FiSTB)
ACHIEVEMENTS
•Tester of the Year in Finland 2021
•EuroSTAR Testing Excellence Award 2021
•Exemplary DevOps Instructor Award 2023 by
DASA
•ISTQB Executive Committee 2015-2021
•Influencing testing since 1996
•Ranked in 100 most influential IT persons in
Finland (Tivi magazine)
•Great number of presentations in Finnish and
international conferences
•TestausOSY/FAST founding member.
•Co-author of Agile Testing Foundations book
•Regular blogger in Tivi-magazine
•Growing Knowit to Testing Leader 2002-2024
Kari Kakkonen, Trainer and Coach
SERVICES
•ISTQB Advanced, Foundation, Agile Testing, AI Testing
•Knowit Quality Professional
•DASA DevOps
•Quality & Test process and organization development,
Metrics, TMMi and other assessments
•Agile testing, Scrum, Kanban, Lean
•Leadership
•Test automation, Mobile, Cloud, DevOps, AI
•Quality, cost, benefits
EDUCATION
•ISTQB Expert Level Test Management & Advanced Full
& Agile Tester certified
•DASA DevOps, Scrum Master and SAFe certified
•TMMi Professional, Assessor, Process Improver certified
•SPICE provisionary assessor certified
•M.Sc.(Eng), Helsinki University of Technology (present Aalto
University), Otaniemi, Espoo
•Marketing studies, University of Wisconsin-Madison,
the USA.
BUSINESS DOMAINS
Wide spread of business domain knowledge: Embedded,
industry, public, training, telecommunications, commerce,
Insurance, banking, pension.
2
twitter.com/kkakkonen
dragonsout.com
act2lead.net
MORE INFORMATION
linkedin.com/in/karikakkonen/
© CopyrightDragons Out Trainings 2024

3. 3

4. Agenda
•What is testing?
•Testing in Agile
•Testing in DevOps
•Workshop
4© CopyrightDragons Out Trainings 2024

5. What is Testing
5© CopyrightDragons Out Trainings 2024

6. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
What is testing?
●Testing = checking + exploring (of the test object and its environment)
○Test object: software, service, code, API, document, data, business idea, understanding…
●Reactive work –done for the test object that exists (partly or fully)
○Testing can be done anytime, as it is not a phase
●Improves quality indirectly by providing relevant information to stakeholders for better decision making
(about quality)
○Information: bugs, observations, risks, improvement ideas…
○Decision about readiness to deploy, how to improve product, bugs to be fixed, schedule…

7. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
What is quality?
●Quality = subjective value to a person who matters (at given time)
○Value is not only monetary value
●Different people have perspectives to quality, for example:
○Supplier (subcontractor): Product works according to predefined requirements
○User: Product is easy and fast to use
○Developer: Product’s source code is easy to understand and maintain
○Tester: Product is easy to test (by humans and automation)
●Focus to good enough quality and improve quality all the time
●Product’s external quality is not enough as we need also focus to product’s internal quality, as well to
team and organizational level quality
●Make sure people knows the quality characteristics and that you need to clarify those regularly, as
those will change as time goes by
●Quality = the ability of a product or service to satisfy STATED or IMPLIED needs

8. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
Quality characteristics (by Rikard Edgren)
http://thetesteye.com/posters/TheTestEye_SoftwareQualityCharacteristics.pdf
●Capability:Can the product perform
valuable functions?
●Reliability: Can you trust the product in
many and difficult situations?
●Usability: Is the product easy to use?
●Charisma: Does the product have “it”?
●Security: Does the product protect against
unwanted usage?
●Performance: Is the product fast enough?
●IT-bility: Is the product easy to install,
maintain and support?
●Compatibility: How well does the product
interact with software and environments?
●Supportability: Can customers’ usage and
problems be supported?
●Testability: Is it easy to check and test the
product?
●Maintainability: Can the product be
maintained and extended at low cost?
●Portability: Is transferring of the product to
different environments enabled?

9. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
ISO 25010 quality characteristics

10. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
What is quality engineering or quality assurance (QA)?
●Quality engineering is about team members and their stakeholders taking joint responsibility to continuously
deliver IT systems with the right quality at the right moment to the businesspeople and their customers. It is a
principle of software engineering concerned with applying quality measures to assure the quality of IT
systems.
●QA = preventive and process-oriented work
○ QA is not the same as testing → Different skills and people may be needed
●Aims to prevent human errors and mistakes causing bad quality
○ For examplemisunderstandings, time pressure and poor skills can cause bugs
●Examples of QA activities:
○Processimprovements
○Reviews, static testing
○Collaborative product creation
○Standards (into coding, testing, etc.)
○Assessments, retrospectivesandlessons learned
○Check if the processes are used
●QA can notassure quality, but it can improve quality indirectly (like testing)
●Quality culture matters as QA is part of quality management (like testing)

11. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
How to make testing diverse: Test levels, test types
Code review
Static code
analysis
Software
Composition
Analysis
Unit
testing
API
testing
Integration
testing
Reviews
Functional
testing
User story
acceptance
testing
User
Acceptance
Testing
Alfa and beta
testing
A/B testing
Testing in
production and
monitoring
Usability
testing
Accessibility
testing
Performance
testing
Security
testing
Browser and
device testing
Other
test types

12. Testing in Agile
12© CopyrightDragons Out Trainings 2024

13. •Testers must understand the values
and principles that underpin Agile
projects
•Testers are an integral part of a
whole-team approach together with
developers and business
representatives
•Testers and other members in an
Agile project communicate with each
other early and frequently, which
helps with removing defects early and
developing a quality product
Agile Tester Works Differently
© Knowit Trainings v. 1.3 -202303 1-13

14. •The whole team, the testers,
developers, and the business
representatives, is responsible for
quality by working together in every
step of the development process
•Testers will work closely with both
developers and business
representatives to ensure that the
desired quality levels are achieved
•to support and collaborate with
business representatives to help
them create suitable acceptance
tests,
•to agree on the testing strategy with
developers
•to decide on test automation
approaches.
Whole Team Approach Responsibilities
© Knowit Trainings v. 1.3 -202303 1-14
Tester
Business
Stakeholders
Developer
Acceptance
tests
Test
Strategy
Automation
Approaches
Collaboration

15. Increment
completed
•Agile projects have short iterations enabling the project team to receive early and
continuous feedback on product quality throughout the development lifecycle
•By getting frequent customer feedback as the project progresses, Agile teams can
incorporate most new changes into the product development process
•Testing takes place all the time, and theirs is lot or repeated tests (regression testing)
Early and Frequent Feedback
© Knowit Trainings v. 1.3 -202303 1-15
Analyze
Design
Code
Testing
Iteration 1
Analyze
Design
Code
Testing
Iteration 2
Analyze
Design
Code
Testing
Iteration 3
Analyze
Design
Code
Testing
Iteration n
Customer
feedback
Increment
completed
Customer
feedback
Increment
completed
Customer
feedback
Product
completed
Customer
feedback
. . .

16. Testing Quadrants, Test Levels, and Testing Types
© Knowit Trainings v. 1.3 -202303 3-16
Source: Lisa Crispin and
Janet Gregory
Functional tests
Examples
Story tests
Prototypes
Simulations
Exploratory testing
Scenarios
Usability testing
UAT (User Acceptance Testing)
Alpha/beta
Unit Tests
Component Tests
Performance & load testing
Security testing
”-ility” testing
Automated &
manual
Automated
Manual
Tools
Technology facing
Business facing
Guide development
Critique Product

17. Testing in DevOps
17© CopyrightDragons Out Trainings 2024

18. Copyrights © Marko Rytkönen and Kari Kakkonen
www.act2lead.net
REACTIVE WORK
Testing
PROACTIVE WORK
Quality Assurance
PASSIVE WORK
Monitoring, surveys…
SHIFT TESTING RIGHTSHIFT TESTING LEFT
Shift left and shift right

19. We test everywhere in DevOps
© CopyrightDragons Out Trainings 2024 19
Source: Dan Ashby

20. Holistic testing –we test everywhere
© CopyrightDragons Out Trainings 2024 20
Source: Lisa Crispin and
Janet Gregory

21. TMAP quality engineering topics
© CopyrightSogeti 2020 21

22. Workshop
22© CopyrightDragons Out Trainings 2024

23. Workshop
•Divide into breakout rooms
•Each breakout room has equivalently
numbered area in Mural
•Each area has one section per DevOps
infinity loop (Plan, Branch…)
•We have several sessions brainstorming
and discussing how quality and/or testing
can be taken into accountin each section,
and come back main room to discuss
together
•The result can be greater than the
written knowledge on the topic so far –
we use the power of cross-functional
teams!
© CopyrightDragons Out Trainings 2024 23
Source: Dan Ashby

24. Sources
•https://danashby.co.uk/2016/10/19/continuous-testing-in-devops/
•https://agiletestingfellow.com/blog/post/holistic-testing-what-it-
means-for-agile-teams
•https://tmap.net/
•https://www.istqb.org/certifications/agile-tester
•Gregory,J.,Crispin,L.(2014).More Agile Testing: Learning Journeys
for the Whole Team.United Kingdom:Pearson Education.
•https://www.dasa.org/products/talent-products/certification-
programs/dasa-devops-professional-specify-and-verify/
© CopyrightDragons Out Trainings 2024 24

25. Any questions?
DASA DevOps Professional Specify & Verify
●https://www.dasa.org/products/talent-products/certification-
programs/dasa-devops-professional-specify-and-verify/
Follow and share Kari:
●https://www.linkedin.com/in/karikakkonen/
●https://www.dragonsout.com
●https://www.act2lead.net/
●[email protected]
Follow and share Rik:
●https://www.linkedin.com/in/rik-marselis/
●https://www.tmap.net/
●[email protected]
25© CopyrightDragons Out Trainings 2024