The Joy of Testing - Deep Dive @ Devoxx Belgium 2024

We spend half of our coding time reading, writing or fixing tests. But who likes that? To quickly fix this chore, we often copy-cat older tests or write new ones without paying much attention to their quality. This deep-dive walks you through a variety of practical testing techniques and tricks you must have in your toolbox to write expressive tests with a higher signal/noise ratio. In the agenda: test structure & naming, mutation testing (vs AI-gen tests), ObjectMother, why never to use assertEquals, nested fixtures, parameterized use/abuse, Gherkin, plus a few surprises. All this through an entertaining and interactive session that puts the joy back into testing.

1. The Joy of Tes*ng
3-hours Deep Dive at
by Victor Rentea
Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24

2. UT 3
- Evan Spiegel
Ever had a great
refactoring idea
but you were afraid
to do it?

3. UT 4
Prove it works today, including edge cases
Fast Feedback to preserve focus: Start Clicking => Stop Thinking!
Refactor without fear tomorrow, be crea7ve, if func'onal-tests(elseJ)
Up-to-date Spec running on CI, if func'onal tests
Clarify requirements, if test-early
Design feedback: Hard to test ≈> poor design, if test-early
For coverage%, to get my PR approved "; Trap: Coverage Rush⚠
Why Write Tests?
✅
"
#
$
%
✨
%

4. ! I'm Victor Rentea " Java Champion, PhD(CS)
# 18 years in Java + most languages
$ 10 years of Training / 150+ companies / 20+ countries:
- Architecture, Refactoring, Unit Tes2ng (today)
- Spring, JPA, Performance, ReacIve
European SoKware CraKers Community (on meetup.com)
YouTube.com/vrentea - events% + conference&
Life += ' + ( + ) + 2**victorrentea.ro
I'm drawing on screen with:
> ScreenBrush (Mac)
> ZoomIt (Win)

5. UT 9!
Quali&es of a Unit Test

6. UT 10
§Sensible: detects changes in behavior
§Func7onal, not coupled to implementaEon (resilient to refactoring)
§Specific: fails for one clear reason
§Expressive: uses domain terms with a high signal/raEo raEo
§Automated: easy to run by anyone with no/minimal manual setup
§Repeatable: produces the same outcome if rerun
§Isolated vs other tests / external system state
§Minimal: includes only relevant stuff (#responsible copy-paste)
§Fast, total build Eme < 15-20 min
Quali0es of a Test
from the book "The Art of Unit Tes.ng"by Roy Osherove

7. UT 12Coverage

8. UT 13
Coverage Rush
Management Goal: Coverage > xx%'vs (Confused Developers or ☠Legacy
The more code in here,
the more untested code
can be in the project.!
Example of Cobra Effect
5000 lines

9. UT 14
Coverage %
Cost
Coverage%100%80%90%60%
= covrigi [RO]
cover critical code first
! Higher goals for 'domain' package
Minimum Coverage%

10. UT 15
Line vs Branch Coverage
§Line Coverage = did my tests executed a line?
§Branch Coverage = did my tests exercised a path? ! ! !
-if (condition) è 50% = means condi4on was only true (or false) during tests
-condition ? expr1 : expr2
-a && b || c è 100% branch coverage = a, b, c were all false + true in tests
-Measured via a -javaagent: that
monitors code executed by tests
IntelliJ setup

11. UT 16
tl;dr
Use Branch Coverage

12. UT 17
Professionals use coverage analysis
to hunt for UNCOVERED code,
NOT to celebrate COVERED lines

13. UT 18
Talk is cheap
Show the me the code_
Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24

14. UT 19
A new instance of the test class is created for each @Test.
So it's ok to leave dirty data in test class' instance fields.
#simplicity
#1 Basic People don't know about JUnit!

15. UT 20
Muta%on Tes%ng

16. UT 21
All tests are green. "#
You break some behavior,
but all tests stay Green
!
Evergreen Tests
Imagine 95% coverage...

17. UT 22
Evergreen Tests
Coverage% = ?
Oups!!
100%
If all tested code is deleted,
the test remains GREEN *
Da-i asa! vs Basculanta
!
lies to you

18. UT 23
§Prod coverage of class X.java = 87% !
§XTest.java = 2000-lines
§67 x @Mock
§200 x Mockito.when "
including when(aMock.getB()).thenReturn(bMock);
§0 x "assert" #
§0 x "verify" #
Cobra ! Effect

19. UT 24
The only way to prove
a test is correct?
tl;dr
it FAILS if (and only if)
you BREAK the tested behavior

20. UT 25
Muta3on Tes3ng
You inten1onally
introduce a bug in
Produc@on CodeYou get a
MUTANT
Tests should FAIL +
(proving they would catch that bug)
Run Tests
Don't forget⚠ to
Undo the bug
Correct tests fail<=>Produc@on Code is altered
The only way to prove
a test is correct?

21. UT 26
§Alters tested bytecode and checks tests fail
-Using a set of mutators, like "Negate Condi@onals"
§But⚠:
-Can report false posi@ves (non-issues)
-Takes long @me to run if applied on lots of code
§Use it:
-To learn
-On super-cri@cal code (target a single package)
Muta3on Tes3ngwith pitest

22. UT 27!"
Ever generated a test with AI (LLM)?
, Dream: Gen-AI tests (chaos) but then check them via muta@on-tes@ng (exact)
Lower entry-cost when tes@ng legacy -

23. UT 28
Check that every new test you write
fails for a bug in tested logic.
(hands-on if not obvious)

24. UT 30Bug-Tests

25. UT 31

26. UT 32
Breakpoint
using step-into, inspect,
evaluate, watch variable...
✅ Use for a 1st explora4on

27. UT 33
Breakpoint-debugging an issue is an<social behavior
Others entering that code
tomorrow will suffer too.
requires deep focus . and
drains lots of brain energy /
Breakpoint

28. UT 35
Fix a bug like a Pro
1.Reproduce the bug [twice]
2.Write a test for it è FAILS❌
(⚠can be very-very hard in complex, terrible legacy code)
3.log.debug("..{}", var) instead of breakpoint
4.Find the cause > Kill bug > Test è GREEN✅
Avoid vola1le
breakpoint sessions
You can later lower the log level in producIon without restart#:
using: JMX, Spring Actuator, or Logback config auto-reload to inves9gate unreproducible bugs

29. UT 36
Why Bother?
Bugs S7ck Together

30. UT 37
refactoring + tes4ng merge
in "islands of safety"
Test Around Bugs
bug
unit-tests around it
Tes4ng another nearby bug
becomes easier tomorrow
Bugs S1ck Together
Unit-tes4ng legacy code
requires some prep-cleaning

31. UT 38 h+p://googletes9ng.blogspot.com
https://drive.google.com/open?id=1YQNHvc0NIzspxKHG4BU9mjdOqDQdks9z

32. UT 40
Bugs in Tests

33. UT 41
A Bug in a Test
assertEquals("Greetings" + name, logic.getGreetings(name));
production codeit's missing a " "!
The developer copied the logic from produc@on to tests0
You want Explicit Tests
assertEquals("Greetings John", logic.getGreetings("John"));
Find the BUG !

34. UT 42
Tests should be Simple and Expressive,
NOT cryp4c, smarty, and hard to read

35. UT 43
Unmaintainable Tests
Can make developers :
Anger Zone
delete test
@Disable
// comment test
// @Test
delete asserts
flip asserts to pass

36. UT 45Test Name

37. UT 47
Test Name
void givenAnInactiveCustomer_whenHePlacesAnOrder_itIsActivated()
void givenAProgramLinkedTo2Records_whenUpdated_2AuditsAreInserted()
GivenThenWhen
!

38. UT 48
AnInactiveCustomerShould
.beActivatedWhenHePlacesAnOrder()
Brain-Friendly Test Names
Fluent naming
hFps://codurance.com/2014/12/13/naming-test-classes-and-methods/
"Given"
the class name explains
the context shared by testsTest
"Then"
start with the expected effect
to capture the reader a+en9on
"When"
... then describe in what condi9on it occurs

39. UT 49AssertJ

40. UT 50
assertEquals
Anyone?

41. UT 51
assertEquals
Never use again
Use assertThat(from AssertJ)

42. UT 54
§A few fields
assertThat(x.getA()).isEqualTo(y.getA());
assertThat(x.getB()).isEqualTo(y.getB());
§Almost all fields
assertThat(x)
.usingReflective(ignoringFields("a")).isEqualTo(y);
§All fields
assertThat(toPrettyJson(x))
.isEqualTo(toPrettyJson(y));
How to compare data structures?

43. UT 58
Parameterized
Tests

44. UT 59
Parameterized Tests – Best Prac&ces
§Orthogonal parameters - best (low mul4collinearity)
Example (for 3 input params):
-Good: a={1,2,3}, b={7,8}, c={T,F} = 3x2x2 = 12 validcombina4ons with 3 params
-Bad if only 3 valid combina4ons: (1,7,true); (1,8,false); (2,8,true)
§Few parameters < 3-5
-More => group them in a class/record with named fields
-Complex input structure * many tests => consider file-based tests
§Avoid boolean-riddled test
-if (param1) excep4on was thrown
if (param2) when(mock)..
if (param3) verify(mock)..

45. UT 60
Tes0ng StylesMaintainability
§Output-Based
-Assert the returned value
-(enough when tested code has no side effects, pure func4on")
§State-Based
-Assert fields of tested stateful object
-Assert fields of collaborator objects (eg. passed as params#)
-Assert external state in DB/Files
§CommunicaEon-Based (mocks)
-Verify interac4on with collaborators

46. UT 61
Approval Tests

47. UT 63
When you can't an4cipate the impact of a change.
§Capture all output of a component in a human readable file
-csv, json, xml, txt
§Change the component behavior.
§Diff 1the new output with the old output saved in files
§If correct => save the new output in the old file = "approve"
Approval Tes3ng

48. UT 64
Tes2ng Legacy

49. UT 65
1000
lines of code
in a single file
00

50. UT 67
Characterization Tests
§Dark Techniques to write temporary, ugly tests
-Break encapsula4on of tested code (private -> public)
-Par4al mock (@Spy) internal methods
-Mock sta4cs & control global state
§Pre-tesEng refactoring (unguarded by tests)
-Exploratory refactoring (eg 30-minute > undo!): look for seams ("fracture planes")
-Tiny, safe baby-steps refactoring with IDE, pairing (2) or mobbing (N)
§Extract and test code with a clear role = maintainable tests!
§CharacterizaEon Tests: capture input/output as a black-box
Legacy Code Tes&ng Prac&ces

51. UT 68
Characterization Tests
Some legacy code with no tests runs in production for many years.
Therefore, we can assume it is correct. = it became its own spec
But we need to refactor that code. # # # #
Our Goal = preserve its behaviour (correct or not), not to find bugs.
Step 1. Identify & capture the inputs/outputs of the prod code (HARD),
example IN= params, results of DB queries/API call, time/random/UUID!"
OUT= returned value, DB insert/update, API calls, sent messages!"
Step 2. Save {inputs + recorded outputs} to files.
Step 3. Identify many different inputs to cover all execution paths
Get line coverage close to 100%; ± manual mutation testing.
PRO$: Record IN/OUT from production = "Golden Master" Technique
(also highlights dead☠ code)
Step 4. Refactor guarded by these tests. When done, remove tests?

52. UT 69
Shadow TesFng
Prod env
Staging env
Clientstraffic
traffic
↓
↓
copy
↓
↓
↓
↓
compare
results
↑
↑

53. UT 74BDD
Libraries:
Cucumber
JBehave
BeHat (php)
SpecFlow (C#)
...Behavior-Driven-Development
(aka Acceptance-Test-Driven-Development)

54. UT 75
Have I understood correctly
what we need to build?

55. UT 76
Bridge the Gap
Make sure you implement the right thing
Gherkin Language can help

56. UT 78
Gherkin Language
PO
Business
// "glue" code can: configure mocks, DB, WireMock, set
fields, load files ...
@When("Player{int} scores {int} points")
public void playerScores (int player, int points) {
for (int i = 0; i < points; i++) {
tennisScore.addPoint(player);
}
}
@Then("Score is {string}")
public void score_is(String expected) throws Throwable {
assertThat(tennisScore.score()).isEqualTo(expected);
}
Cucumber

57. UT 80
Behavior-Driven Development
Before star;ng the development,
developers + testers + domain experts (BA/PO)
write together 2 a series of .feature files that capture
(developer = typist, having sketched a few draV scenarios beforehand)
the acceptance criteria of the stories to develop.
When these .features go green✅ => develop = done-
Later, each change request starts with edi;ng .feature files

58. UT 81
! Developers' Dream "
Business people will maintain the .features
. It never works.
✅ Works: Pair on them (Dev✍ + Biz0)
✅ Works: Automa;on-tester

59. UT 82
When should you use a .feature?
✅ Hesita2ng/conflic2ng requirements (CYA principle)
✅ Complex rules, to beQer understand via more examples
✅ Business-cri2cal features to avoid expen$ive confusions
✅ Table-like test data (Excel-style) => scenario outline
❌ Business disengages: trivial or too technical features?

60. UT 83
Gherkin Best Prac0ces
§Use Ubiquitous Language understood by tech and !tech: PO, BA, QA
§Focus on behaviour (aka "FuncEonal Tests"), hide details behind steps
§Keep steps implementa7on simple; expose relevant parameters
§UI Scenarios (Selenium) can be fragile and slow⚠ => fire at API-level
§DRY: use shared steps, Background:, Scenario Outline, and @tags
§Mock external services for faster and more reliable tests
§Share state between step classes via dep. injecEon: Spring, Guice..
§Prevent state leak: cleanup test environment a]er each scenario
§Version control .features along code
§Runs on CI ⚠ (even if slow)

61. UT 84
WASTE
Maintaining .feature files
that no businessperson approves

62. UT 85
When only developers
work with Gherkin files
consider building a test DSL instead.
WriVen in the same language (java, C#...),
or in a denser one: kt, py, groovy

63. UT 86
Test Helpers
Tes<ng Framework
Test DSL

64. UT 88
Test Domain-Specific Language
= a fluent mini-tes@ng framework:
Click to see implementa9on

65. UT 89 http://spockframework.org/spock/docs/1.3/spock_primer.html
Spock tests (.groovy)

66. UT 90
Test DSL in Kotlin
https://medium.com/porterbuddy/simple-test-dsls-in-kotlin-c6393035a92

67. UT 91
Fine-Grained Tests
Complex Mocking
Fragile vs Refactoring
Focus on implementa;on
Longer Tests
Require Test Helpers
Robust
Close to spec

68. UT 92
JUnit4 Rules
JUnit 5 Extensions
Cucumber Tags
.py decorators
...
Test AOP
Invisible code runs before and a\er your tests
!

69. UT 93
Hierarchical
Test Fixtures

70. UT 94
Not all long tests are like this

71. UT 95
Hierarchical Test Fixtures
Shared fixture for
all tests on this flow
Addi4onal fixture
for some branches
= context
@Nested classes
Test Subclasses
Test AOP

72. UT 96
An Inac4ve Customer
… with an order
… with no orders
Hierarchical Fixtures
(@Nested in JUnit5)
Nested Test Fixtures in JUnit5: https://junit.org/junit5/docs/current/user-guide/#writing-tests-nested

73. UT 97
An Inac4ve Customer
@Nested With an order
@Nested With no orders
An Inac4ve Customer
With an order
… with no orders
@Before
Hierarchical Fixtures
@Before
@Before
@Before
@Before
JUnit5 @NestedExtending a test class

74. UT 98
The Joy of Testing – key points
#1 section
- Why do we write Tests?
- Coverage% ▶ CustomerValidatorTest
- Test Names
- ObjectMother: in-mem vs parse.json
- Mutation Testing
- Bug-testing
- assertThat (AssertJ) ▶
- Test AOP with JUnit Extensions ▶
#2 section
- Parameterized Tests ▶ TennisScoreTest
- Approval Tests (by text inspection%) ▶
- Characterization Tests (for legacy code)
- Gherkin for Functional Tests ▶
- Hierarchical Test Fixtures ▶ CustomerFacade
- Test Qualities

75. UT 99
The Joy of Tes*ng
Please share with others
Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24
Thank you!

76. UT 100
The Joy of Tes*ng
3 Share your thoughts: stalk me or at [email protected]
4 Test-Driven Design Insights, 3h Deep Dive at Devoxx 20234 Tes1ng Microservices: Join the Revolu1on, at Devoxx 20234 YouTube.com/vrentea
5 Private training ± consultancy: victorrentea.ro/catalog
What Next?
% Free monthly webinars: victorrentea.ro/community
Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24