Assessing Mock Classes: An Empirical Study (ICSME 2020)

During testing activities, developers frequently rely on dependencies (e.g., web services, etc) that make the test harder to be implemented. In this scenario, they can use mock objects to emulate the dependencies' behavior, which contributes to make the test fast and isolated. In practice, the emulated dependency can be dynamically created with the support of mocking frameworks or manually hand-coded in mock classes. While the former is well-explored by the research literature, the latter has not yet been studied. Assessing mock classes would provide the basis to better understand how those mocks are created and consumed by developers and to detect novel practices and challenges. In this paper, we provide the first empirical study to assess mock classes. We analyze 12 popular software projects, detect 604 mock classes, and assess their content, design, and usage. We find that mock classes: often emulate domain objects, external dependencies, and web services; are typically part of a hierarchy; are mostly public, but 1/3 are private; and are largely consumed by client projects, particularly to support web testing. Finally, based on our results, we provide implications and insights to researchers and practitioners working with mock classes.

2. Motivation

Mocks are frequently used in software testing

Mocks simulate dependencies making tests fast, isolated and
deterministic

Two ways to use mocks: mock objects and mock classes

3. Mock objects
Object creation based on mocking frameworks

“Canning” behavior/state based on a specification

EASYMOCK o / nibrest-noc
mockit Kress noce

Easy mocking. Better testing.

4. atomy of a mock object

@Test
public void t G r() {
Car car = mock(Car.class)
ar(any(Car.class)))
Return(car)
(null);

when(service

Equals("SUCCESS", controller.getCar("test", car).getS
sals("ERROR", controller.getCar("test", car)

doThrow(IllegalArgumentException.class).when(car).se (
sertEquals("ERROR", controller.getCar("test", car).getSta

5. Anatomy of a mock object

@Test
public void t

(null);

quals("SUCCESS", controller.getCar("test", car).getSta
quals("ERROR", controller.getCar("test", car).getStatu

doThrow(IllegalArgumentException.class).when(car).se
ıals("ERROR", controller.getCar("test", car).get

6. Anatomy of a mock object

@Test
public void te
Car car =
he etchC: Car.class)))

(null);

quals("SUCCESS", controller.getCar("test", car).getSta
quals("ERROR", controller.getCar("test", car).getStatu

doThrow(IllegalArgumentException.class).when(car).se
ıals("ERROR", controller.getCar("test", car).ge

7. Anatomy of a mock object

QTest
public void testGetCar() {
Car car = mock(Car.class);
when(service.fetchCar(any(Car.class)))
.thenReturn(car)
.thenReturn(null);

assertEquals("SUCCESS", controller.getCar("test", car).getStatus().name());
assertEquals("ERROR", controller.getCar("test", car).getStatus().name());

doThrow(IllegalArgumentException.class).when(car).set_id(anyString());
assertEquals("ERROR", controller.getCar("test", car).getStatus().name());

8. Anatomy of a mock object

QTest
public void testGetCar() {
Car car = mock(Car.class);
when(service.fetchCar(any(Car.class)))
.thenReturn(car)
.thenReturn(null);

assertEquals("SUCCESS", controller.getCar("test", car).getStatus().name());
assertEquals("ERROR", controller.getCar("test", car).getStatus().name());

doThrow(IllegalArgumentException.class).when(car).set_id(anyString());
assertEquals("ERROR", controller.getCar("test", car).getStatus().name());

9. Mock classes

Manually creating a class simulating desired behavior/state

public class MockWebRequest extends WebReq
private Url url;
@0verride
public Charset getCharset() {
return Charset. forName("U

}

Override
public Url getClienturl() {
return new Url(url.getSegments(), Collections.<QueryParameter> emptyList());

}

10. Studies about mocks

@ There are studies on framework-based mock objects
(Spadini et. al., EMSE, 2019; Mostafa and Wang, QSIC,
2014)

e But no studies about mock classes!

11. The problem

@ What do mock classes mock?
@ How are mock classes designed?
e Are they any different from regular classes?

e Are mock classes widely used in practice by developers?

12. Assessing mock classes

> provide the basis to know

their usage, related

practices and challenges a
—

e ~ 5

~ -一
=>

13. Study Design

14. Study design

e Select 12 popular Java systems on GitHub

15. pe spring
© u ※ EventBus boot
spring
m -
SZ ucene Solr u elasticsearch 7

00000 ES [주의

16. Study design

e Detect mock classes from the entirety of project’s
defined classes

e Compare with classes mocked via frameworks

e Find usage of the mock classes in the Boa dataset

17. Identifying mock classes

e Isa “mock class” if it has “mock” in its name
e But no “test[s]” or “mockito”

18. Identifying mock classes

private static class Te KServ extends

0046
public tDis r getNamedDispatcher(
if (path.equals(*default") Il path.equals("c
return new MockRequestDispatcher(*/*);

private class Runnable implements

Latch countDownLatch;

MockRunnable(CountD ch countDownLatch) {
this.countDownLatch ntDownLa

}

}
else {
return null;

aoverri
public void run() {
countDownLatch.countDown();

}
}

public getVirtualServerName() {
return 0011

}

Guava - MockRunnable
Spring Framework - TestMockServletContext

18

19. Identifying mock classes

class TestMockservletContext extends Mockservletcontext

RequestDispatcher (String
pat ("default h ("custom

(CountDownLatch

aoverride

override
public String

Guava - MockRunnable
Spring Framework - TestMockServletContext

19

20. Identifying mock classes

pri lass MockRunnable implements Runnable {
RARE niece stent RequestDispatcher (String

p ("default h ("custom

(CountDownLatch

aoverride

override
public String

Guava - MockRunnable
Spring Framework - TestMockServletContext

20

21. In summary, our dataset has...

22. In summary, our dataset has...

12 projects

23. In summary, our dataset has...
12 projects

78,832 classes

24. In summary, our dataset has...
12 projects
78,832 classes

1,519 classes mocked with Mockito

25. In summary, our dataset has...
12 projects
78,832 classes

1,519 classes mocked with Mockito

604 mock classes

26. SELECTED PROJECTS AND DETECTED MOCK CLASSES.

Classes Mocked Mock

Project Stars Classes Using Mockito Classes
Elasticsearch 48.3k 16,097 524 138
Spring Boot 47.1k 7,845 367 33
RxJava 42.4k 2,766 21 0
Guava 36.8k 6,174 19 12
OkHttp 36.8k 210 0 1
Spring Fw. 36.5k 10,980 272 127
Retrofit 35.5k 303 0 4
Dubbo 32k 2,104 60 58
Spark 25.8k 1,850 23 0
EventBus 22.4k 163 0 0
Lucene-Solr 3.4k 11,583 35 132
Camel 3.2k 18,757 198 99

Total - 78,832 1,519 604

27. Research Questions

e RQ1: What type of dependencies are mock classes
emulating?

2

28. Research Questions

e RQ1: What type of dependencies are mock classes
emulating?
e RQ2: How are mock classes designed?

29. Research Questions

e RQ1: What type of dependencies are mock classes
emulating?

e RQ2: How are mock classes designed?

e RQ3: How are mock classes used by developers?

30. Results

31. RQ1 - What is the Content of Mock Classes?

Emulate domain objects, external dependencies, and web
services

Not very different than mocking frameworks!

31

32. RQ1 - What is the Content of Mock Classes?

MOCK CLASS CATEGORIES.

Category # %

Domain object 211 35 MM
External dependency 138 23 A
Web service 93 15 I

Test support 67 11 1
Native Java libraries 53 9 1
Network service 42 7 I
Database 0 0

Total 604 100 MN

32

33. RQ2 - How are mock classes designed?

34. RQ2 - How are mock classes designed?

Often extend other classes or implement interfaces

35. RQ2 - How are mock classes designed?

Often extend other classes or implement interfaces

70% are public / 30% private-package

36. RQ2 - How are mock classes designed?

Often extend other classes or implement interfaces
70% are public / 30% private-package

Overall same number of methods of regular classes

37. RQ2 - How are mock classes designed?

CLASS EXTENSION AND INTERFACE IMPLEMENTATION IN THE MOCK CLASSES.

A Mock Class Interface
Project Classes Extension ió Implementation 2 None =
Elasticsearch 138 88 63.8 56 40.6 13 9.4
Spring Boot 33 8 25.0 18 54.5 8 24.2
Guava 12 4 333 8 66.7 0
OkHttp 1 0 0.0 1 100.0 0
Spring Fw. 127 50 39.4 74 58.3 14 11.0
Retrofit 4 1 25.0 1 25.0 2 50.0
Dubbo 58 10 17.2 48 82.8 1 1.7
Lucene-Solr 132 109 82.6 28 21.2 2 1.5
Apache Camel 99 62 62.6 48 48.5 El 5.
Total 604 332 549 282 46.7 45 7

38. RQ2 - How are mock classes designed?

VISIBILITY IN THE MOCK CLASSES.

Mock

Project Public % Protected % Package % Private %
Classes
Elasticsearch 138 84 60.1 5 37: 17 12.3 32 23.2
Spring Boot 33 15 45.5 0 0 15 45.5 3 9.1
Guava 12 2 16.7 0 0 2 16.7 8 66.7
OkHttp 1 1 100.0 0 0 0 0 0 0
Spring Fw. 12 92 72.4 0 0 9 7.1 26 20.5
Retrofit 4 2 50.0 0 0 2 50.0 0 0
Dubbo 58 54 93.1 0 0 2 3.4 2 3.4
Lucene-Solr 132 77 58.3 2 1.5 24 18.2 29 21.9
Apache Camel 99 86 86.9 0 0 8 8.1 5 5.1

Total 604 413 68.4 7 1.2 13.1 105 17.4

고
N

39. RQ2 - How are mock classes designed?

NUMBER OF METHODS IN THE MOCK AND REGULAR CLASSES.

Methods in Mock Classes Methods in Regular Classes

Project mean median std. dev. mean median std. dev.
Elasticsearch 4.6 2 5.1 33 4 6.3
Spring Boot 2.8 2 2.6 32 2 3.9
Guava 8 5.5 7.4 2.8 2 2.4
Spring Fw. 12.4 6 20.0 5.1 2 8.3
Dubbo 6.1 25 8.1 7.4 4.5 95
Lucene-Solr 3.1. 2 5.6 33 3 5.4
Apache Camel. 10.9 4 21.0 4.9 3 6.8

3 13.8 53 3 6.9

All 7.2

40. RQ3 - How are mock classes used by developers?

41. RQ3 - How are mock classes used by developers?

Largely consumed to support testing

42. RQ3 - How are mock classes used by developers?

Largely consumed to support testing

Web services are the most emulated dependencies

43. RQ3 - How are mock classes used by developers?

Largely consumed to support testing
Web services are the most emulated dependencies

Usage concentrated in a few classes

44. RQ3 - How are mock classes used by developers?

MOST FREQUENT MOCK CATEGORIES IN THE BOA DATASET.

Category #Classes #Clients %

Web service 35 24022 46 MM
Domain object 39 11,879 23 A
External dependency 14 7,624 15 I
Native Java libraries 18 5,339 10 ㅣ

Test support 7 1,899 4 |
Network service 15 1,316 2 |

Total 128 100 MN

52,079

45. Conclusions

46. Conclusions

Mock classes are...

47. Conclusions

Mock classes are...

often created to emulate domain objects, external
dependencies and web services

48. Conclusions

Mock classes are...

often created to emulate domain objects, external
dependencies and web services

often part of a hierarchy, mostly public, not much structurally
different from regular classes

49. Conclusions

Mock classes are...

often created to emulate domain objects, external
dependencies and web services

often part of a hierarchy, mostly public, not much structurally
different from regular classes

largely consumed by client projects to support testing

50. Future work

investigate the reasons for developers to rely on mock classes
instead of mocking frameworks

take a deeper look on the semantics of mock classes

find mock classes in other programming languages to assess
the concerns of different software ecosystems