Badoo: Cross platform Mobile Test Automation and Continuos Delivery

Sathish Gogineni, Badoo, QA-секция CodeFest 2015

1. Badoo:&Cross&pla,orm&mobile&test&
automa3on&and&con3nuous&delivery&&
Sathish'Gogineni'
Mobile'QA'Automa4on'Lead'

3. Raise&your&hands&if….&
• You’re'working'in'Mobile'pla=orm?'
• You'have'apps'on'mul4ple'mobile'pla=orms?'
• You’re'aware'of'the'test'automa4on'tools'in'
Mobile?'
• You'are'using'func4onal'test'automa4on?'
• You’re'using'“Cross'pla=orm”'test'automa4on?'

4. A&bit&of&self&promo3on&

5. Who&is&Badoo?&
• A'social'networking'service'
• 241'million'users'
• Available'in'190'countries.''
• Support'45'languages'on'mobile'
• 250'employees'based'in'London'and'Moscow'

6. More&about&Badoo&
• Badoo'has'2'popular'products'
'
'
• Apps'support'4'mobile'pla=orms'
– Badoo'
– HotOrNot'

7. Mobile&Test&Automa3on&at&Badoo&
• A'single'test'automa4on'code'base'
– Run'across'two'products'
– Run'on'three'pla=orms'
– Run'on'phones'and'tablets'
'
• We'have'
– 68%'func4onal'coverage'for'Badoo'in'Android'&'iOS'('390'
scenarios')'
– 76%'func4onal'coverage'for'HotOrNot'in'Android'&'iOS'('250'
scenarios')'
– 52%'func4onal'coverage'in'MobileWeb'('165'scenarios')'
– And'in'Windows'pla=orm'we'have'0%'

9. A&bit&about&Cross&pla,orm&test&

10. Mobile&cross&pla,orm&test&automa3on?&
• Write'once','Run'anywhere(?)'
'
• Write'test'specifica4ons'
'
• Implement'the'tests'at'high'level'
'
• Implement'the'pla=orm/applica4on'UI'
code'
'

11. Cross&pla,orm&stack&in&BDD&
Scenarios&
Feature&
Step&defini3ons&
Android&Page& iOS&Page&
Test'Specifica4on'
High'level'
implementa4on'
Pla=orm'specific'

12. Why&cross&pla,orm&test&automa3on?&
• Business'logic'is'same.'It'is'only'
interac4on'with'UI'is'change'
'
• Pooled'resources'
'
• More'reusable'code.'Less'
maintenance'
'

13. What&should&be&considered?&
• Who'is'wri4ng'the'test'cases?''
''''App'developers'/'QA'developers?'
'
• Which'mobile'pla=orms?'
• Wri4ng'in'language'other'than'
pla=orm'specific'

15. A&bit&about&Cross&Pla,orm&test&tools&

16. Mobile&Cross&Pla,orm&Tools&
• Instrument'based'tools'
'
'
'
• Image'Recogni4on'tools'
'
– Calabash'
– Appium'
– Monkey'Talk'
– eggPlant'
– seetest'

17. A&bit&about&Calabash&

18. What&is&Calabash?&
'
• A'framework'wriben'in'“Ruby”'
'
• Has'clientcserver'architecture'using'JSON'
'
• Integrated'with'cucumber'
'
• Supports'Android'and'iOS'pla=orms'

19. Calabash&:&Pros&&&Cons&
• Pros'
– Has'well'structured'client'framework'
– Supports'backdoor'opera4ons'to'interact'with'
Applica4on'code'
'
• Cons'
– Can’t'run'on'“Produc4on”'build'in'iOS'
– Have'to'use'Ruby'programming'language'
– Forced'to'write'test'cases'in''BDD'('‘Given'When'
Then’')'

20. A&bit&about&Appium&

21. What&is&Appium?&
• Is'a'Server'wriben'in'Node.js'using'JavaScript'
'
• Uses'pla=ormcprovided'frameworks'under'the'
hood'
• Supports'Web'driver'JSONWire'protocol'
'
• Supports'iOS,'Android,'Mobile'Web'and'Hybrid'
applica4ons'
'
'
'

22. Appium&:&Pros&&&Cons&
• Pros'
– Tests'can'be'wriben'in'any'('most'of')'
programming'languages'
– Good'community'support'
– Can'use'same'produc4on'builds'for'tes4ng'
'
• Cons'
– Lible'bit'“buggy”'

24. A'bit'about'Con4nuous'Delivery'

25. Con3nuous&Delivery&is…&
'
A'prac4ce'of'crea4ng,'tes3ng'
and'releasing'stable'applica4on'
frequently'in'smaller'chunks'

26. What&is&Con3nuous&Delivery?&
Con3nuous&&
Delivery&
=&
Con3nuous&
Integra3on&&&
+&
Con3nuous&
Test&Automa3on&

27. Con3nuous&Test&Automa3on&
Development'Branch'

28. Con3nuous&Test&Automa3on&
Development'Branch'

29. Con3nuous&Test&Automa3on&
Development'Branch'
Implement''
Feature'tests'

30. Con3nuous&Test&Automa3on&
Development'Branch'
Feature&
Tests'Implement''
Feature'tests'

31. Con3nuous&Test&Automa3on&
Development'Branch'
Feature&
Tests'
Smoke'
Tests'
Implement''
Feature'tests'

32. Con3nuous&Test&Automa3on&
Development'Branch'
Feature&
Tests'
Smoke'
Tests'
Implement''
Feature'tests'

33. Con3nuous&Test&Automa3on&
Development'Branch'
Feature&
Tests'
Smoke'
Tests'
Nightly''
Builds'
Nightly''
Builds'
Implement''
Feature'tests'

34. Con3nuous&Test&Automa3on&
Development'Branch'
Feature'2'
Feature'1'

35. Con3nuous&Test&Automa3on&
Development'Branch'
Master'
Feature'2'
Feature'1'

36. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'Branch'
Master'
Feature'2'
Feature'1'

37. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'Branch'
Integra4on'
Tests'
Master'
Feature'1'
Feature'2'

38. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'2'
Integra4on'1'
Master'Branch'

39. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'2'
Integra4on'1'
Master'Branch'
Nightly''
Builds'
Nightly''
Builds'

40. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'2'
Integra4on'1'
Master'Branch'
Release'Branch'

41. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'2'
Release'
Tests'
Integra4on'1'
Master'Branch'
Release'Branch'

42. Con3nuous&Test&Automa3on&
Development'Branch'
Integra4on'2'
Integra4on'1'
Master'Branch'
Release'Branch'
Release'
Tests'
Release'
Tests'

44. Scaling'build'infrastructure'

45. How&we&started?&
• A'single'build'machine'with'Jenkins'
• Running'on'Android'emulator'and'iOS'
simulator'
'
• Run'the''tests'only'on'“master”'branch'
every'night'

47. What&we&have&today?&
• Distributed'CI'server'with'12'maccmini'
agents''
'
• Each'Agent'has'Android'and'iOS'device'
connected'
• Running'the'tests'on'Real'devices'
• Integrated'with'JIRA'
'
'
'

50. What&we&will&do&in&future?&
'
• More'devices'to'each'agent'and'Parallelizing'
test'suite'
• Using'VMs'for'scaling'build'agents'
'
'

51. Summary&
'
• Worth'wri4ng'cross'pla=orms'tests.'It'Ensure'
behavior'is'same'in'every'pla=orm''
'
• Mobile'Test'Automa4on'tools'are's4ll'evolving'
'
• And,'they'have'limita4ons'

52. Nevertheless…&
'
• We'reduced'Regression'tes4ng'to'a'day'for'
every'release'('66%'4me'save')'
• 3X'more'releases'to'the'app'stores'
• More'stable'applica4on.'Crash'rate'reduced'
from'6%'to'2%'

53. hQps://techblog.badoo.com/&
&
Sathish&Gogineni&
[email protected]&
'