A case study for Software engineering improvement. Adopt Agile Methodology and improve quality with end-to-end automated testing.

Case study for process improvement and product strategy for Software engineering.

1. Amplify Product delivery improvement program Developed by Amnuay M. https://linkedin.com/in/amnuay

2. Amplify – Head of Engineering As a Head of Software Engineering. Many process and perspective count towards the efficiency of the team, and target product quality. Below is a case of Head of Engineering at Amplify. Over the past 5 years, Amplify has created a market leading eWallet with a large and loyal user base. Over that time the product has evolved to incorporate several different value streams in areas as diverse as food & beverage delivery to mobile gaming. Amplify has always opted to move fast and capture an evolving market and has built a reputation for being innovative and quick to bring out new products and services. The teams take great pride in their ability to get new products and services into the hands of their customers quickly. The product has evolved from being a simple eWallet application to a lifestyle super app. The backend, mobile, web, and infrastructure teams have all grown to an overall tech team of around 100 people. There is a QA function of mostly manual testers that perform regression tests prior to any release. Their platform is built upon a microservices architecture, a singular MongoDB back-end datastore and native apps for Android and iOS. Over the last 12 months, the rate of delivery has been on a steady decline. Products and features that used to be delivered in two weeks are now taking at least two months to deliver if not longer. There has been a slow rise in production incidents and deployments often need to be rolled back. It now also seems to be a requirement to issue forced updates to the app with each deployment and there is increasingly negative feedback coming from customers regarding this. As part of an external audit, several security controls were found to not be in place and a centralized backlog of issue was created. The security team have been complaining that they are seeing no progress on the burn down of this backlog and there is no clear owner for these items. Finally, the teams are finding it increasingly difficult to detect issues in the system. They are starting to rely on feedback from customers to give them an indication that something is wrong. The teams are having to work harder and longer just to keep things the way they are and overall, the morale of the team is dropping. We are starting to see an increase in attrition and there are 20 roles that are currently waiting to be backfilled.

3. Background and problem statement Market-leading eWallet evolved into Lifestyle Super App Tech team ~100 people (Mobile-Native, Web, Backend, Infra, and QA) Platform was built on microservice architecture Manual QA for regression tests Delivery rate steadily declined from 2 weeks to two months Production incidents are raising, and deployments need to be rolled back Application deployment needs forced updates hence customer satisfaction scores down Security control ineffective. Lack of security audit findings owner Hardly detect issues in the system. Reactive detection from customer feedback. Team morale is dropping, and 20 vacancies are currently waiting to be backfilled. Management reports and dashboard required to represent team productivity EA and KM DevSecOps QA Automation Agile Organization Flight level 2, 3

4. Agile organization As-is Hierarchical business functions Team formed slowly overtime Job descriptions, Job levels, Job Titles Managers own teams and people career Jobs are owned by the manager and not shared People assigned jobs by management People rewarded by level, tenure, experience Inclusion, sustainability, diversity To-be Projects, squads, teams, shared services Teams assemble and stop quickly Assignments, tasks, expert roles Managers manage projects and sponsor people Jobs open in transparent marketplace People sought out based on skills, work on multiple projects People rewarded by outcomes, reputation, sponsorship Citizenship, collective thinking, shared values Current situations. Amplify had aggregated the same type of functionalities in the same group of people while working on their scope Job flow from different team require prioritization and knowledge transfer, requirement sync-up, and job sequential Backlogs and team capability owned by the manager and job assignments also need manager assignments. KPI-based organization where need additional reports and provisioning of outcomes.

5. Enterprise Architect and Knowledge management As-is Project team meeting with additional overhead on the arrangement and explanation Jobs and requirements are conveyed by Email and Word documents Longer time to delivery Application is stored in a separate repository, Build and Deploy were based on manual operations. No deployment redundancy or Deployment strategy. No application roadmap, mapping for all services To-be Knowledge management for Application comments, Application documentation, Unit Tests, API Documentation, Backlogs are communicated within the Kanban board with DOR, DOD definitions, and its dependencies specified Automate deployment shorten time to delivery Mono-repo, Deployment strategy, Blue-green deployment, Canary deployment, Environment sand-box Domain driven design with service mapping in KM Current situations. Project team works as a big team whereas all explanation and meeting sessions are always needed and cost additional time. Microservices are scattered within Kubernetes (K8s) farm. Service discovery is hard to maintain Application is lack of documentation from requirements, the application itself, integration specification, and QA documentation Application deployment still based on traditional deployment, Application design not support

6. DevSecOps As-is Silo work in Developers, Security, IT Operations Slower security detection and incident response Security assessment on the delivered product as preventive Last mile scan cannot identify the root cause of security vuln Too much security debt No actual security awareness and enforcement / Cost incurred during Operation and security process To-be Collaborative work to deliver shared objectives with the tool Security monitoring and Incident response built-in the pipeline Security shifted left and proactive monitoring Granular level of security vulnerability detection Security issues are addressed early in development process Security enforcement point significantly increase security awareness Significantly reduced cost in Development, Security and IT Operation Current situations. Developer, Security, and IT Operation are working on their own silos and perspective Application build and deployment manually, Also rollback manually Security control was ineffective e.g., Secure coding, Vulnerability scan, and fix, are lately detected and late in fixing. No CI/CD pipeline and no automate tools for product quality control (Security Scan, Coverage Scan)

7. QA Automation As-is QA knowledge depends on staff QA process starts at the later phase of product delivery No test management to manage all test cases and result All test cases and data need to be reviewed/revised all the time -> Additional effort Manual QA needs high effort to run through all test cases Production incidents caused by uncovered test scenarios To-be QA Governance setup with policy and process Shift left QA defined and QA SME join since product definition Test management tool manages all test cases and result Test cases, Test data can be re-used / QA Automation reduced Human error, Time, and effort UAT and Regression tests can be re-used, reviewed, and execute on time Current situations. Manual QA and regression test on Mobile application High risk on production incidents and keep growing -> Causing additional workload on deployment rollback QA knowledge depends on staff, No QA governance, policy, and process in place No Test management toolset

8. Management report – Flight level 3 As-is Management report in Email or PowerPoint / Management reports need additional effort to produce Work prioritization is always delayed and no transparency All requirements, data, and documents are hard to access Management and supervisor have less visibility on work in progress, ready to release jobs To-be Epic and Stories report/dashboard from backlog management tools Reports can be pulled out of the system anytime All backlogs can be prioritized in real-time and accessible Requirements and knowledge accessible from anywhere Management and supervisor are effectively managed teams' delivery online Current situations. Amplify is facing a hard time managing people resources and a vacancies fulfillment Job flow always stuck with meeting queue, delayed and equipment communication gaps With a high workload Managers and supervisors are swamped with backlogs and issues Management reports are time-consuming tasks where information and achievement are scattered

9. Q&A Developed by Amnuay M. https://linkedin.com/in/amnuay