How Yieldmo Cut Database Costs and Cloud Dependencies Fast by Todd Coleman

Yieldmo processes hundreds of billions of ad requests daily with subsecond latency. Initially using DynamoDB for its simplicity and stability, they faced rising costs, suboptimal latencies, and cloud provider lock-in. This session explores their journey to ScyllaDB’s DynamoDB-compatible API.

1. A ScyllaDB Community
How Yieldmo Cut Database
Costs and Cloud Dependencies
Fast
Todd Coleman
Chief Architect

2. Todd (he/him)
■Chief Architect and technical co-founder at Yieldmo
■Over 20 years high frequency/low latency architecture
and scale experience
■Deep experience with cloud computing platforms

3. ■Introduction
■The Problem: Challenges with DynamoDB
■Evaluating Alternatives
■Migration process
■Impact of Migration
Presentation Agenda

4. Introduction

5. Who we are
Yieldmo is an advertising platform that helps brands invent creative
experiences through tech and AI.
■We provide ad placement technology for publishers.
■Auction-based ad selection.
■Machine learning to optimize outcomes.
■Innovative ad formats.

6. What we’ll cover today
■The Challenge: Our ad tech business requires ultra-fast,
high-throughput database lookups, but DynamoDB’s costs and
cross-cloud limitations became a concern.
■The Solution: We migrated to ScyllaDB, which provided lower costs,
better performance, and DynamoDB compatibility, making the
transition smooth.

7. Our Database Needs
■Real-time ad auctions require fast database lookups.
■Database must handle billions of requests daily with single-digit
millisecond latency while also supporting constant writes
■Database must have near high uptime for mission critical workflows
Ad Server
DSP Partners
(Advertisers)
DynamoDB
Identity
Lookup
Request Bids
Bids
Request Ad
Ad

8. The Problem: Challenges with
DynamoDB

9. Database Workload Characteristics
■Read-heavy with consistent traffic patterns.
■Data growth aligned with business growth.
■Requires stability, scalability, and high throughput.

10. Limitations of DynamoDB
■Rising costs due to scale.
■Possible caching solutions would
produce limited speed
improvements and higher costs
■Cross-cloud latency issues when
experimenting with GCP.
Rising DynamoDB cost

11. Tipping Point
Cost concerns and need for cross-cloud deployments triggered a
search for alternatives.


■Attempted creating GCP
Amsterdam ad serving colo,
however connecting to AWS
DynamoDB in Dublin was
impractical due to latency

12. Evaluating Alternatives

13. Key Priorities
■Stability, reliability, Speed and Ease of management.
■Cost-reduction.
■Cross-cloud compatibility.

14. Other Options Considered
■Caching Layer with DynamoDB.
■Pros: Could improve speed on some lookups
■Cons: Additional cost and complexity, expect many cache misses
unless we pre-cache. Doesn’t address cloud provider compatibility
issues.
■Aerospike
■Pros: Exceptional speed, multiple cloud provider support
■Cons: High memory requirements and cost, several weeks of
codebase modifications

15. Why ScyllaDB
■Multiple cloud provider support
■Cost-effective
■Good performance
■Minimal code changes due to DynamoDB compatibility
■Support from ScyllaDB’s team during POC made migration easy

16. Migration process

17. Initial Steps
■Proof of concept (POC) to validate performance improvements.
■Collaborative planning with ScyllaDB for migration.

18. Data Migration Challenge
■Migrating multi-terabyte tables with continuous data updates.
■Dual writes to AWS and ScyllaDB
■Using Apache Spark cluster to data loading

19. Process Overview
■Snapshot current Kafka offsets and pause Machine Learning batch job writes
■Run migration job using ScyllaDB tools and Spark Cluster, copying data up until the
point of the Kafka offsets snapshot
■Create write stream from Kafka to ScyllaDB starting from offset snapshot
■Restart Machine Learning batch jobs writing directly to ScyllaDB
Kafka
Spark
Cluster
DynamoDB
Identity Data
Stream
Batch Jobs
Machine
Learning Results
ScyllaDB
Cluster

20. Timeline and Lessons Learned
■First POC, copied single 3.3TB, 28 billion item table in us-east-1 region
■Migrator job took about 10 hours
■After migration job finished, enambed compaction and TTLs
■Worked through issues with node load distribution and data expiration
■Second POC, all tables in all 5 regions
■About a 2 week process to load all tables in all regions
■Refined instance types to only use i4i EC2 family
■Used new ScyllaDB load balancing code for better load distribution.
■Must closely control scale as nodes must be added in sets of 3

21. System Architecture

22. Impact of Migration

23. Benefits
■Cost Savings: Significant reduction in database costs
■Cross-Cloud Compatibility: Enabled GCP integration with reduced
latency
■Performance: Modest speed improvements

24. Additional takeaways
■Smooth transition due to DynamoDB compatibility
■ScyllaDB’s managed service made the transition nearly as
low-overhead as DynamoDB
■Hesitations about leaving DynamoDB were mitigated by ScyllaDB’s
stability and support

25. Stay in Touch
Todd Coleman
[email protected]
linkedin.com/in/todd-coleman-40084