Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL

In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.

1. Beyond Event
Sourcing
May 2024
Embracing CRUD for Wix's High-Growth
Platform

2. Beyond Event Sourcing
Hi,
I’m Natan
Backend Infra Tech Lead @Wix
Yoga enthusiast
Speaker
Blogger
natansilnitsky
www.natansil.com
@NSilnitsky

3. Beyond Event Sourcing @NSilnitsky

4. Beyond Event Sourcing @NSilnitsky
~4B
Daily HTTP
Transactions
4000±
Microservices
in production
~70B
Kafka
messages a day

5. Beyond Event Sourcing @NSilnitsky
Agenda
Event sourcing & CQRS Framework for CRUD services
Dev velocity1
Scalability


2
Performance3
Resilience4
* stores

6. Beyond Event Sourcing @NSilnitsky
Event Sourcing & CQRS
Wix Stores Example

7. Beyond Event Sourcing @NSilnitsky
Add/Update product QueryProduct
Product Catalog

8. Beyond Event Sourcing @NSilnitsky
WriteProduct
Event sourcing and CQRS
Product 123:
Product Created1
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)4
ReadProduct
Name
Price
Description
Stock-level
}
Append-Only
Events
Snapshot

9. Beyond Event Sourcing @NSilnitsky
Event
Store
CreateProduct:
Catalog Write API
Event sourcing and CQRS
Product Created

10. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS
Product 123:
Product Created1
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)4
Event
Store
CreateProduct:
Catalog Write API
Product Created

11. Beyond Event Sourcing @NSilnitsky
Replay
GetProduct
Catalog Read API
Event sourcing and CQRS
Replay
events
Product 123:
Product Created1
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)4
Event
Store
CreateProduct:
Catalog Write API
Product Created

12. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS – Advantages
Debug with
“time travel”
GetProduct
Catalog Read API
Replay
events
Product 123:
Product Created1
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)4
Event
Store
CreateProduct:
Catalog Write API
Product Created

13. Beyond Event Sourcing @NSilnitsky
Product 123:
Product Created1
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)4
…..5
Product Changed (stock-level)30
Event
Store
Event sourcing and CQRS

14. Beyond Event Sourcing @NSilnitsky
Product 123:
Product Created1
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)4
…..5
Product Changed (stock-level)30
Event
Store
Product 123 Snapshot
Snapshot
Repository
Event sourcing and CQRS

15. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Advantages
Catalog Read API
Inventory
snapshot
Product
Snapshot
Product Created
Product Changed (price)2
Product Changed (description)3
Product Changed (stock-level)
Product 123:
1
4

16. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Product
DB
Create/Read Product
Catalog CRUD
Event
Store
CreateProduct:
Catalog Write API
Product Created
Product 123 Snapshot
Snapshot
Repository
Catalog Read API

17. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Product Created (price 4$)1
Product Changed (price 6$)2
Product 123:
Snapshot
Repository
Delayed
Product snapshot ConsumerCatalog Write API

18. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Product 123:
Snapshot
Repository
Product Created (price 4$)

1
Product snapshot ConsumerCatalog Write API
Product Changed2
Product Created (price 4$)1
Product Changed (price 6$)2 Delayed

19. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Product Changed (descriptionA)3
Product Changed (descriptionB)4
Product 123:
Snapshot
Repository
Product Created (price 4$)

1
Product Changed (descriptionB)2
Catalog Write API
Product Changed (descriptionA)3
Product Created (price 4$)1
Product Changed (price 6$)2 Delayed
Product snapshot Consumer

20. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Product Changed (descriptionA)3
Product Changed (descriptionB)4
Product 123:
Snapshot
Repository
Product Created (price 4$)1
Product 123 Snapshot
Product Changed (descriptionB)2
Product
4$
DescriptionA
Product Changed (descriptionA)3
Catalog Write API
Product Created (price 4$)1
Product Changed (price 6$)2 Delayed
Product snapshot Consumer

21. Beyond Event Sourcing @NSilnitsky
Read your own writes
Event sourcing and CQRS - Disadvantages

22. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Confetti Cannon events:
Product Changed (price -1$)
Read your own writes
Catalog Write API
Event
Store

23. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Confetti Cannon events:
Snapshot
Repository
Confetti Cannon
Snapshot
Physical Item
3$
DescriptionA
Product Changed (price -1$)
Read your own writes
Catalog Write API
Event
Store

24. Beyond Event Sourcing @NSilnitsky
Event sourcing and CQRS - Disadvantages
Confetti Cannon events:
Snapshot
Repository
Confetti Cannon
Snapshot
Physical Item
3$
DescriptionA
Product Changed (price -1$)
Read your own writes
Catalog Write API
Event
Store
Product Changed (price -1$)
+

25. Beyond Event Sourcing @NSilnitsky
Product
DB
Update Product
Catalog CRUD
Event sourcing and CQRS - Disadvantages
Read your own writes

26. Beyond Event Sourcing @NSilnitsky
Read your own writes
Event sourcing and CQRS - Disadvantages

27. Beyond Event Sourcing
Event sourcing and
CQRS -
Disadvantages
Complexity
Eventual consistency only
Massive scale
Corrupted snapshot
Read your own writes

28. Beyond Event Sourcing @NSilnitsky
APIs
+
Standardization

29. Beyond Event Sourcing @NSilnitsky
RPC
Product Catalog API
Cart
Service
Internal Wix
Developer
Open Platform - API First
* Center 4000, clear,
focused, comprehensive

30. Beyond Event Sourcing @NSilnitsky
HTTP/SDK
Product Catalog API
Internal Wix
Developer
PoS
App
External App
Developer
Open Platform - API First

31. Beyond Event Sourcing @NSilnitsky
JavaScript
method
Product Catalog API
Internal Wix
Developer
Custom
Filter
External App
Developer
External Wix Site
Developer (Velo)
Open Platform - API First

32. Beyond Event Sourcing @NSilnitsky
Product Catalog API
Internal Wix
Developer
External App
Developer
External Wix Site
Developer (Velo)
Open Platform - API First
* CRUD on ES, consistent,
hard in standard way. Users
fast experience

33. Beyond Event Sourcing @NSilnitsky
Stores
Bookings
Events
Forms
Loyalty
Rewards
Tickets Policies
Checkout
Time
Slots
Schemas
Sub
missions
Guests
Coupons
Calendar
Orders
Waitlist
CartCatalog
Programs
Open Platform - API First
Internal Wix
Developer
External App
Developer
External Wix Site
Developer (Velo)

34. Beyond Event Sourcing @NSilnitsky
Wix’s Open Platform
CRUD

Event sourcing
Was
Independent
“startups”
Now
Single Open
Platform

35. Beyond Event Sourcing @NSilnitsky
Wix’s Open Platform
CRUD

Event sourcing
API First
APIs - TDD
+
FE driven
Was
Independent
“startups”
Now
Single Open
Platform

36. Beyond Event Sourcing
Framework for
CRUD services

Open Platform
API Schema
Validations & Permissions
Scaffold
Persistence + Event publishing
Data projections (Queries)

37. Beyond Event Sourcing @NSilnitsky
CreateProduct
Product
Document Store
Catalog API
Product Catalog - CRUD
ReadProduct
UpdateProduct
DeleteProduct
Unified!

38. Beyond Event Sourcing @NSilnitsky
CreateProduct
ReadProduct
UpdateProduct
DeleteProduct
Catalog API
API First - platformized CRUD

39. Beyond Event Sourcing @NSilnitsky
CreateProduct
ReadProduct
UpdateProduct
DeleteProduct
Catalog API
API First - platformized CRUD

40. Beyond Event Sourcing @NSilnitsky
CreateProduct
ReadProduct
UpdateProduct
DeleteProduct
Catalog API
service ProductService {
option (service_entity).message = "...v3.Product";

rpc CreateProduct (CreateProductRequest) returns
(CreateProductResponse) ...
rpc GetProduct (GetProductRequest) returns
(GetProductResponse) ...
rpc UpdateProduct (UpdateProduct) returns (UpdateProduct)
...
…
}

message Product {
option (entity) = {fqdn: "...v3.product"};
google.protobuf.StringValue id = 1;
google.protobuf.Int64Value revision = 2;
...
repeated Inventory inventory = 25;
...
}


API First - platformized CRUD

41. Beyond Event Sourcing @NSilnitsky
API First - platformized CRUD
CreateProduct
ReadProduct
UpdateProduct
DeleteProduct
Catalog API
service ProductService {
option (service_entity).message = "...v3.Product";

rpc CreateProduct (CreateProductRequest) returns
(CreateProductResponse) ...
rpc GetProduct (GetProductRequest) returns
(GetProductResponse) ...
rpc UpdateProduct (UpdateProduct) returns (UpdateProduct)
...
…
}

message Product {
option (entity) = {fqdn: "...v3.product"};
google.protobuf.StringValue id = 1;
google.protobuf.Int64Value revision = 2;
...
repeated Inventory inventory = 25;
...
}

42. Beyond Event Sourcing @NSilnitsky
API First - platformized CRUD

43. Beyond Event Sourcing @NSilnitsky
API First - platformized CRUD

44. Beyond Event Sourcing @NSilnitsky
Event Driven Architecture
Automatic Domain Events FTW

45. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Product
Document Store
Catalog Service
* DE describes…

46. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Catalog Service
Product
SDL
* SDL is
document based.
no direct SQL

47. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Catalog Service
Product
SDL
* SDL is
document based.
no direct SQL

sdl.insert(request.getProduct.mapTo[ProductDomain])

48. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Product
SDL
Catalog Service
Product
SDL


sdlBuilder =
_.withDomainEventsEnabled(_.mapTo[Product])

49. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
Data
warehouse/Lake
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Catalog Service
Product
SDL
* debugging
corruption

50. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
ebay-bridge Service
CreateProduct
UpdateProduct
DeleteProduct
Catalog Service
Product Created
Product Updated
Product Deleted
Product
SDL

51. Beyond Event Sourcing @NSilnitsky
EDA - Domain Events
CreateProduct
UpdateProduct
DeleteProduct
Product Created
Product Updated
Product Deleted
Catalog Service
Product
SDL

52. Beyond Event Sourcing @NSilnitsky
Data consistency in Wix’s EDA
Resilient Producers and consumers

53. Beyond Event Sourcing @NSilnitsky
Make DB Update & Event Producing Atomic
Catalog
Service
Ebay Bridge
Service
* atomic,
otherwise

54. Beyond Event Sourcing @NSilnitsky
Produce event to S3
Resilient Producer
Catch Unsent Events
Catalog Service

55. Beyond Event Sourcing @NSilnitsky
Produce
to Kafka
Poll
Produce event to S3
Resilient Producer
Fallback to S3 and Heal
Catalog Service Healer Service

56. Beyond Event Sourcing @NSilnitsky
Consumer retries + DLQ
Make DB Update & Event Producing Atomic
Catalog
Service
Ebay Bridge
Service

57. Beyond Event Sourcing @NSilnitsky
Alternative - use outbox pattern and/or CDC
Transaction
Outbox Table
Insert
Product Table
Insert
Update
Delete
Database
Instant read-your-own-writes
consistency in Catalog service
Write to
database
CDC
Read from
Outbox Table
Kafka
Connect
Debezium
connector
Publishes
messages
to brokers
Kafka
Broker
Eventually consistent data
exchange with Ebay Bridge Service
Catalog
Service
Ebay Bridge Service

58. Beyond Event Sourcing @NSilnitsky
Data Projection &
query optimization
Materializer

59. Beyond Event Sourcing @NSilnitsky
Query latency - naive
CreateProduct
ReadProduct
DeleteProduct
Catalog Service
Product
SDL
FilterProductWithInventory
Inventory Service
Inventory
SDL
RPC
1
2

60. Beyond Event Sourcing @NSilnitsky
Multi-step
Query latency - naive
CreateProduct
ReadProduct
DeleteProduct
Catalog Service
Product
SDL
FilterProductWithInventory
Inventory Service
Inventory
SDL
RPC
price < 100 and stock > 4



2



1

61. Beyond Event Sourcing @NSilnitsky
Query latency - DB join
CreateProduct
ReadProduct
DeleteProduct
Catalog Service
Product
SDL
FilterProductWithInventory
Inventory Service
Inventory
SDL
DB level Join

62. Beyond Event Sourcing @NSilnitsky
Query Latency - Materializer
Product +
Inventory
Materializer
Inventory
Inventory updated Event
Catalog
Service
Inventory
Service
FilterProductWithInventory

63. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer

64. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer

65. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer

66. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer

67. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer
* no consistency

68. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer

69. Beyond Event Sourcing @NSilnitsky
Simplicity
Onboarding new team member
Write performance
Read performance
consistency
Audit log/time machine
Projections/queries
Comparing Event sourcing
to Wix’s CRUD based solution
Event Sourcing
CRUD
SDL+Domain Events
Materializer

70. Beyond Event Sourcing @NSilnitsky
Summary
Wix successfully shifted its vast distributed system entirely to
CRUD-based microservices, moving away from a CRUD/event sourcing
hybrid.

.
.
.

71. Beyond Event Sourcing @NSilnitsky
Summary
Wix successfully shifted its vast distributed system entirely to
CRUD-based microservices, moving away from a CRUD/event sourcing
hybrid.

This transformation was driven by a commitment to standardization,
managed infrastructure with automated code generation, and a
decoupled architecture.

72. Beyond Event Sourcing @NSilnitsky
Summary
Advanced tools were also implemented to boost development speed,
ensure system resilience, and optimize for scale and performance.
Domain Events
Resilient Producer Materializer
Simple Data Layer

73. Beyond Event Sourcing
Q & A
Thank you
natansilnitsky www.natansil.com @NSilnitsky
?????? slideshare.net/NatanSilnitsky

74. Q & A
www.natansil.com
@NSilnitsky