Architecting a High-Performance (Open Source) Distributed Message Queuing System in C++

BlazingMQ is a new open source* distributed message queuing system developed at and published by Bloomberg. It provides highly-performant queues to applications for asynchronous, efficient, and reliable communication. This system has been used at scale at Bloomberg for eight years, where it moves terabytes of data and billions of messages across tens of thousands of queues in production every day.

BlazingMQ provides highly-available, fault-tolerant queues courtesy of replication based on the Raft consensus algorithm. In addition, it provides a rich set of enterprise message routing strategies, enabling users to implement a variety of scenarios for message processing.

Written in C++ from the ground up, BlazingMQ has been architected with low latency as one of its core requirements. This has resulted in some unique design and implementation choices at all levels of the system, such as its lock-free threading model, custom memory allocators, compact wire protocol, multi-hop network topology, and more.

This talk will provide an overview of BlazingMQ. We will then delve into the system’s core design principles, architecture, and implementation details in order to explore the crucial role they play in its performance and reliability.

*BlazingMQ will be released as open source between now and P99 (exact timing is still TBD)

1. Architecting a High-Performance Distributed Message Queuing System Vitaly Dzhitenov Senior Engineer at Bloomberg

2. Agenda BlazingMQ overview Distributed architecture Key performance-related ideas Broker architecture Fixing bottlenecks ‹#›

3. What is BlazingMQ? Multi-producer, multi-consumer message queue Physical decoupling, as well as temporal isolation, between the actors Guaranteed acknowledgment Message persistence and replication High availability Transport abstraction Scalability (just add more workers / applications); high fan-out ratio (1:6,000+) ‹#›

4. Distributed architecture ‹#› App App Proxy Node Node Node Node App App Proxy Data Center 2 Data Center 1 Replica Leader Replica Primary

5. Queue trajectories ‹#› Primary Replica Replica Replica Proxy Proxy Proxy Proxy Proxy Consumer Consumer Consumer Consumer Consumer Consumer Consumer Proxy Producer Proxy Producer Producer Replica Replica PUTs PUSHes

6. BlazingMQ at Bloomberg Battle-tested in production for eight (8) years 55,000+ queues Processing billions messages and terabytes of data daily Low Latency For 600,000 msg/sec to no persistence queue w/ fan-out ratio 5, the median is 1.7ms For 150,000 msg/sec over 10 persistent queues, the median is 1.4ms https://bloomberg.github.io/blazingmq/docs/performance/benchmarks/ ‹#›

7. Performance Actor thread model Batching Memory and Object pools, polymorphic allocators No data copying ‹#›

8. Actors Client Reading/writing to client Statistics and validation Queue lookup Queue Storage and replication Data routing Cluster Reading/writing to cluster nodes Cluster health Primary node Queue lookup ‹#›

9. Primary Replica Replica Proxy Proxy Actor Model ‹#› PUT PUSH PUT PUSH PUT PUSH Cluster Cluster Cluster Cluster Cluster Queue Queue Queue Queue Queue Client Client Client Client Producer Consumer

10. CLIENT CLUSTER DispatcherClient type: QUEUE Event Dispatcher ‹#› ThreadPool Queue Processors: Monitored SingleConsumerQueues Clients EventPool

11. Batching Batch builders for every data type Flushing (to the network) on: Size limit, fixed or auto-tuning Dispatcher queue idleness Intelligent batching decisions: Adjustable batch size Interdependent flushing ‹#›

12. Proxy Primary Replica Channel Advanced batching ‹#› Cluster Client Queue network network Channel network network Client Client Cluster Queue network network PUT PUSH Replication PUSH PUSH PUT PUT Queue Cluster PUT PUT PUSH

13. Actor bottleneck ‹#› Cluster Client Queue Client Queue Client Queue

14. The solution Separate Control and Data planes Keep Cluster on the Control Plane and bypass it on the Data Plane Queue takes over Context, Statistics, and Validation work on the Data Plane Queue validates data using lockless synchronization with Cluster AtomicGate Based on one atomic int Multiple lockless, non-blocking AtomicGate::tryEnter Single AtomicGate::open , AtomicGate::closeAndDrain ‹#›

15. Published as Open Source! https://github.com/bloomberg/blazingmq https://bloomberg.github.io/blazingmq https://bloomberg.github.io/blazingmq/docs/performance/benchmarks/ ‹#›

16. Vitaly Dzhitenov [email protected] @TechAtBloomberg Thank you! Let’s connect.