Rust + io_uring + ktls: How Fast Can We Make HTTP?

Working on Fluke: async Rust HTTP1+2 with io_uring & kTLS, sponsored by fly.io & Shopify. Unlike others, Fluke is built from the ground up to fully leverage io_uring, minimizing syscalls with kTLS. A promising future for proxies & apps if a stable API emerges. #Rust #io_uring #kTLS

1. A ScyllaDB Community
Rust, io_uring, ktls:
How fast can we make HTTP?
Amos Wenger
writer, video producer, cat owner
bearcove

2. A ScyllaDB Community
Nobody in the Rust space is going
far enough with io_uring
(as far as I'm aware)
Amos Wenger
writer, video producer, cat owner
bearcove

3. Amos Wenger (they/them) aka @fasterthanlime

writer, video producer, cat owner
■Wrote "Making our own executable packer"
■Teaching Rust since 2019 with Cool Bear
■Fan of TLS (thread-local storage & the other one)
bearcove

6. Define "HTTP"

7. Define "fast"

8. Rust HTTP is already fast

9. hyper on master is ?????? v1.4.1 via ?????? v1.80.1
❯ gl --color=always | tail -5
Commit: 886551681629de812a87555bb4ecd41515e4dee6
Author: Sean McArthur <[email protected]>
Date: 2014-08-30 14:18:28 -0700 (10 years ago)

init

10. HTTP/1.1 200 OK
Date: Fri, 31 Dec 1999 23:59:59 GMT
Content-Type: text/html
Content-Length: 1354

12. /// An HTTP status code (`status-code` in RFC 9110 et al.).
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash )]
pub struct StatusCode(NonZeroU16);

13. mystery winner > itoa stack > itoa heap > std::fmt
criterion bench: format_status_code, avg µs

14. // A string of packed 3-ASCII-digit status code
// values for the supported range of [100, 999]
// (900 codes, 2700 bytes).
const CODE_DIGITS: &str = "\
100101102103104105106107108109110\
✂ ✂ ✂
989990991992993994995996997998999";

15. We're not bickering over
assembly anymore

16. My hypothesis

●spectre, meltdown, etc => mitigations
●mitigations => more expensive syscalls
●more expensive syscalls => io_uring

17. Type systems are hard

18. fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>

19. fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>

20. Lifetimes exist in every language

Rust merely explicits them

21. fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>

22. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>

fn poll_read(
&mut self,
buf: &mut [u8],
) -> Poll<Result<usize>>
evented (O_NONBLOCK)

23. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>

fn poll_read(
&mut self,
buf: &mut [u8],
) -> Poll<Result<usize>>
evented (O_NONBLOCK)

24. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>

fn poll_read(
&mut self,
cx: &mut Context<'_>,
buf: &mut [u8],
) -> Poll<Result<usize>>
evented (O_NONBLOCK)

25. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<usize>>
evented (O_NONBLOCK)

26. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
&mut self,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>>
evented (O_NONBLOCK)

27. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>>
evented (O_NONBLOCK)

29. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<Result<()>>
evented (O_NONBLOCK)
fn read(
&mut self,
buf: &mut [u8]
) -> Read<'_, Self>
where Self: Unpin

async

30. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
fn read(
&mut self,
buf: &mut [u8]
) -> Read<'_, Self>
where Self: Unpin
async

31. blocking
fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
async fn read(
&mut self,
buf: &mut [u8],
) -> Result<usize>
where Self: Unpin { ... }
async

32. async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> {
let mut buf = vec![0u8; 4];
s.read_exact(&mut buf).await?;
Ok(buf)
}

33. async stack trace

read(&mut [u8])
read_exact(&mut [u8])
mhhh()

// (not shown: tokio runtime internals)

34. real stack trace

Read::poll(Pin<&mut Read>, &mut Context<'_>)
ReadExact::poll(Pin<&mut ReadExact>, &mut Context<'_>)
Mhh::poll(Pin<&mut Mhh>, &mut Context<'_>)

// (not shown: tokio runtime internals)

35. async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> {
let mut buf = vec![0u8; 4];
tokio::select! {
result = s.read_exact(&mut buf) => {
result?;
Ok(buf)
}
_ = sleep(Duration::from_secs(1)) => {
Err(timeout_err())
}
}
}

36. async fn mhh(mut s: TcpStream) -> io::Result<Vec<u8>> {
let mut buf = vec![0u8; 4];
tokio::select! {
result = s.read_exact(&mut buf) => {
result?;
Ok(buf)
}
_ = sleep(Duration::from_secs(1)) => {
Err(timeout_err())
}
}
}

37. rio::Uring
pub fn recv<'a, Fd, Buf>(
&'a self,
stream: &'a Fd,
iov: &'a Buf
) -> Completion<'a, usize>

38. rio::Uring
impl<'a, C: FromCqe> Drop
for Completion<'a, C> {
fn drop(&mut self) {
self.wait_inner();
}
}

39. let mut buf = vec![0u8; 4];
let mut read_fut = Box::pin(s.read_exact(&mut buf));

tokio::select! {
_ = &mut read_fut => { todo!() }
_ = sleep(Duration::from_secs(1)) => {
std::mem::forget(read_fut);
Err(timeout_err())
}
}

42. tokio_uring::net::TcpStream
async fn read(&self, buf: T) -> (T, Result<usize>)
where T: BoundedBufMut;

43. Fine, I'll rewrite everything
on top of io-uring then.

docs.rs/loona

45. load testing is hard
■macOS = nice for dev, useless for perf
■P-states
■love your noisy neighbors
■stats are hard (coordinated omission etc.)

46. the plan?
■Intel(R) Xeon(R) CPU E3-1275 v6 @ 3.80GHz
■h2load from another dedicated server
■16 clients virtual clients, max 100 streams per
client
■python for automation (running commands over
SSH, CSV => XLS etc.)
■perf for counting cycles, instructions, branches

49. what's next?
●more/better benchmarks
●…on hardware from this decade
●proxying to HTTP/1.1, serving from disk
●messing with: allocators, buffer size
●io_uring: provided buffers, multishot accept/read
●move off of tokio entirely (no atomics needed, no "write to
unpark thread" needed)

50. how do we make that happen?
●money donations
●hardware donations
●expertise donations
●did I mention money

51. Thank you! Let’s connect.
Amos Wenger
[email protected]
@fasterthanlime
https://fasterthanli.me
bearcove