epoll echo server

grom358 · 08-28-2022, 04:46 AM

Doing some basic benchmarks on linux epoll using https://github.com/frevib/epoll-echo-server

Code:

grom@grom-laptop:~/tmp/echo$ dd if=test.bin bs=4096 count=10240000 | ncat localhost 7777 > /dev/null
^C461+0 records in
460+0 records out
1884160 bytes (1.9 MB, 1.8 MiB) copied, 13.3298 s, 141 kB/s

If compare to io_uring implementation:

Code:

grom@grom-laptop:~/tmp/echo$ dd if=test.bin bs=4096 count=10240000 | ncat localhost 7777 > /dev/null
2560000+0 records in
2560000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 10.3257 s, 1.0 GB/s

This test.bin is just a file with random bytes created from /dev/urandom

Clearly I'm doing something wrong. No way epoll is this slow. I looked at the epoll code and nothing stands out as wrong to me. Compared it with some other code examples.

grom358 · 08-28-2022, 06:20 AM

I went and tested using https://github.com/tokio-rs/tokio/tree/master/tokio which is using epoll under the hood:

Code:

grom@grom-laptop:~/tmp/echo$ dd if=test.bin bs=4096 count=10240000 | ncat localhost 7777 > /dev/null
2560000+0 records in
2560000+0 records out
10485760000 bytes (10 GB, 9.8 GiB) copied, 9.74089 s, 1.1 GB/s

So still confused what is so wrong with the C example.

grom358 · 08-28-2022, 07:43 AM

I come across https://github.com/millken/c-example...poll-example.c

The solution was to have a loop over calling read until EAGAIN

Which is in the man page:

Quote:

An application that employs the EPOLLET flag should use
nonblocking file descriptors to avoid having a blocking read or
write starve a task that is handling multiple file descriptors.
The suggested way to use epoll as an edge-triggered (EPOLLET)
interface is as follows:

a) with nonblocking file descriptors; and

b) by waiting for an event only after read(2) or write(2) return
EAGAIN.