gvsafronov/futriix
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Increase rioConnsetWrite max chunk size to 16K (#817)

Browse Source 
Fixes #796

Currently rioConnWite uses 1024 bytes chunk when feeding the replication
sockets on RDB write. This value seems too small and we will end up with
high syscall overhead.
**This PR sets the max chunk size to 16K.**

Using a simple test program we did not observe any significant
improvement in read/write times going with chunks bigger than 4K but
that might be la bottleneck on network throughput. We did observe sweet
point of CPU utilization at 16K when using TLS.

```
lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 24.04 LTS
Release:	24.04
Codename:	noble
```

```
uname -a
Linux ip-172-31-22-140 6.8.0-1009-aws #9-Ubuntu SMP Fri May 17 14:39:23 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
```
All files were compiled with O3 optimization level.
```
gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
```

**Results:**

Chunk Size | write-time sec | writes total | write cpu-time (usr+sys) |
read-time sec | read total syscalls | read cpu-time (usr+sys)
-- | -- | -- | -- | -- | -- | --
1K | 0.162946 | 102400 | 0.185916 | 0.168479 | 2447 | 0.026945
4K | 0.163036 | 25600 | 0.122629 | 0.168627 | 715 | 0.023382
8K | 0.163942 | 12800 | 0.121131 | 0.168887 | 704 | 0.039388
16K | 0.163614 | 6400 | 0.104742 | 0.168202 | 2483 | 0.025574
64K | 0.16279 | 1600 | 0.098792 | 0.168854 | 1068 | 0.046929
1K - TLS | 0.32648 | 102400 | 0.366961 | 0.330785 | 102400 | 0.337377
4K - TLS | 0.164296 | 25600 | 0.183326 | 0.169032 | 25600 | 0.129952
8K - TLS | 0.163977 | 12800 | 0.163118 | 0.169484 | 12800 | 0.098432
16K - TLS | 0.164861 | 6400 | 0.150666 | 0.169878 | 6383 | 0.094794
64K - TLS | 0.163704 | 6400 | 0.156125 | 0.169323 | 6388 | 0.089971

---------

Signed-off-by: Ran Shidlansik <ranshid@amazon.com>
Signed-off-by: ranshid <88133677+ranshid@users.noreply.github.com>
Signed-off-by: Binbin <binloveplay1314@qq.com>
Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
Co-authored-by: Binbin <binloveplay1314@qq.com>
This commit is contained in:
ranshid 2024-07-24 18:23:06 +03:00 committed by GitHub
parent 3cca26881a
commit 5c073a58e4
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/rio.c
View File

@ -528,7 +528,7 @@ static size_t rioConnsetWrite(rio *r, const void *buf, size_t len) {
* parallelize while the kernel is sending data in background to
* the TCP socket. */
while (len) {
size_t count = len < 1024 ? len : 1024;
size_t count = len < RIO_CONNSET_WRITE_MAX_CHUNK_SIZE ? len : RIO_CONNSET_WRITE_MAX_CHUNK_SIZE;
int broken = 0;
for (j = 0; j < r->io.connset.numconns; j++) {
if (r->io.connset.state[j] != 0) {

5
src/server.h
View File

@ -167,6 +167,11 @@ struct hdr_histogram;
* and hold back additional reading based on this factor. */
#define CHILD_COW_DUTY_CYCLE 100
/* When child process is performing write to connset it iterates on the set
* writing a chunk of the available data to send on each connection.
* This constant defines the maximal size of the chunk to use. */
#define RIO_CONNSET_WRITE_MAX_CHUNK_SIZE 16384
/* Instantaneous metrics tracking. */
#define STATS_METRIC_SAMPLES 16 /* Number of samples per metric. */
#define STATS_METRIC_COMMAND 0 /* Number of commands executed. */