gvsafronov/futriix
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity

Improve CPU usage in tests with IO threads (#823)

Browse Source 
Currently, when running tests with IO threads, we set the
`events-per-io-thread` config to 0. This activated IO threads 100% of
the time, regardless of the number of IO events.

This is causing issues with tests running multiple server instances, as
it drained machine CPU resources. As a result, tests could have very
long runtimes, especially on limited instances.
For example, in
https://github.com/valkey-io/valkey/actions/runs/10066315827/job/27827426986?pr=804,
the `Cluster consistency during live resharding` test ran for 1 hour and
41 minutes.

This PR addresses the issue by:
1. Deactivating IO threads when there are no IO events
2. Continuing to offload all IO events to IO threads

Tested on 16 cores instance, after implementing these changes, the
runtime for the `Cluster consistency during live resharding` test
dropped from 7 minutes an 14 seconds to 3 minutes and 28 seconds.

Signed-off-by: Uri Yagelnik <uriy@amazon.com>
This commit is contained in:
uriyage 2024-07-24 08:24:41 +03:00 committed by GitHub
parent f00c8f6214
commit 3cca26881a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
src/io_threads.c
View File

@ -164,9 +164,9 @@ void waitForClientIO(client *c) {
void adjustIOThreadsByEventLoad(int numevents, int increase_only) {
if (server.io_threads_num == 1) return; /* All I/O is being done by the main thread. */
debugServerAssertWithInfo(NULL, NULL, server.io_threads_num > 1);
int target_threads =
server.events_per_io_thread == 0 ? server.io_threads_num : numevents / server.events_per_io_thread;
/* When events_per_io_thread is set to 0, we offload all events to the IO threads.
* This is used mainly for testing purposes. */
int target_threads = server.events_per_io_thread == 0 ? (numevents + 1) : numevents / server.events_per_io_thread;
target_threads = max(1, min(target_threads, server.io_threads_num));