Lipeng Zhu c21f1dc084

Increase the IO_THREADS_MAX_NUM. (#1220) ...

### Description

This patch try to increase the max number of io-threads from 16(128) to
256 for below reasons:

1. The core number increases a lot in the modern server processors, for
example, the [Sierra
Forest](https://en.wikipedia.org/wiki/Sierra_Forest) processors are
targeted towards with up to **288** cores.
Due to limitation of **_io-threads_** number (16 and 128 ), benchmark
like https://openbenchmarking.org/test/pts/valkey even cannot run on a
high core count server.

2. For some workloads, the bottleneck could be main thread, but for the
other workloads, big key/value which caused heavy io, the bottleneck
could be the io-threads, for example benchmark `memtier_benchmark -s
127.0.0.1 -p 9001 "--data-size" "20000" --ratio 1:0 --key-pattern P:P
--key-minimum=1 --key-maximum 1000000 --test-time 180 -c 50 -t 16
--hide-histogram`. The QPS is still scalable after 16 io-threads.

![image](https://github.com/user-attachments/assets/e980f805-a162-44be-b03e-ab37a9c489cf)
**Fig 1. QPS Scale factor with io-threads number grows.**

Signed-off-by: Lipeng Zhu <lipeng.zhu@intel.com>
Co-authored-by: Wangyang Guo <wangyang.guo@intel.com>

2024-10-27 22:43:23 -07:00

..

commands

Fix BGSAVE CANCEL since and history fields (#1200)

2024-10-21 16:04:47 +02:00

modules

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

unit

Fix undefined-santitizer warning in rax test (#1122)

2024-10-03 17:34:03 +02:00

.clang-format

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

.clang-format-ignore

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

.gitignore

Rename redis.info to valkey.info in LCOV (#259)

2024-04-10 08:59:45 +02:00

acl.c

Fix empty response for ACL CAT category subcommand for module defined categories (#1140)

2024-10-09 21:20:47 -07:00

adlist.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

adlist.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

ae_epoll.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

ae_evport.c

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

ae_kqueue.c

Combine events to eliminate redundant kevent(2) calls (#638)

2024-06-16 21:18:20 -07:00

ae_select.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

ae.c

Change return value of aeTimeProc callback function to long long. (#1057)

2024-09-27 12:20:47 -07:00

ae.h

Change return value of aeTimeProc callback function to long long. (#1057)

2024-09-27 12:20:47 -07:00

anet.c

Add support for setting the group on a unix domain socket (#901)

2024-08-23 11:52:08 -07:00

anet.h

Add support for setting the group on a unix domain socket (#901)

2024-08-23 11:52:08 -07:00

aof.c

Use listLast to replace listIndex -1 (#1163)

2024-10-15 10:29:52 +08:00

asciilogo.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

bio.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

bio.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

bitops.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

blocked.c

Make use of a single listNode pointer for blocking utility lists (#919)

2024-08-20 18:54:53 +08:00

call_reply.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

call_reply.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

childinfo.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

cli_commands.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

cli_commands.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

cli_common.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

cli_common.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

cluster_legacy.c

Limit CLUSTER_CANT_FAILOVER_DATA_AGE log to 10 times period (#1189)

2024-10-24 16:38:47 +08:00

cluster_legacy.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

cluster_slot_stats.c

Cleaned up getSlotOrReply is return -1 instead of C_ERR (#1211)

2024-10-23 17:11:42 +08:00

cluster_slot_stats.h

Handle underflow condition of network out slot stats metric (#840)

2024-07-29 21:50:46 -07:00

cluster.c

Fix bug for CLUSTER SLOTS from EVAL over TLS (#1072)

2024-09-25 03:55:53 -04:00

cluster.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

commands.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

commands.def

Updating command.def by running the generate-command-code.py (#1203)

2024-10-21 13:48:29 -07:00

commands.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

config.c

Increase the IO_THREADS_MAX_NUM. (#1220)

2024-10-27 22:43:23 -07:00

config.h

Increase the IO_THREADS_MAX_NUM. (#1220)

2024-10-27 22:43:23 -07:00

connection.c

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

connection.h

Introduce connection context for Unix socket (#1160)

2024-10-18 17:48:18 -07:00

connhelpers.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

crc16_slottable.h

Remove trademarked language in code comments (#223)

2024-04-09 10:24:03 +02:00

crc16.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

crc64.c

An initial simple unit test framework (#344)

2024-05-02 20:00:04 -07:00

crc64.h

An initial simple unit test framework (#344)

2024-05-02 20:00:04 -07:00

crccombine.c

Update serverpanic output based on 'extended-redis-compatibility' config. (#415)

2024-05-08 12:17:32 -07:00

crccombine.h

CRC64 perf improvements from Redis patches (#350)

2024-04-30 19:32:01 -07:00

crcspeed.c

CRC64 perf improvements from Redis patches (#350)

2024-04-30 19:32:01 -07:00

crcspeed.h

CRC64 perf improvements from Redis patches (#350)

2024-04-30 19:32:01 -07:00

db.c

Fix SORT GET to ignore special pattern # in cluster slot check (#1182)

2024-10-19 14:56:10 +08:00

debug.c

Eliminate hashTypeIterator memory allocation by assigning it on stack (#1105)

2024-10-06 21:34:45 +02:00

debugmacro.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

defrag.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

dict.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

dict.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

endianconv.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

endianconv.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

eval.c

Removing Redis from internal lua function names and comments (#1102)

2024-10-04 12:58:42 -07:00

evict.c

Change return value of aeTimeProc callback function to long long. (#1057)

2024-09-27 12:20:47 -07:00

expire.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

fmacros.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

fmtargs.h

Improve multithreaded performance with memory prefetching (#861)

2024-08-26 21:10:44 -07:00

function_lua.c

Removing Redis from internal lua function names and comments (#1102)

2024-10-04 12:58:42 -07:00

functions.c

Fix module / script call CLUSTER SLOTS / SHARDS fake client check crash (#1063)

2024-09-25 14:50:48 +08:00

functions.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

geo.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

geo.h

…

geohash_helper.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

geohash_helper.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

geohash.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

geohash.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

hyperloglog.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

intrinsics.h

Using intrinsics to optimize counting HyperLogLog trailing bits (#846)

2024-08-27 20:44:32 -07:00

intset.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

intset.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

io_threads.c

fix typo (#1202)

2024-10-21 22:54:40 -04:00

io_threads.h

Io thread work offload (#763)

2024-07-18 19:21:45 -07:00

kvstore.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

kvstore.h

Use metadata to handle the reference relationship between kvstore and dict (#966)

2024-09-02 22:35:24 +08:00

latency.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

latency.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

lazyfree.c

Use metadata to handle the reference relationship between kvstore and dict (#966)

2024-09-02 22:35:24 +08:00

listpack_malloc.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

listpack.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

listpack.h

Migrate listpack.c unit tests to new framework (#949)

2024-09-09 13:01:25 +08:00

localtime.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

logreqres.c

Replace client flags to bitfield (#614)

2024-06-30 11:33:10 -07:00

lolwut5.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

lolwut6.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

lolwut.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

lolwut.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

lzf_c.c

Change lzf to handle values larger than UINT32_MAX (#9776)

2021-11-16 13:12:25 +02:00

lzf_d.c

Change lzf to handle values larger than UINT32_MAX (#9776)

2021-11-16 13:12:25 +02:00

lzf.h

Change lzf to handle values larger than UINT32_MAX (#9776)

2021-11-16 13:12:25 +02:00

lzfP.h

Change lzf to handle values larger than UINT32_MAX (#9776)

2021-11-16 13:12:25 +02:00

Makefile

Fix RDMA build dependence (#1074)

2024-09-25 11:30:45 +02:00

memory_prefetch.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

memory_prefetch.h

Improve multithreaded performance with memory prefetching (#861)

2024-08-26 21:10:44 -07:00

memtest.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

mkreleasehdr.sh

Update Server version to valkey version (#232)

2024-04-05 21:15:57 -07:00

module.c

Add server log when module load fails with busy name (#1084)

2024-10-09 16:10:29 +08:00

monotonic.c

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

monotonic.h

Remove prototypes with empty declarations (#12020)

2023-05-02 17:31:32 -07:00

mt19937-64.c

Fix random element selection for large hash tables. (#8133)

2020-12-23 15:52:07 +02:00

mt19937-64.h

Fix random element selection for large hash tables. (#8133)

2020-12-23 15:52:07 +02:00

multi.c

free client's multi state when it becomes dirty (#961)

2024-08-29 19:20:53 +08:00

networking.c

Trim free space from inline command argument strings to avoid excess memory usage (#1213)

2024-10-23 16:56:32 -07:00

notify.c

Update keyspace notifications link to valkey.io in code comment (#1100)

2024-10-01 04:30:35 -04:00

object.c

fix typo (#1136)

2024-10-08 08:07:51 -07:00

pqsort.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

pqsort.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

pubsub.c

Optimize the per slot dictionary by checking for cluster mode earlier (#995)

2024-09-11 09:53:42 -07:00

quicklist.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

quicklist.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

rand.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

rand.h

Removing Redis from internal lua function names and comments (#1102)

2024-10-04 12:58:42 -07:00

rax_malloc.h

Rax size tracking (#688)

2024-10-02 19:28:55 +02:00

rax.c

Rax size tracking (#688)

2024-10-02 19:28:55 +02:00

rax.h

Rax size tracking (#688)

2024-10-02 19:28:55 +02:00

rdb.c

Maintain return value of rdbSaveDb after writing slot-info aux (#1222)

2024-10-24 09:53:05 -04:00

rdb.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

rdma.c

RDMA: Fix listener priv opaque pointer (#1194)

2024-10-21 10:11:27 +02:00

redismodule.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

release.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

replication.c

Fix minor repldbfd leak in updateReplicasWaitingBgsave if fstat fails (#1226)

2024-10-27 15:23:00 +08:00

resp_parser.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

resp_parser.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

rio.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

rio.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

script_lua.c

Fix FUNCTION KILL error message being displayed as SCRIPT KILL (#1171)

2024-10-15 23:32:42 +08:00

script_lua.h

More rebranding (#606)

2024-06-07 01:40:55 +02:00

script.c

Add cpu-usec metric support under CLUSTER SLOT-STATS command (#20). (#712)

2024-07-22 18:03:28 -07:00

script.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

sds.c

Use sdsAllocSize instead of sdsZmallocSize (#923)

2024-08-27 14:43:01 -07:00

sds.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

sdsalloc.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

sentinel.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

server.c

Do security attack check only when command not found to reduce the critical path (#1212)

2024-10-25 11:13:28 +02:00

server.h

Configurable log and timestamp formats (logfmt, ISO8601) (#1022)

2024-10-25 00:36:32 +02:00

serverassert.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

serverassert.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

setcpuaffinity.c

cpu affinity: DragonFlyBSD support (#7956)

2020-10-25 14:14:05 +02:00

setproctitle.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

sha1.c

Migrate sha1 unit test to new framework (#470)

2024-05-09 19:44:40 -07:00

sha1.h

Migrate sha1 unit test to new framework (#470)

2024-05-09 19:44:40 -07:00

sha256.c

Add sanitizer support and clean up sanitizer findings (#9601)

2021-11-11 13:51:33 +02:00

sha256.h

fix explanation of sha256 (#9220)

2021-07-10 10:04:54 -05:00

siphash.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

slowlog.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

slowlog.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

socket.c

Connection minor fixes (#953)

2024-08-27 16:11:33 +02:00

solarisfixes.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

sort.c

Fix SORT GET to ignore special pattern # in cluster slot check (#1182)

2024-10-19 14:56:10 +08:00

sparkline.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

sparkline.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

stream.h

Remove master and slave from source code (#591)

2024-06-07 14:21:33 -07:00

strl.c

Change strlcat function name from redis to valkey (#440)

2024-05-06 00:09:01 -07:00

syncio.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

syscheck.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

syscheck.h

Add warning for suspected slow system clocksource setting (#10636)

2022-05-22 17:10:31 +03:00

t_hash.c

Move prepareClientToWrite out of loop for HGETALL command (#1119)

2024-10-11 21:28:42 -07:00

t_list.c

Move prepareClientToWrite out of loop for lrange command to reduce the redundant call. (#860)

2024-08-27 19:11:09 -07:00

t_set.c

Use hashtable as the default type of temp set object during sunion/sdiff (#996)

2024-09-10 22:09:18 +02:00

t_stream.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

t_string.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

t_zset.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

testhelp.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

threads_mngr.c

Set up clang-format github action (#538)

2024-05-28 09:27:51 -07:00

threads_mngr.h

Add clang-format configs (#323)

2024-05-22 23:24:12 -07:00

timeout.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

tls.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

tracking.c

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

unix.c

Introduce connection context for Unix socket (#1160)

2024-10-18 17:48:18 -07:00

util.c

Apply CVE patches for CVE-2024-31449, CVE-2024-31227, CVE-2024-31228 (#1115)

2024-10-02 19:22:09 -04:00

util.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

valgrind.sup

Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed

2020-12-06 14:54:34 +02:00

valkey-benchmark.c

Change return value of aeTimeProc callback function to long long. (#1057)

2024-09-27 12:20:47 -07:00

valkey-check-aof.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

valkey-check-rdb.c

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

valkey-cli.c

Remove the restriction that cli --cluster create requires at least 3 primary nodes (#1075)

2024-10-17 13:33:44 +08:00

valkeymodule.h

clang-format: set ColumnLimit to 0 and reformat (#1045)

2024-09-25 01:22:54 +02:00

version.h

Change Redis to Valkey in log messages (#226)

2024-04-17 14:38:21 +02:00

ziplist.c

Clarify the wording from dually to the more common doubly (#1214)

2024-10-23 14:30:42 -07:00

ziplist.h

Copyright update to reflect IP transfer from salvatore to Redis (#740)

2024-08-14 09:20:36 -07:00

zipmap.c

Delete unused parts of zipmap (#973)

2024-08-31 15:42:44 +02:00

zipmap.h

Delete unused parts of zipmap (#973)

2024-08-31 15:42:44 +02:00

zmalloc.c

Fix false sharing issue between main thread and io-threads when access used_memory_thread. (#1179)

2024-10-17 12:37:10 +02:00

zmalloc.h

Rename z{malloc,calloc,realloc,free} into valkey_{malloc,calloc,realloc,free} (#1169)

2024-10-15 13:05:22 +02:00