futriix

History

Optimize the performance of cluster slots for non-continuous slots (#11745 )

This change improves the performance of cluster slots by removing the deferring lengths that are used. Deferring lengths are used in two contexts, the first is for determining the number of replicas that serve a slot (Added in 6.2 as part of a different performance improvement) and the second is for determining the extra networking options for each node (Added in 7.0). For continuous slots, (e.g. 0-8196) this improvement is very negligible, however it becomes more significant when slots are not continuous (e.g. 0 2 4 6 etc) which can happen in production for various users.

The `cluster slots` command is deprecated in favor of `cluster shards`, but since most clients don't support the new command yet I think it's important to not degrade performance here.

Benchmarking shows about 2x improvement, however I wasn't able to get a coherent TPS number since the benchmark process was being saturated long before Redis was, so had to run with multiple benchmarks and merge results. If needed I can add this to our memtier framework. Instead the next section shows the number of usec per call from the benchmark results, which shows significant improvement as well as having a more coherent response in the CoB.

| | New Code | Old Code | % Improvements
|----|----|----- |-----
| Uniform slots| usec_per_call=10.46 | usec_per_call=11.03 | 5.7%
| Worst case (Only even slots)| usec_per_call=963.80 | usec_per_call=2950.99 | 307%

This change also removes some extra white space that I added a when making a code change for adding hostnames.

2023-01-29 18:04:53 -08:00

commands

update sentinel config condition (#11751 )

2023-01-26 10:10:17 +02:00

modules

use $^ instead of $< for linker in module makefile (#10530 )

2022-04-05 17:08:27 +03:00

.gitignore

Ignore gcov/lcov artifacts

2012-04-13 17:52:33 -07:00

acl.c

Retain ACL categories used to generate ACL for displaying them later (#11224 )

2022-11-03 10:14:56 -07:00

adlist.c

Remove duplicate code in listAddNodeTail (#11733 )

2023-01-20 13:18:52 -08:00

adlist.h

optimize unwatchAllKeys() (#11511 )

2022-11-23 17:39:08 +02:00

ae_epoll.c

Fail fast when systemic error occurs in poll (#8749 )

2021-04-26 15:52:06 +03:00

ae_evport.c

Fix cluster bus extensions backwards compatibility (#10206 )

2022-01-30 19:43:37 +02:00

ae_kqueue.c

Fix the timing of read and write events under kqueue (#9416 )

2021-09-02 11:07:51 +03:00

ae_select.c

fix unused argument warning in ae_select.c (#10824 )

2022-06-07 14:47:09 +03:00

ae.c

Avoid spurious wakeup on deleted timer event (#11069 )

2022-11-25 20:36:33 -08:00

ae.h

Add event loop support to the module API (#10001 )

2022-01-18 13:10:07 +02:00

anet.c

Introduce connAddr

2022-08-22 15:01:40 +08:00

anet.h

Introduce connAddr

2022-08-22 15:01:40 +08:00

aof.c

Add explicit error log message for AOF_TRUNCATED status when server load AOF file (#11484 )

2022-11-22 16:18:36 +02:00

asciilogo.h

Changes http to https in texts (#8495 )

2021-03-10 19:11:16 +02:00

atomicvar.h

Adding parentheses and do-while(0) to macros (#11080 )

2022-08-03 19:38:08 +03:00

bio.c

Removing old redundant code from bio.c (#11136 )

2022-08-26 09:09:23 -07:00

bio.h

Removing old redundant code from bio.c (#11136 )

2022-08-26 09:09:23 -07:00

bitops.c

Change compiler optimizations to -O3 -flto (#11207 )

2022-10-02 15:15:14 +03:00

blocked.c

reprocess command when client is unblocked on keys (#11012 )

2023-01-01 23:35:42 +02:00

call_reply.c

Fix broken protocol in MISCONF error, RM_Yield bugs, RM_Call(EVAL) OOM check bug, and new RM_Call checks. (#10786 )

2022-06-01 13:04:22 +03:00

call_reply.h

Add new RM_Call flags for script mode, no writes, and error replies. (#10372 )

2022-03-22 14:13:28 +02:00

childinfo.c

fixes for fork child exit and test: #11463 (#11499 )

2022-11-12 20:35:34 +02:00

cli_common.c

Fix error/warning on Arm due to unsigned char. (#10572 )

2022-04-12 18:55:11 +03:00

cli_common.h

redis-cli: Better --json Unicode support and --quoted-json (#10286 )

2022-03-05 21:25:52 +02:00

cluster.c

Optimize the performance of cluster slots for non-continuous slots (#11745 )

2023-01-29 18:04:53 -08:00

cluster.h

Increase frequency of failover log and emit the status of the election to help debugging (#11665 )

2023-01-11 16:42:23 -08:00

commands.c

update sentinel config condition (#11751 )

2023-01-26 10:10:17 +02:00

config.c

Make dictEntry opaque

2023-01-11 09:59:24 +01:00

config.h

2022-10-02 16:36:31 +03:00

connection.c

Use cached value correctly inside connectionTypeTls() (#11236 )

2022-09-06 09:04:33 +03:00

connection.h

Introduce .is_local method for connection layer (#11672 )

2023-01-04 10:52:56 +02:00

connhelpers.h

Fixed some typos, add a spell check ci and others minor fix (#8890 )

2021-06-10 15:39:33 +03:00

crc16_slottable.h

Added basic support for clusters to redis-benchmark.

2019-03-01 17:53:14 +01:00

crc16.c

RDMF (Redis/Disque merge friendlyness) refactoring WIP 1.

2015-07-26 15:17:18 +02:00

crc64.c

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

crc64.h

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

crcspeed.c

Fixed some typos, add a spell check ci and others minor fix (#8890 )

2021-06-10 15:39:33 +03:00

crcspeed.h

Added crcspeed library

2020-04-24 17:11:21 -07:00

db.c

Fix master client check in expireIfNeeded() for read only replica (#11761 )

2023-01-29 18:00:24 -08:00

debug.c

Make dictEntry opaque

2023-01-11 09:59:24 +01:00

debugmacro.h

Supplement define guards to prevent multiple inclusion (#10246 )

2022-02-06 20:13:34 -08:00

defrag.c

Key as dict entry - memory optimization for sets (#11595 )

2023-01-20 18:45:29 +02:00

dict.c

Key as dict entry - memory optimization for sets (#11595 )

2023-01-20 18:45:29 +02:00

dict.h

Key as dict entry - memory optimization for sets (#11595 )

2023-01-20 18:45:29 +02:00

endianconv.c

Avoid using unsafe C functions (#10932 )

2022-07-18 10:56:26 +03:00

endianconv.h

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

eval.c

Make dictEntry opaque

2023-01-11 09:59:24 +01:00

evict.c

Cleanup: Get rid of server.core_propagates (#11572 )

2022-12-20 09:51:50 +02:00

expire.c

Remove the bucket-cb from dictScan and move dictEntry defrag to dictScanDefrag

2023-01-11 10:25:20 +01:00

fmacros.h

Fix failed tests on Linux Alpine and add a CI job. (#8532 )

2021-02-23 12:57:45 +02:00

function_lua.c

Add missing lua_pop in luaGetFromRegistry (#11097 )

2022-08-14 11:50:18 +03:00

functions.c

Make dictEntry opaque

2023-01-11 09:59:24 +01:00

functions.h

Functions: Move library meta data to be part of the library payload. (#10500 )

2022-04-05 10:27:24 +03:00

geo.c

Speedup GEODIST with fixedpoint_d2string as an optimized version of snprintf %.4f (#11552 )

2022-12-04 10:11:38 +02:00

geo.h

RDMF (Redis/Disque merge friendlyness) refactoring WIP 1.

2015-07-26 15:17:18 +02:00

geohash_helper.c

GEOSEARCH BYBOX: Simplified haversine distance formula when longitude diff is 0 (#11579 )

2022-12-05 15:45:04 +02:00

geohash_helper.h

Delete some unimplemented prototype. (#8882 )

2021-04-29 08:25:10 +03:00

geohash.c

Fix mistake / outdated doc comment (#10521 )

2022-04-04 15:35:49 +03:00

geohash.h

Remove duplicate header file include (#10264 )

2022-02-08 16:49:47 +02:00

help.h

Make PFMERGE source key optional in docs, add tests with one input key, add tests on missing source keys (#11205 )

2022-10-22 20:41:17 +03:00

hyperloglog.c

Hyperloglog avoid allocate more than 'server.hll_sparse_max_bytes' bytes of memory for sparse representation (#11438 )

2022-11-28 17:35:31 +02:00

intset.c

Listpack encoding for sets (#11290 )

2022-11-09 19:50:07 +02:00

intset.h

Listpack encoding for sets (#11290 )

2022-11-09 19:50:07 +02:00

latency.c

Add warning for suspected slow system clocksource setting (#10636 )

2022-05-22 17:10:31 +03:00

latency.h

Add warning for suspected slow system clocksource setting (#10636 )

2022-05-22 17:10:31 +03:00

lazyfree.c

Add listpack encoding for list (#11303 )

2022-11-16 20:29:46 +02:00

listpack_malloc.h

Optimize listpack for stream usage to avoid repeated reallocs (#6281 )

2021-02-16 16:17:38 +02:00

listpack.c

When converting a set to dict, presize for one more element to be added (#11559 )

2022-12-06 11:25:51 +02:00

listpack.h

When converting a set to dict, presize for one more element to be added (#11559 )

2022-12-06 11:25:51 +02:00

localtime.c

fix typos (#10402 )

2022-03-09 13:58:23 +02:00

lolwut5.c

Fixed some typos, add a spell check ci and others minor fix (#8890 )

2021-06-10 15:39:33 +03:00

lolwut6.c

Fixed some typos, add a spell check ci and others minor fix (#8890 )

2021-06-10 15:39:33 +03:00

lolwut.c

Fixed some typos, add a spell check ci and others minor fix (#8890 )

2021-06-10 15:39:33 +03:00

lolwut.h

add include guard for lolwut.h

2020-05-05 23:35:08 -04:00

lzf_c.c

Change lzf to handle values larger than UINT32_MAX (#9776 )

2021-11-16 13:12:25 +02:00

lzf_d.c

Change lzf to handle values larger than UINT32_MAX (#9776 )

2021-11-16 13:12:25 +02:00

lzf.h

Change lzf to handle values larger than UINT32_MAX (#9776 )

2021-11-16 13:12:25 +02:00

lzfP.h

Change lzf to handle values larger than UINT32_MAX (#9776 )

2021-11-16 13:12:25 +02:00

Makefile

Use jemalloc by default also on ARM (#11407 )

2022-11-07 19:11:12 +02:00

memtest.c

Add sanitizer support and clean up sanitizer findings (#9601 )

2021-11-11 13:51:33 +02:00

mkreleasehdr.sh

Build TLS as a loadable module

2022-08-23 12:37:56 +03:00

module.c

Move stat_active_defrag_hits increment to activeDefragAlloc

2023-01-11 10:25:20 +01:00

monotonic.c

Optimization: Use either monotonic or wall-clock to measure command execution time, to regain up to 4% execution time (#10502 )

2022-04-20 14:00:30 +03:00

monotonic.h

Optimization: Use either monotonic or wall-clock to measure command execution time, to regain up to 4% execution time (#10502 )

2022-04-20 14:00:30 +03:00

mt19937-64.c

Fix random element selection for large hash tables. (#8133 )

2020-12-23 15:52:07 +02:00

mt19937-64.h

Fix random element selection for large hash tables. (#8133 )

2020-12-23 15:52:07 +02:00

multi.c

Add redis_ prefix for member2struct, avoid redefined warning in FreeBSD (#11549 )

2022-11-27 10:18:48 +02:00

networking.c

Include peer-addr:port and local-addr:port when logging accept errors (#11622 )

2023-01-04 16:09:27 -08:00

notify.c

Add RM_PublishMessageShard (#10543 )

2022-04-17 15:43:22 +03:00

object.c

Remove the bucket-cb from dictScan and move dictEntry defrag to dictScanDefrag

2023-01-11 10:25:20 +01:00

pqsort.c

Fix null pointer subtraction warning (#10498 )

2022-04-04 18:38:18 +03:00

pqsort.h

BSD license added to every C source and header file.

2012-11-08 18:31:32 +01:00

pubsub.c

Make dictEntry opaque

2023-01-11 09:59:24 +01:00

quicklist.c

Add listpack encoding for list (#11303 )

2022-11-16 20:29:46 +02:00

quicklist.h

Add listpack encoding for list (#11303 )

2022-11-16 20:29:46 +02:00

rand.c

Use 'void' for zero-argument functions

2014-08-08 10:05:32 +02:00

rand.h

BSD license added to every C source and header file.

2012-11-08 18:31:32 +01:00

rax_malloc.h

Cluster: hash slots tracking using a radix tree.

2017-03-27 16:37:22 +02:00

rax.c

code, typo and comment cleanups (#11280 )

2022-10-02 13:56:45 +03:00

rax.h

Squash merging 125 typo/grammar/comment/doc PRs (#7773 )

2020-09-10 13:43:38 +03:00

rdb.c

Fix zuiFind crash / RM_ScanKey hang on SET object listpack encoding (#11581 )

2022-12-09 17:08:01 +02:00

rdb.h

Stream consumers: Re-purpose seen-time, add active-time (#11099 )

2022-11-30 14:21:31 +02:00

redis-benchmark.c

benchmark getRedisConfig exit only when meet NOAUTH error (#11096 )

2022-11-28 20:51:25 +08:00

redis-check-aof.c

Unify repeated code in redis-check-aof (#11456 )

2022-11-06 14:49:55 +02:00

redis-check-rdb.c

Listpack encoding for sets (#11290 )

2022-11-09 19:50:07 +02:00

redis-cli.c

Make dictEntry opaque

2023-01-11 09:59:24 +01:00

redis-trib.rb

Redis-trib deprecated: it no longer works and it

2018-07-13 10:51:58 +02:00

redisassert.c

Add sanitizer support and clean up sanitizer findings (#9601 )

2021-11-11 13:51:33 +02:00

redisassert.h

Fix a prototype inconsitency of _serverAssert between redisassert.h and redis.h (#10872 )

2022-06-19 08:42:12 +03:00

redismodule.h

Add a special notification unlink available only for modules (#9406 )

2022-11-30 11:56:36 +02:00

release.c

Build TLS as a loadable module

2022-08-23 12:37:56 +03:00

replication.c

reprocess command when client is unblocked on keys (#11012 )

2023-01-01 23:35:42 +02:00

resp_parser.c

Unified Lua and modules reply parsing and added RESP3 support to RM_Call (#9202 )

2021-08-04 16:28:07 +03:00

resp_parser.h

Fix an mistake in comment (#10560 )

2022-04-10 09:29:50 +03:00

rio.c

optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (#10587 )

2022-10-15 12:17:41 +03:00

rio.h

Adds isolated netstats for replication. (#10062 )

2022-05-31 08:07:33 +03:00

script_lua.c

Fix potential issue with Lua argv caching, module command filter and libc realloc (#11652 )

2023-01-04 11:03:55 +02:00

script_lua.h

Protect any table which is reachable from globals and added globals white list.

2022-04-27 00:37:40 +03:00

script.c

reprocess command when client is unblocked on keys (#11012 )

2023-01-01 23:35:42 +02:00

script.h

Reintroduce lua argument cache in luaRedisGenericCommand removed in v7.0 (#11541 )

2022-12-05 08:33:53 +02:00

sds.c

Optimize the performance of sdscatrepr in printable characters (#11725 )

2023-01-22 09:16:17 +02:00

sds.h

Fix additional AOF filename issues. (#10110 )

2022-01-18 12:52:27 +02:00

sdsalloc.h

Sanitize dump payload: fail RESTORE if memory allocation fails

2020-12-06 14:54:34 +02:00

sentinel.c

update sentinel config condition (#11751 )

2023-01-26 10:10:17 +02:00

server.c

Key as dict entry - memory optimization for sets (#11595 )

2023-01-20 18:45:29 +02:00

server.h

Move stat_active_defrag_hits increment to activeDefragAlloc

2023-01-11 10:25:20 +01:00

setcpuaffinity.c

cpu affinity: DragonFlyBSD support (#7956 )

2020-10-25 14:14:05 +02:00

setproctitle.c

Fix failed tests on Linux Alpine and add a CI job. (#8532 )

2021-02-23 12:57:45 +02:00

sha1.c

Ignore -Wstringop-overread warning for SHA1Transform() on GCC 12 (#11538 )

2022-11-24 15:27:16 +02:00

sha1.h

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

sha256.c

Add sanitizer support and clean up sanitizer findings (#9601 )

2021-11-11 13:51:33 +02:00

sha256.h

fix explanation of sha256 (#9220 )

2021-07-10 10:04:54 -05:00

siphash.c

Add sanitizer support and clean up sanitizer findings (#9601 )

2021-11-11 13:51:33 +02:00

slowlog.c

slowlog get command supports passing in -1 to get all logs. (#9018 )

2021-06-14 16:46:45 +03:00

slowlog.h

Auto-generate the command table from JSON files (#9656 )

2021-12-15 21:23:15 +02:00

socket.c

Introduce .is_local method for connection layer (#11672 )

2023-01-04 10:52:56 +02:00

solarisfixes.h

Check for __sun macro in solarisfixes.h, not in includers.

2015-01-09 11:23:22 +01:00

sort.c

Avoid integer overflows in SETRANGE and SORT (CVE-2022-35977) (#11720 )

2023-01-16 13:49:30 +02:00

sparkline.c

Squash merging 125 typo/grammar/comment/doc PRs (#7773 )

2020-09-10 13:43:38 +03:00

sparkline.h

LATENCY GRAPH implemented.

2014-07-02 16:31:22 +02:00

stream.h

Stream consumers: Re-purpose seen-time, add active-time (#11099 )

2022-11-30 14:21:31 +02:00

strl.c

Avoid using unsafe C functions (#10932 )

2022-07-18 10:56:26 +03:00

syncio.c

syncWithMaster(): non blocking state machine.

2015-08-06 18:12:20 +02:00

syscheck.c

fix typos in syscheck (#11710 )

2023-01-22 16:32:20 +02:00

syscheck.h

Add warning for suspected slow system clocksource setting (#10636 )

2022-05-22 17:10:31 +03:00

t_hash.c

Obuf limit, exit during loop in *RAND* commands and KEYS (#11676 )

2023-01-16 13:51:18 +02:00

t_list.c

reprocess command when client is unblocked on keys (#11012 )

2023-01-01 23:35:42 +02:00

t_set.c

Key as dict entry - memory optimization for sets (#11595 )

2023-01-20 18:45:29 +02:00

t_stream.c

reprocess command when client is unblocked on keys (#11012 )

2023-01-01 23:35:42 +02:00

t_string.c

Avoid integer overflows in SETRANGE and SORT (CVE-2022-35977) (#11720 )

2023-01-16 13:49:30 +02:00

t_zset.c

Obuf limit, exit during loop in *RAND* commands and KEYS (#11676 )

2023-01-16 13:51:18 +02:00

testhelp.h

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

timeout.c

Blocking command with a 0.001 seconds timeout blocks indefinitely (#11688 )

2023-01-08 01:02:48 -08:00

tls.c

Introduce .is_local method for connection layer (#11672 )

2023-01-04 10:52:56 +02:00

tracking.c

Cleanup: Get rid of server.core_propagates (#11572 )

2022-12-20 09:51:50 +02:00

unix.c

Introduce .is_local method for connection layer (#11672 )

2023-01-04 10:52:56 +02:00

util.c

Fixed small distance replies on GEODIST and GEO commands WITHDIST (#11631 )

2022-12-15 22:25:38 +02:00

util.h

Speedup GEODIST with fixedpoint_d2string as an optimized version of snprintf %.4f (#11552 )

2022-12-04 10:11:38 +02:00

valgrind.sup

Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed

2020-12-06 14:54:34 +02:00

version.h

Add Module API for version and compatibility checks (#7865 )

2020-10-11 17:21:58 +03:00

ziplist.c

fix arm build warning due to new compiler optimizations (#11362 )

2022-10-07 21:24:54 +03:00

ziplist.h

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

zipmap.c

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

zipmap.h

Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784 )

2021-11-16 08:55:10 +02:00

zmalloc.c

Key as dict entry - memory optimization for sets (#11595 )

2023-01-20 18:45:29 +02:00

zmalloc.h

zmalloc api set malloc attributes for api giving non aliased pointers. (#11196 )

2022-09-05 16:09:28 +03:00