gvsafronov/futriix
1
0
Fork 0
Code Issues Pull Requests Actions 8 Packages Projects Releases Wiki Activity
futriix/src
History
Madelyn Olson 2bb29e4aa3
Always compact nodes in stream listpacks after creating new nodes (#11885) 
This change attempts to alleviate a minor memory usage degradation for Redis 6.2 and onwards when using rather large objects (~2k) in streams. Introduced in #6281, we pre-allocate the head nodes of a stream to be 4kb, to limit the amount of unnecessary initial reallocations that are done. However, if we only ever allocate one object because 2 objects exceeds the max_stream_entry_size, we never actually shrink it to fit the single item. This can lead to a lot of excessive memory usage. For smaller item sizes this becomes less of an issue, as the overhead decreases as the items become smaller in size.

This commit also changes the MEMORY USAGE of streams, since it was reporting the lpBytes instead of the allocated size. This introduced an observability issue when diagnosing the memory issue, since Redis reported the same amount of used bytes pre and post change, even though the new implementation allocated more memory.
2023-03-07 15:06:53 -08:00
..
commands
Add missing since filed for new CLIENT NO-TOUCH command (#11829)
2023-02-23 10:56:52 +02:00
modules
use $^ instead of $< for linker in module makefile (#10530)
2022-04-05 17:08:27 +03:00
.gitignore
…
acl.c
Cleanup around script_caller, fix tracking of scripts and ACL logging for RM_Call (#11770)
2023-02-16 08:07:35 +02:00
adlist.c
Remove duplicate code in listAddNodeTail (#11733)
2023-01-20 13:18:52 -08:00
adlist.h
optimize unwatchAllKeys() (#11511)
2022-11-23 17:39:08 +02:00
ae_epoll.c
Fail fast when systemic error occurs in poll (#8749)
2021-04-26 15:52:06 +03:00
ae_evport.c
Fix cluster bus extensions backwards compatibility (#10206)
2022-01-30 19:43:37 +02:00
ae_kqueue.c
Fix the timing of read and write events under kqueue (#9416)
2021-09-02 11:07:51 +03:00
ae_select.c
fix unused argument warning in ae_select.c (#10824)
2022-06-07 14:47:09 +03:00
ae.c
Avoid spurious wakeup on deleted timer event (#11069)
2022-11-25 20:36:33 -08:00
ae.h
Add event loop support to the module API (#10001)
2022-01-18 13:10:07 +02:00
anet.c
Introduce connAddr
2022-08-22 15:01:40 +08:00
anet.h
Introduce connAddr
2022-08-22 15:01:40 +08:00
aof.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
asciilogo.h
Changes http to https in texts (#8495)
2021-03-10 19:11:16 +02:00
atomicvar.h
Adding parentheses and do-while(0) to macros (#11080)
2022-08-03 19:38:08 +03:00
bio.c
Reclaim page cache of RDB file (#11248)
2023-02-12 09:23:29 +02:00
bio.h
Reclaim page cache of RDB file (#11248)
2023-02-12 09:23:29 +02:00
bitops.c
Change compiler optimizations to -O3 -flto (#11207)
2022-10-02 15:15:14 +03:00
blocked.c
Match REDISMODULE_OPEN_KEY_* flags to LOOKUP_* flags (#11772)
2023-02-09 14:59:05 +02:00
call_reply.c
Fix broken protocol in MISCONF error, RM_Yield bugs, RM_Call(EVAL) OOM check bug, and new RM_Call checks. (#10786)
2022-06-01 13:04:22 +03:00
call_reply.h
Add new RM_Call flags for script mode, no writes, and error replies. (#10372)
2022-03-22 14:13:28 +02:00
childinfo.c
fixes for fork child exit and test: #11463 (#11499)
2022-11-12 20:35:34 +02:00
cli_common.c
Fix error/warning on Arm due to unsigned char. (#10572)
2022-04-12 18:55:11 +03:00
cli_common.h
redis-cli: Better --json Unicode support and --quoted-json (#10286)
2022-03-05 21:25:52 +02:00
cluster.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
cluster.h
Propagate message to a node only if the cluster link is healthy. (#11752)
2023-02-02 09:06:24 -08:00
commands.c
Add missing since filed for new CLIENT NO-TOUCH command (#11829)
2023-02-23 10:56:52 +02:00
config.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
config.h
Add GNUC minor version check for redis_unreachable (#11882)
2023-03-05 15:28:50 +02:00
connection.c
Use cached value correctly inside connectionTypeTls() (#11236)
2022-09-06 09:04:33 +03:00
connection.h
Introduce .is_local method for connection layer (#11672)
2023-01-04 10:52:56 +02:00
connhelpers.h
Fixed some typos, add a spell check ci and others minor fix (#8890)
2021-06-10 15:39:33 +03:00
crc16_slottable.h
…
crc16.c
…
crc64.c
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
crc64.h
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
crcspeed.c
Fixed some typos, add a spell check ci and others minor fix (#8890)
2021-06-10 15:39:33 +03:00
crcspeed.h
Added crcspeed library
2020-04-24 17:11:21 -07:00
db.c
Add CLIENT NO-TOUCH for clients to run commands without affecting LRU/LFU of keys (#11483)
2023-02-23 09:07:49 +02:00
debug.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
debugmacro.h
Supplement define guards to prevent multiple inclusion (#10246)
2022-02-06 20:13:34 -08:00
defrag.c
Key as dict entry - memory optimization for sets (#11595)
2023-01-20 18:45:29 +02:00
dict.c
Key as dict entry - memory optimization for sets (#11595)
2023-01-20 18:45:29 +02:00
dict.h
Key as dict entry - memory optimization for sets (#11595)
2023-01-20 18:45:29 +02:00
endianconv.c
Avoid using unsafe C functions (#10932)
2022-07-18 10:56:26 +03:00
endianconv.h
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
eval.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
evict.c
Cleanup: Get rid of server.core_propagates (#11572)
2022-12-20 09:51:50 +02:00
expire.c
add test case and comments for active expiry in the writeable replica (#11789)
2023-02-20 10:23:25 +02:00
fmacros.h
Fix failed tests on Linux Alpine and add a CI job. (#8532)
2021-02-23 12:57:45 +02:00
function_lua.c
Add missing lua_pop in luaGetFromRegistry (#11097)
2022-08-14 11:50:18 +03:00
functions.c
Make dictEntry opaque
2023-01-11 09:59:24 +01:00
functions.h
Functions: Move library meta data to be part of the library payload. (#10500)
2022-04-05 10:27:24 +03:00
geo.c
Speedup GEODIST with fixedpoint_d2string as an optimized version of snprintf %.4f (#11552)
2022-12-04 10:11:38 +02:00
geo.h
…
geohash_helper.c
GEOSEARCH BYBOX: Simplified haversine distance formula when longitude diff is 0 (#11579)
2022-12-05 15:45:04 +02:00
geohash_helper.h
Delete some unimplemented prototype. (#8882)
2021-04-29 08:25:10 +03:00
geohash.c
Fix mistake / outdated doc comment (#10521)
2022-04-04 15:35:49 +03:00
geohash.h
Remove duplicate header file include (#10264)
2022-02-08 16:49:47 +02:00
help.h
Make PFMERGE source key optional in docs, add tests with one input key, add tests on missing source keys (#11205)
2022-10-22 20:41:17 +03:00
hyperloglog.c
Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766)
2023-01-31 17:26:35 +02:00
intset.c
Listpack encoding for sets (#11290)
2022-11-09 19:50:07 +02:00
intset.h
Listpack encoding for sets (#11290)
2022-11-09 19:50:07 +02:00
latency.c
Add warning for suspected slow system clocksource setting (#10636)
2022-05-22 17:10:31 +03:00
latency.h
Add warning for suspected slow system clocksource setting (#10636)
2022-05-22 17:10:31 +03:00
lazyfree.c
Add listpack encoding for list (#11303)
2022-11-16 20:29:46 +02:00
listpack_malloc.h
Optimize listpack for stream usage to avoid repeated reallocs (#6281)
2021-02-16 16:17:38 +02:00
listpack.c
When converting a set to dict, presize for one more element to be added (#11559)
2022-12-06 11:25:51 +02:00
listpack.h
When converting a set to dict, presize for one more element to be added (#11559)
2022-12-06 11:25:51 +02:00
localtime.c
fix typos (#10402)
2022-03-09 13:58:23 +02:00
lolwut5.c
Fixed some typos, add a spell check ci and others minor fix (#8890)
2021-06-10 15:39:33 +03:00
lolwut6.c
Fixed some typos, add a spell check ci and others minor fix (#8890)
2021-06-10 15:39:33 +03:00
lolwut.c
Fixed some typos, add a spell check ci and others minor fix (#8890)
2021-06-10 15:39:33 +03:00
lolwut.h
add include guard for lolwut.h
2020-05-05 23:35:08 -04:00
lzf_c.c
Change lzf to handle values larger than UINT32_MAX (#9776)
2021-11-16 13:12:25 +02:00
lzf_d.c
Change lzf to handle values larger than UINT32_MAX (#9776)
2021-11-16 13:12:25 +02:00
lzf.h
Change lzf to handle values larger than UINT32_MAX (#9776)
2021-11-16 13:12:25 +02:00
lzfP.h
Change lzf to handle values larger than UINT32_MAX (#9776)
2021-11-16 13:12:25 +02:00
Makefile
Use jemalloc by default also on ARM (#11407)
2022-11-07 19:11:12 +02:00
memtest.c
Add sanitizer support and clean up sanitizer findings (#9601)
2021-11-11 13:51:33 +02:00
mkreleasehdr.sh
Build TLS as a loadable module
2022-08-23 12:37:56 +03:00
module.c
Try to trim strings only when applicable (#11817)
2023-02-28 19:38:58 +02:00
monotonic.c
Optimization: Use either monotonic or wall-clock to measure command execution time, to regain up to 4% execution time (#10502)
2022-04-20 14:00:30 +03:00
monotonic.h
Optimization: Use either monotonic or wall-clock to measure command execution time, to regain up to 4% execution time (#10502)
2022-04-20 14:00:30 +03:00
mt19937-64.c
Fix random element selection for large hash tables. (#8133)
2020-12-23 15:52:07 +02:00
mt19937-64.h
Fix random element selection for large hash tables. (#8133)
2020-12-23 15:52:07 +02:00
multi.c
Fix possible memory corruption in FLUSHALL when a client watches more than one key (#11854)
2023-02-28 12:02:55 +02:00
networking.c
Add CLIENT NO-TOUCH for clients to run commands without affecting LRU/LFU of keys (#11483)
2023-02-23 09:07:49 +02:00
notify.c
Add RM_PublishMessageShard (#10543)
2022-04-17 15:43:22 +03:00
object.c
Always compact nodes in stream listpacks after creating new nodes (#11885)
2023-03-07 15:06:53 -08:00
pqsort.c
Fix null pointer subtraction warning (#10498)
2022-04-04 18:38:18 +03:00
pqsort.h
…
pubsub.c
Make dictEntry opaque
2023-01-11 09:59:24 +01:00
quicklist.c
Add listpack encoding for list (#11303)
2022-11-16 20:29:46 +02:00
quicklist.h
Add listpack encoding for list (#11303)
2022-11-16 20:29:46 +02:00
rand.c
…
rand.h
…
rax_malloc.h
…
rax.c
code, typo and comment cleanups (#11280)
2022-10-02 13:56:45 +03:00
rax.h
Squash merging 125 typo/grammar/comment/doc PRs (#7773)
2020-09-10 13:43:38 +03:00
rdb.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
rdb.h
Reclaim page cache of RDB file (#11248)
2023-02-12 09:23:29 +02:00
redis-benchmark.c
adding the ability to add streams to the pre-defined redis-benchmark tests (#11762)
2023-02-02 09:18:22 -08:00
redis-check-aof.c
Unify repeated code in redis-check-aof (#11456)
2022-11-06 14:49:55 +02:00
redis-check-rdb.c
Listpack encoding for sets (#11290)
2022-11-09 19:50:07 +02:00
redis-cli.c
Dont COMMANDS DOCS if not TTY (not interactive) (#11850)
2023-03-03 10:28:55 +02:00
redis-trib.rb
…
redisassert.c
Add sanitizer support and clean up sanitizer findings (#9601)
2021-11-11 13:51:33 +02:00
redisassert.h
Fix a prototype inconsitency of _serverAssert between redisassert.h and redis.h (#10872)
2022-06-19 08:42:12 +03:00
redismodule.h
Match REDISMODULE_OPEN_KEY_* flags to LOOKUP_* flags (#11772)
2023-02-09 14:59:05 +02:00
release.c
Build TLS as a loadable module
2022-08-23 12:37:56 +03:00
replication.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
resp_parser.c
Unified Lua and modules reply parsing and added RESP3 support to RM_Call (#9202)
2021-08-04 16:28:07 +03:00
resp_parser.h
Fix an mistake in comment (#10560)
2022-04-10 09:29:50 +03:00
rio.c
Reclaim page cache of RDB file (#11248)
2023-02-12 09:23:29 +02:00
rio.h
Reclaim page cache of RDB file (#11248)
2023-02-12 09:23:29 +02:00
script_lua.c
Try to trim strings only when applicable (#11817)
2023-02-28 19:38:58 +02:00
script_lua.h
Protect any table which is reachable from globals and added globals white list.
2022-04-27 00:37:40 +03:00
script.c
Cleanup around script_caller, fix tracking of scripts and ACL logging for RM_Call (#11770)
2023-02-16 08:07:35 +02:00
script.h
Cleanup around script_caller, fix tracking of scripts and ACL logging for RM_Call (#11770)
2023-02-16 08:07:35 +02:00
sds.c
Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766)
2023-01-31 17:26:35 +02:00
sds.h
Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766)
2023-01-31 17:26:35 +02:00
sdsalloc.h
Sanitize dump payload: fail RESTORE if memory allocation fails
2020-12-06 14:54:34 +02:00
sentinel.c
Demoting some of the non-warning messages to notice (#10715)
2023-02-19 16:33:19 +02:00
server.c
assert in case resize output buffer will attempt to shrink too much (#11839)
2023-02-26 11:54:29 +02:00
server.h
Try to trim strings only when applicable (#11817)
2023-02-28 19:38:58 +02:00
setcpuaffinity.c
cpu affinity: DragonFlyBSD support (#7956)
2020-10-25 14:14:05 +02:00
setproctitle.c
Fix failed tests on Linux Alpine and add a CI job. (#8532)
2021-02-23 12:57:45 +02:00
sha1.c
Ignore -Wstringop-overread warning for SHA1Transform() on GCC 12 (#11538)
2022-11-24 15:27:16 +02:00
sha1.h
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
sha256.c
Add sanitizer support and clean up sanitizer findings (#9601)
2021-11-11 13:51:33 +02:00
sha256.h
fix explanation of sha256 (#9220)
2021-07-10 10:04:54 -05:00
siphash.c
Add sanitizer support and clean up sanitizer findings (#9601)
2021-11-11 13:51:33 +02:00
slowlog.c
slowlog get command supports passing in -1 to get all logs. (#9018)
2021-06-14 16:46:45 +03:00
slowlog.h
Auto-generate the command table from JSON files (#9656)
2021-12-15 21:23:15 +02:00
socket.c
Introduce .is_local method for connection layer (#11672)
2023-01-04 10:52:56 +02:00
solarisfixes.h
…
sort.c
Avoid integer overflows in SETRANGE and SORT (CVE-2022-35977) (#11720)
2023-01-16 13:49:30 +02:00
sparkline.c
Squash merging 125 typo/grammar/comment/doc PRs (#7773)
2020-09-10 13:43:38 +03:00
sparkline.h
…
stream.h
Stream consumers: Re-purpose seen-time, add active-time (#11099)
2022-11-30 14:21:31 +02:00
strl.c
Avoid using unsafe C functions (#10932)
2022-07-18 10:56:26 +03:00
syncio.c
…
syscheck.c
fix typos in syscheck (#11710)
2023-01-22 16:32:20 +02:00
syscheck.h
Add warning for suspected slow system clocksource setting (#10636)
2022-05-22 17:10:31 +03:00
t_hash.c
Integer Overflow in RAND commands can lead to assertion (CVE-2023-25155) (#11857)
2023-02-28 15:15:46 +02:00
t_list.c
reprocess command when client is unblocked on keys (#11012)
2023-01-01 23:35:42 +02:00
t_set.c
Integer Overflow in RAND commands can lead to assertion (CVE-2023-25155) (#11857)
2023-02-28 15:15:46 +02:00
t_stream.c
Always compact nodes in stream listpacks after creating new nodes (#11885)
2023-03-07 15:06:53 -08:00
t_string.c
Avoid integer overflows in SETRANGE and SORT (CVE-2022-35977) (#11720)
2023-01-16 13:49:30 +02:00
t_zset.c
Integer Overflow in RAND commands can lead to assertion (CVE-2023-25155) (#11857)
2023-02-28 15:15:46 +02:00
testhelp.h
skip new page cache reclame unit test when running in valgrind (#11808)
2023-02-16 10:50:58 +02:00
timeout.c
Blocking command with a 0.001 seconds timeout blocks indefinitely (#11688)
2023-01-08 01:02:48 -08:00
tls.c
Introduce .is_local method for connection layer (#11672)
2023-01-04 10:52:56 +02:00
tracking.c
Prevent Redis from crashing from key tracking invalidations (#11814)
2023-02-21 08:14:41 -08:00
unix.c
Introduce .is_local method for connection layer (#11672)
2023-01-04 10:52:56 +02:00
util.c
String pattern matching had exponential time complexity on pathological patterns (CVE-2022-36021) (#11858)
2023-02-28 15:15:26 +02:00
util.h
Reclaim page cache of RDB file (#11248)
2023-02-12 09:23:29 +02:00
valgrind.sup
Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed
2020-12-06 14:54:34 +02:00
version.h
Add Module API for version and compatibility checks (#7865)
2020-10-11 17:21:58 +03:00
ziplist.c
fix arm build warning due to new compiler optimizations (#11362)
2022-10-07 21:24:54 +03:00
ziplist.h
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
zipmap.c
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
zipmap.h
Add --large-memory flag for REDIS_TEST to enable tests that consume more than 100mb (#9784)
2021-11-16 08:55:10 +02:00
zmalloc.c
Key as dict entry - memory optimization for sets (#11595)
2023-01-20 18:45:29 +02:00
zmalloc.h
zmalloc api set malloc attributes for api giving non aliased pointers. (#11196)
2022-09-05 16:09:28 +03:00