9599 Commits

Author SHA1 Message Date
Yossi Gottlieb
2a2d5a7a1f Fix use-after-free issue in spt_copyenv. (#8088)
Seems to have gone unnoticed for a long time, because at least with
glibc it will only be triggered if setenv() was called before spt_init,
which Redis doesn't.

Fixes #8064.

(cherry picked from commit 7e5a6313f0add995c723351532d994118e3e8a6d)
2021-01-12 16:25:37 +02:00
Wang Yuan
a60ed4a599 Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108)
When replica diskless-load type is swapdb in cluster mode, we didn't backup
keys to slots map, so we will lose keys to slots map if fail to sync.
Now we backup keys to slots map at first, and restore it properly when fail.

This commit includes a refactory/cleanup of the backups mechanism (moving it to db.c and re-structuring it a bit).

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit b55a827ea2e19bd6cd48f216e8e6caa34299f9b9)
2021-01-12 16:25:37 +02:00
Wang Yuan
5ed6662145 Reset average ttl when empty databases (#8106)
On FLUSHDB or full sync, reset old average TTL stat.
This Stat is incrementally collected by the master over time when it searches for expired keys. 

(cherry picked from commit c85bf2352d2a3dd04872124d33066403b114a7e7)
2021-01-12 16:25:37 +02:00
Oran Agra
f885e364ba Fix bug with module GIL being released prematurely (#8061)
This is hopefully usually harmles.
The server.ready_keys will usually be empty so the code after releasing
the GIL will soon be done.
The only case where it'll actually process things is when a module
releases a client (or module) blocked on a key, by triggering this NOT
from within a command (e.g. a timer event).

This bug was introduced in redis 6.0.9, see #7903

(cherry picked from commit e6fa47380a5274119ed37c7a5ea7455d4b7dbdcc)
2021-01-12 16:25:37 +02:00
Oran Agra
3ac666b3e6 Fix oom-score-adj-values range, abs options, and bug when used in config file (#8046)
Fix: When oom-score-adj-values is provided in the config file after
oom-score-adj yes, it'll take an immediate action, before
readOOMScoreAdj was acquired, resulting in an error (out of range score
due to uninitialized value. delay the reaction the real call is made by
main().

Since the values are clamped to -1000..1000, and they're
applied as an offset from the value at startup (which may be -1000), we
need to allow the offsets to reach to +2000 so that a value of +1000 is
achievable in case the value at startup was -1000.

Adding an option for absolute values rather than relative ones.

(cherry picked from commit 61954951edbda670bfbae8be0147daa64df95f26)
2021-01-12 16:25:37 +02:00
guybe7
4b37eb13dc EXISTS should not alter LRU, OBJECT should not reveal expired keys on replica (#8016)
The bug was introduced by #5021 which only attempted avoid EXIST on an
already expired key from returning 1 on a replica.

Before that commit, dbExists was used instead of
lookupKeyRead (which had an undesired effect to "touch" the LRU/LFU)

Other than that, this commit fixes OBJECT to also come empty handed on
expired keys in replica.

And DEBUG DIGEST-VALUE to behave like DEBUG OBJECT (get the data from
the key regardless of it's expired state)

(cherry picked from commit f8ae991717f10c837c1a76b2954dae56ecb0e6bc)
2021-01-12 16:25:37 +02:00
Wang Yuan
3a13c654e6 Disable rehash when redis has child process (#8007)
In redisFork(), we don't set child pid, so updateDictResizePolicy()
doesn't take effect, that isn't friendly for copy-on-write.

The bug was introduced this in redis 6.0: 56258c6

(cherry picked from commit 89c78a980807aa1de0a6d0ccde6042450333a783)
2021-01-12 16:25:37 +02:00
John Sully
cbc331be74 Fix intermittent module test failure, resizing the repl backlog is not good enough to clear the backlog
Former-commit-id: d50e4bb8193a349eb3295a53e3111e764676ddcf
2021-01-12 00:29:07 +00:00
benschermel
812959637b created ascii logo image
Former-commit-id: 7f72b214e58164a9731508da6f95241874174210
2021-01-11 11:40:00 -05:00
John Sully
828cf80c94 Merge branch 'repl_perf' into unstable
Former-commit-id: bc8cc1adb301d51e926f2552ce1eb28165ed569b
2020-12-24 02:22:57 +00:00
John Sully
761dd95d2a Don't write in the lock
Former-commit-id: 93b7364aaecbe578be4b75c18c840c4b396c7d4e
2020-12-17 06:52:59 +00:00
John Sully
07a52ceb67 Merge branch 'unstable' into keydbpro
Former-commit-id: 53a062a6b796ccbd43560fed1b05367ba759da96
2020-12-11 18:39:30 +00:00
John Sully
513da06a6f Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: d03c2de9017b73b5d339fe76dd99292f27dfcd52
2020-12-11 03:59:50 +00:00
John Sully
59a1d5085f don't try and free a client that can't be freed
Former-commit-id: 7d3c5f1e64a79f47a103ce97c6991aa473fb697a
2020-12-11 03:59:39 +00:00
John Sully
51ef4343c4 Prevent use after free
Former-commit-id: 81573de393eef083d2209117b270d088a7b3f819
2020-12-11 03:59:26 +00:00
John Sully
fb5b1e8e7d Merge branch 'unstable' into keydbpro
Former-commit-id: e59c3945d3ed0b05a3e42713b091b1b5c96a9db2
2020-12-11 00:49:54 +00:00
John Sully
69f375c45b Remove AE_ASSERT and use the proper assert infra
Former-commit-id: a80dd5d0010d87bd7dbfb7e2ccbca98a8511db69
2020-12-11 00:24:27 +00:00
John Sully
ffd99133ef Fix crash freeing nested hash with list
Former-commit-id: 1eb035e6ad23048df8103300c65fd219297ceff7
2020-12-11 00:23:14 +00:00
John Sully
2a18091d73 Fix test timeouts
Former-commit-id: 6f3feee67678b594fbe9fd592f9f823a7a07b770
2020-12-10 20:09:19 +00:00
John Sully
95181438dd Merge branch 'repl_perf' into keydbpro
Former-commit-id: c4431a6be7a1ddc05abdd4510630fac2a2a6664e
2020-12-10 18:45:52 +00:00
John Sully
706ba4e1dc Merge branch 'unstable' into keydbpro
Former-commit-id: b7a1e16c0f04e8aeb3764e3681fa14fc0f97f6a3
2020-12-10 02:37:28 +00:00
Kajaruban Surendran
8586dfb15c Addressing PR comments
Former-commit-id: bdd14ac517fb1921dd6c398ad10346c74324740d
2020-12-04 14:54:06 -05:00
Kajaruban Surendran
d46ed5d770 Addressing PR comments
Former-commit-id: 63e5d00bdfdb054353c7f3fb7a5890d98e7a7f92
2020-12-04 14:54:06 -05:00
Kajaruban Surendran
5886c1b985 Handling of aeResizeSetSize's return code for current thread
Former-commit-id: c25fb3d74fb9e7adb6ad7ef730355e325e982cd2
2020-12-04 14:54:06 -05:00
Kajaruban Surendran
123aec9bed check the aeResizeSetSize return code
Former-commit-id: 572d41325b9fd5322ab0e41f88e384d69f52f41d
2020-12-04 14:54:06 -05:00
Kajaruban Surendran
024739dee7 Adding thread index to the error message
Former-commit-id: e02120cdf7ca5adec58aa8e84aece89b01d6b751
2020-12-04 14:54:06 -05:00
Kajaruban Surendran
98f0c6b599 Call aePostFunction to change the setsize of each threads by that thread
Former-commit-id: 7b1221a74d06616149436fd44d67a2ad83048e44
2020-12-04 14:54:06 -05:00
Kajaruban Surendran
63d79cc297 Calling initServerThread function cserver.cthreads(server-threads) times than MAX_EVENT_LOOPS(constant)
Former-commit-id: d6b4f994ad776efab6d647947fe1543359ec9401
2020-12-04 14:52:28 -05:00
christianEQ
71b4a7b6d3 Issue: #204 Allocate 8 MB to thread stack
Former-commit-id: 66a41dafc47a20251f5f6776625780dfa26ee505
2020-12-01 14:50:03 -05:00
John Sully
300eca8ff6 Issue: #260 Don't use C++ static_assert in C files
Former-commit-id: f19b9c155ef5b957f7cd9a3017c54aab742ffb2f
2020-11-30 23:01:20 +00:00
John Sully
ecad9741bc code coverage reports should test multithreaded
Former-commit-id: 5d016fdb6b11404d0eddbab651ed17180526a991
2020-11-30 22:59:00 +00:00
VivekSainiEQ
6c16fe69e7 Added remainder of runtest scripts to make lcov command. Code coverage goes from 57.9% to 72.1%
Former-commit-id: f99cba0b92d921b1a2d350f2c7545138485c2a74
2020-11-30 17:47:04 -05:00
VivekSainiEQ
6632bafced Added KeyDB.info to .gitignore file
Former-commit-id: e10dc0dace6b152418aad2bd3ee212a471bca9e8
2020-11-30 17:47:04 -05:00
Kajaruban Surendran
fd03008bbf Configurable option for MOTD
Former-commit-id: 49a89d636ba698dbd0858d5059d3d6387c8c1fc7
2020-11-30 10:42:48 -05:00
VivekSainiEQ
f99824faf4 Added references to KeyDB.info instead of redis.info
Former-commit-id: b94f91b9b25b5476d62bdbe2f054cc61691de9f5
2020-11-25 18:49:20 -05:00
VivekSainiEQ
7ad88d0bac Properly escaped $ in lcov command
Former-commit-id: 06a17c8511cfa8e07962401cc01045878525dc27
2020-11-25 18:49:20 -05:00
VivekSainiEQ
2ab3385733 Updated 'make lcov' command to exclude external dependancies and generate final code coverage percentage
Former-commit-id: eebd70aaa6bec0362ea90148174263fa6ffcc169
2020-11-25 18:49:20 -05:00
VivekSainiEQ
47a18f5131 Made NULL pointer check explicit, issue #257
Former-commit-id: 1d4d2925c2a11bb7fdffe92c17a315c2df011a54
2020-11-25 17:21:37 -05:00
VivekSainiEQ
d4a2ccd50b Added NULL check to function objFromAllocPtr, now NULL input will map to NULL output even if active replica is enabled
Former-commit-id: b67db7c2d5396892cc0c91c3c822a5b9a547badf
2020-11-25 17:21:37 -05:00
John Sully
3d75ddeb87 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 18953b2998e5527260d161ce882b55a9684efab5
2020-11-23 02:01:54 +00:00
John Sully
c179c98870 Fix issue where active replication doesn't replicate RDB data
Former-commit-id: 527b7eb0742567302e0343e3acbed9814c0cbb95
2020-11-23 02:01:40 +00:00
Kajaruban Surendran
3ee8edc8b0 fix for the server crash when the maxclients increased via config set
Former-commit-id: 34a97c447f8cb5753e0747b435c7ce1a5d381c47
2020-11-20 15:20:46 -05:00
John Sully
e8753d1b4b Blocking clients should not crash if an active replica loads a remote RDB with a key in the blocklist
Former-commit-id: 1c525e20b10e0a47af687a0d46faf75229a1cbf5
2020-11-19 23:28:01 +00:00
John Sully
8d8f2a3b9f Merge branch 'RELEASE_6' into keydbpro
Former-commit-id: 2ee33cb050b26cbe34f0b390b8807cdc1b908ded
2020-11-19 20:38:47 +00:00
John Sully
4694161550 Ensure multithreaded tests pass
Former-commit-id: 0bcce502859647529b8835a4a101b64ddf0d625c
2020-11-13 21:08:10 +00:00
John Sully
9120d36bfc Improve replication performance in multithreaded scenarios
Former-commit-id: 96e0b2a2b19df220975e61131cbc535b0c34a828
2020-11-13 21:08:10 +00:00
John Sully
da2a3dbf99 Add ability to compile KeyDB without MOTD and curl
Former-commit-id: f3794d2c420423ca7dcc51688c478f8ada290786
2020-11-13 20:51:30 +00:00
John Sully
e085772d01 Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
Oran Agra
25214bd7dc Redis 6.0.9. 2020-10-27 09:12:01 +02:00
Yossi Gottlieb
3cc673bdd8 Fix wrong zmalloc_size() assumption. (#7963)
When using a system with no malloc_usable_size(), zmalloc_size() assumed
that the heap allocator always returns blocks that are long-padded.

This may not always be the case, and will result with zmalloc_size()
returning a size that is bigger than allocated. At least in one case
this leads to out of bound write, process crash and a potential security
vulnerability.

Effectively this does not affect the vast majority of users, who use
jemalloc or glibc.

This problem along with a (different) fix was reported by Drew DeVault.

(cherry picked from commit 9824fe3e392caa04dc1b4071886e9ac402dd6d95)
2020-10-27 09:12:01 +02:00