27431 Commits

Author SHA1 Message Date
Oran Agra
7cd645a043 Fix redis-cli / redis-sential overflow on some platforms (CVE-2021-32762)
The redis-cli command line tool and redis-sentinel service may be vulnerable
to integer overflow when parsing specially crafted large multi-bulk network
replies. This is a result of a vulnerability in the underlying hiredis
library which does not perform an overflow check before calling the calloc()
heap allocation function.

This issue only impacts systems with heap allocators that do not perform their
own overflow checks. Most modern systems do and are therefore not likely to
be affected. Furthermore, by default redis-sentinel uses the jemalloc allocator
which is also not vulnerable.
2021-10-04 13:59:40 +03:00
Oran Agra
04ba485042 Fix redis-cli / redis-sential overflow on some platforms (CVE-2021-32762)
The redis-cli command line tool and redis-sentinel service may be vulnerable
to integer overflow when parsing specially crafted large multi-bulk network
replies. This is a result of a vulnerability in the underlying hiredis
library which does not perform an overflow check before calling the calloc()
heap allocation function.

This issue only impacts systems with heap allocators that do not perform their
own overflow checks. Most modern systems do and are therefore not likely to
be affected. Furthermore, by default redis-sentinel uses the jemalloc allocator
which is also not vulnerable.
2021-10-04 13:59:40 +03:00
Oran Agra
59c94dedb2 Fix Integer overflow issue with intsets (CVE-2021-32687)
The vulnerability involves changing the default set-max-intset-entries
configuration parameter to a very large value and constructing specially
crafted commands to manipulate sets
2021-10-04 13:59:40 +03:00
Oran Agra
b1149a49b2 Fix Integer overflow issue with intsets (CVE-2021-32687)
The vulnerability involves changing the default set-max-intset-entries
configuration parameter to a very large value and constructing specially
crafted commands to manipulate sets
2021-10-04 13:59:40 +03:00
John Sully
ac22f3c60f disable key cache during load if necessary
Former-commit-id: 68dcf66909e2138da4902bdec98985f4fcd737cf
2021-10-04 07:35:36 +00:00
John Sully
66af52120c disable key cache during load if necessary
Former-commit-id: 68dcf66909e2138da4902bdec98985f4fcd737cf
2021-10-04 07:35:36 +00:00
John Sully
9d78b8bb08 Handle the case where the key cache exceeds maxmemory
Former-commit-id: 01febf902267fec7fe87e6437b0b81fd08b50963
2021-10-04 07:34:05 +00:00
John Sully
db351b697a Handle the case where the key cache exceeds maxmemory
Former-commit-id: 01febf902267fec7fe87e6437b0b81fd08b50963
2021-10-04 07:34:05 +00:00
John Sully
86ec032e2c Don't expand the dictionary if a storage provider is set as we won't use the whole thing
Former-commit-id: 1f07b01144397cec59ec2d94f41c85eceb7248e2
2021-10-04 07:33:38 +00:00
John Sully
e580edabac Don't expand the dictionary if a storage provider is set as we won't use the whole thing
Former-commit-id: 1f07b01144397cec59ec2d94f41c85eceb7248e2
2021-10-04 07:33:38 +00:00
John Sully
d29df021b1 Use the concurrentqueue for multithread load instead of mutex and cvs
Former-commit-id: d5a59113dbfedaf7b62a650cff58a2e8ec01826f
2021-10-04 07:33:03 +00:00
John Sully
4650e051bd Use the concurrentqueue for multithread load instead of mutex and cvs
Former-commit-id: d5a59113dbfedaf7b62a650cff58a2e8ec01826f
2021-10-04 07:33:03 +00:00
John Sully
8a2f2bcb91 Add in the concurrentqueue
Former-commit-id: 319cad462be502b0b7a8c45b634d578b2c1c4e9d
2021-10-02 18:27:21 +00:00
John Sully
a4b8d420de Add in the concurrentqueue
Former-commit-id: 319cad462be502b0b7a8c45b634d578b2c1c4e9d
2021-10-02 18:27:21 +00:00
jsully
1f323d743f Merge branch 'improve_overwrite_performance' into 'keydbpro'
Improve overwrite performance

See merge request external-collab/keydb-pro-6!4

Former-commit-id: 7e59e889f28e30fca409dfb951c31f8cb47effbd
2021-09-28 18:13:12 +00:00
jsully
7cf8433745 Merge branch 'improve_overwrite_performance' into 'keydbpro'
Improve overwrite performance

See merge request external-collab/keydb-pro-6!4

Former-commit-id: 7e59e889f28e30fca409dfb951c31f8cb47effbd
2021-09-28 18:13:12 +00:00
John Sully
663f0bca3e Revert "Don't zero out potentially long buffers" - It has no benefit above the noise floor and potentially is the cause of a multithread slowdown
This reverts commit 9a9841afb3ffac4ebece0ec911b35ce1fe1c7e35 [formerly 91e76ab4a00546278100baf6dfe61c13e802b40e].


Former-commit-id: f651cf67d731a10c3a52e30261c8d8b9d291576c
2021-09-28 18:08:15 +00:00
John Sully
617519e021 Revert "Don't zero out potentially long buffers" - It has no benefit above the noise floor and potentially is the cause of a multithread slowdown
This reverts commit 711c08c62491d0670b329fa7760c6938689910f1 [formerly 91e76ab4a00546278100baf6dfe61c13e802b40e].


Former-commit-id: f651cf67d731a10c3a52e30261c8d8b9d291576c
2021-09-28 18:08:15 +00:00
malavan
81be792183 add hmset and hmget to keydb-benchmark
Former-commit-id: e05466c362ba01e61cf007ad8bfc26e1af21ec4f
2021-09-23 20:53:27 +00:00
malavan
4b43f12a97 add hmset and hmget to keydb-benchmark
Former-commit-id: e05466c362ba01e61cf007ad8bfc26e1af21ec4f
2021-09-23 20:53:27 +00:00
malavan
33c0ae4084 add async tag to hgwtall hkeys hvals hscan
Former-commit-id: 8534de27603bd4cbd10a087840bcb1a90651afa1
2021-09-23 20:18:38 +00:00
malavan
a41ee0af81 add async tag to hgwtall hkeys hvals hscan
Former-commit-id: 8534de27603bd4cbd10a087840bcb1a90651afa1
2021-09-23 20:18:38 +00:00
malavan
cd4864e69e add async lookup to hmget
Former-commit-id: a967b47b60e3f4efbc945dc44a0903f47e07993f
2021-09-23 19:57:18 +00:00
malavan
1a9dc0d312 add async lookup to hmget
Former-commit-id: a967b47b60e3f4efbc945dc44a0903f47e07993f
2021-09-23 19:57:18 +00:00
malavan
e3c09a2e79 Merge branch 'async_commands' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into async_commands
Former-commit-id: a19112ec839da4684aeb1a9d1b41906f4c698944
2021-09-23 17:48:33 +00:00
malavan
d502437cf3 Merge branch 'async_commands' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into async_commands
Former-commit-id: a19112ec839da4684aeb1a9d1b41906f4c698944
2021-09-23 17:48:33 +00:00
John Sully
c1332c89ca Rework the tuning to be better for mixed read/write workloads
Former-commit-id: a4fdd3e3cb41160b20e92e1b1f4f4ebd2ee86a4a
2021-09-23 17:46:56 +00:00
John Sully
154c9d1f79 Rework the tuning to be better for mixed read/write workloads
Former-commit-id: a4fdd3e3cb41160b20e92e1b1f4f4ebd2ee86a4a
2021-09-23 17:46:56 +00:00
John Sully
48ba79618e Move the snapshot stale threshold to a named constant
Former-commit-id: 46d2aaf17abb0fb021aa6d7b393ffc143493d339
2021-09-23 17:45:49 +00:00
John Sully
3294e4f788 Move the snapshot stale threshold to a named constant
Former-commit-id: 46d2aaf17abb0fb021aa6d7b393ffc143493d339
2021-09-23 17:45:49 +00:00
John Sully
0594060e75 prefetch crashes intermittently when a snapshot exists
Former-commit-id: 4a2657023d5d8218c815ce77c2676fd53a634b3f
2021-09-23 17:44:06 +00:00
John Sully
99ff67283c prefetch crashes intermittently when a snapshot exists
Former-commit-id: 4a2657023d5d8218c815ce77c2676fd53a634b3f
2021-09-23 17:44:06 +00:00
malavan
1f4c9e6a75 prev_error_count needs to be thread local
Former-commit-id: 4abede1c40db1dfb4235b9aa1e3177678c5304b1
2021-09-23 17:41:28 +00:00
malavan
4297a3fb32 prev_error_count needs to be thread local
Former-commit-id: 4abede1c40db1dfb4235b9aa1e3177678c5304b1
2021-09-23 17:41:28 +00:00
John Sully
938d318596 Async commands incorrectly think they are dirty
Former-commit-id: 4874247931425767156ab3da934c00a4d4832bcf
2021-09-23 05:24:28 +00:00
John Sully
bba9b516bf Async commands incorrectly think they are dirty
Former-commit-id: 4874247931425767156ab3da934c00a4d4832bcf
2021-09-23 05:24:28 +00:00
John Sully
4a0b8534f4 Create snapshots less often, and use them if they exist
Former-commit-id: ba8a00074171d346813247de0b218e08c8f07b92
2021-09-23 00:19:52 +00:00
John Sully
34d7a95ea6 Create snapshots less often, and use them if they exist
Former-commit-id: ba8a00074171d346813247de0b218e08c8f07b92
2021-09-23 00:19:52 +00:00
John Sully
c3e01cbf9e Clients should initialize with an mvcc checkpoint
Former-commit-id: 5183cb721774be1b769d130359e4bddb1eb8224b
2021-09-23 00:15:47 +00:00
John Sully
96c825d300 Clients should initialize with an mvcc checkpoint
Former-commit-id: 5183cb721774be1b769d130359e4bddb1eb8224b
2021-09-23 00:15:47 +00:00
malavan
ef1ca1be9b update expire tests for async commands
Former-commit-id: 088ecd30a7b360e63d701f811f2fc0b9d77f5eb0
2021-09-22 21:49:30 +00:00
malavan
0a92bbbdfb update expire tests for async commands
Former-commit-id: 088ecd30a7b360e63d701f811f2fc0b9d77f5eb0
2021-09-22 21:49:30 +00:00
malavan
b16b003c13 make keyIsExpired check earlier on async read
Former-commit-id: 32dbfbb3a69681553b65afa2d05ea47058833550
2021-09-22 21:16:36 +00:00
malavan
2e5c69f2d8 make keyIsExpired check earlier on async read
Former-commit-id: 32dbfbb3a69681553b65afa2d05ea47058833550
2021-09-22 21:16:36 +00:00
malavan
dd2d3aca1e remove unneeded mgetCore function
Former-commit-id: 7961d4253b5a356e24d17ab07ac79ba1bf191efd
2021-09-22 21:15:47 +00:00
malavan
3857c65dd9 remove unneeded mgetCore function
Former-commit-id: 7961d4253b5a356e24d17ab07ac79ba1bf191efd
2021-09-22 21:15:47 +00:00
malavan
f9b88a79ce check if key is expired for async lookup
Former-commit-id: d4e3a2688e903fe426e791f8608495fca1c76df6
2021-09-22 19:30:38 +00:00
malavan
5ede3928ca check if key is expired for async lookup
Former-commit-id: d4e3a2688e903fe426e791f8608495fca1c76df6
2021-09-22 19:30:38 +00:00
malavan
acfa705d48 Merge branch 'async_commands' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into async_commands
Former-commit-id: 393add1f3d70f7d1e09ecbce60c65508bc06394d
2021-09-22 19:28:37 +00:00
malavan
0e19299966 Merge branch 'async_commands' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into async_commands
Former-commit-id: 393add1f3d70f7d1e09ecbce60c65508bc06394d
2021-09-22 19:28:37 +00:00