1304 Commits

Author SHA1 Message Date
malavan
d658891bf0 refactor of asyncCommand to include snapshot creation
Former-commit-id: c0908362162b5f2834b90cd9ce84fd1ee6768834
2021-08-26 13:10:59 +00:00
malavan
65a29f9322 refactor async to offload common code to client
Former-commit-id: 9a7547bfaa0ceff76e604262913fb11a64c627d8
2021-08-25 20:06:06 +00:00
MalavanEQAlpha
e0af0bcb95 Merge pull request #313 from MalavanEQAlpha/fixissue295
Resolve Issue #295 by speeding up keyIsExpired and setting timeout on subkey expires.

Former-commit-id: 9e5e6c2f86548b18ae27b4e1ac20c72517392b07
2021-08-18 17:34:18 -04:00
christianEQ
8bccf9bf12 fixed overly strict assert for 32bit
Former-commit-id: ce4891b33d65038bb0543eb2d3526c5310fee59b
2021-08-13 11:05:12 -04:00
malavan
67ffa578d7 Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: cf40fa5da29ba9574d20d706d0a41ca7001df95a
2021-08-11 20:41:54 +00:00
VivekSainiEQ
2598493a62 Prevent invalid mvcc timestamps from causing critical errors
Former-commit-id: 6f2dbb00119b1d0a1f5a2543d2c6af05f83ef5de
2021-08-11 15:06:14 -04:00
John Sully
a560bcc4a2 Merge branch 'keydbpro_collab' into multithread_load
Former-commit-id: 8016c20f1f9a648e658c816e2f6777c5718d5e19
2021-08-09 20:20:34 +00:00
Christian Legge
9e562f1fe5 Add REPLPING command for use during replication (#329)
* added replping command for initiating replication

* backwards compatibility for replping (retry if not recognized)

* don't allow ping during loading (load balancer fix)

* changed replping warning to notice

Former-commit-id: d7f6bc16145206e96ffeb9941398d564c3dba6a9
2021-07-29 15:50:30 -04:00
John Sully
73560dc7b2 Initialize el so we can detect if it is null
Former-commit-id: ec0f833ea17c668971893aa8f198d22da2e1d289
2021-07-23 19:31:22 +00:00
Huang Zhw
835d15b536 On 32 bit platform, the bit position of GETBIT/SETBIT/BITFIELD/BITCOUNT,BITPOS may overflow (see CVE-2021-32761) (#9191)
GETBIT, SETBIT may access wrong address because of wrap.
BITCOUNT and BITPOS may return wrapped results.
BITFIELD may access the wrong address but also allocate insufficient memory and segfault (see CVE-2021-32761).

This commit uses `uint64_t` or `long long` instead of `size_t`.
related https://github.com/redis/redis/pull/8096

At 32bit platform:
> setbit bit 4294967295 1
(integer) 0
> config set proto-max-bulk-len 536870913
OK
> append bit "\xFF"
(integer) 536870913
> getbit bit 4294967296
(integer) 0

When the bit index is larger than 4294967295, size_t can't hold bit index. In the past,  `proto-max-bulk-len` is limit to 536870912, so there is no problem.

After this commit, bit position is stored in `uint64_t` or `long long`. So when `proto-max-bulk-len > 536870912`, 32bit platforms can still be correct.

For 64bit platform, this problem still exists. The major reason is bit pos 8 times of byte pos. When proto-max-bulk-len is very larger, bit pos may overflow.
But at 64bit platform, we don't have so long string. So this bug may never happen.

Additionally this commit add a test cost `512MB` memory which is tag as `large-memory`. Make freebsd ci and valgrind ci ignore this test.

(cherry picked from commit 71d452876ebf8456afaadd6b3c27988abadd1148)
2021-07-21 21:06:49 +03:00
Yossi Gottlieb
79fa5618f1 Fix CLIENT UNBLOCK crashing modules. (#9167)
Modules that use background threads with thread safe contexts are likely
to use RM_BlockClient() without a timeout function, because they do not
set up a timeout.

Before this commit, `CLIENT UNBLOCK` would result with a crash as the
`NULL` timeout callback is called. Beyond just crashing, this is also
logically wrong as it may throw the module into an unexpected client
state.

This commits makes `CLIENT UNBLOCK` on such clients behave the same as
any other client that is not in a blocked state and therefore cannot be
unblocked.

(cherry picked from commit aa139e2f02292d668370afde8c91575363c2d611)
2021-07-21 21:06:49 +03:00
Oran Agra
6cd84b64f0 Test infra, handle RESP3 attributes and big-numbers and bools (#9235)
- promote the code in DEBUG PROTOCOL to addReplyBigNum
- DEBUG PROTOCOL ATTRIB skips the attribute when client is RESP2
- networking.c addReply for push and attributes generate assertion when
  called on a RESP2 client, anything else would produce a broken
  protocol that clients can't handle.

(cherry picked from commit 6a5bac309e868deef749c36949723b415de2496f)
2021-07-21 21:06:49 +03:00
perryitay
3f4f9b6331 Fail EXEC command in case a watched key is expired (#9194)
There are two issues fixed in this commit: 
1. we want to fail the EXEC command in case there is a watched key that's logically
   expired but not yet deleted by active expire or lazy expire.
2. we saw that currently cache time is update in every `call()` (including nested calls),
   this time is being also being use for the isKeyExpired comparison, we want to update
   the cache time only in the first call (execCommand)

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit ac8b1df8850cc80fbf9ce8c2fbde0c1d3a1b4e91)
2021-07-21 21:06:49 +03:00
John Sully
26750791b3 Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: e4e5c6696c6d831924f314a198b266b10d831e14
2021-07-19 20:51:27 +00:00
John Sully
fc7221739c Info command should show how many keys are cached in RAM vs storage provider
Former-commit-id: 08597bee69bc16ca7c3d5ff31020472774c6eec9
2021-07-19 15:11:33 +00:00
John Sully
29ef9708fa Return the ring buffer to its original size if we temporarily resized it
Former-commit-id: a12ce4a0d105bf7d6ccff95f7dc0044c4676b0a7
2021-07-18 20:45:32 +00:00
John Sully
ca920fed38 Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: 8eec3e948ffd204bb2d6170ad3ca42fa8a2c6d8b
2021-07-09 05:25:04 +00:00
John Sully
6bb2075d3c Merge branch 'unstable' into keydbpro
Former-commit-id: 81ded8a35daa5100cac7299a7d0b5f43ee7ac74f
2021-07-09 04:41:47 +00:00
John Sully
da2aceabcf Merge tag '6.2.3' into unstable
Former-commit-id: 1895dbb7680fa9aadf6040912e89c733abc8c706
2021-07-09 04:40:31 +00:00
John Sully
46c1969997 Merge branch 'repl_backlog_rework' into keydbpro_collab
Former-commit-id: 0983e195757d92d6cbcb63436a1332a6b80e275c
2021-06-30 06:36:45 +00:00
John Sully
d55bcf23bd Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: ecc69952dfd1f145e1aff12bca56a4b4e102d669
2021-06-25 06:21:58 +00:00
vivek
e68f9cfea1 Primitive implementation of bypassing client buffer, stats are all messed up and print statements everywhere
Former-commit-id: 59b2ae8ff451f8a5ac2f3baf3c7b509f6872895e
2021-06-25 03:10:56 +00:00
John Sully
53d7d09f07 Remove fPendingReplicaWrite flag which can instead be calculated on demand
Former-commit-id: ae26afd13f955eb230b5c2cab20ec90db9b714ad
2021-06-25 01:54:38 +00:00
VivekSainiEQ
4ac475ea20 More code cleanup
Former-commit-id: 8e9962b9b7b9093399451bf93d30e5b5d26e3d33
2021-06-16 19:41:55 +00:00
VivekSainiEQ
0b08909024 Merged in 6.2.2
Former-commit-id: 3fb9bb5b4fdfb8e09430c7bef6bf9421065adf0f
2021-06-10 20:11:14 +00:00
VivekSainiEQ
f7d8a0950d Cleaned up code a bit, need to rewrite some comments to reflect new behaviour
Former-commit-id: 850ec766cd71614ce9e61c12414545cd212d3878
2021-06-08 23:10:53 +00:00
VivekSainiEQ
9e344f2577 Seems to pass multithreaded test cases, thank the lord
Former-commit-id: 6cbf70cfff5735f3d4ef2e980945b4b1a1f85971
2021-06-04 20:09:47 +00:00
VivekSainiEQ
09c4939287 Removed unused variables
Former-commit-id: 48663bc480f7279a94c68aeebdd9721ca64f7038
2021-06-03 21:47:33 +00:00
VivekSainiEQ
615fbeb10f Sync works single threaded properly, passes all but one testcase (which hangs)
Former-commit-id: 9a6ca3a5d906b9d87fe70652d218decbb2775ac1
2021-06-02 23:41:36 +00:00
John Sully
6fdd8449ff Merge branch 'keydbpro_collab' into multithread_load
Former-commit-id: b580a5561220bc6dc7bc073135f3045ba0cdda51
2021-06-02 04:24:49 +00:00
John Sully
075ce8db9e Merge branch 'merge_6.2.2' into keydbpro_collab
Former-commit-id: 54fe037e4e215b1e5bdb42b567c6df04a69ba150
2021-06-02 02:05:44 +00:00
John Sully
1d45cc0476 Reduce lock contention when loading to a storage provider
Former-commit-id: 58bc777f2215918043325753b6e2bf89dc3108f7
2021-06-01 20:01:41 +00:00
Madelyn Olson
875a1f07d8 Hide migrate command from slowlog if they include auth (#8859)
Redact commands that include sensitive data from slowlog and monitor

(cherry picked from commit a59e75a475782d86d7ce2b5b9c6f5bb4a5ef0bd6)
2021-06-01 17:03:36 +03:00
yoav-steinberg
15c078df61 Enforce client output buffer soft limit when no traffic. (#8833)
When client breached the output buffer soft limit but then went idle,
we didn't disconnect on soft limit timeout, now we do.
Note this also resolves some sporadic test failures in due to Linux
buffering data which caused tests to fail if during the test we went
back under the soft COB limit.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: sundb <sundbcn@gmail.com>
(cherry picked from commit 152fce5e2cbf947a389da414a431f7331981a374)
2021-06-01 17:03:36 +03:00
John Sully
721e572275 Merge remote-tracking branch 'collab/keydbpro' into multithread_load
Former-commit-id: a09454b3a4b295b2d04bdb7d742db7f9a3e17be7
2021-05-31 01:16:01 +00:00
John Sully
11cd2d6776 Merge branch 'unstable' into keydbpro
Former-commit-id: 14c24ce161ddcbddb701a20062659261397cb0e4
2021-05-29 06:49:05 +00:00
John Sully
97d6875862 Fix failover command test failures
Former-commit-id: d3c37c7159a92319759a33851669862a82cf1b28
2021-05-29 01:19:12 +00:00
John Sully
6b4f686f5f Merge branch 'unstable' into keydbpro
Former-commit-id: 205d8f18d2bb8df5253bab40578b006b7aa73fd5
2021-05-28 23:32:46 +00:00
VivekSainiEQ
4d85f3f6b0 Added more synchronization and fixed some data races
Former-commit-id: 183e015dac6f85df1c94d0761e89bc23d9f53319
2021-05-27 18:57:23 +00:00
VivekSainiEQ
8a7cf8a608 Added transmitted RDB lock
Former-commit-id: 4b32167afc85742d85ff9b47b2c2e0b6b02e140a
2021-05-26 20:10:33 +00:00
John Sully
caba5639f4 Fix TLS Build Errors
Former-commit-id: aea86c5977c01debb4f4a4340c866aa5c5f20f04
2021-05-25 16:55:59 +00:00
John Sully
5267928381 Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17adec5d406d2e30a5b9ea71c07fce5c
2021-05-21 05:54:39 +00:00
John Sully
fe8efa916b Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0edaa724b9d060a6bb8edc5a6de65fa
2021-05-19 02:59:48 +00:00
Malavan
7a799faa04 Merge remote-tracking branch 'upstream/unstable' into fixissue295
Former-commit-id: 9110747194f1a3627adc1a672600a1c6445b8de4
2021-05-17 21:35:43 +00:00
Malavan
d0de1741ed Speedup keyIsExpired by removing subkey search
Former-commit-id: a01564158e40300ab4a0338c0a6e924972385407
2021-05-07 09:40:23 +00:00
John Sully
61e054f826 Fix test hang
Former-commit-id: 23647390e628de07759f8e7d8768a7f638edf01d
2021-05-07 00:28:10 +00:00
John Sully
10383a8234 Add endurance testing to better detect threading bugs
Former-commit-id: 945e428aa110968479fdcdfc2d5c5308a99eadc3
2021-05-06 00:09:07 +00:00
John Sully
f41b79cea6 Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: 446daf8f4c33005096372ad9e29ae9b3d465c64e
2021-05-03 22:58:56 +00:00
jsully
fe9f276f95 Merge branch 'cluster_bugfix' into 'keydbpro'
Set repl_down_since in freeClientAsync

See merge request keydb-dev/KeyDB-Pro!12

Former-commit-id: bd7591fe50b1e5565da2fa521438b88058d4da07
2021-05-03 21:41:05 +00:00
christian
12044bab09 Set repl_down_since in freeClientAsync
Former-commit-id: fb4bd2984651837fadd1dbbbb56327e8452b4a2a
2021-05-03 21:41:04 +00:00