15157 Commits

Author SHA1 Message Date
John Sully
a0ea81d682 Prevent unnecessary copies in replication scenarios
Former-commit-id: b152a9bd88c081ce98eebe9a7af49649e60e5523
2021-03-28 22:43:15 +00:00
Steve
54fd3d4024
Fix redis-cli handling of rediss:// URL scheme (#8705) 2021-03-29 00:25:30 +03:00
John Sully
84e07e5d24 Make some asserts debug only for perf
Former-commit-id: dc66209f2cf8eadb794dad302bd1ea92890e75b0
2021-03-28 21:08:48 +00:00
John Sully
2e3fd28dcb Excessive rehashing adds latency
Former-commit-id: ee5a4528d61420a18f89a07f4ac63e2181a19738
2021-03-28 20:22:01 +00:00
John Sully
15d9f45b4e Fix thread safety issues with the cache prefetch logic
Former-commit-id: 4892122fc02109d98684a350bd732a0b08a8c7b4
2021-03-28 20:21:44 +00:00
John Sully
01fb2a99bd Prefetch keys even in pure RAM scenarios
Former-commit-id: d7219de186d60a5a437c1828ac97117eaad34819
2021-03-28 20:21:31 +00:00
John Sully
1da03185e6 Eliminate unnecessary lookup in ensure when there is no snapshot
Former-commit-id: 1f363ed7c13c186f0c120ab4f3e321144667f50f
2021-03-28 20:19:15 +00:00
John Sully
f0f4377822 Enable LTO
Former-commit-id: 3ec75184bae92c0e7af579eda8cbe6cfa2375327
2021-03-28 20:13:54 +00:00
John Sully
bb629c6f1e Enable LTO
Former-commit-id: 7241bfc25525e8b27193fb0d4f9eae2e5e98e8d6
2021-03-28 18:27:14 +00:00
John Sully
526761b6f9 Make some asserts debug only for perf
Former-commit-id: c22740df4f7c1468fbe19fd97dfaec20e011b6f1
2021-03-28 18:27:00 +00:00
John Sully
5a28256b80 Excessive rehashing adds latency
Former-commit-id: 9db2277b0fea032abcde089f617fe7e4f5ce12e3
2021-03-28 17:59:02 +00:00
John Sully
73bc5aec95 Fix thread safety issues with the cache prefetch logic
Former-commit-id: a80a128bb64b81115c095d6dd91896ff73048b3d
2021-03-28 17:58:43 +00:00
Huang Zhw
a79bb86856
Remove redundant defrag dictEntry in activeDefragSdsListAndDict. (#8713)
In activeDefragSdsListAndDict when defrag list key sucess, the key
in dict is also replaced. Then the corresponding dictEntry is
defraged. But the dictEntry will be defraged in next step by defrag
the dict values too.
2021-03-28 15:16:30 +03:00
benschermel
2245b1db00 bump version
Former-commit-id: f660cd93574ba96d707c1cf7853e1e5074b0ec3f
2021-03-27 18:30:52 +00:00
benschermel
7f3d6976e7 bump version
Former-commit-id: 33b75e0a2cf8712c976e7af74e2a615bb991c356
2021-03-27 18:26:11 +00:00
John Sully
c356822897 Prefetch keys even in pure RAM scenarios
Former-commit-id: 54c0f11db0b35f34c9dfc3c8b4b76b7b7921bacd
2021-03-26 23:48:24 +00:00
John Sully
1a2803a1ac Eliminate unnecessary lookup in ensure when there is no snapshot
Former-commit-id: 04540531b95ebadf6d60c90c6681c11060eaf907
2021-03-26 23:44:42 +00:00
benschermel
d42ad8f837 bump version
Former-commit-id: 4272a7de24eca935bfe1e13d44bdc0ece58efb55
2021-03-26 22:57:19 +00:00
benschermel
7554a9ed9c update rpm script for centos8
Former-commit-id: f52815d30956d4283452598d33a7fa6466c5e9c5
2021-03-26 22:55:12 +00:00
Huang Zhw
e138698e54
make processCommand check publish channel permissions. (#8534)
Add publish channel permissions check in processCommand.

processCommand didn't check publish channel permissions, so we can
queue a publish command in a transaction. But when exec the transaction,
it will fail with -NOPERM.

We also union keys/commands/channels permissions check togegher in
ACLCheckAllPerm. Remove pubsubCheckACLPermissionsOrReply in 
publishCommand/subscribeCommand/psubscribeCommand. Always 
check permissions in processCommand/execCommand/
luaRedisGenericCommand.
2021-03-26 14:10:01 +03:00
John Sully
72e40cc1c2 Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: 7e0bcf58224921abf2046dc1f58606ed18ebb379
2021-03-26 02:54:50 +00:00
John Sully
931a5cce44 Fix issue where timethread wakes too slowly
Former-commit-id: 60f875f5eb7cb541e537f0f440ce8d5aeac57862
2021-03-26 02:54:33 +00:00
John Sully
95400d3082 Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: c180de1c6d3ef25877ec3b4c50698624dfd3320d
2021-03-26 02:36:27 +00:00
jsully
69dd623558 Merge branch 'timethread_idle' into 'keydbpro'
Sleep the time thread when server sleeps

See merge request keydb-dev/KeyDB-Pro!15

Former-commit-id: 0864c46e38f85f03ea69a18162e3797370c17c91
2021-03-26 02:36:01 +00:00
sundb
db6655deb4
Fix crash when hz config out of range (#8658)
Fix crash when hz config out of range
2021-03-25 18:59:52 -07:00
John Sully
43bb519cc3 Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: 20d2c4fd62bb1867b8970b2bb282bd2ed64bcf6e
2021-03-26 01:50:51 +00:00
John Sully
3d501dfb11 Fix bug where we skip valid dict elements in dictGetRandomKey
Former-commit-id: c25a9a3b84c967428b3598c99a65b14ed2417571
2021-03-26 01:18:39 +00:00
John Sully
788ff61b4b Prevent unnecessary copies in replication scenarios
Former-commit-id: 84c671cdafae7e84beef722fcc8a90b92918c89d
2021-03-25 23:14:48 +00:00
Igor
cf0a909e2d
Introduce fast path to bypass expensive serveClientsBlockedOnKeyByModule call (#8689)
Introduce fast path to bypass expensive serveClientsBlockedOnKeyByModule call
2021-03-25 07:16:03 -07:00
Wang Yuan
1976acfe3e
Remove deprecated comments about saving expire time (#8692) 2021-03-25 15:09:12 +02:00
Oran Agra
497351ad07
Fix SLOWLOG for blocked commands (#8632)
* SLOWLOG didn't record anything for blocked commands because the client
  was reset and argv was already empty. there was a fix for this issue
  specifically for modules, now it works for all blocked clients.
* The original command argv (before being re-written) was also reset
  before adding the slowlog on behalf of the blocked command.
* Latency monitor is now updated regardless of the slowlog flags of the
  command or its execution (their purpose is to hide sensitive info from
  the slowlog, not hide the fact the latency happened).
* Latency monitor now uses real_cmd rather than c->cmd (which may be
  different if the command got re-written, e.g. GEOADD)

Changes:
* Unify shared code between slowlog insertion in call() and
  updateStatsOnUnblock(), hopefully prevent future bugs from happening
  due to the later being overlooked.
* Reset CLIENT_PREVENT_LOGGING in resetClient rather than after command
  processing.
* Add a test for SLOWLOG and BLPOP

Notes:
- real_cmd == c->lastcmd, except inside MULTI and Lua.
- blocked commands never happen in these cases (MULTI / Lua)
- real_cmd == c->cmd, except for when the command is rewritten (e.g.
  GEOADD)
- blocked commands (currently) are never rewritten
- other than the command's CLIENT_PREVENT_LOGGING, and the
  execution flag CLIENT_PREVENT_LOGGING, other cases that we want to
  avoid slowlog are on AOF loading (specifically CMD_CALL_SLOWLOG will
  be off when executed from execCommand that runs from an AOF)
2021-03-25 10:20:27 +02:00
hzzb
4a2c4477c9
Fix misleading comment (#8690)
REPLICAOF actually specifies which master to attach.
2021-03-24 19:39:31 -07:00
Madelyn Olson
98d2e0017d
Skip unecessary check for pong recieved (#8585) 2021-03-24 19:37:38 -07:00
christianEQ
de376f6b93 moved time thread code to end of beforeSleep
Former-commit-id: ac1022c772c7357571829f24c87aa3dc2deade72
2021-03-24 21:16:21 +00:00
christianEQ
ce8705451a serverAsserts on sleeping_threads, remove bare locks
Former-commit-id: 8c64ca2333d8eb2ee92d835907474ec63e127b62
2021-03-24 20:44:40 +00:00
christianEQ
9dde193c1d counter for time thread sleeping threads
Former-commit-id: 6270939d6a2d7820fecac9c5aedf0ff985531f6b
2021-03-24 20:33:26 +00:00
John Sully
d71956e1ce Fix bug where we skip valid dict elements in dictGetRandomKey
Former-commit-id: 626b56b00824573660af0c47b210fd1e8d2cfeb2
2021-03-24 20:26:33 +00:00
christianEQ
ecf2d0aeb5 removed extraneous wakeTimeThread cal
Former-commit-id: cfb5c5d7dc1e09c51ca416a2dbb19eab4bbaa77a
2021-03-24 20:24:07 +00:00
John Sully
0c414b80c1 Don't run code in evict unless we really have to
Former-commit-id: b665b1c2b2df96883a6e2237f7bf3f9b1bec2a89
2021-03-24 20:13:42 +00:00
John Sully
e94095913d Bump RocksDB version for better perf
Former-commit-id: ab4ae61b9c54b3c28dc5fd775d0df3d377c4846a
2021-03-24 20:12:43 +00:00
John Sully
d048d763b6 Fix bug where we skip valid dict elements in dictGetRandomKey
Former-commit-id: 291a3610a679cb1d17caadf6ab067cad41885935
2021-03-24 19:58:51 +00:00
christianEQ
301e4f6a16 added condition variable to time thread; wake on afterSleep, sleep on beforeSleep
Former-commit-id: ff2f2a3aceff2ba4a74951197348d67fc39568b2
2021-03-24 19:27:39 +00:00
Qu Chen
7de6451818
Properly initialize variable to make valgrind happy in checkChildrenDone(). Removed usage for the obsolete wait3() and wait4() in favor of waitpid(), and properly check for the exit status code. (#8666) 2021-03-24 08:41:05 -07:00
yoav-steinberg
3060de88ce
Remove cron saving during BGSAVE test. (#8688)
This fixes a race where a bgsave can start during the test after we verified no bgsave is running.
2021-03-24 15:14:47 +02:00
Oran Agra
f6e1a94e03
Corrupt stream key access to uninitialized memory (#8681)
the corrupt-dump-fuzzer test found a case where an access to a corrupt
stream would have caused accessing to uninitialized memory.
now it'll panic instead.

The issue was that there was a stream that says it has more than 0
records, but looking for the max ID came back empty handed.

p.s. when sanitize-dump-payload is used, this corruption is detected,
and the RESTORE command is gracefully rejected.
2021-03-24 11:33:49 +02:00
Itamar Haber
dc39b0c190
Fixes a typo in DEBUG HELP (#8638) 2021-03-24 11:11:38 +02:00
chenyangyang
ccc39f64f8
Add RM_GetAbsExpire/RM_SetAbsExpire for module. (#8564)
Add a check to ensure that the expire parameters in RM_SetExpire and RM_SetAbsExpire must be positive.
2021-03-24 11:10:15 +02:00
Oran Agra
b45b0d81bb
Fix crash report killed by message (#8683)
We sometimes see the crash report saying we were killed by a random
process even in cases where the crash was spontanius in redis.
for instance, crashes found by the corrupt-dump test.

It looks like this si_pid is sometimes left uninitialized, and a good
way to tell if the crash originated in redis or trigged by outside is to
look at si_code, real signal codes are always > 0, and ones generated by
kill are have si_code of 0 or below.
2021-03-24 08:33:24 +02:00
yoav-steinberg
d026647f4f
Avoid evaluating log arguments when log filtered by level. (#8685) 2021-03-24 08:22:12 +02:00
John Sully
9102c4e610 Disable assert
Former-commit-id: b5d5eeee7cb1900eba192704e5d68c90b9f2b9f2
2021-03-23 23:50:48 +00:00