7838 Commits

Author SHA1 Message Date
John Sully
00934cd341 Add snapshot depth to info string
Former-commit-id: 66aef678e9d6d7ab6c28622d54ada0ea8cdd2a99
2020-08-16 00:27:10 +00:00
John Sully
4a356ac13f Don't try and consolidate snapshots with a depth of 1
Former-commit-id: 26c298bd9bc4e2c6981de5c20284120ea54580c3
2020-08-16 00:26:05 +00:00
John Sully
3093d9e117 Only save MVCC tstamps if its an active replica
Former-commit-id: 5aded7dbad055308dc3932f7797fc71684b60966
2020-08-16 00:24:18 +00:00
John Sully
cf4e74006f Don't free snapshot objects in a critical path (under the AE lock)
Former-commit-id: d0da3d3cb74334cc8a2d14f4bdaef7935181700a
2020-08-16 00:13:19 +00:00
John Sully
311b286d41 BUG: Cannot have an EMBSTR bigger than 255
Former-commit-id: e2d6e2d9d585cb7a73f469a8580f9cb0ec71a429
2020-08-15 23:30:22 +00:00
John Sully
82989540c4 remove unnecessary allocation in rdb save
Former-commit-id: dfa76f04fd862e0cc7b4d3284b3922e3a5e5549d
2020-08-15 23:27:32 +00:00
John Sully
ce69a765f8 Remove unnecessary key comparisons in perf critical snapshot paths
Former-commit-id: 40f8a8d102fdca9443399ef03a47df609b146d58
2020-08-15 23:25:58 +00:00
John Sully
1055543817 dictMerge perf improvements
Former-commit-id: 48401ec369c5693689ef658cca518dc94ab1402e
2020-08-15 23:14:29 +00:00
John Sully
eb572c57e2 Rehash efficiency
Former-commit-id: fab383156626ec683881101c22eb2f6c2cea4c5d
2020-08-15 23:05:56 +00:00
John Sully
6c6ca43eef Prevent unnecessary copy when overwriting a value from a snapshot
Former-commit-id: 654a7bc6ea82f4ac45a1c1a25c794e1c27c0d902
2020-08-15 22:59:01 +00:00
John Sully
c804b28446 Merge branch 'unstable' into keydbpro
Former-commit-id: ae482585f0dc470efd73833f74111c2f87a172c5
2020-08-15 21:29:00 +00:00
John Sully
6b8e979434 Prehash the tombstone for cleanup
Former-commit-id: c9d97a7c7448fc769486175bea1648589487c87c
2020-08-14 16:05:39 +00:00
John Sully
9090e26aca Add build flag to disable MVCC tstamps
Former-commit-id: f17d178d03f44abcdaddd851a313dd3f7ec87ed5
2020-08-10 06:10:24 +00:00
John Sully
9928562dad Fix lock after free in module API
Former-commit-id: d88fd1588d292bffc0aa53c299cb52e7a4e91015
2020-08-10 06:10:24 +00:00
John Sully
0cb8d4ca63 MVCC Perf fixes
Former-commit-id: 5a4afe5fb4231bec34d434f9e3214a7320842091
2020-08-10 05:45:56 +00:00
John Sully
4037ee98a4 Fix assert caused by freeTombstoneObjects and null check in consolidate_children
Former-commit-id: 8565a02b331cd2bba2a1c7c6693dfb3f6e61c845
2020-08-10 05:01:36 +00:00
John Sully
848057ff19 RocksDB Read Performance Improvements
Former-commit-id: 80cca4869888e048e10e11f1f20796c482c3e5b3
2020-08-09 23:36:20 +00:00
John Sully
80dc68e497 Start off storage cache with a larger size
Former-commit-id: 5f6fb970a81cc73586ba595b35564e7865e7262d
2020-08-09 00:57:56 +00:00
John Sully
ef7198e24d keydbpro unstable version should be 0.0.0
Former-commit-id: 4a6e0772f36d5dd721ad2e9b83f9fccce2e38ff0
2020-08-06 02:13:23 +00:00
John Sully
2c9c4a98ea Latency normalization with SCAN
Former-commit-id: 237f25d854e70d4d7a3095fdf56aaa80770e492e
2020-08-04 04:37:16 +00:00
Oran Agra
1c356d2084 Redis 6.0.6. 2020-07-20 21:08:26 +03:00
yoav-steinberg
575ffeb879 Support passing stack allocated module strings to moduleCreateArgvFromUserFormat (#7528)
Specifically, the key passed to the module aof_rewrite callback is a stack allocated robj. When passing it to RedisModule_EmitAOF (with appropriate "s" fmt string) redis used to panic when trying to inc the ref count of the stack allocated robj. Now support such robjs by coying them to a new heap robj. This doesn't affect performance because using the alternative "c" or "b" format strings also copies the input to a new heap robj.

(cherry picked from commit 8a2b0472a78c09398e4416c06b7c5f343348f96b)
2020-07-20 21:08:26 +03:00
Luke Palmer
2871eb287a Send null for invalidate on flush (#7469)
(cherry picked from commit df4c74ef07139d51b06e3d250107c6f71264c33c)
2020-07-20 21:08:26 +03:00
dmurnane
81d36bc6c8 Notify systemd on sentinel startup (#7168)
Co-authored-by: Daniel Murnane <dmurnane@eitccorp.com>
(cherry picked from commit c292d43fec492f555e6889b6f53b7f380f54bccc)
2020-07-20 21:08:26 +03:00
Developer-Ecosystem-Engineering
1e144396e0 Add registers dump support for Apple silicon (#7453)
Export following environment variables before building on macOS on Apple silicon

export ARCH_FLAGS="-arch arm64"
export SDK_NAME=macosx
export SDK_PATH=$(xcrun --show-sdk-path --sdk $SDK_NAME)
export CFLAGS="$ARCH_FLAGS -isysroot $SDK_PATH -I$SDK_PATH/usr/include"
export CXXFLAGS=$CFLAGS
export LDFLAGS="$ARCH_FLAGS"
export CC="$(xcrun -sdk $SDK_PATH --find clang) $CFLAGS"
export CXX="$(xcrun -sdk $SDK_PATH --find clang++) $CXXFLAGS"
export LD="$(xcrun -sdk $SDK_PATH --find ld) $LDFLAGS"

make
make test
..
All tests passed without errors!

Backtrack logging assumes x86 and required updating

(cherry picked from commit 004479c184f62324cb495b75e222b84e5544311e)
2020-07-20 21:08:26 +03:00
Wen Hui
b06b71a442 correct error msg for num connections reaching maxclients in cluster mode (#7444)
(cherry picked from commit 6c5f98b24ba3cefa20fdfe17c530bcce20f8c151)
2020-07-20 21:08:26 +03:00
WuYunlong
8b20802a09 Fix command help for unexpected options (#7476)
(cherry picked from commit e5166eccee3396a24dfd3a79d3211943e5a3d25e)
2020-07-20 21:08:26 +03:00
WuYunlong
bcf9ca362c Refactor RM_KeyType() by using macro. (#7486)
(cherry picked from commit 7da8c062d5ee8eef8663fe3740267561bf65086d)
2020-07-20 21:08:26 +03:00
Oran Agra
b0f08a04e0 diskless master disconnect replicas when rdb child failed (#7518)
in case the rdb child failed, crashed or terminated unexpectedly redis
would have marked the replica clients with repl_put_online_on_ack and
then kill them only after a minute when no ack was received.

it would not stream anything to these connections, so the only effect of
this bug is a delay of 1 minute in the replicas attempt to re-connect.

(cherry picked from commit a3df70923431bee4aaac0efc46004484a63cb167)
2020-07-20 21:08:26 +03:00
Qu Chen
2c8ecc9713 Replica always reports master's config epoch in CLUSTER NODES output. (#7235)
(cherry picked from commit a517043c7acadbbc0e760092c1abce465636773e)
2020-07-20 21:08:26 +03:00
Oran Agra
90b6ed7a2c RESTORE ABSTTL skip expired keys - leak (#7511)
(cherry picked from commit c10eabeeca9b79a6583ac2db75e7b1aed6c4dceb)
2020-07-20 21:08:26 +03:00
jimgreen2013
8610a17696 fix description about ziplist, the code is ok (#6318)
* fix description about ZIP_BIG_PREVLEN(the code is ok), it's similar to
antirez#4705

* fix description about ziplist entry encoding field (the code is ok),
the max length should be 2^32 - 1 when encoding is 5 bytes

(cherry picked from commit c05d8a0a31286688249de14e42c5a69b553563db)
2020-07-20 21:08:26 +03:00
马永泽
796cbb68c9 fix benchmark in cluster mode fails to authenticate (#7488)
Co-authored-by: Oran Agra <oran@redislabs.com> (styling)
(cherry picked from commit 40e930dab5d9943f760c65021a2b195d402af3ca)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb
27d44fbf73 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.

(cherry picked from commit c611a836f630ecf358b5cfb0d3c5e21c9f0bc105)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb
8bacdf8b48 TLS: Ignore client cert when tls-auth-clients off. (#7457)
(cherry picked from commit 6dfbccc212b0977482fa1aba1dfe7a474ac34f2d)
2020-07-20 21:08:26 +03:00
James Hilliard
e76db7509d Use pkg-config to properly detect libssl and libcrypto libraries (#7452)
(cherry picked from commit 5f16b65a5fae28d2343dd002fcfa2dc802dd18f7)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb
6af3d57beb TLS: Add missing redis-cli options. (#7456)
* Tests: fix and reintroduce redis-cli tests.

These tests have been broken and disabled for 10 years now!

* TLS: add remaining redis-cli support.

This adds support for the redis-cli --pipe, --rdb and --replica options
previously unsupported in --tls mode.

* Fix writeConn().

(cherry picked from commit 99b920534f7710d544c38b870fd10c6053283d99)
2020-07-20 21:08:26 +03:00
Oran Agra
1f9793c996 redis-cli --hotkeys fixed to handle non-printable key names
(cherry picked from commit a11940b112577b5290a3bf4ef7bea7e3028c7d76)
2020-07-20 21:08:26 +03:00
Oran Agra
86e45052c3 redis-cli --bigkeys fixed to handle non-printable key names
(cherry picked from commit 3d760eaea2a32282d0a7edb2ebe88ebf2c07d47a)
2020-07-20 21:08:26 +03:00
Oran Agra
da96665c04 RESTORE ABSTTL won't store expired keys into the db (#7472)
Similarly to EXPIREAT with TTL in the past, which implicitly deletes the
key and return success, RESTORE should not store key that are already
expired into the db.
When used together with REPLACE it should emit a DEL to keyspace
notification and replication stream.

(cherry picked from commit 1c35f8741baa0def2f87eeab17898c79f0147d11)
2020-07-20 21:08:26 +03:00
huangzhw
5728ec50d3 defrag.c activeDefragSdsListAndDict when defrag sdsele, We can't use (#7492)
it to calculate hash, we should use newsds.

(cherry picked from commit 84bef6691bbd5c517bafdd09a3ce57fc17d1d66c)
2020-07-20 21:08:26 +03:00
Oran Agra
298e93c360 tests/valgrind: don't use debug restart (#7404)
* tests/valgrind: don't use debug restart

DEBUG REATART causes two issues:
1. it uses execve which replaces the original process and valgrind doesn't
   have a chance to check for errors, so leaks go unreported.
2. valgrind report invalid calls to close() which we're unable to resolve.

So now the tests use restart_server mechanism in the tests, that terminates
the old server and starts a new one, new PID, but same stdout, stderr.

since the stderr can contain two or more valgrind report, it is not enough
to just check for the absence of leaks, we also need to check for some known
errors, we do both, and fail if we either find an error, or can't find a
report saying there are no leaks.

other changes:
- when killing a server that was already terminated we check for leaks too.
- adding DEBUG LEAK which was used to test it.
- adding --trace-children to valgrind, although no longer needed.
- since the stdout contains two or more runs, we need slightly different way
  of checking if the new process is up (explicitly looking for the new PID)
- move the code that handles --wait-server to happen earlier (before
  watching the startup message in the log), and serve the restarted server too.

* squashme - CR fixes

(cherry picked from commit 8d4f055e43ab554adfce617c971f10c4b6423484)
2020-07-20 21:08:26 +03:00
Oran Agra
b71330af50 change references to the github repo location (#7479)
(cherry picked from commit 8df988207500e0c76e2276a733be54ad443d78bb)
2020-07-20 21:08:26 +03:00
zhaozhao.zz
a17f059d21 BITOP: propagate only when it really SET or DEL targetkey (#5783)
For example:
    BITOP not targetkey sourcekey

If targetkey and sourcekey doesn't exist, BITOP has no effect,
we do not propagate it, thus can save aof and replica flow.

(cherry picked from commit 2cf11ce5ca6804df9ff65bc90dbd1dfc5e2e497c)
2020-07-20 21:08:26 +03:00
antirez
ae2bcd49db Update comment to clarify change in #7398.
(cherry picked from commit 7bb0342201aa317415da7b512fb69737581bf822)
2020-07-20 21:08:26 +03:00
antirez
ecb2743e8a LPOS: option FIRST renamed RANK.
(cherry picked from commit 5b16c2f1744abf2640549e36fa2744417d427b7c)
2020-07-20 21:08:26 +03:00
Oran Agra
8510588213 EXEC always fails with EXECABORT and multi-state is cleared
In order to support the use of multi-exec in pipeline, it is important that
MULTI and EXEC are never rejected and it is easy for the client to know if the
connection is still in multi state.

It was easy to make sure MULTI and DISCARD never fail (done by previous
commits) since these only change the client state and don't do any actual
change in the server, but EXEC is a different story.

Since in the past, it was possible for clients to handle some EXEC errors and
retry the EXEC, we now can't affort to return any error on EXEC other than
EXECABORT, which now carries with it the real reason for the abort too.

Other fixes in this commit:
- Some checks that where performed at the time of queuing need to be re-
  validated when EXEC runs, for instance if the transaction contains writes
  commands, it needs to be aborted. there was one check that was already done
  in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
  -NOREPLICAS, -MASTERDOWN
- When a command is rejected by processCommand it was rejected with addReply,
  which was not recognized as an error in case the bad command came from the
  master. this will enable to count or MONITOR these errors in the future.
- make it easier for tests to create additional (non deferred) clients.
- add tests for the fixes of this commit.

(cherry picked from commit fe8d6fe74920798c146a587810ee91ff049a9093)
2020-07-20 21:08:26 +03:00
antirez
63073e6a44 Include cluster.h for getClusterConnectionsCount().
(cherry picked from commit 79e15d1686462c750073ed12c1bfab7fa8b090c8)
2020-07-20 21:08:26 +03:00
antirez
5f889001cf Fix BITFIELD i64 type handling, see #7417.
(cherry picked from commit 448a8bb0ed9e970a1c6db594a5dc88ded1036695)
2020-07-20 21:08:26 +03:00
antirez
93fa64f65a Clarify maxclients and cluster in conf. Remove myself too.
(cherry picked from commit c10a7a040f10e32f4a27288499c0c1e104754a1b)
2020-07-20 21:08:26 +03:00