6732 Commits

Author SHA1 Message Date
Wen Hui
3e11f9d967 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.

(cherry picked from commit b7236f0002bcaa15f3a487def9c5069b6c422e65)
2020-09-01 09:27:58 +03:00
Oran Agra
b091e0059d Trim trailing spaces in error replies coming from rejectCommand (#7668)
fe8d6fe749 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.

(cherry picked from commit 05a4af3464b16e42b31dfb1ea62e2a66dc032fb2)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
d13c44583c Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 2ec11f941ae41188e517670fc3224b12c7666541)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
3ed7d0af20 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.

(cherry picked from commit 24efd22e894c90f380aa05a5fa77134bb9423ad3)
2020-09-01 09:27:58 +03:00
Madelyn Olson
e0f6c0de33 Fixed timer warning (#5953)
(cherry picked from commit 46ce35c55e3c6d5c3b2f43194de7694c27ad1d5d)
2020-09-01 09:27:58 +03:00
Wagner Francisco Mezaroba
4772370a87 allow --pattern to be used along with --bigkeys (#3586)
Adds --pattern option to cli's --bigkeys, --hotkeys & --scan modes

(cherry picked from commit 3ec7f8e915fb15634ea5babf217b8a8ecb3e3647)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
a42e7f6876 redis-benchmark: fix wrong random key for hset (#4895)
(cherry picked from commit b1a242b84a7ff4aadf8c47ea83a658db9cf0e98b)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
f573b23187 CLIENT_MASTER should ignore server.proto_max_bulk_len
(cherry picked from commit 0061dbba04918b3f62f943f469f41590d4200919)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
f0f4ef4423 config: proto-max-bulk-len must be 1mb or greater
(cherry picked from commit 46323cec56793c66273005b72f1f29146c108840)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
0d78615532 using proto-max-bulk-len in checkStringLength for SETRANGE and APPEND
(cherry picked from commit 912b48e4b9d9a0738ab3010c4f30e47a3beec3fc)
2020-09-01 09:27:58 +03:00
WuYunlong
5240ad73ef see #7250, fix signature of RedisModule_DeauthenticateAndCloseClient (#7645)
In redismodule.h, RedisModule_DeauthenticateAndCloseClient returns void
`void REDISMODULE_API_FUNC(RedisModule_DeauthenticateAndCloseClient)(RedisModuleCtx *ctx, uint64_t client_id);`
But in module.c, RM_DeauthenticateAndCloseClient returns int
`int RM_DeauthenticateAndCloseClient(RedisModuleCtx *ctx, uint64_t client_id)`

It it safe to change return value from `void` to `int` from the user's perspective.

(cherry picked from commit dd08aec539f8a8b35ccb8b8c953a76c07c739d62)
2020-09-01 09:27:58 +03:00
Wang Yuan
753ff31fa3 Print error info if failed opening config file (#6943)
(cherry picked from commit 514a1e223df5b62cd1fc6aff38126d4a74ebef7e)
2020-09-01 09:27:58 +03:00
Wen Hui
9b7da52c3b fix memory leak in ACLLoadFromFile error handling (#7623)
(cherry picked from commit 30ead1edaee1cea93ee5cdeb975880df12b41f5c)
2020-09-01 09:27:58 +03:00
Oran Agra
6f6658ab60 redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`

(cherry picked from commit f519dcb21626e5fd214960c7d83ee4fab7a3929d)
2020-09-01 09:27:58 +03:00
Frank Meier
d5f7ec90bc reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db

(cherry picked from commit c6ac2588db27de9a71effa6dc21417f7c787ac69)
2020-09-01 09:27:58 +03:00
Oran Agra
3a4ee4b6d6 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail

(cherry picked from commit c5d85c69c75438f98f84e549877c2999a2e450a8)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
2428b64d5a Clarify RM_BlockClient() error condition. (#6093)
(cherry picked from commit 342d9a642f2d21fe906bf1be24d841fa6f314601)
2020-09-01 09:27:58 +03:00
namtsui
3d98418f4a Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
(cherry picked from commit 8c03eb90da3951ed92d8d2729fc6d2ce4feb9e45)
2020-09-01 09:27:58 +03:00
Wen Hui
46686af7f5 Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562)
(cherry picked from commit 0a2b019b79831e6c593c2de943af61ceef4327e1)
2020-09-01 09:27:58 +03:00
Wen Hui
9f873c6c2a fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop

(cherry picked from commit 2afa308306fc641204f10a2bbe2fe35e28b6d259)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
ebdfa71584 TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
(cherry picked from commit bc450c5f63d39d0f0b8c97fa91d15bb8d688b86d)
2020-09-01 09:27:58 +03:00
Jiayuan Chen
7b2af98316 Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit 198770751fdc4c46eb4971ead9b5787fd6ce39fd)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
ba559dc903 TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.

(cherry picked from commit b76a93c362091daafd8a8d15a45d527b7437d013)
2020-09-01 09:27:58 +03:00
grishaf
53f36dc5b6 Fix prepareForShutdown function declaration (#7566)
(cherry picked from commit f8751d03ba9635064c89844d5915c2ec2dcdc827)
2020-09-01 09:27:58 +03:00
Madelyn Olson
32d225c644 Properly reset errno for rdbLoad (#7542)
(cherry picked from commit 9615c7480de53c920baadf6279b527b60de8f0d4)
2020-09-01 09:27:58 +03:00
Wen Hui
fb9b6e0919 Add missing calls to raxStop (#7532)
Since the dynamic allocations in raxIterator are only used for deep walks, memory
leak due to missing call to raxStop can only happen for rax with key names longer
than 32 bytes.

Out of all the missing calls, the only ones that may lead to a leak are the rax
for consumer groups and consumers, and these were only in AOFRW and rdbSave, which
normally only happen in fork or at shutdown.

(cherry picked from commit 0b8d47a9857142203c03846fc4284746695d3dc3)
2020-09-01 09:27:58 +03:00
Wen Hui
25ab97cbe3 add missing caching command in client help (#7399)
(cherry picked from commit e035e5218f4245f5fa5eb0cafc56c572d9f4fa15)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
8739408147 replication: need handle -NOPERM error after send ping (#7538)
(cherry picked from commit 57fbe4cbafb07aa9f036d7e2f5cf88830c27a921)
2020-09-01 09:27:58 +03:00
Oran Agra
1c356d2084 Redis 6.0.6. 2020-07-20 21:08:26 +03:00
yoav-steinberg
575ffeb879 Support passing stack allocated module strings to moduleCreateArgvFromUserFormat (#7528)
Specifically, the key passed to the module aof_rewrite callback is a stack allocated robj. When passing it to RedisModule_EmitAOF (with appropriate "s" fmt string) redis used to panic when trying to inc the ref count of the stack allocated robj. Now support such robjs by coying them to a new heap robj. This doesn't affect performance because using the alternative "c" or "b" format strings also copies the input to a new heap robj.

(cherry picked from commit 8a2b0472a78c09398e4416c06b7c5f343348f96b)
2020-07-20 21:08:26 +03:00
Luke Palmer
2871eb287a Send null for invalidate on flush (#7469)
(cherry picked from commit df4c74ef07139d51b06e3d250107c6f71264c33c)
2020-07-20 21:08:26 +03:00
dmurnane
81d36bc6c8 Notify systemd on sentinel startup (#7168)
Co-authored-by: Daniel Murnane <dmurnane@eitccorp.com>
(cherry picked from commit c292d43fec492f555e6889b6f53b7f380f54bccc)
2020-07-20 21:08:26 +03:00
Developer-Ecosystem-Engineering
1e144396e0 Add registers dump support for Apple silicon (#7453)
Export following environment variables before building on macOS on Apple silicon

export ARCH_FLAGS="-arch arm64"
export SDK_NAME=macosx
export SDK_PATH=$(xcrun --show-sdk-path --sdk $SDK_NAME)
export CFLAGS="$ARCH_FLAGS -isysroot $SDK_PATH -I$SDK_PATH/usr/include"
export CXXFLAGS=$CFLAGS
export LDFLAGS="$ARCH_FLAGS"
export CC="$(xcrun -sdk $SDK_PATH --find clang) $CFLAGS"
export CXX="$(xcrun -sdk $SDK_PATH --find clang++) $CXXFLAGS"
export LD="$(xcrun -sdk $SDK_PATH --find ld) $LDFLAGS"

make
make test
..
All tests passed without errors!

Backtrack logging assumes x86 and required updating

(cherry picked from commit 004479c184f62324cb495b75e222b84e5544311e)
2020-07-20 21:08:26 +03:00
Wen Hui
b06b71a442 correct error msg for num connections reaching maxclients in cluster mode (#7444)
(cherry picked from commit 6c5f98b24ba3cefa20fdfe17c530bcce20f8c151)
2020-07-20 21:08:26 +03:00
WuYunlong
8b20802a09 Fix command help for unexpected options (#7476)
(cherry picked from commit e5166eccee3396a24dfd3a79d3211943e5a3d25e)
2020-07-20 21:08:26 +03:00
WuYunlong
bcf9ca362c Refactor RM_KeyType() by using macro. (#7486)
(cherry picked from commit 7da8c062d5ee8eef8663fe3740267561bf65086d)
2020-07-20 21:08:26 +03:00
Oran Agra
b0f08a04e0 diskless master disconnect replicas when rdb child failed (#7518)
in case the rdb child failed, crashed or terminated unexpectedly redis
would have marked the replica clients with repl_put_online_on_ack and
then kill them only after a minute when no ack was received.

it would not stream anything to these connections, so the only effect of
this bug is a delay of 1 minute in the replicas attempt to re-connect.

(cherry picked from commit a3df70923431bee4aaac0efc46004484a63cb167)
2020-07-20 21:08:26 +03:00
Qu Chen
2c8ecc9713 Replica always reports master's config epoch in CLUSTER NODES output. (#7235)
(cherry picked from commit a517043c7acadbbc0e760092c1abce465636773e)
2020-07-20 21:08:26 +03:00
Oran Agra
90b6ed7a2c RESTORE ABSTTL skip expired keys - leak (#7511)
(cherry picked from commit c10eabeeca9b79a6583ac2db75e7b1aed6c4dceb)
2020-07-20 21:08:26 +03:00
jimgreen2013
8610a17696 fix description about ziplist, the code is ok (#6318)
* fix description about ZIP_BIG_PREVLEN(the code is ok), it's similar to
antirez#4705

* fix description about ziplist entry encoding field (the code is ok),
the max length should be 2^32 - 1 when encoding is 5 bytes

(cherry picked from commit c05d8a0a31286688249de14e42c5a69b553563db)
2020-07-20 21:08:26 +03:00
马永泽
796cbb68c9 fix benchmark in cluster mode fails to authenticate (#7488)
Co-authored-by: Oran Agra <oran@redislabs.com> (styling)
(cherry picked from commit 40e930dab5d9943f760c65021a2b195d402af3ca)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb
27d44fbf73 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.

(cherry picked from commit c611a836f630ecf358b5cfb0d3c5e21c9f0bc105)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb
8bacdf8b48 TLS: Ignore client cert when tls-auth-clients off. (#7457)
(cherry picked from commit 6dfbccc212b0977482fa1aba1dfe7a474ac34f2d)
2020-07-20 21:08:26 +03:00
James Hilliard
e76db7509d Use pkg-config to properly detect libssl and libcrypto libraries (#7452)
(cherry picked from commit 5f16b65a5fae28d2343dd002fcfa2dc802dd18f7)
2020-07-20 21:08:26 +03:00
Yossi Gottlieb
6af3d57beb TLS: Add missing redis-cli options. (#7456)
* Tests: fix and reintroduce redis-cli tests.

These tests have been broken and disabled for 10 years now!

* TLS: add remaining redis-cli support.

This adds support for the redis-cli --pipe, --rdb and --replica options
previously unsupported in --tls mode.

* Fix writeConn().

(cherry picked from commit 99b920534f7710d544c38b870fd10c6053283d99)
2020-07-20 21:08:26 +03:00
Oran Agra
1f9793c996 redis-cli --hotkeys fixed to handle non-printable key names
(cherry picked from commit a11940b112577b5290a3bf4ef7bea7e3028c7d76)
2020-07-20 21:08:26 +03:00
Oran Agra
86e45052c3 redis-cli --bigkeys fixed to handle non-printable key names
(cherry picked from commit 3d760eaea2a32282d0a7edb2ebe88ebf2c07d47a)
2020-07-20 21:08:26 +03:00
Oran Agra
da96665c04 RESTORE ABSTTL won't store expired keys into the db (#7472)
Similarly to EXPIREAT with TTL in the past, which implicitly deletes the
key and return success, RESTORE should not store key that are already
expired into the db.
When used together with REPLACE it should emit a DEL to keyspace
notification and replication stream.

(cherry picked from commit 1c35f8741baa0def2f87eeab17898c79f0147d11)
2020-07-20 21:08:26 +03:00
huangzhw
5728ec50d3 defrag.c activeDefragSdsListAndDict when defrag sdsele, We can't use (#7492)
it to calculate hash, we should use newsds.

(cherry picked from commit 84bef6691bbd5c517bafdd09a3ce57fc17d1d66c)
2020-07-20 21:08:26 +03:00
Oran Agra
298e93c360 tests/valgrind: don't use debug restart (#7404)
* tests/valgrind: don't use debug restart

DEBUG REATART causes two issues:
1. it uses execve which replaces the original process and valgrind doesn't
   have a chance to check for errors, so leaks go unreported.
2. valgrind report invalid calls to close() which we're unable to resolve.

So now the tests use restart_server mechanism in the tests, that terminates
the old server and starts a new one, new PID, but same stdout, stderr.

since the stderr can contain two or more valgrind report, it is not enough
to just check for the absence of leaks, we also need to check for some known
errors, we do both, and fail if we either find an error, or can't find a
report saying there are no leaks.

other changes:
- when killing a server that was already terminated we check for leaks too.
- adding DEBUG LEAK which was used to test it.
- adding --trace-children to valgrind, although no longer needed.
- since the stdout contains two or more runs, we need slightly different way
  of checking if the new process is up (explicitly looking for the new PID)
- move the code that handles --wait-server to happen earlier (before
  watching the startup message in the log), and serve the restarted server too.

* squashme - CR fixes

(cherry picked from commit 8d4f055e43ab554adfce617c971f10c4b6423484)
2020-07-20 21:08:26 +03:00