20927 Commits

Author SHA1 Message Date
Oran Agra
58ca0b0be2 Assertion and panic, print crash log without generating SIGSEGV
This makes it possible to add tests that generate assertions, and run
them with valgrind, making sure that there are no memory violations
prior to the assertion.

New config options:
- crash-log-enabled - can be disabled for cleaner core dumps
- crash-memcheck-enabled - useful for faster termination after a crash
- use-exit-on-panic - to be used by the test suite so that valgrind can
  detect leaks and memory corruptions

Other changes:
- Crash log is printed even on system that dont HAVE_BACKTRACE, i.e. in
  both SIGSEGV and assert / panic
- Assertion and panic won't print registers and code around EIP (which
  was useless), but will do fast memory test (which may still indicate
  that the assertion was due to memory corrpution)

I had to reshuffle code in order to re-use it, so i extracted come code
into function without actually doing any changes to the code:
- logServerInfo
- logModulesInfo
- doFastMemoryTest (with the exception of it being conditional)
- dumpCodeAroundEIP

changes to the crash report on segfault:
- logRegisters is called right after the stack trace (before info) done
  just in order to have more re-usable code
- stack trace skips the first two items on the stack (the crash log and
  signal handler functions)
2020-08-06 16:47:27 +03:00
Oran Agra
90b717e723 Assertion and panic, print crash log without generating SIGSEGV
This makes it possible to add tests that generate assertions, and run
them with valgrind, making sure that there are no memory violations
prior to the assertion.

New config options:
- crash-log-enabled - can be disabled for cleaner core dumps
- crash-memcheck-enabled - useful for faster termination after a crash
- use-exit-on-panic - to be used by the test suite so that valgrind can
  detect leaks and memory corruptions

Other changes:
- Crash log is printed even on system that dont HAVE_BACKTRACE, i.e. in
  both SIGSEGV and assert / panic
- Assertion and panic won't print registers and code around EIP (which
  was useless), but will do fast memory test (which may still indicate
  that the assertion was due to memory corrpution)

I had to reshuffle code in order to re-use it, so i extracted come code
into function without actually doing any changes to the code:
- logServerInfo
- logModulesInfo
- doFastMemoryTest (with the exception of it being conditional)
- dumpCodeAroundEIP

changes to the crash report on segfault:
- logRegisters is called right after the stack trace (before info) done
  just in order to have more re-usable code
- stack trace skips the first two items on the stack (the crash log and
  signal handler functions)
2020-08-06 16:47:27 +03:00
ShooterIT
e6c811cd85 [Redis-benchmark] Support zset type 2020-08-06 15:36:28 +08:00
ShooterIT
e5a50ed3c4 [Redis-benchmark] Support zset type 2020-08-06 15:36:28 +08:00
John Sully
ef7198e24d keydbpro unstable version should be 0.0.0
Former-commit-id: 4a6e0772f36d5dd721ad2e9b83f9fccce2e38ff0
2020-08-06 02:13:23 +00:00
John Sully
b388da300b keydbpro unstable version should be 0.0.0
Former-commit-id: 4a6e0772f36d5dd721ad2e9b83f9fccce2e38ff0
2020-08-06 02:13:23 +00:00
Itamar Haber
04db7fd8e9 Merge pull request #7092 from itamarhaber/fix-5629
Prevents default save configuration being reset...
2020-08-05 21:16:38 +03:00
Itamar Haber
24c539251f
Merge pull request #7092 from itamarhaber/fix-5629
Prevents default save configuration being reset...
2020-08-05 21:16:38 +03:00
Oran Agra
f519dcb216 redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`
2020-08-05 18:30:43 +03:00
Oran Agra
1aa31e4da9 redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`
2020-08-05 18:30:43 +03:00
Frank Meier
c6ac2588db reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db
2020-08-05 18:30:43 +03:00
Frank Meier
51077c8212 reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db
2020-08-05 18:30:43 +03:00
Tyson Andre
486e39e86e Add a ZMSCORE command returning an array of scores. (#7593)
Syntax: `ZMSCORE KEY MEMBER [MEMBER ...]`

This is an extension of #2359
amended by Tyson Andre to work with the changed unstable API,
add more tests, and consistently return an array.

- It seemed as if it would be more likely to get reviewed
  after updating the implementation.

Currently, multi commands or lua scripting to call zscore multiple times
would almost definitely be less efficient than a native ZMSCORE
for the following reasons:

- Need to fetch the set from the string every time instead of reusing the C
  pointer.
- Using pipelining or multi-commands would result in more bytes sent by
  the client for the repeated `ZMSCORE KEY` sections.
- Need to specially encode the data and decode it from the client
  for lua-based solutions.
- The fastest solution I've seen for large sets(thousands or millions)
  involves lua and a variadic ZADD, then a ZINTERSECT, then a ZRANGE 0 -1,
  then UNLINK of a temporary set (or lua). This is still inefficient.

Co-authored-by: Tyson Andre <tysonandre775@hotmail.com>
2020-08-04 17:49:33 +03:00
Tyson Andre
f11f26cc53
Add a ZMSCORE command returning an array of scores. (#7593)
Syntax: `ZMSCORE KEY MEMBER [MEMBER ...]`

This is an extension of #2359
amended by Tyson Andre to work with the changed unstable API,
add more tests, and consistently return an array.

- It seemed as if it would be more likely to get reviewed
  after updating the implementation.

Currently, multi commands or lua scripting to call zscore multiple times
would almost definitely be less efficient than a native ZMSCORE
for the following reasons:

- Need to fetch the set from the string every time instead of reusing the C
  pointer.
- Using pipelining or multi-commands would result in more bytes sent by
  the client for the repeated `ZMSCORE KEY` sections.
- Need to specially encode the data and decode it from the client
  for lua-based solutions.
- The fastest solution I've seen for large sets(thousands or millions)
  involves lua and a variadic ZADD, then a ZINTERSECT, then a ZRANGE 0 -1,
  then UNLINK of a temporary set (or lua). This is still inefficient.

Co-authored-by: Tyson Andre <tysonandre775@hotmail.com>
2020-08-04 17:49:33 +03:00
John Sully
2c9c4a98ea Latency normalization with SCAN
Former-commit-id: 237f25d854e70d4d7a3095fdf56aaa80770e492e
2020-08-04 04:37:16 +00:00
John Sully
299794b32c Latency normalization with SCAN
Former-commit-id: 237f25d854e70d4d7a3095fdf56aaa80770e492e
2020-08-04 04:37:16 +00:00
hujie
46bff240bf remove duplicate semicolon (#7438) 2020-08-02 13:59:51 +03:00
hujie
2f454e5ae8
remove duplicate semicolon (#7438) 2020-08-02 13:59:51 +03:00
Oran Agra
c5d85c69c7 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail
2020-07-31 13:14:29 +03:00
Oran Agra
f7e7775990
module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail
2020-07-31 13:14:29 +03:00
Oran Agra
27a8ded3db Remove dead code from update_zmalloc_stat_alloc (#7589)
this seems like leftover from before 9d8ceca
2020-07-31 13:01:39 +03:00
Oran Agra
50f5181488
Remove dead code from update_zmalloc_stat_alloc (#7589)
this seems like leftover from before 6eb51bf
2020-07-31 13:01:39 +03:00
fayadexinqing
323d92b689 broadcast a PONG message when slot's migration is over, which may reduce the moved request of clients (#7571) 2020-07-29 18:05:27 -07:00
fayadexinqing
4faad81f50
broadcast a PONG message when slot's migration is over, which may reduce the moved request of clients (#7571) 2020-07-29 18:05:27 -07:00
Kevin McGehee
b6acefddb7 Call propagate instead of writing directly to AOF/replicas (#6658)
Use higher-level API to funnel all generic propagation through
single function call.
2020-07-29 17:54:37 -07:00
Kevin McGehee
b55c1602b5
Call propagate instead of writing directly to AOF/replicas (#6658)
Use higher-level API to funnel all generic propagation through
single function call.
2020-07-29 17:54:37 -07:00
Yossi Gottlieb
342d9a642f Clarify RM_BlockClient() error condition. (#6093) 2020-07-29 17:03:38 +03:00
Yossi Gottlieb
7af05f07ff
Clarify RM_BlockClient() error condition. (#6093) 2020-07-29 17:03:38 +03:00
Arun Ranganathan
444b53e640 Show threading configuration in INFO output (#7446)
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-07-29 08:46:44 +03:00
Arun Ranganathan
f6cad30bb6
Show threading configuration in INFO output (#7446)
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-07-29 08:46:44 +03:00
namtsui
8c03eb90da Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
2020-07-29 08:25:56 +03:00
namtsui
63dae52324
Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
2020-07-29 08:25:56 +03:00
Wen Hui
0a2b019b79 Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562) 2020-07-29 08:22:54 +03:00
Wen Hui
f33acb3f02
Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562) 2020-07-29 08:22:54 +03:00
Wen Hui
2afa308306 fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop
2020-07-28 22:05:48 +03:00
Wen Hui
c69a9b2f61
fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop
2020-07-28 22:05:48 +03:00
Yossi Gottlieb
bc450c5f63 TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
2020-07-28 11:32:47 +03:00
Yossi Gottlieb
784ceeb90d
TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
2020-07-28 11:32:47 +03:00
Jiayuan Chen
198770751f Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-07-28 10:45:21 +03:00
Jiayuan Chen
f31260b044
Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-07-28 10:45:21 +03:00
Yossi Gottlieb
b76a93c362 TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.
2020-07-27 13:26:02 +03:00
Yossi Gottlieb
c75512d89d TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.
2020-07-27 13:26:02 +03:00
grishaf
f8751d03ba Fix prepareForShutdown function declaration (#7566) 2020-07-26 08:27:30 +03:00
grishaf
4126ca466f
Fix prepareForShutdown function declaration (#7566) 2020-07-26 08:27:30 +03:00
zhaozhao.zz
f4e2525c86 more strict check in rioConnRead (#7564) 2020-07-24 14:40:19 +08:00
zhaozhao.zz
da840e9851
more strict check in rioConnRead (#7564) 2020-07-24 14:40:19 +08:00
Meir Shpilraien (Spielrein)
73198c5019 This PR introduces a new loaded keyspace event (#7536)
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-07-23 12:38:51 +03:00
Meir Shpilraien (Spielrein)
8d82639319
This PR introduces a new loaded keyspace event (#7536)
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-07-23 12:38:51 +03:00
Oran Agra
4a118ab30b Fix harmless bug in rioConnRead (#7557)
this code is in use only if the master is disk-based, and the replica is
diskless. In this case we use a buffered reader, but we must avoid reading
past the rdb file, into the command stream. which Luckly rdb.c doesn't
really attempt to do (it knows how much it should read).

When rioConnRead detects that the extra buffering attempt reaches beyond
the read limit it should read less, but if the caller actually requested
more, then it should return with an error rather than a short read. the
bug would have resulted in short read.

in order to fix it, the code must consider the real requested size, and
not the extra buffering size.
2020-07-23 12:37:43 +03:00
Oran Agra
40d7fca368
Fix harmless bug in rioConnRead (#7557)
this code is in use only if the master is disk-based, and the replica is
diskless. In this case we use a buffered reader, but we must avoid reading
past the rdb file, into the command stream. which Luckly rdb.c doesn't
really attempt to do (it knows how much it should read).

When rioConnRead detects that the extra buffering attempt reaches beyond
the read limit it should read less, but if the caller actually requested
more, then it should return with an error rather than a short read. the
bug would have resulted in short read.

in order to fix it, the code must consider the real requested size, and
not the extra buffering size.
2020-07-23 12:37:43 +03:00