27289 Commits

Author SHA1 Message Date
John Sully
34c014fb41 Merge branch 'unstable' into keydbpro
Former-commit-id: e59c3945d3ed0b05a3e42713b091b1b5c96a9db2
2020-12-11 00:49:54 +00:00
John Sully
fb5b1e8e7d Merge branch 'unstable' into keydbpro
Former-commit-id: e59c3945d3ed0b05a3e42713b091b1b5c96a9db2
2020-12-11 00:49:54 +00:00
John Sully
aab452cf61 Remove AE_ASSERT and use the proper assert infra
Former-commit-id: a80dd5d0010d87bd7dbfb7e2ccbca98a8511db69
2020-12-11 00:24:27 +00:00
John Sully
69f375c45b Remove AE_ASSERT and use the proper assert infra
Former-commit-id: a80dd5d0010d87bd7dbfb7e2ccbca98a8511db69
2020-12-11 00:24:27 +00:00
John Sully
3e8ac111b5 Prevent test crash due to unimplemented save for nested hash
Former-commit-id: 7e0077f293c5f91f240ab401e5f167e487afbfe0
2020-12-11 00:23:44 +00:00
John Sully
367067c98a Prevent test crash due to unimplemented save for nested hash
Former-commit-id: 7e0077f293c5f91f240ab401e5f167e487afbfe0
2020-12-11 00:23:44 +00:00
John Sully
68665aeeb3 Fix crash freeing nested hash with list
Former-commit-id: 1eb035e6ad23048df8103300c65fd219297ceff7
2020-12-11 00:23:14 +00:00
John Sully
ffd99133ef Fix crash freeing nested hash with list
Former-commit-id: 1eb035e6ad23048df8103300c65fd219297ceff7
2020-12-11 00:23:14 +00:00
John Sully
856dd45bf6 Fix test timeouts
Former-commit-id: 6f3feee67678b594fbe9fd592f9f823a7a07b770
2020-12-10 20:09:19 +00:00
John Sully
2a18091d73 Fix test timeouts
Former-commit-id: 6f3feee67678b594fbe9fd592f9f823a7a07b770
2020-12-10 20:09:19 +00:00
John Sully
6b6ae228e1 Merge branch 'repl_perf' into keydbpro
Former-commit-id: c4431a6be7a1ddc05abdd4510630fac2a2a6664e
2020-12-10 18:45:52 +00:00
John Sully
95181438dd Merge branch 'repl_perf' into keydbpro
Former-commit-id: c4431a6be7a1ddc05abdd4510630fac2a2a6664e
2020-12-10 18:45:52 +00:00
John Sully
5c832c5ad0 Merge branch 'unstable' into keydbpro
Former-commit-id: b7a1e16c0f04e8aeb3764e3681fa14fc0f97f6a3
2020-12-10 02:37:28 +00:00
John Sully
706ba4e1dc Merge branch 'unstable' into keydbpro
Former-commit-id: b7a1e16c0f04e8aeb3764e3681fa14fc0f97f6a3
2020-12-10 02:37:28 +00:00
Yossi Gottlieb
ad304dbcf0 Add module data-type support for COPY. (#8112)
This adds a copy callback for module data types, in order to make
modules compatible with the new COPY command.

The callback is optional and COPY will fail for keys with data types
that do not implement it.
2020-12-09 20:22:45 +02:00
Yossi Gottlieb
4e064fbab4
Add module data-type support for COPY. (#8112)
This adds a copy callback for module data types, in order to make
modules compatible with the new COPY command.

The callback is optional and COPY will fail for keys with data types
that do not implement it.
2020-12-09 20:22:45 +02:00
Yossi Gottlieb
4a433dab8d Fix double close on IPv6 setup error. (#8168)
Fixes #8165.
2020-12-09 20:20:55 +02:00
Yossi Gottlieb
cf3d79d4c1
Fix double close on IPv6 setup error. (#8168)
Fixes #8165.
2020-12-09 20:20:55 +02:00
Oran Agra
dd98830417 zset full dump sanitization bug (dup score instead of field) (#8167) 2020-12-09 17:05:05 +02:00
Oran Agra
6a3c4ac50a
zset full dump sanitization bug (dup score instead of field) (#8167) 2020-12-09 17:05:05 +02:00
Oran Agra
11e0c4739b Handle output buffer limits for Module blocked clients (#8141)
Module blocked clients cache the response in a temporary client,
the reply list in this client would be affected by the recent fix
in #7202, but when the reply is later copied into the real client,
it would have bypassed all the checks for output buffer limit, which
would have resulted in both: responding with a partial response to
the client, and also not disconnecting it at all.
2020-12-08 16:41:20 +02:00
Oran Agra
48efc25f74
Handle output buffer limits for Module blocked clients (#8141)
Module blocked clients cache the response in a temporary client,
the reply list in this client would be affected by the recent fix
in #7202, but when the reply is later copied into the real client,
it would have bypassed all the checks for output buffer limit, which
would have resulted in both: responding with a partial response to
the client, and also not disconnecting it at all.
2020-12-08 16:41:20 +02:00
Oran Agra
080ad5b0f2 Improve stability of new CSC eviction test (#8160)
b640e2944 added a test that now fails with valgrind
it fails for two resons:
1) the test samples the used memory and then limits the maxmemory to
   that value, but it turns out this is not atomic and on slow machines
   the background cron process that clean out old query buffers reduces
   the memory so that the setting doesn't cause eviction.
2) the dbsize was tested late, after reading some invalidation messages
   by that time more and more keys got evicted, partially draining the
   db. this is not the focus of this fix (still a known limitation)
2020-12-08 16:33:09 +02:00
Oran Agra
a102b21d17
Improve stability of new CSC eviction test (#8160)
c4fdf09c0 added a test that now fails with valgrind
it fails for two resons:
1) the test samples the used memory and then limits the maxmemory to
   that value, but it turns out this is not atomic and on slow machines
   the background cron process that clean out old query buffers reduces
   the memory so that the setting doesn't cause eviction.
2) the dbsize was tested late, after reading some invalidation messages
   by that time more and more keys got evicted, partially draining the
   db. this is not the focus of this fix (still a known limitation)
2020-12-08 16:33:09 +02:00
Wang Yuan
8c9752b8f7 Minor improvements for list-2 test (#8156)
had some unused variables.
now some are used to assert that they match, others were useless.
2020-12-08 16:26:38 +02:00
Wang Yuan
1acc315cea
Minor improvements for list-2 test (#8156)
had some unused variables.
now some are used to assert that they match, others were useless.
2020-12-08 16:26:38 +02:00
Yossi Gottlieb
3cbe16602f Fix failing macOS tests due to wc differences. (#8161) 2020-12-08 16:22:16 +02:00
Yossi Gottlieb
00db1b5579
Fix failing macOS tests due to wc differences. (#8161) 2020-12-08 16:22:16 +02:00
Itamar Haber
ea20139700 Adds exclusive range query intervals to XPENDING (#8130) 2020-12-08 11:43:00 +02:00
Itamar Haber
37f45d9e56
Adds exclusive range query intervals to XPENDING (#8130) 2020-12-08 11:43:00 +02:00
Yossi Gottlieb
9f272197ea Fix setproctitle related crashes. (#8150)
Makes spt_init more careful with assumptions about what memory regions
may be overwritten. It will now only consider a contiguous block of argv
and envp elements and mind any gaps.
2020-12-08 11:27:30 +02:00
Yossi Gottlieb
ec02c761aa
Fix setproctitle related crashes. (#8150)
Makes spt_init more careful with assumptions about what memory regions
may be overwritten. It will now only consider a contiguous block of argv
and envp elements and mind any gaps.
2020-12-08 11:27:30 +02:00
Wen Hui
326487f83e use getPositiveLongFromObjectOrReply for positive check in spop (#8146)
This is a cleanup commit that doesn't fix any bug, however it does change the error response text in case the user provided a negative number.
2020-12-08 11:15:39 +02:00
Wen Hui
09b7f6c2a1
use getPositiveLongFromObjectOrReply for positive check in spop (#8146)
This is a cleanup commit that doesn't fix any bug, however it does change the error response text in case the user provided a negative number.
2020-12-08 11:15:39 +02:00
guybe7
9f6805294c More efficient self-XCLAIM (#8098)
when the same consumer re-claim an entry that it already has, there's
no need to remove-and-insert if it's the same rax.
we do need to update the idle time though.
this commit only improves efficiency (doesn't change behavior).
2020-12-07 21:31:35 +02:00
guybe7
6bb5503524
More efficient self-XCLAIM (#8098)
when the same consumer re-claim an entry that it already has, there's
no need to remove-and-insert if it's the same rax.
we do need to update the idle time though.
this commit only improves efficiency (doesn't change behavior).
2020-12-07 21:31:35 +02:00
filipe oliveira
c6c5dd15d8 Enable configuring OpenSSL using the standard openssl.cnf (#8143) 2020-12-07 14:30:12 +02:00
filipe oliveira
b5e99bd064
Enable configuring OpenSSL using the standard openssl.cnf (#8143) 2020-12-07 14:30:12 +02:00
Yossi Gottlieb
ed3a6c70dc Add CLIENT INFO and CLIENT LIST [id]. (#8113)
* Add CLIENT INFO subcommand.

The output is identical to CLIENT LIST but provides a single line for
the current client only.

* Add CLIENT LIST ID [id...].

Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-12-07 14:24:05 +02:00
Yossi Gottlieb
bccbc5509a
Add CLIENT INFO and CLIENT LIST [id]. (#8113)
* Add CLIENT INFO subcommand.

The output is identical to CLIENT LIST but provides a single line for
the current client only.

* Add CLIENT LIST ID [id...].

Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-12-07 14:24:05 +02:00
David CARLIER
e61af1b5b8 Solaris based system rss size report. (#8138) 2020-12-06 15:30:29 +02:00
David CARLIER
ec951cdc15
Solaris based system rss size report. (#8138) 2020-12-06 15:30:29 +02:00
Oran Agra
149ee4c0da Sanitize dump payload: performance optimizations and tuning
First, if the ziplist header is surely inside the ziplist, do fast path
decoding rather than the careful one.

In that case, streamline the encoding if-else chain to be executed only
once, and the encoding validity tested at the end.

encourage inlining

likely / unlikely hints for speculative execution

Assertion used _exit(1) to tell the compiler that the code after them is
not reachable and get rid of warnings.

But in some cases assertions are placed inside tight loops, and any
piece of code in them can slow down execution (code cache and other
reasons), instead using either abort() or better yet, unreachable
builtin.
2020-12-06 14:54:34 +02:00
Oran Agra
e288430c05 Sanitize dump payload: performance optimizations and tuning
First, if the ziplist header is surely inside the ziplist, do fast path
decoding rather than the careful one.

In that case, streamline the encoding if-else chain to be executed only
once, and the encoding validity tested at the end.

encourage inlining

likely / unlikely hints for speculative execution

Assertion used _exit(1) to tell the compiler that the code after them is
not reachable and get rid of warnings.

But in some cases assertions are placed inside tight loops, and any
piece of code in them can slow down execution (code cache and other
reasons), instead using either abort() or better yet, unreachable
builtin.
2020-12-06 14:54:34 +02:00
Oran Agra
595cff7fd4 Sanitize dump payload: fail RESTORE if memory allocation fails
When RDB input attempts to make a huge memory allocation that fails,
RESTORE should fail gracefully rather than die with panic
2020-12-06 14:54:34 +02:00
Oran Agra
7ca00d694d Sanitize dump payload: fail RESTORE if memory allocation fails
When RDB input attempts to make a huge memory allocation that fails,
RESTORE should fail gracefully rather than die with panic
2020-12-06 14:54:34 +02:00
Oran Agra
7006f02599 Sanitize dump payload: validate no duplicate records in hash/zset/intset
If RESTORE passes successfully with full sanitization, we can't affort
to crash later on assertion due to duplicate records in a hash when
converting it form ziplist to dict.
This means that when doing full sanitization, we must make sure there
are no duplicate records in any of the collections.
2020-12-06 14:54:34 +02:00
Oran Agra
3716950cfc Sanitize dump payload: validate no duplicate records in hash/zset/intset
If RESTORE passes successfully with full sanitization, we can't affort
to crash later on assertion due to duplicate records in a hash when
converting it form ziplist to dict.
This means that when doing full sanitization, we must make sure there
are no duplicate records in any of the collections.
2020-12-06 14:54:34 +02:00
Oran Agra
387c5048b9 testsuite: fix fd leak, prevent port clashing when using --baseport
when using --baseport to run two tests suite in parallel (different
folders), we need to also make sure the port used by the testsuite to
communicate with it's workers is unique. otherwise the attept to find a
free port connects to the other test suite and messes it.

maybe one day we need to attempt to bind, instead of connect when tring
to find a free port.
2020-12-06 14:54:34 +02:00
Oran Agra
5b44631397 testsuite: fix fd leak, prevent port clashing when using --baseport
when using --baseport to run two tests suite in parallel (different
folders), we need to also make sure the port used by the testsuite to
communicate with it's workers is unique. otherwise the attept to find a
free port connects to the other test suite and messes it.

maybe one day we need to attempt to bind, instead of connect when tring
to find a free port.
2020-12-06 14:54:34 +02:00