27356 Commits

Author SHA1 Message Date
John Sully
c6c1084dd8 Initial implementation
Former-commit-id: 958f2c00c8efc15dc91fdeec2ff2e2ae2016c124
2021-02-07 19:11:05 -05:00
John Sully
5c884b6fd8 Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: 4a269c3382083f6ad7f697ef07eebc552e0de9d8
2021-02-08 00:02:54 +00:00
John Sully
a997b2d5ad Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: 4a269c3382083f6ad7f697ef07eebc552e0de9d8
2021-02-08 00:02:54 +00:00
John Sully
f4c7da7284 Merge branch 'unstable' into RELEASE_6
Former-commit-id: 0b7b5332f2a3bb5a6f4310643c343f53f546bd81
2021-02-07 23:55:58 +00:00
John Sully
2da3216228 Merge branch 'unstable' into RELEASE_6
Former-commit-id: 0b7b5332f2a3bb5a6f4310643c343f53f546bd81
2021-02-07 23:55:58 +00:00
John Sully
aa47e643b0 Fix memory leak in mvccRestore
Former-commit-id: 165333b0fc648c79e66f04d9c8c4a1d0059fe57a
2021-02-07 23:38:49 +00:00
John Sully
e55ca3603c Fix memory leak in mvccRestore
Former-commit-id: 165333b0fc648c79e66f04d9c8c4a1d0059fe57a
2021-02-07 23:38:49 +00:00
John Sully
e2f634aef5 Fix cluster test failure
Former-commit-id: 230fde39882766426d892fd30d0c8349f5cca912
2021-02-07 23:38:25 +00:00
John Sully
4c9f912c3b Fix cluster test failure
Former-commit-id: 230fde39882766426d892fd30d0c8349f5cca912
2021-02-07 23:38:25 +00:00
John Sully
6a7b193fc1 Fix memory leak in mvccRestore
Former-commit-id: 92da1cbd0a08b66264f170a9b57ef7270f270157
2021-02-07 23:38:09 +00:00
John Sully
ea74c705a6 Fix memory leak in mvccRestore
Former-commit-id: 92da1cbd0a08b66264f170a9b57ef7270f270157
2021-02-07 23:38:09 +00:00
John Sully
52c1d17127 Update test to handle forkless bgsave
Former-commit-id: fc8c6de8252e6b802688413050f1a823093d0a1d
2021-02-07 21:41:19 +00:00
John Sully
483b43156e Update test to handle forkless bgsave
Former-commit-id: fc8c6de8252e6b802688413050f1a823093d0a1d
2021-02-07 21:41:19 +00:00
John Sully
b20ab1fd21 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into keydbpro
Former-commit-id: 8bad923169b2a73f249fd8b3153236d6a74100c9
2021-02-07 20:53:07 +00:00
John Sully
886fcf6387 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into keydbpro
Former-commit-id: 8bad923169b2a73f249fd8b3153236d6a74100c9
2021-02-07 20:53:07 +00:00
jsully
1c0c0ebd3f Merge branch 'opensourcemerge_2021-02-01' into 'keydbpro'
Merge from open source Feb 5 2021

See merge request keydb-dev/KeyDB-Pro!9

Former-commit-id: 1a2e2b99cc0123b1b03b78b5f28107df395decdb
2021-02-07 20:49:08 +00:00
jsully
ab45347ed6 Merge branch 'opensourcemerge_2021-02-01' into 'keydbpro'
Merge from open source Feb 5 2021

See merge request keydb-dev/KeyDB-Pro!9

Former-commit-id: 1a2e2b99cc0123b1b03b78b5f28107df395decdb
2021-02-07 20:49:08 +00:00
Oran Agra
11fe272c1a Optimize HRANDFIELD and ZRANDMEMBER case 4 when ziplist encoded (#8444)
It is inefficient to repeatedly pick a single random element from a
ziplist.
For CASE4, which is when the user requested a low number of unique
random picks from the collectoin, we used thta pattern.

Now we use a different algorithm that picks unique elements from a
ziplist, and guarentee no duplicate but doesn't provide random order
(which is only needed in the non-unique random picks case)

Unrelated changes:
* change ziplist count and indexes variables to unsigned
* solve compilation warnings about uninitialized vars in gcc 10.2

Co-authored-by: xinluton <xinluton@qq.com>
2021-02-07 16:55:11 +02:00
Oran Agra
62b1f32062
Optimize HRANDFIELD and ZRANDMEMBER case 4 when ziplist encoded (#8444)
It is inefficient to repeatedly pick a single random element from a
ziplist.
For CASE4, which is when the user requested a low number of unique
random picks from the collectoin, we used thta pattern.

Now we use a different algorithm that picks unique elements from a
ziplist, and guarentee no duplicate but doesn't provide random order
(which is only needed in the non-unique random picks case)

Unrelated changes:
* change ziplist count and indexes variables to unsigned
* solve compilation warnings about uninitialized vars in gcc 10.2

Co-authored-by: xinluton <xinluton@qq.com>
2021-02-07 16:55:11 +02:00
Oran Agra
8bfac022f8 solve race in replication-2 test (#8461)
use SIGSTOP instead of DEBUG SLEEP, reduces the test
time by some 2 seconds and avoids failures on slow machines
2021-02-07 16:22:30 +02:00
Oran Agra
02ab14cc2e
solve race in replication-2 test (#8461)
use SIGSTOP instead of DEBUG SLEEP, reduces the test
time by some 2 seconds and avoids failures on slow machines
2021-02-07 16:22:30 +02:00
Oran Agra
cd99db31f1 Update CI on Ubuntu to tcl8.6 (since 20.04 is now used) (#8460)
Github started shifting some repositoreis to use ubuntu 20.04 by default
tcl8.5 is missing in these, but 8.6 exists in both 20.04 and 18.04
2021-02-07 15:41:49 +02:00
Oran Agra
018f7b7378
Update CI on Ubuntu to tcl8.6 (since 20.04 is now used) (#8460)
Github started shifting some repositoreis to use ubuntu 20.04 by default
tcl8.5 is missing in these, but 8.6 exists in both 20.04 and 18.04
2021-02-07 15:41:49 +02:00
Yossi Gottlieb
b07ddb2c04 Add --dump-logs tests option. (#8459)
Dump the entire server log if a test failed, to easy troubleshooting
with no access to log files.
2021-02-07 12:37:24 +02:00
Yossi Gottlieb
5b8350aaaa
Add --dump-logs tests option. (#8459)
Dump the entire server log if a test failed, to easy troubleshooting
with no access to log files.
2021-02-07 12:37:24 +02:00
Yossi Gottlieb
830c207254 Add --insecure option to command line tools. (#8416)
Disable certificate validation, making it possible to connect to servers
without configuring full trust chain.

The use of this option is insecure and makes the connection vulnerable
to man in the middle attacks.
2021-02-07 12:36:56 +02:00
Yossi Gottlieb
be83bb13a8
Add --insecure option to command line tools. (#8416)
Disable certificate validation, making it possible to connect to servers
without configuring full trust chain.

The use of this option is insecure and makes the connection vulnerable
to man in the middle attacks.
2021-02-07 12:36:56 +02:00
Viktor Söderqvist
64737ea8f1 RM_ZsetRem: Delete key if empty (#8453)
Without this fix, RM_ZsetRem can leave empty sorted sets which are
not allowed to exist.

Removing from a sorted set while iterating seems to work (while
inserting causes failed assetions). RM_ZsetRangeEndReached is
modified to return 1 if the key doesn't exist, to terminate
iteration when the last element has been removed.
2021-02-05 19:54:01 +02:00
Viktor Söderqvist
aea6e71ef8
RM_ZsetRem: Delete key if empty (#8453)
Without this fix, RM_ZsetRem can leave empty sorted sets which are
not allowed to exist.

Removing from a sorted set while iterating seems to work (while
inserting causes failed assetions). RM_ZsetRangeEndReached is
modified to return 1 if the key doesn't exist, to terminate
iteration when the last element has been removed.
2021-02-05 19:54:01 +02:00
filipe oliveira
d627014049 Fix compiler warning on implicit declaration of ‘nanosleep’ . Removed unused variable (#8454) 2021-02-05 19:51:31 +02:00
filipe oliveira
b3bdcd2278
Fix compiler warning on implicit declaration of ‘nanosleep’ . Removed unused variable (#8454) 2021-02-05 19:51:31 +02:00
sundb
ef211f440b RAND* commands: fix risk of OOM panic in hash and zset, use fair random in hash, and add tests for even distribution to all (#8429)
Changes to HRANDFIELD and ZRANDMEMBER:
* Fix risk of OOM panic when client query a very big negative count (avoid allocating huge temporary buffer).
* Fix uneven random distribution in HRANDFIELD with negative count (wasn't using dictGetFairRandomKey).
* Add tests to check an even random distribution (HRANDFIELD, SRANDMEMBER, ZRANDMEMBER).

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-02-05 15:56:20 +02:00
sundb
18ac41973b
RAND* commands: fix risk of OOM panic in hash and zset, use fair random in hash, and add tests for even distribution to all (#8429)
Changes to HRANDFIELD and ZRANDMEMBER:
* Fix risk of OOM panic when client query a very big negative count (avoid allocating huge temporary buffer).
* Fix uneven random distribution in HRANDFIELD with negative count (wasn't using dictGetFairRandomKey).
* Add tests to check an even random distribution (HRANDFIELD, SRANDMEMBER, ZRANDMEMBER).

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-02-05 15:56:20 +02:00
christianEQ
fc83515309 fixed up backup stuff
Former-commit-id: 8a23162c24dc13a4268928c5defd85b0b5249d39
2021-02-04 22:36:32 +00:00
christianEQ
511f01bd14 fixed up backup stuff
Former-commit-id: 8a23162c24dc13a4268928c5defd85b0b5249d39
2021-02-04 22:36:32 +00:00
Yang Bodong
81da73ff31 Fix GEOSEARCH tcl test error (#8451)
Issue with new test due to longitude wraparound.
2021-02-04 19:39:07 +02:00
Yang Bodong
b7b23a0ff5
Fix GEOSEARCH tcl test error (#8451)
Issue with new test due to longitude wraparound.
2021-02-04 19:39:07 +02:00
Yang Bodong
e05f15cf9a GEOSEARCH bybox bug fixes and new fuzzy tester (#8445)
Fix errors of GEOSEARCH bybox search due to:
1. projection of the box to a trapezoid (when the meter box is converted to long / lat it's no longer a box).
2. width and height mismatch

Changes:
- New GEOSEARCH point in rectangle algorithm
- Fix GEOSEARCH bybox width and height mismatch bug
- Add GEOSEARCH bybox testing to the existing "GEOADD + GEORANGE randomized test"
- Add new fuzzy test to stress test the bybox corners and edges
- Add some tests for edge cases of the bybox algorithm

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-02-04 18:08:35 +02:00
Yang Bodong
ded1655d49
GEOSEARCH bybox bug fixes and new fuzzy tester (#8445)
Fix errors of GEOSEARCH bybox search due to:
1. projection of the box to a trapezoid (when the meter box is converted to long / lat it's no longer a box).
2. width and height mismatch

Changes:
- New GEOSEARCH point in rectangle algorithm
- Fix GEOSEARCH bybox width and height mismatch bug
- Add GEOSEARCH bybox testing to the existing "GEOADD + GEORANGE randomized test"
- Add new fuzzy test to stress test the bybox corners and edges
- Add some tests for edge cases of the bybox algorithm

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-02-04 18:08:35 +02:00
Yossi Gottlieb
d6c0c9de4a Fix 32-bit test modules build. (#8448) 2021-02-04 11:37:28 +02:00
Yossi Gottlieb
52fb306535
Fix 32-bit test modules build. (#8448) 2021-02-04 11:37:28 +02:00
christianEQ
ed2a86c98f removed bgsave test, not applicable to keydb pro
Former-commit-id: d017c160662702ebcdbf063df239b907c2a193c5
2021-02-03 19:13:53 +00:00
christianEQ
0fd43cdc82 removed bgsave test, not applicable to keydb pro
Former-commit-id: d017c160662702ebcdbf063df239b907c2a193c5
2021-02-03 19:13:53 +00:00
christianEQ
c69b2845fd Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: 5bad058733de2c217340bb9ee48f02b07d754808
2021-02-03 18:10:27 +00:00
christianEQ
cca8d186aa Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: 5bad058733de2c217340bb9ee48f02b07d754808
2021-02-03 18:10:27 +00:00
Yossi Gottlieb
9b564e6bc0 Fix FreeBSD tests and CI Daily issues. (#8438)
* Add bash temporarily to allow sentinel fd leaks test to run.
* Use vmactions-freebsd rdist sync to work around bind permission denied
  and slow execution issues.
* Upgrade to tcl8.6 to be aligned with latest Ubuntu envs.
* Concat all command executions to avoid ignoring failures.
* Skip intensive fuzzer on FreeBSD. For some yet unknown reason, generate_fuzzy_traffic_on_key causes TCL to significantly bloat on FreeBSD resulting with out of memory.
2021-02-03 17:35:28 +02:00
Yossi Gottlieb
de6f3ad017
Fix FreeBSD tests and CI Daily issues. (#8438)
* Add bash temporarily to allow sentinel fd leaks test to run.
* Use vmactions-freebsd rdist sync to work around bind permission denied
  and slow execution issues.
* Upgrade to tcl8.6 to be aligned with latest Ubuntu envs.
* Concat all command executions to avoid ignoring failures.
* Skip intensive fuzzer on FreeBSD. For some yet unknown reason, generate_fuzzy_traffic_on_key causes TCL to significantly bloat on FreeBSD resulting with out of memory.
2021-02-03 17:35:28 +02:00
Jonah H. Harris
058ee191ec Optimizing sorted GEORADIUS COUNT with partial sorting. (#8326)
This commit provides an optimization, in terms of time, for all GEORADIUS*
and GEOSEARCH* searches which utilize the default, sorted, COUNT clause.
This is commonly used for nearest-neighbor (top-K points closest to a given lat/lon)
searches. While the previous implementation appends all matching points to the
geoPoint array and performs pruning after-the-fact via a full sort and [0, count)-based
for-loop, this PR sorts only the required number of elements.

This optimization provides a 5-20% improvement in runtime depending on the
density of points of interest (POI) as well as the radius searched.
No performance degradation has been observed.
2021-02-02 10:57:12 +02:00
Jonah H. Harris
a3718cde06
Optimizing sorted GEORADIUS COUNT with partial sorting. (#8326)
This commit provides an optimization, in terms of time, for all GEORADIUS*
and GEOSEARCH* searches which utilize the default, sorted, COUNT clause.
This is commonly used for nearest-neighbor (top-K points closest to a given lat/lon)
searches. While the previous implementation appends all matching points to the
geoPoint array and performs pruning after-the-fact via a full sort and [0, count)-based
for-loop, this PR sorts only the required number of elements.

This optimization provides a 5-20% improvement in runtime depending on the
density of points of interest (POI) as well as the radius searched.
No performance degradation has been observed.
2021-02-02 10:57:12 +02:00
Huang Zw
54ca166f1c Cleanup: addReplyAggregateLen and addReplyBulkLen remove redundant check (#8431)
addReplyLongLongWithPrefix, has a check against negative length, and the code
flow removed in this commit bypasses the check.
addReplyAggregateLen has an assertion for negative length, but addReplyBulkLen
does not, so this commit fixes theoretical case of access violation (probably
unreachable though)
2021-02-02 10:54:19 +02:00