14685 Commits

Author SHA1 Message Date
Gnanesh
0772098b1b
EXPIRE, EXPIREAT, SETEX, GETEX: Return error when expire time overflows (#8287)
Respond with error if expire time overflows from positive to negative of vice versa.

* `SETEX`, `SET EX`, `GETEX` etc would have already error on negative value,
but now they would also error on overflows (i.e. when the input was positive but
after the manipulation it becomes negative, which would have passed before)
* `EXPIRE` and `EXPIREAT` was ok taking negative values (would implicitly delete
the key), we keep that, but we do error if the user provided a value that changes
sign when manipulated (except the case of changing sign when `basetime` is added)

Signed-off-by: Gnanesh <gnaneshkunal@outlook.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-02-21 09:09:54 +02:00
Jim Brunner
06966d2a0e
dict: pause rehash, minor readability refactor (#8515)
The `dict` field `iterators` is misleading and incorrect. 
This variable is used for 1 purpose - to pause rehashing.

The current `iterators` field doesn't actually count "iterators".
It counts "safe iterators".  But - it doesn't actually count safe iterators
either.  For one, it's only incremented once the safe iterator begins to
iterate, not when it's created.  It's also incremented in `dictScan` to
prevent rehashing (and commented to make it clear why `iterators` is
being incremented during a scan).

This update renames the field as `pauserehash` and creates 2 helper
macros `dictPauseRehashing(d)` and `dictResumeRehashing(d)`
2021-02-20 12:56:30 +02:00
sundb
46346e9e3a
Fix timing error oom-score-adj test (#8513)
fixes timing issue, fork didn't always get to set the oom score before the test verified it.
2021-02-19 13:01:25 +02:00
John Sully
7fb41ba1bd Implement maxstorage config
Former-commit-id: 79e07859dfec14edf2a39646013e1a82db033d4f
2021-02-19 05:17:41 +00:00
Harkrishn Patro
303465af35
Remove redundant pubsub list to store the patterns. (#8472)
Remove redundant pubsub list to store the patterns.
2021-02-17 14:13:50 -08:00
Oran Agra
71ab81ec69
solve valgrind warning in child_info (#8505)
Valgrind warns about `write` accessing uninitialized memory, which was the struct padding.
2021-02-17 12:30:29 +02:00
John Sully
dea064712b Disable async rehash with single threads, it causes slowdowns as the rehash never completes
Former-commit-id: 5d08dbdf76c0fd1e0cfcf86b97ef3e656f0e4f5d
2021-02-16 21:07:43 +00:00
yihuang
aab479f8cf
Optimize listpack for stream usage to avoid repeated reallocs (#6281)
Avoid repeated reallocs growing the listpack while entries are being added.
This is done by pre-allocating the listpack to near maximum size, and using
malloc_size to check if it needs realloc or not.
When the listpack reaches the maximum number of entries, we shrink it to fit it's used size.

Co-authored-by: Viktor Söderqvist <viktor@zuiderkwast.se>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-02-16 16:17:38 +02:00
uriyage
fd052d2a86
Adds INFO fields to track fork child progress (#8414)
* Adding current_save_keys_total and current_save_keys_processed info fields.
  Present in replication, BGSAVE and AOFRW.
* Changing RM_SendChildCOWInfo() to RM_SendChildHeartbeat(double progress)
* Adding new info field current_fork_perc. Present in Replication, BGSAVE, AOFRW,
  and module forks.
2021-02-16 16:06:51 +02:00
Viktor Söderqvist
acb32d472d Add ziplistReplace, in-place optimized for elements of same size
Avoids memmove and reallocs when replacing a ziplist element of the
same encoded size as the new value.

Affects HSET, HINRBY, HINCRBYFLOAT (via hashTypeSet) and LSET (via
quicklistReplaceAtIndex).
2021-02-16 13:01:14 +02:00
Viktor Söderqvist
683e530cf3 Use stack for decoding integer-encoded values in list push
Less heap allocations when commands like LMOVE push integer values.
2021-02-16 13:01:14 +02:00
Viktor Söderqvist
f521498b43 Avoid useless copying in LINDEX command for reply 2021-02-16 13:01:14 +02:00
Oran Agra
fb3457d157
minor test suite cleanup, revive old test (#8497)
There are two tests in other.tcl that were dependant of the sha1 package
import which meant that they didn't usually run.
The reason it was like that was that prior to the creation of DEBUG
DIGEST, the test suite used to have an equivalent function, but that's
no longer the case and this dependency isn't needed.

The other change is to revert config changes done by the test before the
test suite continues. can be useful if using `--host` to run multiple
units against the same server
2021-02-15 17:20:03 +02:00
Yossi Gottlieb
b1929bb2b6
Fix incorrect shared ping length. (#8498) 2021-02-15 17:09:33 +02:00
Yossi Gottlieb
141ac8df59
Escape unsafe field name characters in INFO. (#8492)
Fixes #8489
2021-02-15 17:08:53 +02:00
Oran Agra
30775bc3e3
solve race in replication-2 test - again (#8491)
this should make it timing independent and also faster in most cases
2021-02-15 12:50:23 +02:00
Viktor Söderqvist
0bc8c9c8f9
Modules: In RM_HashSet, add COUNT_ALL flag and set errno (#8446)
The added flag affects the return value of RM_HashSet() to include
the number of inserted fields, in addition to updated and deleted
fields.

errno is set on errors, tests are added and documentation updated.
2021-02-15 11:40:05 +02:00
filipe oliveira
efccd6353b
redis-benchmark: Fix broken protocol when used with -a or --dbnum (#8486)
Fix the pointers to the slot hash tags in the case of prefixed commands usage
i.e. AUTH / SELECT

It adjusts the pointers to the slot hash tags in the case of prefixed commands
usage as soon as we get the 1st reply (same like we already did for the random
strings within the command )
2021-02-14 14:42:41 +02:00
John Sully
f3853b6485 Fix deadlock in storage prefetch
Former-commit-id: 9e6162de08248f7726b97b73aee2f23808ff4e7b
2021-02-12 19:38:57 +00:00
Yossi Gottlieb
8c42d1257f
Fix errors with sentinel leaked fds test. (#8482)
* Don't run test script on non-Linux.
* Verify that reported fds do indeed exist also in parent, to avoid
  false negatives on some systems (namely CentOS).

Co-authored-by: Andy Pan <panjf2000@gmail.com>
2021-02-11 15:25:01 +02:00
Yossi Gottlieb
94bc26e652
Fix duplicate replicas issue. (#8481)
We need to store replicas referenced by their announced address (IP or
address). Before that, if hostnames were used and the IP address
changed, duplicate entries would have been created.
2021-02-11 11:50:47 +02:00
John Sully
bb767ad61d Enable prefetch with storage providers
Former-commit-id: 21f9c52c446ad1b460a340c3d873839b2355728f
2021-02-11 00:50:21 +00:00
Yossi Gottlieb
29ac9aea5d
Fix Sentinel configuration rewrite. (#8480)
The `resolve-hostnames` and `announce-hostnames` parameters were not
specified correctly according to the new convention introduced by
1aad55b66.
2021-02-10 18:38:10 +02:00
filipe oliveira
b5ca1e9e53
Removed time sensitive checks from block on background tests. Fixed uninitialized variable (#8479)
- removes time sensitive checks from block on background tests during leak checks.
- fix uninitialized variable on RedisModuleBlockedClient() when calling
  RM_BlockedClientMeasureTimeEnd() without RM_BlockedClientMeasureTimeStart()
2021-02-10 08:59:07 +02:00
Madelyn Olson
899c85ae67
Moved most static strings into the shared structure (#8411)
Moved most static strings into the shared structure
2021-02-09 11:52:28 -08:00
Itamar Haber
4554a49d32
Adds code of conduct (#8471) 2021-02-09 14:38:09 +02:00
WuYunlong
203f357c32
Cleanup in redis-cli and tests: release memory on exit, change dup test name (#8475)
1. Rename 18-cluster-nodes-slots.tcl to 19-cluster-nodes-slots.tcl.
  it was conflicting with another test prefixed by 18
2. Release memory on exit in redis-cli.c.
3. Fix freeConvertedSds indentation.
2021-02-09 12:36:09 +02:00
John Sully
6ecabe823f Fix polarity issue in async rehash
Former-commit-id: dfea168499f0b7829607faec49c491d5072511bc
2021-02-09 05:10:22 +00:00
John Sully
2c6616bd82 Fix polarity issue in async rehash
Former-commit-id: 9eefba49d4fcde7f12929929d7aeb36f5186a63d
2021-02-09 02:41:44 +00:00
John Sully
f6b9c13c54 Fix deadlock with disconnecting client and storage provider
Former-commit-id: 0831745323e425f463322e8c0dc27fc25854868e
2021-02-08 19:02:01 +00:00
John Sully
e5343f47c2 Merge branch 'unstable' into keydbpro
Former-commit-id: e2140793f2bf565972ada799af73bf4457e2718d
2021-02-08 18:17:09 +00:00
Huang Zw
8f9958dc24
Fix typo and some out of date comments (#8449)
Fix typo and some out of date comments
2021-02-08 09:29:32 -08:00
Yossi Gottlieb
dbcc0a85d0
Fix and cleanup Sentinel leaked fds test. (#8469)
* For consistency, use tclsh for the script as well
* Ignore leaked fds that originate from grandparent process, since we
  only care about fds redis-sentinel itself is responsible for
* Check every test iteration to catch problems early
* Some cleanups, e.g. parameterization of file name, etc.
2021-02-08 17:02:46 +02:00
filipe oliveira
b2351ea0dc
[fix] Increasing block on background timeout time to avoid test failure (#8470)
The test failed from time to time on Github actions.
We think it's possible that on the module's blocking timeout
time tracking test, the timeout is happening prior we issue the
RedisModule_BlockedClientMeasureTimeStart(bc) on the
background thread. If that is the case one possible solution
is to increase the timeout.
Increasing to 200ms to 500ms to see if nightly stops failing.
2021-02-08 16:24:00 +02:00
sundb
1f12be3072
Fix random probability check in ziplistRandomPairsUnique (#8467)
When (remaining == (total_size - index)), element will definitely be random to.
But when rand() == RAND_MAX, the element will miss, this will trigger assert
in serverAssert(ziplistRandomPairsUnique(zsetobj->ptr, count, keys, vals) == count).
2021-02-08 12:41:16 +02:00
Andy Pan
88272cf7ac
Fix typos in comments (#8466) 2021-02-08 12:09:39 +02:00
John Sully
bb733b0aab Fix bug in fastlock stack metadata
Former-commit-id: 241d1bd7e1ed64885adbb07653a5c7e8ef882607
2021-02-07 19:11:05 -05:00
John Sully
7d588e637a Disable multithreaded rehash when active defrag is enabled. The two are not compatible
Former-commit-id: 56addcd17262ffbaefea26c0097cde53c616ece2
2021-02-07 19:11:05 -05:00
John Sully
3970f4a5a7 The server should recalibrate periodically. Also reduce log noise
Former-commit-id: 8d8e1810d49da6aa921d2327cb4ea250c2b5b234
2021-02-07 19:11:05 -05:00
John Sully
8ce03686ed Ensure the C lock implementation behaves the same as the ASM
Former-commit-id: b5ddc11c46d2eabd28fae4c69927c356dd18bf6e
2021-02-07 19:11:05 -05:00
John Sully
7c700f1987 Ensure rehash completes even when we're in a long running task
Former-commit-id: f107746e90f7a8ff3c7094145ee1ad438911e8c2
2021-02-07 19:11:05 -05:00
John Sully
bef72e5f6a Implement rehash during spinlock
Former-commit-id: f68a26381a35b27948046d46c2c7bcfbdc21143d
2021-02-07 19:11:05 -05:00
John Sully
5ab1095022 Allow multiple threads to rehash simultaneously
Former-commit-id: 5a2cc90786dfd1bfd341dbf5713bcde01f0cfff3
2021-02-07 19:11:05 -05:00
John Sully
5d161ca2f2 Fix races in free
Former-commit-id: 3881d5c1f22f022855c0abcf7b0c2070e204c0a3
2021-02-07 19:11:05 -05:00
John Sully
c6c1084dd8 Initial implementation
Former-commit-id: 958f2c00c8efc15dc91fdeec2ff2e2ae2016c124
2021-02-07 19:11:05 -05:00
John Sully
e55ca3603c Fix memory leak in mvccRestore
Former-commit-id: 165333b0fc648c79e66f04d9c8c4a1d0059fe57a
2021-02-07 23:38:49 +00:00
John Sully
4c9f912c3b Fix cluster test failure
Former-commit-id: 230fde39882766426d892fd30d0c8349f5cca912
2021-02-07 23:38:25 +00:00
John Sully
ea74c705a6 Fix memory leak in mvccRestore
Former-commit-id: 92da1cbd0a08b66264f170a9b57ef7270f270157
2021-02-07 23:38:09 +00:00
John Sully
483b43156e Update test to handle forkless bgsave
Former-commit-id: fc8c6de8252e6b802688413050f1a823093d0a1d
2021-02-07 21:41:19 +00:00
John Sully
886fcf6387 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/keydb-dev/KeyDB-Pro into keydbpro
Former-commit-id: 8bad923169b2a73f249fd8b3153236d6a74100c9
2021-02-07 20:53:07 +00:00