27610 Commits

Author SHA1 Message Date
Wen Hui
d96f47cf06
use getPositiveLongFromObjectOrReply for positive check of args (#8750)
just a cleanup
2021-04-07 10:28:53 +03:00
Yang Bodong
79793c3d55 Fix out of range confusing error messages (XAUTOCLAIM, RPOP count) (#8746)
Fix out of range error messages to be clearer (avoid mentioning 9223372036854775807)
* Fix XAUTOCLAIM COUNT option confusing error msg
* Fix other RPOP and alike error message to mention positive
2021-04-07 10:01:28 +03:00
Yang Bodong
4c14e8668c
Fix out of range confusing error messages (XAUTOCLAIM, RPOP count) (#8746)
Fix out of range error messages to be clearer (avoid mentioning 9223372036854775807)
* Fix XAUTOCLAIM COUNT option confusing error msg
* Fix other RPOP and alike error message to mention positive
2021-04-07 10:01:28 +03:00
jsully
c168c8fbbf Merge branch 'flash_read_perf' into 'keydbpro'
Flash read perf

See merge request external-collab/keydb-pro-6!1

Former-commit-id: f6d7277f307b78f079dd99bf532cd26a7b62018d
2021-04-07 06:12:55 +00:00
jsully
59a9e93abb Merge branch 'flash_read_perf' into 'keydbpro'
Flash read perf

See merge request external-collab/keydb-pro-6!1

Former-commit-id: f6d7277f307b78f079dd99bf532cd26a7b62018d
2021-04-07 06:12:55 +00:00
Cloud User
6601905d8c Make high pri time thread configurable
Former-commit-id: 7e94207765d2166f46792aea0919786f3f30d7b3
2021-04-06 22:39:20 +00:00
Cloud User
6684f150d6 Make high pri time thread configurable
Former-commit-id: 7e94207765d2166f46792aea0919786f3f30d7b3
2021-04-06 22:39:20 +00:00
Oran Agra
7af6fff7b9 update help.h (#8747) 2021-04-06 12:42:18 +03:00
Oran Agra
4de1031d8a
update help.h (#8747) 2021-04-06 12:42:18 +03:00
yjph
1ea4f128d3 Update the location information in some URLs (#8595) 2021-04-06 12:29:02 +03:00
yjph
073598ed8f
Update the location information in some URLs (#8595) 2021-04-06 12:29:02 +03:00
Bonsai
15e2b68214 Fix: server will crash if rdbload or rdbsave method is not provided in module (#8670)
With this fix, module data type registration will fail if the load or save callbacks are not defined, or the optional aux load and save callbacks are not either both defined or both missing.
2021-04-06 12:09:36 +03:00
Bonsai
808f3004f0
Fix: server will crash if rdbload or rdbsave method is not provided in module (#8670)
With this fix, module data type registration will fail if the load or save callbacks are not defined, or the optional aux load and save callbacks are not either both defined or both missing.
2021-04-06 12:09:36 +03:00
Yossi Gottlieb
e9ae932bc7 Clean up and stabilize cluster migration tests. (#8745)
This is work in progress, focusing on two main areas:
* Avoiding race conditions with cluster configuration propagation.
* Ignoring limitations with redis-cli --cluster fix which makes it hard
  to distinguish real errors (e.g. failure to fix) from expected
  conditions in this test (e.g. nodes not agreeing on configuration).
2021-04-06 11:57:57 +03:00
Yossi Gottlieb
4724dd439e
Clean up and stabilize cluster migration tests. (#8745)
This is work in progress, focusing on two main areas:
* Avoiding race conditions with cluster configuration propagation.
* Ignoring limitations with redis-cli --cluster fix which makes it hard
  to distinguish real errors (e.g. failure to fix) from expected
  conditions in this test (e.g. nodes not agreeing on configuration).
2021-04-06 11:57:57 +03:00
Huang Zhw
f5cc88e65d Fix "default" and overwritten / reset users will not have pubsub channels permissions by default. (#8723)
Background:
Redis 6.2 added ACL control for pubsub channels (#7993), which were supposed
to be permissive by default to retain compatibility with redis 6.0 ACL. 
But due to a bug, only newly created users got this `acl-pubsub-default` applied,
while overwritten (updated) users got reset to `resetchannels` (denied).

Since the "default" user exists before loading the config file,
any ACL change to it, results in an update / overwrite.

So when a "default" user is loaded from config file or include ACL
file with no channels related rules, the user will not have any
permissions to any channels. But other users will have default
permissions to any channels.

When upgraded from 6.0 with config rewrite, this will lead to
"default" user channels permissions lost.
When users are loaded from include file, then call "acl load", users
will also lost channels permissions.

Similarly, the `reset` ACL rule, would have reset the user to be denied
access to any channels, ignoring `acl-pubsub-default` and breaking
compatibility with redis 6.0.

The implication of this fix is that it regains compatibility with redis 6.0,
but breaks compatibility with redis 6.2.0 and 2.0.1. e.g. after the upgrade,
the default user will regain access to pubsub channels.

Other changes:
Additionally this commit rename server.acl_pubusub_default to
server.acl_pubsub_default and fix typo in acl tests.
2021-04-05 23:13:20 +03:00
Huang Zhw
3b74b55084
Fix "default" and overwritten / reset users will not have pubsub channels permissions by default. (#8723)
Background:
Redis 6.2 added ACL control for pubsub channels (#7993), which were supposed
to be permissive by default to retain compatibility with redis 6.0 ACL. 
But due to a bug, only newly created users got this `acl-pubsub-default` applied,
while overwritten (updated) users got reset to `resetchannels` (denied).

Since the "default" user exists before loading the config file,
any ACL change to it, results in an update / overwrite.

So when a "default" user is loaded from config file or include ACL
file with no channels related rules, the user will not have any
permissions to any channels. But other users will have default
permissions to any channels.

When upgraded from 6.0 with config rewrite, this will lead to
"default" user channels permissions lost.
When users are loaded from include file, then call "acl load", users
will also lost channels permissions.

Similarly, the `reset` ACL rule, would have reset the user to be denied
access to any channels, ignoring `acl-pubsub-default` and breaking
compatibility with redis 6.0.

The implication of this fix is that it regains compatibility with redis 6.0,
but breaks compatibility with redis 6.2.0 and 2.0.1. e.g. after the upgrade,
the default user will regain access to pubsub channels.

Other changes:
Additionally this commit rename server.acl_pubusub_default to
server.acl_pubsub_default and fix typo in acl tests.
2021-04-05 23:13:20 +03:00
christianEQ
ec11cc177e check for musl in logging.tcl as backtrace() is not available with it
Former-commit-id: 2c8239f9cb30aa32de936be09522e6429aa40326
2021-04-05 11:09:49 -04:00
christianEQ
689a5d2a00 check for musl in logging.tcl as backtrace() is not available with it
Former-commit-id: 2c8239f9cb30aa32de936be09522e6429aa40326
2021-04-05 11:09:49 -04:00
Huang Zhw
84414bb33f redis-cli --bigkeys / memkeys, report detailed error on dbsize failure (#8740)
When DBSIZE failed (e.g. on AUTH error), the printed error didn't reflect the reason.
2021-04-05 08:30:41 +03:00
Huang Zhw
a3da3e592d
redis-cli --bigkeys / memkeys, report detailed error on dbsize failure (#8740)
When DBSIZE failed (e.g. on AUTH error), the printed error didn't reflect the reason.
2021-04-05 08:30:41 +03:00
Sokolov Yura
0d7a17b252 Add cluster-allow-replica-migration option. (#5285)
Previously (and by default after commit) when master loose its last slot
(due to migration, for example), its replicas will migrate to new last slot
holder.

There are cases where this is not desired:
* Consolidation that results with removed nodes (including the replica, eventually).
* Manually configured cluster topologies, which the admin wishes to preserve.

Needlessly migrating a replica triggers a full synchronization and can have a negative impact, so
we prefer to be able to avoid it where possible.

This commit adds 'cluster-allow-replica-migration' configuration option that is
enabled by default to preserve existed behavior. When disabled, replicas will
not be auto-migrated.

Fixes #4896

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-04 09:43:24 +03:00
Sokolov Yura
1cab962098
Add cluster-allow-replica-migration option. (#5285)
Previously (and by default after commit) when master loose its last slot
(due to migration, for example), its replicas will migrate to new last slot
holder.

There are cases where this is not desired:
* Consolidation that results with removed nodes (including the replica, eventually).
* Manually configured cluster topologies, which the admin wishes to preserve.

Needlessly migrating a replica triggers a full synchronization and can have a negative impact, so
we prefer to be able to avoid it where possible.

This commit adds 'cluster-allow-replica-migration' configuration option that is
enabled by default to preserve existed behavior. When disabled, replicas will
not be auto-migrated.

Fixes #4896

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-04 09:43:24 +03:00
John Sully
6b2567298d Don't ensure if we don't have to
Former-commit-id: b7b678a3ada531890e67313c867b7b49b01fe41e
2021-04-01 18:17:05 +00:00
John Sully
f8cf5d3e88 Don't ensure if we don't have to
Former-commit-id: b7b678a3ada531890e67313c867b7b49b01fe41e
2021-04-01 18:17:05 +00:00
jsully
7b61c83b3e Merge branch 'keydbpro' into 'flash_read_perf'
# Conflicts:
#   src/db.cpp
#   src/networking.cpp

Former-commit-id: e16c846b2b9e70f20981172287b19e585f81d73d
2021-04-01 17:25:31 +00:00
jsully
c6cea9d84f Merge branch 'keydbpro' into 'flash_read_perf'
# Conflicts:
#   src/db.cpp
#   src/networking.cpp

Former-commit-id: e16c846b2b9e70f20981172287b19e585f81d73d
2021-04-01 17:25:31 +00:00
Wang Yuan
689b9ce067 Handle remaining fsync errors (#8419)
In `aof.c`, we call fsync when stop aof, and now print a log to let user know that if fail.
In `cluster.c`, we now return error, the calling function already handles these write errors.
In `redis-cli.c`, users hope to save rdb, we now print a message if fsync failed.
In `rio.c`, we now treat fsync errors like we do for write errors. 
In `server.c`, we try to fsync aof file when shutdown redis, we only can print one log if fail.
In `bio.c`, if failing to fsync aof file, we will set `aof_bio_fsync_status` to error , and reject writing just like last writing aof error,  moreover also set INFO command field `aof_last_write_status` to error.
2021-04-01 12:45:15 +03:00
Wang Yuan
1eb85249e7
Handle remaining fsync errors (#8419)
In `aof.c`, we call fsync when stop aof, and now print a log to let user know that if fail.
In `cluster.c`, we now return error, the calling function already handles these write errors.
In `redis-cli.c`, users hope to save rdb, we now print a message if fsync failed.
In `rio.c`, we now treat fsync errors like we do for write errors. 
In `server.c`, we try to fsync aof file when shutdown redis, we only can print one log if fail.
In `bio.c`, if failing to fsync aof file, we will set `aof_bio_fsync_status` to error , and reject writing just like last writing aof error,  moreover also set INFO command field `aof_last_write_status` to error.
2021-04-01 12:45:15 +03:00
Valentino Geron
f2a8041cd5 Fix XAUTOCLAIM response to return the next available id as the cursor (#8725)
This command used to return the last scanned entry id as the cursor,
instead of the next one to be scanned.
so in the next call, the user could / should have sent `(cursor` and not
just `cursor` if he wanted to avoid scanning the same record twice.

Scanning the record twice would look odd if someone is checking what
exactly was scanned, but it also has a side effect of incrementing the
delivery count twice.
2021-04-01 12:13:55 +03:00
Valentino Geron
44d8b039e8
Fix XAUTOCLAIM response to return the next available id as the cursor (#8725)
This command used to return the last scanned entry id as the cursor,
instead of the next one to be scanned.
so in the next call, the user could / should have sent `(cursor` and not
just `cursor` if he wanted to avoid scanning the same record twice.

Scanning the record twice would look odd if someone is checking what
exactly was scanned, but it also has a side effect of incrementing the
delivery count twice.
2021-04-01 12:13:55 +03:00
Oran Agra
9ab2625ce2 Solve sentinel test issue in TLS due to recent tests change. (#8728)
371d7f120 added a change that configures the tcp (plaintext) port
alongside the tls port, this causes the INFO command for tcp_port
to return that instead of the tls port when running in tls, and that broke
the sentinel tests that query it.

the fix is to add a method that gets the right port from CONFIG instead
of relying on the tcp_port info field.
2021-04-01 09:44:44 +03:00
Oran Agra
370ab4c4db
Solve sentinel test issue in TLS due to recent tests change. (#8728)
5629dbe71 added a change that configures the tcp (plaintext) port
alongside the tls port, this causes the INFO command for tcp_port
to return that instead of the tls port when running in tls, and that broke
the sentinel tests that query it.

the fix is to add a method that gets the right port from CONFIG instead
of relying on the tcp_port info field.
2021-04-01 09:44:44 +03:00
guybe7
41832aee11 zsetAdd: Fix wrong reply in case of INCR and GT/LT (#8717)
If GT/LT fails the operation we need to reply with
nill (like failure due to NX).

Other changes:
Add the missing $encoding suffix to many zset tests

Note: there's a behavior change just in case of INCR + GT/LT that fails.
The old code was replying with the wrong (rejected) score, and now it'll reply with nil.

Note that that's anyway a corner case so this "behavior change" shouldn't have too much affect.
Using GT/LT with INCR has a predictable result even before we run the command
(INCR GT will only only / always fail if the increment is negative).
2021-04-01 09:33:53 +03:00
guybe7
843f769b96
zsetAdd: Fix wrong reply in case of INCR and GT/LT (#8717)
If GT/LT fails the operation we need to reply with
nill (like failure due to NX).

Other changes:
Add the missing $encoding suffix to many zset tests

Note: there's a behavior change just in case of INCR + GT/LT that fails.
The old code was replying with the wrong (rejected) score, and now it'll reply with nil.

Note that that's anyway a corner case so this "behavior change" shouldn't have too much affect.
Using GT/LT with INCR has a predictable result even before we run the command
(INCR GT will only only / always fail if the increment is negative).
2021-04-01 09:33:53 +03:00
Wen Hui
b37ac422e7 generalize config file check for sentinel (#8730)
The implications of this change is just that in the past when a config file was missing,
in some cases it was exiting before printing the sever startup prints and sometimes after,
and now it'll always exit before printing them.
2021-04-01 09:01:05 +03:00
Wen Hui
d5935bb0a4
generalize config file check for sentinel (#8730)
The implications of this change is just that in the past when a config file was missing,
in some cases it was exiting before printing the sever startup prints and sometimes after,
and now it'll always exit before printing them.
2021-04-01 09:01:05 +03:00
wuYin
38c3522388 reuse existing range comparators in the zset (#8714)
There are 2 common range comparators for skiplist: zslValueGteMin and
zslValueLteMax, but they're not being reused in zslDeleteRangeByScore

This is a small change to make code cleaner.
2021-04-01 08:50:23 +03:00
wuYin
e5d50b236c
reuse existing range comparators in the zset (#8714)
There are 2 common range comparators for skiplist: zslValueGteMin and
zslValueLteMax, but they're not being reused in zslDeleteRangeByScore

This is a small change to make code cleaner.
2021-04-01 08:50:23 +03:00
Dvir Volk
7225b3cb53 Added macros for RM_log logging levels (#4246)
Added macros for RM_log logging levels, to avoid typos and the need
to memorize the level strings by heart
2021-04-01 08:44:57 +03:00
Dvir Volk
07f39ae47b
Added macros for RM_log logging levels (#4246)
Added macros for RM_log logging levels, to avoid typos and the need
to memorize the level strings by heart
2021-04-01 08:44:57 +03:00
sundb
a7812a6f95 Use chi-square for random distributivity verification in test (#8709)
Problem:
Currently, when performing random distribution verification, we determine
the probability of each element occurring in the sum, but the probability is
only an estimate, these tests had rare sporadic failures, and we cannot verify
what the probability of failure will be.

Solution:
Using the chi-square distribution instead of the original random distribution
validation makes the test more reasonable and easier to find problems.
2021-04-01 08:20:15 +03:00
sundb
569a3f4548
Use chi-square for random distributivity verification in test (#8709)
Problem:
Currently, when performing random distribution verification, we determine
the probability of each element occurring in the sum, but the probability is
only an estimate, these tests had rare sporadic failures, and we cannot verify
what the probability of failure will be.

Solution:
Using the chi-square distribution instead of the original random distribution
validation makes the test more reasonable and easier to find problems.
2021-04-01 08:20:15 +03:00
yoav-steinberg
e40f2219b1 Always use prev node if there's some room in it setDeferredReply (#8729)
this is a followup PR for #8699
instead of copying the deferred reply data to the previous node only if it has room for the entire thing,
we can now split the new payload, put part of it into the spare space in the prev node,
and the rest may fit into the next node.
2021-04-01 00:07:19 +03:00
yoav-steinberg
636aa8de76
Always use prev node if there's some room in it setDeferredReply (#8729)
this is a followup PR for #8699
instead of copying the deferred reply data to the previous node only if it has room for the entire thing,
we can now split the new payload, put part of it into the spare space in the prev node,
and the rest may fit into the next node.
2021-04-01 00:07:19 +03:00
guybe7
1f88ae6212 Initialize RedisModuleDigest with zeroes (#8731)
In case the module's digest function doesn't modify
'md' it'll contain garbage and result in wrong
DEBUG DIGEST
2021-03-31 23:56:04 +03:00
guybe7
68ba4d43ac
Initialize RedisModuleDigest with zeroes (#8731)
In case the module's digest function doesn't modify
'md' it'll contain garbage and result in wrong
DEBUG DIGEST
2021-03-31 23:56:04 +03:00
John Sully
e5d5b41be1 Fix issue #300
Former-commit-id: 22d0bca3a51bef7d42264b518b1c470df51fdc5e
2021-03-30 23:45:07 +00:00
John Sully
68b0098ee3 Fix issue #300
Former-commit-id: 22d0bca3a51bef7d42264b518b1c470df51fdc5e
2021-03-30 23:45:07 +00:00
John Sully
40fdf5670c Fix issue #300
Former-commit-id: bee7f398c9c3b4c01b687d26194913bf215dca36
2021-03-30 23:44:34 +00:00