9555 Commits

Author SHA1 Message Date
Huang Zhw
f5cc88e65d Fix "default" and overwritten / reset users will not have pubsub channels permissions by default. (#8723)
Background:
Redis 6.2 added ACL control for pubsub channels (#7993), which were supposed
to be permissive by default to retain compatibility with redis 6.0 ACL. 
But due to a bug, only newly created users got this `acl-pubsub-default` applied,
while overwritten (updated) users got reset to `resetchannels` (denied).

Since the "default" user exists before loading the config file,
any ACL change to it, results in an update / overwrite.

So when a "default" user is loaded from config file or include ACL
file with no channels related rules, the user will not have any
permissions to any channels. But other users will have default
permissions to any channels.

When upgraded from 6.0 with config rewrite, this will lead to
"default" user channels permissions lost.
When users are loaded from include file, then call "acl load", users
will also lost channels permissions.

Similarly, the `reset` ACL rule, would have reset the user to be denied
access to any channels, ignoring `acl-pubsub-default` and breaking
compatibility with redis 6.0.

The implication of this fix is that it regains compatibility with redis 6.0,
but breaks compatibility with redis 6.2.0 and 2.0.1. e.g. after the upgrade,
the default user will regain access to pubsub channels.

Other changes:
Additionally this commit rename server.acl_pubusub_default to
server.acl_pubsub_default and fix typo in acl tests.
2021-04-05 23:13:20 +03:00
Huang Zhw
84414bb33f redis-cli --bigkeys / memkeys, report detailed error on dbsize failure (#8740)
When DBSIZE failed (e.g. on AUTH error), the printed error didn't reflect the reason.
2021-04-05 08:30:41 +03:00
Sokolov Yura
0d7a17b252 Add cluster-allow-replica-migration option. (#5285)
Previously (and by default after commit) when master loose its last slot
(due to migration, for example), its replicas will migrate to new last slot
holder.

There are cases where this is not desired:
* Consolidation that results with removed nodes (including the replica, eventually).
* Manually configured cluster topologies, which the admin wishes to preserve.

Needlessly migrating a replica triggers a full synchronization and can have a negative impact, so
we prefer to be able to avoid it where possible.

This commit adds 'cluster-allow-replica-migration' configuration option that is
enabled by default to preserve existed behavior. When disabled, replicas will
not be auto-migrated.

Fixes #4896

Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-04 09:43:24 +03:00
John Sully
6b2567298d Don't ensure if we don't have to
Former-commit-id: b7b678a3ada531890e67313c867b7b49b01fe41e
2021-04-01 18:17:05 +00:00
jsully
7b61c83b3e Merge branch 'keydbpro' into 'flash_read_perf'
# Conflicts:
#   src/db.cpp
#   src/networking.cpp

Former-commit-id: e16c846b2b9e70f20981172287b19e585f81d73d
2021-04-01 17:25:31 +00:00
Wang Yuan
689b9ce067 Handle remaining fsync errors (#8419)
In `aof.c`, we call fsync when stop aof, and now print a log to let user know that if fail.
In `cluster.c`, we now return error, the calling function already handles these write errors.
In `redis-cli.c`, users hope to save rdb, we now print a message if fsync failed.
In `rio.c`, we now treat fsync errors like we do for write errors. 
In `server.c`, we try to fsync aof file when shutdown redis, we only can print one log if fail.
In `bio.c`, if failing to fsync aof file, we will set `aof_bio_fsync_status` to error , and reject writing just like last writing aof error,  moreover also set INFO command field `aof_last_write_status` to error.
2021-04-01 12:45:15 +03:00
Valentino Geron
f2a8041cd5 Fix XAUTOCLAIM response to return the next available id as the cursor (#8725)
This command used to return the last scanned entry id as the cursor,
instead of the next one to be scanned.
so in the next call, the user could / should have sent `(cursor` and not
just `cursor` if he wanted to avoid scanning the same record twice.

Scanning the record twice would look odd if someone is checking what
exactly was scanned, but it also has a side effect of incrementing the
delivery count twice.
2021-04-01 12:13:55 +03:00
guybe7
41832aee11 zsetAdd: Fix wrong reply in case of INCR and GT/LT (#8717)
If GT/LT fails the operation we need to reply with
nill (like failure due to NX).

Other changes:
Add the missing $encoding suffix to many zset tests

Note: there's a behavior change just in case of INCR + GT/LT that fails.
The old code was replying with the wrong (rejected) score, and now it'll reply with nil.

Note that that's anyway a corner case so this "behavior change" shouldn't have too much affect.
Using GT/LT with INCR has a predictable result even before we run the command
(INCR GT will only only / always fail if the increment is negative).
2021-04-01 09:33:53 +03:00
Wen Hui
b37ac422e7 generalize config file check for sentinel (#8730)
The implications of this change is just that in the past when a config file was missing,
in some cases it was exiting before printing the sever startup prints and sometimes after,
and now it'll always exit before printing them.
2021-04-01 09:01:05 +03:00
wuYin
38c3522388 reuse existing range comparators in the zset (#8714)
There are 2 common range comparators for skiplist: zslValueGteMin and
zslValueLteMax, but they're not being reused in zslDeleteRangeByScore

This is a small change to make code cleaner.
2021-04-01 08:50:23 +03:00
Dvir Volk
7225b3cb53 Added macros for RM_log logging levels (#4246)
Added macros for RM_log logging levels, to avoid typos and the need
to memorize the level strings by heart
2021-04-01 08:44:57 +03:00
yoav-steinberg
e40f2219b1 Always use prev node if there's some room in it setDeferredReply (#8729)
this is a followup PR for #8699
instead of copying the deferred reply data to the previous node only if it has room for the entire thing,
we can now split the new payload, put part of it into the spare space in the prev node,
and the rest may fit into the next node.
2021-04-01 00:07:19 +03:00
guybe7
1f88ae6212 Initialize RedisModuleDigest with zeroes (#8731)
In case the module's digest function doesn't modify
'md' it'll contain garbage and result in wrong
DEBUG DIGEST
2021-03-31 23:56:04 +03:00
John Sully
e5d5b41be1 Fix issue #300
Former-commit-id: 22d0bca3a51bef7d42264b518b1c470df51fdc5e
2021-03-30 23:45:07 +00:00
John Sully
40fdf5670c Fix issue #300
Former-commit-id: bee7f398c9c3b4c01b687d26194913bf215dca36
2021-03-30 23:44:34 +00:00
John Sully
577ba39c70 Fix issue #300
Former-commit-id: e9551c9e8d196f37e3742dfc7df824e164181d60
2021-03-30 23:30:41 +00:00
John Sully
ad8a706d4a Fix issue #300
Former-commit-id: bf5b2d78d98b61dedb8fa09930b70651ba97028f
2021-03-30 23:30:17 +00:00
John Sully
0b6a66ca55 Fix crash in RDB save
Former-commit-id: b032809b3e978fe571b791179d32ecdc9c067045
2021-03-30 20:49:33 +00:00
guybe7
7d01b47442 XAUTOCLAIM: JUSTID should prevent incrementing delivery_count (#8724)
To align with XCLAIM and the XAUTOCLAIM docs
2021-03-30 23:48:01 +03:00
John Sully
123d7ad148 Fix crash in RDB save
Former-commit-id: ef1e122fcb039d25f60752ca27c9feb5e8cfbb31
2021-03-30 20:44:22 +00:00
Jérôme Loyet
1ac84a68c7 Add replica-announced config option (#8653)
The 'sentinel replicas <master>' command will ignore replicas with
`replica-announced` set to no.

The goal of disabling the config setting replica-announced is to allow ghost
replicas. The replica is in the cluster, synchronize with its master, can be
promoted to master and is not exposed to sentinel clients. This way, it is
acting as a live backup or living ghost.

In addition, to prevent the replica to be promoted as master, set
replica-priority to 0.
2021-03-30 23:40:22 +03:00
Viktor Söderqvist
371d7f120f Add support for plaintext clients in TLS cluster (#8587)
The cluster bus is established over TLS or non-TLS depending on the configuration tls-cluster. The client ports distributed in the cluster and sent to clients are assumed to be TLS or non-TLS also depending on tls-cluster.

The cluster bus is now extended to also contain the non-TLS port of clients in a TLS cluster, when available. The non-TLS port of a cluster node, when available, is sent to clients connected without TLS in responses to CLUSTER SLOTS, CLUSTER NODES, CLUSTER SLAVES and MOVED and ASK redirects, instead of the TLS port.

The user was able to override the client port by defining cluster-announce-port. Now cluster-announce-tls-port is added, so the user can define an alternative announce port for both TLS and non-TLS clients.

Fixes #8134
2021-03-30 23:11:32 +03:00
yoav-steinberg
c77d031ede Minor client output buffer optimizations (#8699)
* Avoid checking output limits if buffer didn't grow.
* Use previouse node in case it has room in deferred output node.
2021-03-30 23:06:29 +03:00
JunhuaY
265f82d08f re-fix config rewrite for empty save directive (#8722)
the bug was also discussed in #8716, and was solved in #8719, but incompletely:
when the server is started, and the save option is default, if you issue the " config set save "" "
to change the save option, and then issue the “config rewrite” command, the " save "" " won't be saved.
2021-03-30 22:49:06 +03:00
Hanif Ariffin
4b204b6776 Fix typo in error string
Former-commit-id: 0ac80086bcef36b2abd2eb67f0d9830e3172ab17
2021-03-30 13:52:47 -04:00
VivekSainiEQ
239bb9cf65 Added logic back to only acquire/release GIL if modules are enabled, without causing deadlocks
Former-commit-id: 9ab36ddc36e1d12e41d2eca917ee24a44a82df52
2021-03-30 13:46:03 -04:00
VivekSainiEQ
662d5faa8b now acquire and release GIL every time you go to/wake from sleep
Former-commit-id: a6bc34c09b58f8c87da1611cd89638245d428a96
2021-03-30 13:46:03 -04:00
VivekSainiEQ
cb43eda420 Created and initialized seperate thread variables for modules
Former-commit-id: 3bb6b16c4a8f692b46040b72a51bef57fa03f1e6
2021-03-30 13:46:03 -04:00
Yossi Gottlieb
394ce70f1d Fix config rewrite with an empty "save" parameter. (#8719) 2021-03-29 18:53:20 +03:00
Meir Shpilraien (Spielrein)
b0c017a7bd Restore old client 'processCommandAndResetClient' to fix false dead client indicator (#8715)
'processCommandAndResetClient' returns 1 if client is dead. It does it
by checking if serve.current_client is NULL. On script timeout, Redis will re-enter
'processCommandAndResetClient' and when finish we will set server.current_client
to NULL. This will cause later to falsely return 1 and think that the client that
sent the timed-out script is dead (Redis to stop reading from the client buffer).
2021-03-29 13:34:16 +03:00
John Sully
c0d7601aa4 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 2910cee32ba6e4ef4b79b83bec2980c582a9310c
2021-03-29 00:49:34 +00:00
John Sully
3027a4cbb4 Enable LTO
Former-commit-id: a6edb104b7142b8e76048e0534e342048ade08d6
2021-03-29 00:47:50 +00:00
John Sully
e15f035bfb Prevent unnecessary copies in replication scenarios
Former-commit-id: b152a9bd88c081ce98eebe9a7af49649e60e5523
2021-03-28 22:43:15 +00:00
Steve
03c61de3f6 Fix redis-cli handling of rediss:// URL scheme (#8705) 2021-03-29 00:25:30 +03:00
John Sully
999dba6619 Make some asserts debug only for perf
Former-commit-id: dc66209f2cf8eadb794dad302bd1ea92890e75b0
2021-03-28 21:08:48 +00:00
John Sully
22b1ac1e8d Excessive rehashing adds latency
Former-commit-id: ee5a4528d61420a18f89a07f4ac63e2181a19738
2021-03-28 20:22:01 +00:00
John Sully
33197a128d Fix thread safety issues with the cache prefetch logic
Former-commit-id: 4892122fc02109d98684a350bd732a0b08a8c7b4
2021-03-28 20:21:44 +00:00
John Sully
afeb4db219 Prefetch keys even in pure RAM scenarios
Former-commit-id: d7219de186d60a5a437c1828ac97117eaad34819
2021-03-28 20:21:31 +00:00
John Sully
39f4615893 Eliminate unnecessary lookup in ensure when there is no snapshot
Former-commit-id: 1f363ed7c13c186f0c120ab4f3e321144667f50f
2021-03-28 20:19:15 +00:00
John Sully
aa8800abaf Enable LTO
Former-commit-id: 3ec75184bae92c0e7af579eda8cbe6cfa2375327
2021-03-28 20:13:54 +00:00
John Sully
8377da61ed Enable LTO
Former-commit-id: 7241bfc25525e8b27193fb0d4f9eae2e5e98e8d6
2021-03-28 18:27:14 +00:00
John Sully
6cf736535d Make some asserts debug only for perf
Former-commit-id: c22740df4f7c1468fbe19fd97dfaec20e011b6f1
2021-03-28 18:27:00 +00:00
John Sully
b8c03fd014 Excessive rehashing adds latency
Former-commit-id: 9db2277b0fea032abcde089f617fe7e4f5ce12e3
2021-03-28 17:59:02 +00:00
John Sully
74d73b4ca3 Fix thread safety issues with the cache prefetch logic
Former-commit-id: a80a128bb64b81115c095d6dd91896ff73048b3d
2021-03-28 17:58:43 +00:00
Huang Zhw
36424b8e0f Remove redundant defrag dictEntry in activeDefragSdsListAndDict. (#8713)
In activeDefragSdsListAndDict when defrag list key sucess, the key
in dict is also replaced. Then the corresponding dictEntry is
defraged. But the dictEntry will be defraged in next step by defrag
the dict values too.
2021-03-28 15:16:30 +03:00
benschermel
8130a7fa59 bump version
Former-commit-id: f660cd93574ba96d707c1cf7853e1e5074b0ec3f
2021-03-27 18:30:52 +00:00
benschermel
2f80810544 bump version
Former-commit-id: 33b75e0a2cf8712c976e7af74e2a615bb991c356
2021-03-27 18:26:11 +00:00
John Sully
0d69b7684e Prefetch keys even in pure RAM scenarios
Former-commit-id: 54c0f11db0b35f34c9dfc3c8b4b76b7b7921bacd
2021-03-26 23:48:24 +00:00
John Sully
a66295ed98 Eliminate unnecessary lookup in ensure when there is no snapshot
Former-commit-id: 04540531b95ebadf6d60c90c6681c11060eaf907
2021-03-26 23:44:42 +00:00
benschermel
9a8d93708a bump version
Former-commit-id: 4272a7de24eca935bfe1e13d44bdc0ece58efb55
2021-03-26 22:57:19 +00:00