14679 Commits

Author SHA1 Message Date
Yossi Gottlieb
84e0489562
Remove end-of-life CentOS 6 workflows. (#8121) 2020-12-02 13:47:25 +02:00
luhuachao
7885faf18b
Modify help msg PING_BULK to PING_MBULK in benchmark (#8109)
As described in redis-benchamrk help message 'The test names are the same as the ones produced as output.', In redis-benchmark output, we can only see PING_BULK, but the cmd `redis-benchmark -t ping_bulk` is not supported. We  have to run it with ping_mbulk which is not user friendly.
2020-12-02 13:17:25 +02:00
christianEQ
71b4a7b6d3 Issue: #204 Allocate 8 MB to thread stack
Former-commit-id: 66a41dafc47a20251f5f6776625780dfa26ee505
2020-12-01 14:50:03 -05:00
Madelyn Olson
69b7113bb5
Getset fix (#8118)
* Fixed SET GET executing on wrong type

Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2020-12-01 11:46:45 -08:00
sundb
3ba2281f96
Improve dbid range check for SELECT, MOVE, COPY (#8085)
SELECT used to read the index into a `long` variable, and then pass it to a function
that takes an `int`, possibly causing an overflow before the range check.

Now all these commands use better and cleaner range check, and that also results in
a slight change of the error response in case of an invalid database index.

SELECT:
in the past it would have returned either `-ERR invalid DB index` (if not a number),
or `-ERR DB index is out of range` (if not between 1..16 or alike).
now it'll return either `-ERR value is out of range` (if not a number), or
`-ERR value is out of range, value must between -2147483648 and 2147483647`
(if not in the range for an int), or `-ERR DB index is out of range`
(if not between 0..16 or alike)


MOVE:
in the past it would only fail with `-ERR index out of range` no matter the reason.
now return the same errors as the new ones for SELECT mentioned above.
(i.e. unlike for SELECT even for a value like 17 we changed the error message)

COPY:
doesn't really matter how it behaved in the past (new command), new behavior is
like the above two.
2020-12-01 21:41:26 +02:00
Itamar Haber
c1b1e8c329
Adds pub/sub channel patterns to ACL (#7993)
Fixes #7923.

This PR appropriates the special `&` symbol (because `@` and `*` are taken),
followed by a literal value or pattern for describing the Pub/Sub patterns that
an ACL user can interact with. It is similar to the existing key patterns
mechanism in function (additive) and implementation (copy-pasta). It also adds
the allchannels and resetchannels ACL keywords, naturally.

The default user is given allchannels permissions, whereas new users get
whatever is defined by the acl-pubsub-default configuration directive. For
backward compatibility in 6.2, the default of this directive is allchannels but
this is likely to be changed to resetchannels in the next major version for
stronger default security settings.

Unless allchannels is set for the user, channel access permissions are checked
as follows :
* Calls to both PUBLISH and SUBSCRIBE will fail unless a pattern matching the
  argumentative channel name(s) exists for the user.
* Calls to PSUBSCRIBE will fail unless the pattern(s) provided as an argument
  literally exist(s) in the user's list.

Such failures are logged to the ACL log.

Runtime changes to channel permissions for a user with existing subscribing
clients cause said clients to disconnect unless the new permissions permit the
connections to continue. Note, however, that PSUBSCRIBErs' patterns are matched
literally, so given the change bar:* -> b*, pattern subscribers to bar:* will be
disconnected.

Notes/questions:
* UNSUBSCRIBE, PUNSUBSCRIBE and PUBSUB remain unprotected due to lack of reasons
  for touching them.
2020-12-01 14:21:39 +02:00
John Sully
300eca8ff6 Issue: #260 Don't use C++ static_assert in C files
Former-commit-id: f19b9c155ef5b957f7cd9a3017c54aab742ffb2f
2020-11-30 23:01:20 +00:00
John Sully
ecad9741bc code coverage reports should test multithreaded
Former-commit-id: 5d016fdb6b11404d0eddbab651ed17180526a991
2020-11-30 22:59:00 +00:00
VivekSainiEQ
52561f1729 Added module data type load/save tests
Former-commit-id: cae9924fd9eefcd88cef1c964f0bc8bce7dd4242
2020-11-30 17:47:04 -05:00
VivekSainiEQ
3d15a7b65a Removed use of module datatypes, now should work if tests/modules is not built
Former-commit-id: 089f28e8ba91ed2b875048efa0b28ceb32010541
2020-11-30 17:47:04 -05:00
VivekSainiEQ
54b36868ba Added tests for saving various data types to disk and loading them back, and for loading data types from redis to maintain compatibility
Former-commit-id: dcb44d3a09a4021f05079bedbac690e33ec7f39e
2020-11-30 17:47:04 -05:00
VivekSainiEQ
6c16fe69e7 Added remainder of runtest scripts to make lcov command. Code coverage goes from 57.9% to 72.1%
Former-commit-id: f99cba0b92d921b1a2d350f2c7545138485c2a74
2020-11-30 17:47:04 -05:00
VivekSainiEQ
6632bafced Added KeyDB.info to .gitignore file
Former-commit-id: e10dc0dace6b152418aad2bd3ee212a471bca9e8
2020-11-30 17:47:04 -05:00
Wang Yuan
c85bf2352d
Reset average ttl when empty databases (#8106)
On FLUSHDB or full sync, reset old average TTL stat.
This Stat is incrementally collected by the master over time when it searches for expired keys.
2020-11-30 23:15:14 +02:00
sundb
04056b767f
BITOP speedup when or/and output is 0/255, stop processing further keys (#8110)
when performing the and operation, if the output is 0, we can jump out of the loop.
when performing an or operation, if the output is 0xff, we can jump out of the loop.
2020-11-30 23:03:53 +02:00
Kajaruban Surendran
fd03008bbf Configurable option for MOTD
Former-commit-id: 49a89d636ba698dbd0858d5059d3d6387c8c1fc7
2020-11-30 10:42:48 -05:00
Itamar Haber
d322e7baba
Adds support for nested args in cli's help (#8074)
See https://github.com/redis/redis-doc/pull/1443

Also allows nameless commands.
2020-11-30 15:00:16 +02:00
guybe7
ada2ac9ae2
XPENDING with IDLE (#7972)
Used to filter stream pending entries by their idle-time,
useful for XCLAIMing entries that have not been processed
for some time
2020-11-29 12:08:47 +02:00
Oran Agra
cb5eadb33b
bitops limited to proto_max_bulk_len rather than 512MB (#8096)
we recently did that for SETRANGE and APPEND
2020-11-26 10:58:01 +02:00
VivekSainiEQ
f99824faf4 Added references to KeyDB.info instead of redis.info
Former-commit-id: b94f91b9b25b5476d62bdbe2f054cc61691de9f5
2020-11-25 18:49:20 -05:00
VivekSainiEQ
7ad88d0bac Properly escaped $ in lcov command
Former-commit-id: 06a17c8511cfa8e07962401cc01045878525dc27
2020-11-25 18:49:20 -05:00
VivekSainiEQ
2ab3385733 Updated 'make lcov' command to exclude external dependancies and generate final code coverage percentage
Former-commit-id: eebd70aaa6bec0362ea90148174263fa6ffcc169
2020-11-25 18:49:20 -05:00
VivekSainiEQ
86ca4300c8 Added test for issue #257
Former-commit-id: 779c775df9e8be36064689404e3ff51bead8a364
2020-11-25 17:21:37 -05:00
VivekSainiEQ
47a18f5131 Made NULL pointer check explicit, issue #257
Former-commit-id: 1d4d2925c2a11bb7fdffe92c17a315c2df011a54
2020-11-25 17:21:37 -05:00
VivekSainiEQ
d4a2ccd50b Added NULL check to function objFromAllocPtr, now NULL input will map to NULL output even if active replica is enabled
Former-commit-id: b67db7c2d5396892cc0c91c3c822a5b9a547badf
2020-11-25 17:21:37 -05:00
Oran Agra
bbc2c44541
INFO client_recent_max_input_buffer includes argv array (#8065)
this metric already includes the argv bytes, like what clientsCronTrackClientsMemUsage does, but it's missing the array itself.

p.s. For the purpose of tracking expensive clients we don't need to include the size of the client struct and the static reply buffer in it.
2020-11-25 23:39:01 +02:00
kukey
cf88779527
Merge two aeDeleteFileEvent refs into one (#7521)
Merge two aeDeleteFileEvent refs into one
2020-11-25 13:37:54 -08:00
Dipankar Achinta
79a7c17176
Fix typo in ae.c file (#7895) 2020-11-25 13:36:21 -08:00
David CARLIER
0719388cfb
raspberry build fix. (#8095)
__ILP32__ is 32 bits ABI and does not imply x86, this patch resolves this.
2020-11-25 12:15:32 -08:00
sundb
25f457c7f6
Avoid excessive malloc and free in copyCommand robj creation (#8067)
Avoid multiple conditional judgments
Avoid allocating robj->ptr when we're gonna replace it right after.
2020-11-24 21:40:58 +02:00
Yossi Gottlieb
7e5a6313f0
Fix use-after-free issue in spt_copyenv. (#8088)
Seems to have gone unnoticed for a long time, because at least with
glibc it will only be triggered if setenv() was called before spt_init,
which Redis doesn't.

Fixes #8064.
2020-11-24 17:58:10 +02:00
David CARLIER
f16b52cb7d
redis_set_thread_title support for Haiku. (#8060) 2020-11-23 16:14:33 +02:00
Itamar Haber
0963edbc65
Adds tests for XADD/XTRIM's MAXLEN arguments (#8083) 2020-11-23 14:37:58 +02:00
John Sully
3d75ddeb87 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 18953b2998e5527260d161ce882b55a9684efab5
2020-11-23 02:01:54 +00:00
John Sully
c179c98870 Fix issue where active replication doesn't replicate RDB data
Former-commit-id: 527b7eb0742567302e0343e3acbed9814c0cbb95
2020-11-23 02:01:40 +00:00
xindoo
0b1d89d7ae
fix comment error about zslDeleteRangeByScore range (#8075)
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-11-22 14:56:45 +02:00
Yossi Gottlieb
08d3e929e5
Clean up building with USE_SYSTEMD. (#8073)
When USE_SYSTEMD=yes is specified, try to use pkg-config to determine
libsystemd linker flags. If not found, silently fall back to simply
using "-lsystemd".

We now use a LIBSYSTEMD_LIBS variable so users can explicitly override
it and specify their own library.

If USE_SYSTEMD is unspecified the old behavior of auto-enabling it if
both pkg-config and libsystemd are available is retained.
2020-11-22 14:40:38 +02:00
Wang Yuan
f207e1682f
Fix diskless replication failure when has non-rdb child process (#8070)
If we enable diskless replication, set repl-diskless-sync-delay to 0,
and master has non-rdb child process such as rewrite aof child, master
will try to start to a new BGSAVE but fails immediately (before fork)
when replicas ask for full synchronization, and master always fails
to start a new BGSAVE and disconnects with replicas until non-rdb
child process exists.

this bug was introduced in #6271 (not yet released in 6.0.x)
2020-11-22 14:12:45 +02:00
Oran Agra
e6fa47380a
Fix bug with module GIL being released prematurely (#8061)
This is hopefully usually harmles.
The server.ready_keys will usually be empty so the code after releasing
the GIL will soon be done.
The only case where it'll actually process things is when a module
releases a client (or module) blocked on a key, by triggering this NOT
from within a command (e.g. a timer event).

This bug was introduced in redis 6.0.9, see #7903
2020-11-22 14:00:51 +02:00
Oran Agra
61954951ed
Fix oom-score-adj-values range, abs options, and bug when used in config file (#8046)
Fix: When oom-score-adj-values is provided in the config file after
oom-score-adj yes, it'll take an immediate action, before
readOOMScoreAdj was acquired, resulting in an error (out of range score
due to uninitialized value. delay the reaction the real call is made by
main().

Since the values are clamped to -1000..1000, and they're
applied as an offset from the value at startup (which may be -1000), we
need to allow the offsets to reach to +2000 so that a value of +1000 is
achievable in case the value at startup was -1000.

Adding an option for absolute values rather than relative ones.
2020-11-22 13:57:56 +02:00
Kajaruban Surendran
3ee8edc8b0 fix for the server crash when the maxclients increased via config set
Former-commit-id: 34a97c447f8cb5753e0747b435c7ce1a5d381c47
2020-11-20 15:20:46 -05:00
Rosen Penev
a221a313ae
fix compilation with uClibc-ng (#8054)
backtrace can be compile time disabled.
2020-11-20 18:13:11 +02:00
John Sully
e8753d1b4b Blocking clients should not crash if an active replica loads a remote RDB with a key in the blocklist
Former-commit-id: 1c525e20b10e0a47af687a0d46faf75229a1cbf5
2020-11-19 23:28:01 +00:00
John Sully
8d8f2a3b9f Merge branch 'RELEASE_6' into keydbpro
Former-commit-id: 2ee33cb050b26cbe34f0b390b8807cdc1b908ded
2020-11-19 20:38:47 +00:00
guybe7
f8ae991717
EXISTS should not alter LRU, OBJECT should not reveal expired keys on replica (#8016)
The bug was introduced by #5021 which only attempted avoid EXIST on an
already expired key from returning 1 on a replica.

Before that commit, dbExists was used instead of
lookupKeyRead (which had an undesired effect to "touch" the LRU/LFU)

Other than that, this commit fixes OBJECT to also come empty handed on
expired keys in replica.

And DEBUG DIGEST-VALUE to behave like DEBUG OBJECT (get the data from
the key regardless of it's expired state)
2020-11-18 11:16:21 +02:00
Meir Shpilraien (Spielrein)
d87a0d0286
Unified MULTI, LUA, and RM_Call with respect to blocking commands (#8025)
Blocking command should not be used with MULTI, LUA, and RM_Call. This is because,
the caller, who executes the command in this context, expects a reply.

Today, LUA and MULTI have a special (and different) treatment to blocking commands:

LUA   - Most commands are marked with no-script flag which are checked when executing
and command from LUA, commands that are not marked (like XREAD) verify that their
blocking mode is not used inside LUA (by checking the CLIENT_LUA client flag).
MULTI - Command that is going to block, first verify that the client is not inside
multi (by checking the CLIENT_MULTI client flag). If the client is inside multi, they
return a result which is a match to the empty key with no timeout (for example blpop
inside MULTI will act as lpop)
For modules that perform RM_Call with blocking command, the returned results type is
REDISMODULE_REPLY_UNKNOWN and the caller can not really know what happened.

Disadvantages of the current state are:

No unified approach, LUA, MULTI, and RM_Call, each has a different treatment
Module can not safely execute blocking command (and get reply or error).
Though It is true that modules are not like LUA or MULTI and should be smarter not
to execute blocking commands on RM_Call, sometimes you want to execute a command base
on client input (for example if you create a module that provides a new scripting
language like javascript or python).
While modules (on modules command) can check for REDISMODULE_CTX_FLAGS_LUA or
REDISMODULE_CTX_FLAGS_MULTI to know not to block the client, there is no way to
check if the command came from another module using RM_Call. So there is no way
for a module to know not to block another module RM_Call execution.

This commit adds a way to unify the treatment for blocking clients by introducing
a new CLIENT_DENY_BLOCKING client flag. On LUA, MULTI, and RM_Call the new flag
turned on to signify that the client should not be blocked. A blocking command
verifies that the flag is turned off before blocking. If a blocking command sees
that the CLIENT_DENY_BLOCKING flag is on, it's not blocking and return results
which are matches to empty key with no timeout (as MULTI does today).

The new flag is checked on the following commands:

List blocking commands: BLPOP, BRPOP, BRPOPLPUSH, BLMOVE,
Zset blocking commands: BZPOPMIN, BZPOPMAX
Stream blocking commands: XREAD, XREADGROUP
SUBSCRIBE, PSUBSCRIBE, MONITOR
In addition, the new flag is turned on inside the AOF client, we do not want to
block the AOF client to prevent deadlocks and commands ordering issues (and there
is also an existing assert in the code that verifies it).

To keep backward compatibility on LUA, all the no-script flags on existing commands
were kept untouched. In addition, a LUA special treatment on XREAD and XREADGROUP was kept.

To keep backward compatibility on MULTI (which today allows SUBSCRIBE, and PSUBSCRIBE).
We added a special treatment on those commands to allow executing them on MULTI.

The only backward compatibility issue that this PR introduces is that now MONITOR
is not allowed inside MULTI.

Tests were added to verify blocking commands are not blocking the client on LUA, MULTI,
or RM_Call. Tests were added to verify the module can check for CLIENT_DENY_BLOCKING flag.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-11-17 18:58:55 +02:00
thomaston
39f716a121
ZREVRANGEBYSCORE Optimization for out of range offset (#5773)
ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]
When the offset is too large, the query is very slow. Especially when the offset is greater than the length of zset it is easy to determine whether the offset is greater than the length of zset at first, and If it exceed the length of zset, then return directly.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-11-17 18:35:56 +02:00
Yossi Gottlieb
d638b05834
Improve and clean up supervised process support. (#8036)
* Configuration file default should now be "auto".
* Expose "process_supervised" as an info field.
* Log messages improvements (clarify required systemd config, report
  auto-detected supervision mode, etc.)
* Set server.supervised properly, so it can take precedence of
  "daemonize" configuration.
* Produce clear warning if systemd is detected/requested but executable
  is compiled without support for it, instead of silently ignoring.
* Handle systemd notification error on startup, and turn off supervised
  mode if it failed.
2020-11-17 12:52:49 +02:00
swamp0407
ea7cf737a1
Add COPY command (#7953)
Syntax:
COPY <key> <new-key> [DB <dest-db>] [REPLACE]

No support for module keys yet.

Co-authored-by: tmgauss
Co-authored-by: Itamar Haber <itamar@redislabs.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-11-17 12:03:05 +02:00
Oran Agra
9812e88959
Fix memory leaks in newly added ZDIFF (#8056) 2020-11-16 16:37:15 +02:00