9555 Commits

Author SHA1 Message Date
Yossi Gottlieb
ed23d94107 Improve redis-cli non-binary safe string handling. (#8566)
* The `redis-cli --scan` output should honor output mode (set explicitly or implicitly), and quote key names when not in raw mode.
  * Technically this is a breaking change, but it should be very minor since raw mode is by default on for non-tty output.
  * It should only affect  TTY output (human users) or non-tty output if `--no-raw` is specified.

* Added `--quoted-input` option to treat all arguments as potentially quoted strings.
* Added `--quoted-pattern` option to accept a potentially quoted pattern.

Unquoting is applied to potentially quoted input only if single or double quotes are used. 

Fixes #8561, #8563
2021-03-04 15:03:49 +02:00
sundb
141c6e1716 Fix memory overlap in quicklistRotate (#8599)
When the length of the quicklist is 1(only one zipmap), the rotate operation will cause
memory overlap when moving an entity from the tail of the zipmap to the head.
quicklistRotate is a dead code, so it has no impact on the existing code.
2021-03-04 13:12:00 +02:00
YaacovHazan
ead6f01882 Fix RedisModule_IsAOFClient Redis Module API (#8596)
Since the API declared (as #define) in redismodule.h and uses
the CLIENT_ID_AOF that declared in the server.h, when
a module will want to make use of this API, it will get a compilation
error (module doesn't include the server.h).

The API was broken by fd914fd (failed attempt for a cleanup).
Revert to the original version of RedisModule_IsAOFClient
that uses UINT64_MAX instead of CLIENT_ID_AOF
2021-03-04 13:02:23 +02:00
John Sully
902264efb7 Load perf fixes with a storage provider set
Former-commit-id: 861b19de00c75c9167cc25031292284ad1c21893
2021-03-04 07:41:06 +00:00
John Sully
50ce24a10c Fix compile warnings
Former-commit-id: c314cab888e9c8b3e825b2dfe0c0392ee998bdc4
2021-03-03 22:12:51 +00:00
jsully
ba5b7a6da1 Merge branch 'time_thread_priority' into 'keydbpro'
time thread priority + nanosleep

See merge request keydb-dev/KeyDB-Pro!11

Former-commit-id: 423b76a2bbbd568f525f6469ce829ab053972536
2021-03-03 18:55:07 +00:00
Yossi Gottlieb
31ebf191ba Improve SSL cleanup handling. (#8589)
This solves the problem of /dev/random and /dev/urandom open file
descriptors leaking to childs with some versions of OpenSSL.
2021-03-03 10:08:06 +02:00
yoav-steinberg
667b36adac Fix client_recent_max_input/output_buffer in 'INFO CLIENTS' when all clients drop. (#8588)
When no connected clients variables stopped being updated instead of being zeroed over time.

The other changes are cleanup and optimization (avoiding an unnecessary division per client)
2021-03-03 09:36:27 +02:00
John Sully
08b9f21d71 Data loss issue due to async rehash interfering with snapshots
Former-commit-id: 5245ca9a67c74b34139cafc9754543d1b8bed90a
2021-03-03 07:06:19 +00:00
John Sully
ea6ba8370d In low load async rehash may not complete in time, do it in the cron
Former-commit-id: 0a27d30753d887b6251e645abe26118068c55587
2021-03-03 07:05:51 +00:00
John Sully
4066ce8f3a Don't let dictionaries shrink too rapidly. It can cause massive perf issues while in the shrink rehash
Former-commit-id: a7ad346e4f03c85d22a29c8268d35471e86283aa
2021-03-03 07:05:12 +00:00
Wen Hui
8264f8bdbd generalize error check for SENTINEL MONITOR rumtime config (#8590) 2021-03-03 08:36:08 +02:00
Yossi Gottlieb
fe76001425 Fix errors when loading RDB with missing modules. (#8579)
Fixes #8574
2021-03-02 09:39:37 +02:00
John Sully
76698beeaf Drastically improve perf when loading an RDB with a storage provider
Former-commit-id: 0133b42d54676e8fac2c5cb006cc87988dced268
2021-03-02 04:16:20 +00:00
John Sully
6065f276c4 Key prefetch error handling instead of crashign
Former-commit-id: b322a14efb1b897134b229c1726b45264b57783f
2021-03-02 01:38:12 +00:00
VivekSainiEQ
d855afdcac Fixed bug where make USE_SYSTEMD=yes would not pass in correct flags, issue #226
Former-commit-id: 69dc0a06bfd34516c5c18a6e22985ef01a472339
2021-03-01 21:31:33 +00:00
Oran Agra
003cff7b0a Redis 6.2.1 2021-03-01 21:02:05 +02:00
Oran Agra
c83db6bf5a Merge unstable into 6.2 2021-03-01 20:54:02 +02:00
Oran Agra
e978fc37ed fix stream deep sanitization with deleted records (#8568)
When sanitizing the stream listpack, we need to count the deleted records too.
otherwise the last line that checks the next pointer fails.

Add test to cover that state in the stream tests.
2021-03-01 17:23:29 +02:00
YaacovHazan
c0de6fbd29 Make port, tls-port and bind configurations modifiable (#8510)
Add ability to modify port, tls-port and bind configurations by CONFIG SET command.

To simplify the code and make it cleaner, a new structure
added, socketFds, which contains the file descriptors array and its counter,
and used for TCP, TLS and Cluster sockets file descriptors.
2021-03-01 16:04:44 +02:00
Bonsai
34fa7b137e fix: call CLIENT INFO from redis module will crash the server (#8560)
Because when the RM_Call is invoked. It will create a faker client.
The point is client connection is NULL, so server will crash in connGetInfo

Co-authored-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
2021-03-01 08:18:14 +02:00
David Gilman
c6fb349f05 Fix compliation on arm64 Mac with jemalloc (#8458)
The arm_thread_state64_get_pc used later in the file is defined
in mach kernel headers. Apparently they get included if you use
the system malloc but not if you use jemalloc.
2021-03-01 08:15:26 +02:00
Qu Chen
adfb3a52c6 Make dbid range check for SWAPDB command consistent with SELECT, MOVE, and COPY. (#8555)
DB ID used to be parsed as a long for SWAPDB command, now
make it into an int to be consistent with other commands that parses
the DB ID argument like SELECT, MOVE, COPY. See #8085

The implication is that the error message when the provided db index
is greater than 4M changes slightly.
2021-03-01 07:54:52 +02:00
John Sully
eabfae81b3 Bump version
Former-commit-id: 31392562b27a6fc1576476626a27a3defe13795f
2021-02-28 21:02:25 +00:00
John Sully
0cba72ba14 Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: 7609673e282b46a760a3d0e0b42d685af9b4d056
2021-02-28 20:46:08 +00:00
Pavlo Yatsukhnenko
ca3b1cd7f5 Fix div by 0 in redis-cli cluster creation (#8553)
This could happen on an invalid use, when trying to create a cluster with
a single node and provide it's address 3 time to satisfy redis-cli requirements.
2021-02-28 15:28:09 +02:00
Bonsai
956949e7a1 Module API for getting user name of a client (#8508)
Adding RM_GetClientUserNameById to get the ACL user name of a client connection.
2021-02-28 14:36:37 +02:00
Viktor Söderqvist
f24d4002c0 Shared reusable client for RM_Call() (#8516)
A single client pointer is added in the server struct. This is
initialized by the first RM_Call() and reused for every subsequent
RM_Call() except if it's already in use, which means that it's not
used for (recursive) module calls to modules. For these, a new
"fake" client is created each time.

Other changes:
* Avoid allocating a dict iterator in pubsubUnsubscribeAllChannels
  when not needed
2021-02-28 14:11:18 +02:00
christianEQ
175b266c34 time thread priority + nanosleep
Former-commit-id: c84f296edabc001a1836ab8437f746dcff811148
2021-02-26 20:17:34 +00:00
John Sully
bbb310f419 Eliminate needless lock
Former-commit-id: 60f972d463f202edb33ff9a25bc2bd3e2558105c
2021-02-26 05:40:56 +00:00
Madelyn Olson
45cd457aa7 Moved requirepass and querybuf length to generic configs (#8557)
Moved additional configs to generic infrastructure.
2021-02-25 21:00:27 -08:00
John Sully
6ee9d69763 Eliminate the need for an mfence by tricking the CPU into ordering the futex read
Former-commit-id: 340e6f5bc94cd1c3b0c6fb6da833e8504acaf23a
2021-02-26 01:28:05 +00:00
John Sully
4f632962c1 eliminate syscall in call()
Former-commit-id: 3ee111a2e50bc29818ba85ae4fb786171d695c37
2021-02-26 01:03:10 +00:00
jsully
1459d88a6d Merge branch 'timethread' into 'keydbpro'
Offload updating cached time to dedicated thread

See merge request keydb-dev/KeyDB-Pro!10

Former-commit-id: 31786a995d0220ec0cb4502917a8cc0f192da4cb
2021-02-26 00:50:22 +00:00
christian
53b7b83af6 Offload updating cached time to dedicated thread
Former-commit-id: 9bfc8a43952481b5b54a7b051d44b8bece4a18dd
2021-02-26 00:50:22 +00:00
Wen Hui
63bba04778 redis-cli prompt: fix db number and transaction state inconsistencies after reconnect (#8551)
After reconnect, the prompt was showing the db ID and multi state of the previous connection.
2021-02-25 20:08:24 +02:00
Yossi Gottlieb
a21cfa197a Cleanup usage of malloc_usable_size. (#8554)
* Add better control of malloc_usable_size() usage.
* Use malloc_usable_size on alpine libc daily job.
* Add no-malloc-usable-size daily jobs.
* Fix zmalloc(0) when HAVE_MALLOC_SIZE is undefined.

In order to align with the jemalloc behavior, this should never return
NULL or OOM panic.
2021-02-25 09:24:41 +02:00
sundb
c65e914491 Use addReplyErrorObject with shared.noscripterr (#8544) 2021-02-24 08:45:13 -08:00
guybe7
a4e3ddd33c XTRIM: Parse args before lookupKey (#8550)
This aligns better with other commands, specifically XADD
2021-02-24 16:41:50 +02:00
guybe7
c307ce826a XTRIM should take at least 4 args (#8549)
Fix command arity.
Other than key name it must at least take the trimming strategy and it's limit
2021-02-24 16:41:00 +02:00
Huang Zw
7230957b5f Fix quicklistDelRange dead code, delete range wrong (#8257)
This commit fixes a bug in what's currently dead code in redis.

In quicklistDelRange when delete entry from entry.offset to node tail,
extent only need gte node->count - entry.offset, not node->count

Co-authored-by: Yoav Steinberg <yoav@redislabs.com>
2021-02-24 13:18:54 +02:00
Yossi Gottlieb
ba7a99485a Cleanup clang warnings. (#8546) 2021-02-24 10:10:02 +02:00
Yossi Gottlieb
0e5e931839 Use malloc_usable_size() on FreeBSD. (#8545) 2021-02-24 09:48:04 +02:00
Huang Zw
9bf31f67f0 In luaRedisGenericCommand check channel return is ACL_DENIED_CHANNEL (#8535)
not ACL_DENIED_AUTH
2021-02-23 16:55:10 -08:00
Yossi Gottlieb
1cb0f85172 Fix compile errors with no HAVE_MALLOC_SIZE. (#8533)
Also adds a new daily CI test, relying on the fact that we don't use malloc_size() on alpine libmusl.

Fixes #8531
2021-02-23 17:08:49 +02:00
Yossi Gottlieb
665da8a056 Fix failed tests on Linux Alpine and add a CI job. (#8532)
* Remove linux/version.h dependency.

This introduces unnecessary dependencies, and generally not a good idea
as the platform we build on may be different than the platform we run
on.

To determine if sync_file_range exists we can simply rely on header file
hints.

* Fix setproctitle() on libmusl.

The previous ifdef checks were a bit too strict for no apparent
reason.

* Fix tests failure on Linux with no backtrace.

* Add alpine daily CI job.
2021-02-23 12:57:45 +02:00
Oran Agra
c5a3527200 Redis 6.2.0 GA 2021-02-22 15:56:55 +02:00
Oran Agra
e8b616c8ce Merge origin/unstable into 6.2 2021-02-22 15:48:42 +02:00
Oran Agra
8e9579a308 update help.h from redis.io commands.json (#8524) 2021-02-22 15:48:17 +02:00
Yossi Gottlieb
9cee879367 Fix integer overflow (CVE-2021-21309). (#8522)
On 32-bit systems, setting the proto-max-bulk-len config parameter to a high value may result with integer overflow and a subsequent heap overflow when parsing an input bulk (CVE-2021-21309).

This fix has two parts:

Set a reasonable limit to the config parameter.
Add additional checks to prevent the problem in other potential but unknown code paths.
2021-02-22 15:41:32 +02:00