21065 Commits

Author SHA1 Message Date
Huang Zw
f395119ede
Fix bug in activeDefragSdsListAndDict dead code (#8403)
In activeDefragSdsListAndDict when dict_val_type is DEFRAG_SDS_DICT_VAL_VOID_PTR, it should update de->v.val not ln->value.
Because this code path will never be executed, so this bug never happened.
2021-01-27 18:56:43 +02:00
Wang Yuan
a32f87de07 Implement rdb-only replication (#8303)
In some scenarios, such as remote backup, we only want to get remote
redis server db snapshot. Currently, redis-cli acts as a replica and
sends SYNC to redis, but redis still accumulates replication buffer
in the replica client output buffer, that may result in using vast
memory, or failing to transfer RDB because of client-output-buffer-limit.
In this commit, we add 'replconf rdb-only 0|1', redis doesn't send
incremental replication buffer to them if they send 'replconf rdb-only 1',
so we can reduce used memory and improve success of getting RDB.
2021-01-27 17:13:10 +02:00
Wang Yuan
ad7d4c6b70
Implement rdb-only replication (#8303)
In some scenarios, such as remote backup, we only want to get remote
redis server db snapshot. Currently, redis-cli acts as a replica and
sends SYNC to redis, but redis still accumulates replication buffer
in the replica client output buffer, that may result in using vast
memory, or failing to transfer RDB because of client-output-buffer-limit.
In this commit, we add 'replconf rdb-only 0|1', redis doesn't send
incremental replication buffer to them if they send 'replconf rdb-only 1',
so we can reduce used memory and improve success of getting RDB.
2021-01-27 17:13:10 +02:00
christianEQ
5668a5d02d fixed uninitialized variables in ACLSetUserCommandBit
Former-commit-id: 7319953ae389817679ce74c44b8baa631a4eafab
2021-01-26 23:29:15 +00:00
christianEQ
295106c2b2 fixed uninitialized variables in ACLSetUserCommandBit
Former-commit-id: 7319953ae389817679ce74c44b8baa631a4eafab
2021-01-26 23:29:15 +00:00
christianEQ
b840a1a3eb fixed strncpy off-by-one errors
Former-commit-id: b8e106a72e71bf5d8abd7d6032ff950f9722db59
2021-01-26 23:27:28 +00:00
christianEQ
13dfc4f228 fixed strncpy off-by-one errors
Former-commit-id: b8e106a72e71bf5d8abd7d6032ff950f9722db59
2021-01-26 23:27:28 +00:00
christianEQ
59ffc79765 removed unimplemented killIOThreads function declaration
Former-commit-id: f30f8eba2f0a5cb247478c2104a4f748659a6cf2
2021-01-26 22:27:22 +00:00
christianEQ
8f132802d8 removed unimplemented killIOThreads function declaration
Former-commit-id: f30f8eba2f0a5cb247478c2104a4f748659a6cf2
2021-01-26 22:27:22 +00:00
christianEQ
8d7142488b fixed server.cpp mentions
Former-commit-id: 7ed2296fc5d10a1c41020558b4bc445953531bdd
2021-01-26 22:22:19 +00:00
christianEQ
396f8316e8 fixed server.cpp mentions
Former-commit-id: 7ed2296fc5d10a1c41020558b4bc445953531bdd
2021-01-26 22:22:19 +00:00
christianEQ
237f3e8524 fixed race condition in connSocketWrite
Former-commit-id: d7f01c1f7d1bb2d6e0b004f532f7b98ed968d389
2021-01-26 22:08:07 +00:00
christianEQ
be2991db56 fixed race condition in connSocketWrite
Former-commit-id: d7f01c1f7d1bb2d6e0b004f532f7b98ed968d389
2021-01-26 22:08:07 +00:00
christianEQ
c068f2cd3d Merge tag 'tags/6.0.10' into redismerge_2021-01-20
Former-commit-id: dadce055f897cee83946c2d3e5cbb76341b94230
2021-01-26 21:43:09 +00:00
christianEQ
358debebfa Merge tag 'tags/6.0.10' into redismerge_2021-01-20
Former-commit-id: dadce055f897cee83946c2d3e5cbb76341b94230
2021-01-26 21:43:09 +00:00
Oran Agra
58cb3ab7b3 Add tests for RESP3 responce of ZINTER and ZRANGE (#8391)
It was confusing as to why these don't return a map type.
the reason is that order matters, so we need to make sure the client
library knows to respect it.
Added comments in the implementation and tests to cover it.
2021-01-26 17:55:32 +02:00
Oran Agra
9e56d3969a
Add tests for RESP3 responce of ZINTER and ZRANGE (#8391)
It was confusing as to why these don't return a map type.
the reason is that order matters, so we need to make sure the client
library knows to respect it.
Added comments in the implementation and tests to cover it.
2021-01-26 17:55:32 +02:00
Wen Hui
ec447de18e Sentinel: Fix Config Dependency and Rewrite Sequence (#8271)
This commit fixes a well known and an annoying issue in Sentinel mode.

Cause of this issue:
Currently, Redis rewrite process works well in server mode, however in sentinel mode,
the sentinel config has variant semantics for different configurations, in example configuration
https://github.com/redis/redis/blob/unstable/sentinel.conf, we put comments on these.
However the rewrite process only treat the sentinel config as a single option. During rewrite
process, it will mess up with the lines and comments.

Approaches:
In order to solve this issue, we need to differentiate different subconfig options in sentinel separately,
for example, sentinel monitor <master-name> <ip> <redis-port> <quorum>
we can treat it as sentinel monitor option, instead of the sentinel option.

This commit also fixes the dependency issue when putting configurations in sentinel.conf.
For example before this commit,we must put
`sentinel monitor <master-name> <ip> <redis-port> <quorum>` before
`sentinel auth-pass <master-name> <password>` for a single master,
otherwise the server cannot start and will return error. This commit fixes this issue, as long as
the monitoring master was configured, no matter the sequence is, the sentinel can start and run properly.
2021-01-26 09:31:54 +02:00
Wen Hui
1aad55b66f
Sentinel: Fix Config Dependency and Rewrite Sequence (#8271)
This commit fixes a well known and an annoying issue in Sentinel mode.

Cause of this issue:
Currently, Redis rewrite process works well in server mode, however in sentinel mode,
the sentinel config has variant semantics for different configurations, in example configuration
https://github.com/redis/redis/blob/unstable/sentinel.conf, we put comments on these.
However the rewrite process only treat the sentinel config as a single option. During rewrite
process, it will mess up with the lines and comments.

Approaches:
In order to solve this issue, we need to differentiate different subconfig options in sentinel separately,
for example, sentinel monitor <master-name> <ip> <redis-port> <quorum>
we can treat it as sentinel monitor option, instead of the sentinel option.

This commit also fixes the dependency issue when putting configurations in sentinel.conf.
For example before this commit,we must put
`sentinel monitor <master-name> <ip> <redis-port> <quorum>` before
`sentinel auth-pass <master-name> <password>` for a single master,
otherwise the server cannot start and will return error. This commit fixes this issue, as long as
the monitoring master was configured, no matter the sequence is, the sentinel can start and run properly.
2021-01-26 09:31:54 +02:00
Vladimir Maksimovski
4574ffe44d Remove check leading to duplicate branches (#8398)
Remove check leading to duplicate branches and unused withscores parameter
2021-01-25 20:58:03 -08:00
Vladimir Maksimovski
70789cf4bb
Remove check leading to duplicate branches (#8398)
Remove check leading to duplicate branches and unused withscores parameter
2021-01-25 20:58:03 -08:00
zhaozhao.zz
a7d6105602 latency: add expire-del event (#8394) 2021-01-26 09:45:53 +08:00
zhaozhao.zz
951d25ff66
latency: add expire-del event (#8394) 2021-01-26 09:45:53 +08:00
Huang Zw
ae52e0c11d In dbOverwrite moduleNotifyKeyUnlink should use old val (#8381)
The module notification was using the wrong value.
2021-01-22 09:37:25 +02:00
Huang Zw
0232778f35
In dbOverwrite moduleNotifyKeyUnlink should use old val (#8381)
The module notification was using the wrong value.
2021-01-22 09:37:25 +02:00
Yossi Gottlieb
5a5b4246df Fix anetCloexec for Sentinel TLS conns. (#8377)
The flag should be set before TLS negotiation begins to avoid a race
condition where a fork+exec before it is completed ends up leaking the
file descriptor.
2021-01-21 19:55:42 +02:00
Yossi Gottlieb
f28fccd2db
Fix anetCloexec for Sentinel TLS conns. (#8377)
The flag should be set before TLS negotiation begins to avoid a race
condition where a fork+exec before it is completed ends up leaking the
file descriptor.
2021-01-21 19:55:42 +02:00
Yossi Gottlieb
34b97f32dc Better error message const correctness. (#8348) 2021-01-21 11:56:08 +02:00
Yossi Gottlieb
3f3cb9d09b
Better error message const correctness. (#8348) 2021-01-21 11:56:08 +02:00
daemyungkang
218197ccf1 change log for invalid cluster port (#8343)
Update log message to correctly indicate cluster port range
2021-01-20 22:08:05 -08:00
daemyungkang
53cdf8a3d1
change log for invalid cluster port (#8343)
Update log message to correctly indicate cluster port range
2021-01-20 22:08:05 -08:00
Yossi Gottlieb
3c0fd785c4 CONFIG REWRITE should honor umask settings. (#8371)
Fixes a regression introduced due to a new (safer) way of rewriting configuration files. In the past the file was simply overwritten (same inode), but now Redis creates a new temporary file and later renames it over the old one.

The temp file typically gets created with 0600 permissions so we later fchmod it to fix that. Unlike open with O_CREAT, fchmod doesn't consider umask so we have to do that explicitly.

Fixes #8369
2021-01-20 21:57:24 +02:00
Yossi Gottlieb
b548ffabbe
CONFIG REWRITE should honor umask settings. (#8371)
Fixes a regression introduced due to a new (safer) way of rewriting configuration files. In the past the file was simply overwritten (same inode), but now Redis creates a new temporary file and later renames it over the old one.

The temp file typically gets created with 0600 permissions so we later fchmod it to fix that. Unlike open with O_CREAT, fchmod doesn't consider umask so we have to do that explicitly.

Fixes #8369
2021-01-20 21:57:24 +02:00
Guy Korland
4ac682da27 Add CI for FreeBSD (#8292)
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-20 14:07:09 +02:00
Guy Korland
ac5f21d613
Add CI for FreeBSD (#8292)
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-20 14:07:09 +02:00
guybe7
a4f54043a9 Fix firstkey,lastkey,step in COMMAND command for some commands (#8367)
The output for COMMAND command was wrong for some commands.
clients can use firstkey,lastkey,step to find (some) key name arguments, and the
"movablekeys" flag to know that they can't know all (or any) of the key name arguments.

These commands had the wrong output:
1. GEORADIUS*_RO used to have "movablekeys" (which it doesn't really need)
2. XREAD and XREADGROUP used to have (1,1,1). but that's completely wrong.
3. Z*STORE used to have (0,0,0) but it can at lest give the index of the dstkey (1,1,1)
2021-01-20 13:56:45 +02:00
guybe7
baf92f3f1c
Fix firstkey,lastkey,step in COMMAND command for some commands (#8367)
The output for COMMAND command was wrong for some commands.
clients can use firstkey,lastkey,step to find (some) key name arguments, and the
"movablekeys" flag to know that they can't know all (or any) of the key name arguments.

These commands had the wrong output:
1. GEORADIUS*_RO used to have "movablekeys" (which it doesn't really need)
2. XREAD and XREADGROUP used to have (1,1,1). but that's completely wrong.
3. Z*STORE used to have (0,0,0) but it can at lest give the index of the dstkey (1,1,1)
2021-01-20 13:56:45 +02:00
Viktor Söderqvist
82707d6828 More modules API ref formatting fixes (#8344)
Fix broken formatting in `RM_Call` and `RM_CreateDataType`,
`RM_SubscribeToServerEvent` (nested lists, etc. in list items).

Unhide docs of `RM_LoadDataTypeFromString` and
`RM_SaveDataTypeToString` by removing blank line between docs and
function.

Clarification added to `RM__Assert`: Recommentation to use the
`RedisModule_Assert` macro instead.

All names containing underscores (variable and macro names) are
wrapped in backticks (if not already wrapped in backticks). This
prevents underscore from being interpreted as italics in some
cases.

Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in
backticks (and RM replaced by RedisModule in this case). This
prevents the * from being interpreted as an italics marker.

A list item with a sublist, a paragraph and another sublist is a
combination which seems impossible to achieve with RedCarped
markdown, so the one occurrence of this is rewritten.

Various trivial changes (typos, backticks, etc.).

Ruby script:

* Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct
  when refering to the C code but RedisModule is correct in the
  API docs.)
* Automatic backquotes around C functions like `malloc()`.
* Turn URLs into links. The link text is the URL itself.
* Don't add backticks inside bold (**...**)
2021-01-20 11:47:06 +02:00
Viktor Söderqvist
16258f21d1
More modules API ref formatting fixes (#8344)
Fix broken formatting in `RM_Call` and `RM_CreateDataType`,
`RM_SubscribeToServerEvent` (nested lists, etc. in list items).

Unhide docs of `RM_LoadDataTypeFromString` and
`RM_SaveDataTypeToString` by removing blank line between docs and
function.

Clarification added to `RM__Assert`: Recommentation to use the
`RedisModule_Assert` macro instead.

All names containing underscores (variable and macro names) are
wrapped in backticks (if not already wrapped in backticks). This
prevents underscore from being interpreted as italics in some
cases.

Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in
backticks (and RM replaced by RedisModule in this case). This
prevents the * from being interpreted as an italics marker.

A list item with a sublist, a paragraph and another sublist is a
combination which seems impossible to achieve with RedCarped
markdown, so the one occurrence of this is rewritten.

Various trivial changes (typos, backticks, etc.).

Ruby script:

* Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct
  when refering to the C code but RedisModule is correct in the
  API docs.)
* Automatic backquotes around C functions like `malloc()`.
* Turn URLs into links. The link text is the URL itself.
* Don't add backticks inside bold (**...**)
2021-01-20 11:47:06 +02:00
Andy Pan
3e671c62d5 Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
Andy Pan
fb66e2e249
Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
filipe oliveira
e8eecf7e90 Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
filipe oliveira
d42ea9e88a
Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
Viktor Söderqvist
c0f85db2b5 Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Viktor Söderqvist
4985c11bd6
Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Andy Pan
6fc0b5d2b6 Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Andy Pan
5198b513d3
Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Raghav Muddur
c5b13b7815 Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Raghav Muddur
366a16ff05
Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Wen Hui
75dbf9d141 [Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00