27610 Commits

Author SHA1 Message Date
Yossi Gottlieb
b548ffabbe
CONFIG REWRITE should honor umask settings. (#8371)
Fixes a regression introduced due to a new (safer) way of rewriting configuration files. In the past the file was simply overwritten (same inode), but now Redis creates a new temporary file and later renames it over the old one.

The temp file typically gets created with 0600 permissions so we later fchmod it to fix that. Unlike open with O_CREAT, fchmod doesn't consider umask so we have to do that explicitly.

Fixes #8369
2021-01-20 21:57:24 +02:00
Guy Korland
4ac682da27 Add CI for FreeBSD (#8292)
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-20 14:07:09 +02:00
Guy Korland
ac5f21d613
Add CI for FreeBSD (#8292)
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-20 14:07:09 +02:00
guybe7
c4ecac1160 Fix misleading module test (#8366)
the test was misleading because the module would actually woke up on a wrong type and
re-blocked, while the test name suggests the module doesn't not wake up at all on a wrong type..

i changed the name of the test + added verification that indeed the module wakes up and gets
re-blocked after it understand it's the wrong type
2021-01-20 14:03:38 +02:00
guybe7
5a77d015be
Fix misleading module test (#8366)
the test was misleading because the module would actually woke up on a wrong type and
re-blocked, while the test name suggests the module doesn't not wake up at all on a wrong type..

i changed the name of the test + added verification that indeed the module wakes up and gets
re-blocked after it understand it's the wrong type
2021-01-20 14:03:38 +02:00
guybe7
a4f54043a9 Fix firstkey,lastkey,step in COMMAND command for some commands (#8367)
The output for COMMAND command was wrong for some commands.
clients can use firstkey,lastkey,step to find (some) key name arguments, and the
"movablekeys" flag to know that they can't know all (or any) of the key name arguments.

These commands had the wrong output:
1. GEORADIUS*_RO used to have "movablekeys" (which it doesn't really need)
2. XREAD and XREADGROUP used to have (1,1,1). but that's completely wrong.
3. Z*STORE used to have (0,0,0) but it can at lest give the index of the dstkey (1,1,1)
2021-01-20 13:56:45 +02:00
guybe7
baf92f3f1c
Fix firstkey,lastkey,step in COMMAND command for some commands (#8367)
The output for COMMAND command was wrong for some commands.
clients can use firstkey,lastkey,step to find (some) key name arguments, and the
"movablekeys" flag to know that they can't know all (or any) of the key name arguments.

These commands had the wrong output:
1. GEORADIUS*_RO used to have "movablekeys" (which it doesn't really need)
2. XREAD and XREADGROUP used to have (1,1,1). but that's completely wrong.
3. Z*STORE used to have (0,0,0) but it can at lest give the index of the dstkey (1,1,1)
2021-01-20 13:56:45 +02:00
Viktor Söderqvist
82707d6828 More modules API ref formatting fixes (#8344)
Fix broken formatting in `RM_Call` and `RM_CreateDataType`,
`RM_SubscribeToServerEvent` (nested lists, etc. in list items).

Unhide docs of `RM_LoadDataTypeFromString` and
`RM_SaveDataTypeToString` by removing blank line between docs and
function.

Clarification added to `RM__Assert`: Recommentation to use the
`RedisModule_Assert` macro instead.

All names containing underscores (variable and macro names) are
wrapped in backticks (if not already wrapped in backticks). This
prevents underscore from being interpreted as italics in some
cases.

Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in
backticks (and RM replaced by RedisModule in this case). This
prevents the * from being interpreted as an italics marker.

A list item with a sublist, a paragraph and another sublist is a
combination which seems impossible to achieve with RedCarped
markdown, so the one occurrence of this is rewritten.

Various trivial changes (typos, backticks, etc.).

Ruby script:

* Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct
  when refering to the C code but RedisModule is correct in the
  API docs.)
* Automatic backquotes around C functions like `malloc()`.
* Turn URLs into links. The link text is the URL itself.
* Don't add backticks inside bold (**...**)
2021-01-20 11:47:06 +02:00
Viktor Söderqvist
16258f21d1
More modules API ref formatting fixes (#8344)
Fix broken formatting in `RM_Call` and `RM_CreateDataType`,
`RM_SubscribeToServerEvent` (nested lists, etc. in list items).

Unhide docs of `RM_LoadDataTypeFromString` and
`RM_SaveDataTypeToString` by removing blank line between docs and
function.

Clarification added to `RM__Assert`: Recommentation to use the
`RedisModule_Assert` macro instead.

All names containing underscores (variable and macro names) are
wrapped in backticks (if not already wrapped in backticks). This
prevents underscore from being interpreted as italics in some
cases.

Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in
backticks (and RM replaced by RedisModule in this case). This
prevents the * from being interpreted as an italics marker.

A list item with a sublist, a paragraph and another sublist is a
combination which seems impossible to achieve with RedCarped
markdown, so the one occurrence of this is rewritten.

Various trivial changes (typos, backticks, etc.).

Ruby script:

* Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct
  when refering to the C code but RedisModule is correct in the
  API docs.)
* Automatic backquotes around C functions like `malloc()`.
* Turn URLs into links. The link text is the URL itself.
* Don't add backticks inside bold (**...**)
2021-01-20 11:47:06 +02:00
Andy Pan
2824100803 Fix sentinel FD leak test, checking the wrong OS name (#8364) 2021-01-20 10:17:20 +02:00
Andy Pan
6401920d70
Fix sentinel FD leak test, checking the wrong OS name (#8364) 2021-01-20 10:17:20 +02:00
Andy Pan
4cbfe0f1ab Fix sentinel FD leak test, not printing the list of leaks (#8363) 2021-01-20 09:58:02 +02:00
Andy Pan
1be29606c5
Fix sentinel FD leak test, not printing the list of leaks (#8363) 2021-01-20 09:58:02 +02:00
Andy Pan
3e671c62d5 Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
Andy Pan
fb66e2e249
Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
Itamar Haber
ad24a3e39f Fix description of the save conf directive (#8337)
The line that said:

Note: you can disable saving completely by commenting out all "save" lines

was incorrect.

Also, note that the save values used in the default redis.conf file are different than the server's defaults, so this PR also addresses this and introduces a potential behavior change for some users.
2021-01-19 20:31:30 +02:00
Itamar Haber
aaf71b380e
Fix description of the save conf directive (#8337)
The line that said:

Note: you can disable saving completely by commenting out all "save" lines

was incorrect.

Also, note that the save values used in the default redis.conf file are different than the server's defaults, so this PR also addresses this and introduces a potential behavior change for some users.
2021-01-19 20:31:30 +02:00
Oran Agra
ce079b898b Add tests to make sure that relative EXPIRE is propagated to replicas (#8357)
This commit adds tests to make sure that relative and absolute expire commands
are propagated as is to replicas and stop any future attempt to change that without
a proper discussion. see #8327 and #5171

Additionally it slightly improve the AOF test that tests the opposite (always
propagating absolute times), by covering more commands, and shaving 2
seconds from the test time.
2021-01-19 18:49:26 +02:00
Oran Agra
a29aec9abb
Add tests to make sure that relative EXPIRE is propagated to replicas (#8357)
This commit adds tests to make sure that relative and absolute expire commands
are propagated as is to replicas and stop any future attempt to change that without
a proper discussion. see #8327 and #5171

Additionally it slightly improve the AOF test that tests the opposite (always
propagating absolute times), by covering more commands, and shaving 2
seconds from the test time.
2021-01-19 18:49:26 +02:00
filipe oliveira
e8eecf7e90 Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
filipe oliveira
d42ea9e88a
Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
Viktor Söderqvist
c0f85db2b5 Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Viktor Söderqvist
4985c11bd6
Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Andy Pan
6fc0b5d2b6 Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Andy Pan
5198b513d3
Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Raghav Muddur
c5b13b7815 Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Raghav Muddur
366a16ff05
Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Yossi Gottlieb
e4d0ba933d Add io-thread daily CI tests. (#8232)
This adds basic coverage to IO threads by running the cluster and few selected Redis test suite tests with the IO threads enabled.

Also provides some necessary additional improvements to the test suite:

* Add --config to sentinel/cluster tests for arbitrary configuration.
* Fix --tags whitelisting which was broken.
* Add a `network` tag to some tests that are more network intensive. This is work in progress and more tests should be properly tagged in the future.
2021-01-17 15:48:48 +02:00
Yossi Gottlieb
522d93607a
Add io-thread daily CI tests. (#8232)
This adds basic coverage to IO threads by running the cluster and few selected Redis test suite tests with the IO threads enabled.

Also provides some necessary additional improvements to the test suite:

* Add --config to sentinel/cluster tests for arbitrary configuration.
* Fix --tags whitelisting which was broken.
* Add a `network` tag to some tests that are more network intensive. This is work in progress and more tests should be properly tagged in the future.
2021-01-17 15:48:48 +02:00
Wen Hui
75dbf9d141 [Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00
Wen Hui
f5cf1e46a4
[Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00
Yang Bodong
8df851176b Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
2021-01-15 15:32:58 +02:00
Yang Bodong
294f93af97
Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
2021-01-15 15:32:58 +02:00
Viktor Söderqvist
c694535f5f Rename non-API RM-prefixed functions to hide them from API docs
The prefix is changed from `RM_` to `module` on the following
internal functions, to prevent them from appearing in the API docs:

    RM_LogRaw -> moduleLogRaw
    RM_FreeCallReplyRec -> moduleFreeCallReplyRec
    RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
    RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
fcb3dfe56d Rename non-API RM-prefixed functions to hide them from API docs
The prefix is changed from `RM_` to `module` on the following
internal functions, to prevent them from appearing in the API docs:

    RM_LogRaw -> moduleLogRaw
    RM_FreeCallReplyRec -> moduleFreeCallReplyRec
    RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
    RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
3ec23f0b6e Modules API reference formatting fixes
Fixes markdown formatting errors and some functions not showing
up in the generated documentation at all.

Ruby script (gendoc.rb) fixes:

* Modified automatic instertion of backquotes:
  * Don't add backquotes around names which are already preceded by a
    backquote. Fixes for example \`RedisModule_Reply\*\` which turning
    into \`\`RedisModule_Reply\`\*\` messes up the formatting.
  * Add backquotes around types such as RedisModuleString (in addition
    to function names `RedisModule_[A-z()]*` and macro names
    `REDISMODULE_[A-z]*`).
  * Require 4 spaces indentation for disabling automatic backquotes, i.e.
    code blocks. Fixes continuations of list items (indented 2 spaces).
* More permissive extraction of doc comments:
  * Allow doc comments starting with `/**`.
  * Make space before `*` on each line optional.
  * Make space after `/*` and `/**` optional (needed when appearing on
    its own line).

Markdown fixes in module.c:

* Fix code blocks not indented enough (4 spaces needed).
* Add black line before code blocks and lists where missing (needed).
* Enclose special markdown characters `_*^<>` in backticks to prevent them
  from messing up formatting.
* Lists with `1)` changed to `1.` for proper markdown lists.
* Remove excessive indentation which causes text to be unintentionally
  rendered as code blocks.
* Other minor formatting fixes.

Other fixes in module.c:

* Remove blank lines between doc comment and function definition. A blank
  line here makes the Ruby script exclude the function in docs.
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
ebf20b83b2 Modules API reference formatting fixes
Fixes markdown formatting errors and some functions not showing
up in the generated documentation at all.

Ruby script (gendoc.rb) fixes:

* Modified automatic instertion of backquotes:
  * Don't add backquotes around names which are already preceded by a
    backquote. Fixes for example \`RedisModule_Reply\*\` which turning
    into \`\`RedisModule_Reply\`\*\` messes up the formatting.
  * Add backquotes around types such as RedisModuleString (in addition
    to function names `RedisModule_[A-z()]*` and macro names
    `REDISMODULE_[A-z]*`).
  * Require 4 spaces indentation for disabling automatic backquotes, i.e.
    code blocks. Fixes continuations of list items (indented 2 spaces).
* More permissive extraction of doc comments:
  * Allow doc comments starting with `/**`.
  * Make space before `*` on each line optional.
  * Make space after `/*` and `/**` optional (needed when appearing on
    its own line).

Markdown fixes in module.c:

* Fix code blocks not indented enough (4 spaces needed).
* Add black line before code blocks and lists where missing (needed).
* Enclose special markdown characters `_*^<>` in backticks to prevent them
  from messing up formatting.
* Lists with `1)` changed to `1.` for proper markdown lists.
* Remove excessive indentation which causes text to be unintentionally
  rendered as code blocks.
* Other minor formatting fixes.

Other fixes in module.c:

* Remove blank lines between doc comment and function definition. A blank
  line here makes the Ruby script exclude the function in docs.
2021-01-15 13:33:56 +02:00
christianEQ
ab80e23152 changed loading process config types to be consistent across environments
Former-commit-id: d2867580aba70bbfa703578cdb40093a4e1a7147
2021-01-14 21:12:36 -05:00
christianEQ
247c916f69 changed loading process config types to be consistent across environments
Former-commit-id: d2867580aba70bbfa703578cdb40093a4e1a7147
2021-01-14 21:12:36 -05:00
christianEQ
464a08226d renamed loaded key counter to be more clear
Former-commit-id: 8dc238efae8f6ba6b4ddc9b169e7195bcb43920b
2021-01-14 13:06:57 -05:00
christianEQ
588a37583e renamed loaded key counter to be more clear
Former-commit-id: 8dc238efae8f6ba6b4ddc9b169e7195bcb43920b
2021-01-14 13:06:57 -05:00
christianEQ
81f4f73c30 removed 'comment arguments' from config options (doesn't ignore inline comments)
Former-commit-id: 7cd4b5e11c9c92e88cb551f7d83efc668aa259c7
2021-01-14 13:06:57 -05:00
christianEQ
b113977e94 removed 'comment arguments' from config options (doesn't ignore inline comments)
Former-commit-id: 7cd4b5e11c9c92e88cb551f7d83efc668aa259c7
2021-01-14 13:06:57 -05:00
christianEQ
047e1174d9 fixed signedness warning
Former-commit-id: 00043f95099803f6b06a208ac3b19e2cef9abf4d
2021-01-14 13:06:57 -05:00
christianEQ
6f97805577 fixed signedness warning
Former-commit-id: 00043f95099803f6b06a208ac3b19e2cef9abf4d
2021-01-14 13:06:57 -05:00
christianEQ
9262739be2 added section to keydb.conf for new options, tweaked default
Former-commit-id: 9b81f3f263de8cffab20ef4d20c2ac4935f63a10
2021-01-14 13:06:57 -05:00
christianEQ
e68cee23a1 added section to keydb.conf for new options, tweaked default
Former-commit-id: 9b81f3f263de8cffab20ef4d20c2ac4935f63a10
2021-01-14 13:06:57 -05:00
christianEQ
327c44f8eb ping replicas during key load
Former-commit-id: 2747d1a1a99d3415d5486a472fe0e39135b79bc5
2021-01-14 13:06:57 -05:00
christianEQ
d4ad045535 ping replicas during key load
Former-commit-id: 2747d1a1a99d3415d5486a472fe0e39135b79bc5
2021-01-14 13:06:57 -05:00
christianEQ
28a0cdc572 added config option for loading callback interval keys
Former-commit-id: 9f7ed0b2327924563668105ead09f52833902d0b
2021-01-14 13:06:57 -05:00