27350 Commits

Author SHA1 Message Date
Andy Pan
6401920d70
Fix sentinel FD leak test, checking the wrong OS name (#8364) 2021-01-20 10:17:20 +02:00
Andy Pan
4cbfe0f1ab Fix sentinel FD leak test, not printing the list of leaks (#8363) 2021-01-20 09:58:02 +02:00
Andy Pan
1be29606c5
Fix sentinel FD leak test, not printing the list of leaks (#8363) 2021-01-20 09:58:02 +02:00
Andy Pan
3e671c62d5 Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
Andy Pan
fb66e2e249
Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
Itamar Haber
ad24a3e39f Fix description of the save conf directive (#8337)
The line that said:

Note: you can disable saving completely by commenting out all "save" lines

was incorrect.

Also, note that the save values used in the default redis.conf file are different than the server's defaults, so this PR also addresses this and introduces a potential behavior change for some users.
2021-01-19 20:31:30 +02:00
Itamar Haber
aaf71b380e
Fix description of the save conf directive (#8337)
The line that said:

Note: you can disable saving completely by commenting out all "save" lines

was incorrect.

Also, note that the save values used in the default redis.conf file are different than the server's defaults, so this PR also addresses this and introduces a potential behavior change for some users.
2021-01-19 20:31:30 +02:00
Oran Agra
ce079b898b Add tests to make sure that relative EXPIRE is propagated to replicas (#8357)
This commit adds tests to make sure that relative and absolute expire commands
are propagated as is to replicas and stop any future attempt to change that without
a proper discussion. see #8327 and #5171

Additionally it slightly improve the AOF test that tests the opposite (always
propagating absolute times), by covering more commands, and shaving 2
seconds from the test time.
2021-01-19 18:49:26 +02:00
Oran Agra
a29aec9abb
Add tests to make sure that relative EXPIRE is propagated to replicas (#8357)
This commit adds tests to make sure that relative and absolute expire commands
are propagated as is to replicas and stop any future attempt to change that without
a proper discussion. see #8327 and #5171

Additionally it slightly improve the AOF test that tests the opposite (always
propagating absolute times), by covering more commands, and shaving 2
seconds from the test time.
2021-01-19 18:49:26 +02:00
filipe oliveira
e8eecf7e90 Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
filipe oliveira
d42ea9e88a
Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
Viktor Söderqvist
c0f85db2b5 Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Viktor Söderqvist
4985c11bd6
Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Andy Pan
6fc0b5d2b6 Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Andy Pan
5198b513d3
Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Raghav Muddur
c5b13b7815 Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Raghav Muddur
366a16ff05
Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Yossi Gottlieb
e4d0ba933d Add io-thread daily CI tests. (#8232)
This adds basic coverage to IO threads by running the cluster and few selected Redis test suite tests with the IO threads enabled.

Also provides some necessary additional improvements to the test suite:

* Add --config to sentinel/cluster tests for arbitrary configuration.
* Fix --tags whitelisting which was broken.
* Add a `network` tag to some tests that are more network intensive. This is work in progress and more tests should be properly tagged in the future.
2021-01-17 15:48:48 +02:00
Yossi Gottlieb
522d93607a
Add io-thread daily CI tests. (#8232)
This adds basic coverage to IO threads by running the cluster and few selected Redis test suite tests with the IO threads enabled.

Also provides some necessary additional improvements to the test suite:

* Add --config to sentinel/cluster tests for arbitrary configuration.
* Fix --tags whitelisting which was broken.
* Add a `network` tag to some tests that are more network intensive. This is work in progress and more tests should be properly tagged in the future.
2021-01-17 15:48:48 +02:00
Wen Hui
75dbf9d141 [Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00
Wen Hui
f5cf1e46a4
[Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00
Yang Bodong
8df851176b Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
2021-01-15 15:32:58 +02:00
Yang Bodong
294f93af97
Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
2021-01-15 15:32:58 +02:00
Viktor Söderqvist
c694535f5f Rename non-API RM-prefixed functions to hide them from API docs
The prefix is changed from `RM_` to `module` on the following
internal functions, to prevent them from appearing in the API docs:

    RM_LogRaw -> moduleLogRaw
    RM_FreeCallReplyRec -> moduleFreeCallReplyRec
    RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
    RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
fcb3dfe56d Rename non-API RM-prefixed functions to hide them from API docs
The prefix is changed from `RM_` to `module` on the following
internal functions, to prevent them from appearing in the API docs:

    RM_LogRaw -> moduleLogRaw
    RM_FreeCallReplyRec -> moduleFreeCallReplyRec
    RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
    RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
3ec23f0b6e Modules API reference formatting fixes
Fixes markdown formatting errors and some functions not showing
up in the generated documentation at all.

Ruby script (gendoc.rb) fixes:

* Modified automatic instertion of backquotes:
  * Don't add backquotes around names which are already preceded by a
    backquote. Fixes for example \`RedisModule_Reply\*\` which turning
    into \`\`RedisModule_Reply\`\*\` messes up the formatting.
  * Add backquotes around types such as RedisModuleString (in addition
    to function names `RedisModule_[A-z()]*` and macro names
    `REDISMODULE_[A-z]*`).
  * Require 4 spaces indentation for disabling automatic backquotes, i.e.
    code blocks. Fixes continuations of list items (indented 2 spaces).
* More permissive extraction of doc comments:
  * Allow doc comments starting with `/**`.
  * Make space before `*` on each line optional.
  * Make space after `/*` and `/**` optional (needed when appearing on
    its own line).

Markdown fixes in module.c:

* Fix code blocks not indented enough (4 spaces needed).
* Add black line before code blocks and lists where missing (needed).
* Enclose special markdown characters `_*^<>` in backticks to prevent them
  from messing up formatting.
* Lists with `1)` changed to `1.` for proper markdown lists.
* Remove excessive indentation which causes text to be unintentionally
  rendered as code blocks.
* Other minor formatting fixes.

Other fixes in module.c:

* Remove blank lines between doc comment and function definition. A blank
  line here makes the Ruby script exclude the function in docs.
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
ebf20b83b2 Modules API reference formatting fixes
Fixes markdown formatting errors and some functions not showing
up in the generated documentation at all.

Ruby script (gendoc.rb) fixes:

* Modified automatic instertion of backquotes:
  * Don't add backquotes around names which are already preceded by a
    backquote. Fixes for example \`RedisModule_Reply\*\` which turning
    into \`\`RedisModule_Reply\`\*\` messes up the formatting.
  * Add backquotes around types such as RedisModuleString (in addition
    to function names `RedisModule_[A-z()]*` and macro names
    `REDISMODULE_[A-z]*`).
  * Require 4 spaces indentation for disabling automatic backquotes, i.e.
    code blocks. Fixes continuations of list items (indented 2 spaces).
* More permissive extraction of doc comments:
  * Allow doc comments starting with `/**`.
  * Make space before `*` on each line optional.
  * Make space after `/*` and `/**` optional (needed when appearing on
    its own line).

Markdown fixes in module.c:

* Fix code blocks not indented enough (4 spaces needed).
* Add black line before code blocks and lists where missing (needed).
* Enclose special markdown characters `_*^<>` in backticks to prevent them
  from messing up formatting.
* Lists with `1)` changed to `1.` for proper markdown lists.
* Remove excessive indentation which causes text to be unintentionally
  rendered as code blocks.
* Other minor formatting fixes.

Other fixes in module.c:

* Remove blank lines between doc comment and function definition. A blank
  line here makes the Ruby script exclude the function in docs.
2021-01-15 13:33:56 +02:00
christianEQ
ab80e23152 changed loading process config types to be consistent across environments
Former-commit-id: d2867580aba70bbfa703578cdb40093a4e1a7147
2021-01-14 21:12:36 -05:00
christianEQ
247c916f69 changed loading process config types to be consistent across environments
Former-commit-id: d2867580aba70bbfa703578cdb40093a4e1a7147
2021-01-14 21:12:36 -05:00
christianEQ
464a08226d renamed loaded key counter to be more clear
Former-commit-id: 8dc238efae8f6ba6b4ddc9b169e7195bcb43920b
2021-01-14 13:06:57 -05:00
christianEQ
588a37583e renamed loaded key counter to be more clear
Former-commit-id: 8dc238efae8f6ba6b4ddc9b169e7195bcb43920b
2021-01-14 13:06:57 -05:00
christianEQ
81f4f73c30 removed 'comment arguments' from config options (doesn't ignore inline comments)
Former-commit-id: 7cd4b5e11c9c92e88cb551f7d83efc668aa259c7
2021-01-14 13:06:57 -05:00
christianEQ
b113977e94 removed 'comment arguments' from config options (doesn't ignore inline comments)
Former-commit-id: 7cd4b5e11c9c92e88cb551f7d83efc668aa259c7
2021-01-14 13:06:57 -05:00
christianEQ
047e1174d9 fixed signedness warning
Former-commit-id: 00043f95099803f6b06a208ac3b19e2cef9abf4d
2021-01-14 13:06:57 -05:00
christianEQ
6f97805577 fixed signedness warning
Former-commit-id: 00043f95099803f6b06a208ac3b19e2cef9abf4d
2021-01-14 13:06:57 -05:00
christianEQ
9262739be2 added section to keydb.conf for new options, tweaked default
Former-commit-id: 9b81f3f263de8cffab20ef4d20c2ac4935f63a10
2021-01-14 13:06:57 -05:00
christianEQ
e68cee23a1 added section to keydb.conf for new options, tweaked default
Former-commit-id: 9b81f3f263de8cffab20ef4d20c2ac4935f63a10
2021-01-14 13:06:57 -05:00
christianEQ
327c44f8eb ping replicas during key load
Former-commit-id: 2747d1a1a99d3415d5486a472fe0e39135b79bc5
2021-01-14 13:06:57 -05:00
christianEQ
d4ad045535 ping replicas during key load
Former-commit-id: 2747d1a1a99d3415d5486a472fe0e39135b79bc5
2021-01-14 13:06:57 -05:00
christianEQ
28a0cdc572 added config option for loading callback interval keys
Former-commit-id: 9f7ed0b2327924563668105ead09f52833902d0b
2021-01-14 13:06:57 -05:00
christianEQ
81536a31c8 added config option for loading callback interval keys
Former-commit-id: 9f7ed0b2327924563668105ead09f52833902d0b
2021-01-14 13:06:57 -05:00
christianEQ
fc3388acfe configurized loading_process_events_interval_bytes
Former-commit-id: c176ea73664a182af9caacda9d489fb2b344b2de
2021-01-14 13:06:57 -05:00
christianEQ
4fa25cbc57 configurized loading_process_events_interval_bytes
Former-commit-id: c176ea73664a182af9caacda9d489fb2b344b2de
2021-01-14 13:06:57 -05:00
charsyam
b2526c5040 Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress (#8319)
Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress
2021-01-13 16:44:04 -08:00
charsyam
108afbc0bb
Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress (#8319)
Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress
2021-01-13 16:44:04 -08:00
Wang Yuan
b3477eddf9 Optimize performance of clusterGenNodesDescription for large clusters (#8182)
Optimize the performance of clusterGenNodesDescription by only checking slot ownership of each slot once, instead of checking each slot for each node.
2021-01-13 12:36:03 -08:00
Wang Yuan
9cb9f98d2f
Optimize performance of clusterGenNodesDescription for large clusters (#8182)
Optimize the performance of clusterGenNodesDescription by only checking slot ownership of each slot once, instead of checking each slot for each node.
2021-01-13 12:36:03 -08:00
houzj.fnst
8da27781f4 remove some unnecessary checks (#7431)
Remove several checks that always evaluate to true.
2021-01-13 12:00:55 -08:00
houzj.fnst
f5577fdbd8
remove some unnecessary checks (#7431)
Remove several checks that always evaluate to true.
2021-01-13 12:00:55 -08:00
sundb
60423a3e5a Fix use of lookupKeyRead and lookupKeyWrite in zrangeGenericCommand, zunionInterDiffGenericCommand (#8316)
* Change zunionInterDiffGenericCommand to use lookupKeyRead if dstkey is null
* Change zrangeGenericCommand to use lookupKey Write if dstkey isn't null

ZRANGESTORE and UNION, ZINTER, ZDIFF are all new commands (6.2 RC1 and RC2).
In redis 6.0 the ZRANGE was using lookupKeyRead, and ZUNIONSTORE / ZINTERSTORE were using lookupKeyWrite.
So there bugs are introduced in 6.2 and will be resolved before it is released.

the implications of this bug are also not big:
The sole difference between LookupKeyRead and LookupKeyWrite is for command executed on a replica, which are not received from its master client. (for the master, and for the master client on the replica, these two functions behave the same)!
2021-01-13 11:58:12 +02:00