20927 Commits

Author SHA1 Message Date
Viktor Söderqvist
16258f21d1
More modules API ref formatting fixes (#8344)
Fix broken formatting in `RM_Call` and `RM_CreateDataType`,
`RM_SubscribeToServerEvent` (nested lists, etc. in list items).

Unhide docs of `RM_LoadDataTypeFromString` and
`RM_SaveDataTypeToString` by removing blank line between docs and
function.

Clarification added to `RM__Assert`: Recommentation to use the
`RedisModule_Assert` macro instead.

All names containing underscores (variable and macro names) are
wrapped in backticks (if not already wrapped in backticks). This
prevents underscore from being interpreted as italics in some
cases.

Names including a wildcard star, e.g. RM_Defrag*(), is wrapped in
backticks (and RM replaced by RedisModule in this case). This
prevents the * from being interpreted as an italics marker.

A list item with a sublist, a paragraph and another sublist is a
combination which seems impossible to achieve with RedCarped
markdown, so the one occurrence of this is rewritten.

Various trivial changes (typos, backticks, etc.).

Ruby script:

* Replace `RM_Xyz` with `RedisModule_Xyz` in docs. (RM is correct
  when refering to the C code but RedisModule is correct in the
  API docs.)
* Automatic backquotes around C functions like `malloc()`.
* Turn URLs into links. The link text is the URL itself.
* Don't add backticks inside bold (**...**)
2021-01-20 11:47:06 +02:00
Andy Pan
3e671c62d5 Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
Andy Pan
fb66e2e249
Use FD_CLOEXEC in Sentinel, so that FDs don't leak to the scripts it runs (#8242)
Sentinel uses execve to run scripts, so it needs to use FD_CLOEXEC
on all file descriptors, so that they're not accessible by the script it runs.

This commit includes a change to the sentinel tests, which verifies no
FDs are left opened when the script is executed.
2021-01-19 22:57:30 +02:00
filipe oliveira
e8eecf7e90 Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
filipe oliveira
d42ea9e88a
Removing unnecessary runtime tio_debug checks (#8250)
These statements were dead code.
2021-01-19 17:24:49 +02:00
Viktor Söderqvist
c0f85db2b5 Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Viktor Söderqvist
4985c11bd6
Bugfix: Make modules blocked on keys unblock on commands like LPUSH (#8356)
This was a regression from #7625 (only in 6.2 RC2).

This makes it possible again to implement blocking list and zset
commands using the modules API.

This commit also includes a test case for the reverse: A module
unblocks a client blocked on BLPOP by inserting elements using
RedisModule_ListPush(). This already works, but it was untested.
2021-01-19 13:15:33 +02:00
Andy Pan
6fc0b5d2b6 Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Andy Pan
5198b513d3
Add a precheck before the actual call to fcntl (#8360)
Don't bother to call fcntl if the flags are not gonna be changed.
2021-01-19 11:36:21 +02:00
Raghav Muddur
c5b13b7815 Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Raghav Muddur
366a16ff05
Update getTimeZone to long (#8346) 2021-01-18 11:37:05 +02:00
Wen Hui
75dbf9d141 [Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00
Wen Hui
f5cf1e46a4
[Sentinel] more specific config errors for duplicate known-replica and known-sentinel (#8332)
Previously invalid configuration errors were not very specific and in some cases hard to understand.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2021-01-17 15:43:21 +02:00
Yang Bodong
8df851176b Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
2021-01-15 15:32:58 +02:00
Yang Bodong
294f93af97
Add lazyfree-lazy-user-flush config to control default behavior of FLUSH[ALL|DB], SCRIPT FLUSH (#8258)
* Adds ASYNC and SYNC arguments to SCRIPT FLUSH
* Adds SYNC argument to FLUSHDB and FLUSHALL
* Adds new config to control the default behavior of FLUSHDB, FLUSHALL and SCRIPT FLUASH.

the new behavior is as follows:
* FLUSH[ALL|DB],SCRIPT FLUSH: Determine sync or async according to the
  value of lazyfree-lazy-user-flush.
* FLUSH[ALL|DB],SCRIPT FLUSH ASYNC: Always flushes the database in an async manner.
* FLUSH[ALL|DB],SCRIPT FLUSH SYNC: Always flushes the database in a sync manner.
2021-01-15 15:32:58 +02:00
Viktor Söderqvist
c694535f5f Rename non-API RM-prefixed functions to hide them from API docs
The prefix is changed from `RM_` to `module` on the following
internal functions, to prevent them from appearing in the API docs:

    RM_LogRaw -> moduleLogRaw
    RM_FreeCallReplyRec -> moduleFreeCallReplyRec
    RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
    RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
fcb3dfe56d Rename non-API RM-prefixed functions to hide them from API docs
The prefix is changed from `RM_` to `module` on the following
internal functions, to prevent them from appearing in the API docs:

    RM_LogRaw -> moduleLogRaw
    RM_FreeCallReplyRec -> moduleFreeCallReplyRec
    RM_ZsetAddFlagsToCoreFlags -> moduleZsetAddFlagsToCoreFlags
    RM_ZsetAddFlagsFromCoreFlags -> moduleZsetAddFlagsFromCoreFlags
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
3ec23f0b6e Modules API reference formatting fixes
Fixes markdown formatting errors and some functions not showing
up in the generated documentation at all.

Ruby script (gendoc.rb) fixes:

* Modified automatic instertion of backquotes:
  * Don't add backquotes around names which are already preceded by a
    backquote. Fixes for example \`RedisModule_Reply\*\` which turning
    into \`\`RedisModule_Reply\`\*\` messes up the formatting.
  * Add backquotes around types such as RedisModuleString (in addition
    to function names `RedisModule_[A-z()]*` and macro names
    `REDISMODULE_[A-z]*`).
  * Require 4 spaces indentation for disabling automatic backquotes, i.e.
    code blocks. Fixes continuations of list items (indented 2 spaces).
* More permissive extraction of doc comments:
  * Allow doc comments starting with `/**`.
  * Make space before `*` on each line optional.
  * Make space after `/*` and `/**` optional (needed when appearing on
    its own line).

Markdown fixes in module.c:

* Fix code blocks not indented enough (4 spaces needed).
* Add black line before code blocks and lists where missing (needed).
* Enclose special markdown characters `_*^<>` in backticks to prevent them
  from messing up formatting.
* Lists with `1)` changed to `1.` for proper markdown lists.
* Remove excessive indentation which causes text to be unintentionally
  rendered as code blocks.
* Other minor formatting fixes.

Other fixes in module.c:

* Remove blank lines between doc comment and function definition. A blank
  line here makes the Ruby script exclude the function in docs.
2021-01-15 13:33:56 +02:00
Viktor Söderqvist
ebf20b83b2 Modules API reference formatting fixes
Fixes markdown formatting errors and some functions not showing
up in the generated documentation at all.

Ruby script (gendoc.rb) fixes:

* Modified automatic instertion of backquotes:
  * Don't add backquotes around names which are already preceded by a
    backquote. Fixes for example \`RedisModule_Reply\*\` which turning
    into \`\`RedisModule_Reply\`\*\` messes up the formatting.
  * Add backquotes around types such as RedisModuleString (in addition
    to function names `RedisModule_[A-z()]*` and macro names
    `REDISMODULE_[A-z]*`).
  * Require 4 spaces indentation for disabling automatic backquotes, i.e.
    code blocks. Fixes continuations of list items (indented 2 spaces).
* More permissive extraction of doc comments:
  * Allow doc comments starting with `/**`.
  * Make space before `*` on each line optional.
  * Make space after `/*` and `/**` optional (needed when appearing on
    its own line).

Markdown fixes in module.c:

* Fix code blocks not indented enough (4 spaces needed).
* Add black line before code blocks and lists where missing (needed).
* Enclose special markdown characters `_*^<>` in backticks to prevent them
  from messing up formatting.
* Lists with `1)` changed to `1.` for proper markdown lists.
* Remove excessive indentation which causes text to be unintentionally
  rendered as code blocks.
* Other minor formatting fixes.

Other fixes in module.c:

* Remove blank lines between doc comment and function definition. A blank
  line here makes the Ruby script exclude the function in docs.
2021-01-15 13:33:56 +02:00
christianEQ
ab80e23152 changed loading process config types to be consistent across environments
Former-commit-id: d2867580aba70bbfa703578cdb40093a4e1a7147
2021-01-14 21:12:36 -05:00
christianEQ
247c916f69 changed loading process config types to be consistent across environments
Former-commit-id: d2867580aba70bbfa703578cdb40093a4e1a7147
2021-01-14 21:12:36 -05:00
christianEQ
464a08226d renamed loaded key counter to be more clear
Former-commit-id: 8dc238efae8f6ba6b4ddc9b169e7195bcb43920b
2021-01-14 13:06:57 -05:00
christianEQ
588a37583e renamed loaded key counter to be more clear
Former-commit-id: 8dc238efae8f6ba6b4ddc9b169e7195bcb43920b
2021-01-14 13:06:57 -05:00
christianEQ
047e1174d9 fixed signedness warning
Former-commit-id: 00043f95099803f6b06a208ac3b19e2cef9abf4d
2021-01-14 13:06:57 -05:00
christianEQ
6f97805577 fixed signedness warning
Former-commit-id: 00043f95099803f6b06a208ac3b19e2cef9abf4d
2021-01-14 13:06:57 -05:00
christianEQ
9262739be2 added section to keydb.conf for new options, tweaked default
Former-commit-id: 9b81f3f263de8cffab20ef4d20c2ac4935f63a10
2021-01-14 13:06:57 -05:00
christianEQ
e68cee23a1 added section to keydb.conf for new options, tweaked default
Former-commit-id: 9b81f3f263de8cffab20ef4d20c2ac4935f63a10
2021-01-14 13:06:57 -05:00
christianEQ
327c44f8eb ping replicas during key load
Former-commit-id: 2747d1a1a99d3415d5486a472fe0e39135b79bc5
2021-01-14 13:06:57 -05:00
christianEQ
d4ad045535 ping replicas during key load
Former-commit-id: 2747d1a1a99d3415d5486a472fe0e39135b79bc5
2021-01-14 13:06:57 -05:00
christianEQ
28a0cdc572 added config option for loading callback interval keys
Former-commit-id: 9f7ed0b2327924563668105ead09f52833902d0b
2021-01-14 13:06:57 -05:00
christianEQ
81536a31c8 added config option for loading callback interval keys
Former-commit-id: 9f7ed0b2327924563668105ead09f52833902d0b
2021-01-14 13:06:57 -05:00
christianEQ
fc3388acfe configurized loading_process_events_interval_bytes
Former-commit-id: c176ea73664a182af9caacda9d489fb2b344b2de
2021-01-14 13:06:57 -05:00
christianEQ
4fa25cbc57 configurized loading_process_events_interval_bytes
Former-commit-id: c176ea73664a182af9caacda9d489fb2b344b2de
2021-01-14 13:06:57 -05:00
charsyam
b2526c5040 Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress (#8319)
Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress
2021-01-13 16:44:04 -08:00
charsyam
108afbc0bb
Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress (#8319)
Replace many calling zrealloc to one zmalloc in sentinelResetMasterAndChangeAddress
2021-01-13 16:44:04 -08:00
Wang Yuan
b3477eddf9 Optimize performance of clusterGenNodesDescription for large clusters (#8182)
Optimize the performance of clusterGenNodesDescription by only checking slot ownership of each slot once, instead of checking each slot for each node.
2021-01-13 12:36:03 -08:00
Wang Yuan
9cb9f98d2f
Optimize performance of clusterGenNodesDescription for large clusters (#8182)
Optimize the performance of clusterGenNodesDescription by only checking slot ownership of each slot once, instead of checking each slot for each node.
2021-01-13 12:36:03 -08:00
houzj.fnst
8da27781f4 remove some unnecessary checks (#7431)
Remove several checks that always evaluate to true.
2021-01-13 12:00:55 -08:00
houzj.fnst
f5577fdbd8
remove some unnecessary checks (#7431)
Remove several checks that always evaluate to true.
2021-01-13 12:00:55 -08:00
sundb
60423a3e5a Fix use of lookupKeyRead and lookupKeyWrite in zrangeGenericCommand, zunionInterDiffGenericCommand (#8316)
* Change zunionInterDiffGenericCommand to use lookupKeyRead if dstkey is null
* Change zrangeGenericCommand to use lookupKey Write if dstkey isn't null

ZRANGESTORE and UNION, ZINTER, ZDIFF are all new commands (6.2 RC1 and RC2).
In redis 6.0 the ZRANGE was using lookupKeyRead, and ZUNIONSTORE / ZINTERSTORE were using lookupKeyWrite.
So there bugs are introduced in 6.2 and will be resolved before it is released.

the implications of this bug are also not big:
The sole difference between LookupKeyRead and LookupKeyWrite is for command executed on a replica, which are not received from its master client. (for the master, and for the master client on the replica, these two functions behave the same)!
2021-01-13 11:58:12 +02:00
sundb
8a81ed1b5a
Fix use of lookupKeyRead and lookupKeyWrite in zrangeGenericCommand, zunionInterDiffGenericCommand (#8316)
* Change zunionInterDiffGenericCommand to use lookupKeyRead if dstkey is null
* Change zrangeGenericCommand to use lookupKey Write if dstkey isn't null

ZRANGESTORE and UNION, ZINTER, ZDIFF are all new commands (6.2 RC1 and RC2).
In redis 6.0 the ZRANGE was using lookupKeyRead, and ZUNIONSTORE / ZINTERSTORE were using lookupKeyWrite.
So there bugs are introduced in 6.2 and will be resolved before it is released.

the implications of this bug are also not big:
The sole difference between LookupKeyRead and LookupKeyWrite is for command executed on a replica, which are not received from its master client. (for the master, and for the master client on the replica, these two functions behave the same)!
2021-01-13 11:58:12 +02:00
sundb
3e8c7315b7 Fix units in log message of copy-on-write (#8320) 2021-01-13 11:38:02 +02:00
sundb
593cde16bc
Fix units in log message of copy-on-write (#8320) 2021-01-13 11:38:02 +02:00
Oran Agra
93fbd5d13e Redis 6.0.10 2021-01-12 16:25:37 +02:00
Oran Agra
8d70d498d5 Redis 6.0.10 2021-01-12 16:25:37 +02:00
Bob Li
5041095cad Fix overflow of rdbWriteRaw return value (#8306)
Saving string of more than 2GB to the RDB file, can result in corrupt RDB, or failure in rdbSave.
S

(cherry picked from commit 3fb4197a742d064236ae4fdccf9dc00ed3b538d3)
2021-01-12 16:25:37 +02:00
Bob Li
203feb6ea2 Fix overflow of rdbWriteRaw return value (#8306)
Saving string of more than 2GB to the RDB file, can result in corrupt RDB, or failure in rdbSave.
S

(cherry picked from commit 542455ce2afbaffc065afbfdf6578399b8faecda)
2021-01-12 16:25:37 +02:00
guybe7
2d537b376c Modules: Fix an integer sign bug in moduleTimerHandler (#8131)
bug was introduced in 9cbdc8dcdb

(cherry picked from commit de339c29635091c5ce41b7a061b5aff723952a8b)
2021-01-12 16:25:37 +02:00
guybe7
08c3f0bca6 Modules: Fix an integer sign bug in moduleTimerHandler (#8131)
bug was introduced in 1a91a2700b

(cherry picked from commit 2f41a3856845265ffc6cc3a35524883a8690cff7)
2021-01-12 16:25:37 +02:00
Egor Seredin
85bcc60165 Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260)
This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2");

On large string, the previous code would incrementally retry to double the output buffer.
now it uses the the return value of snprintf and grows to the right size in one step.

and also avoids an excessive strlen in sdscat at the end.

(cherry picked from commit 1ad4e18394eac68ab8bae1ef5a2920086ce0f9ba)
2021-01-12 16:25:37 +02:00