20927 Commits

Author SHA1 Message Date
Itamar Haber
5d0c6b0868
Adds help for '--cluster-yes' (#8035) 2020-11-10 22:02:09 +02:00
sundb
428b809649 optimization src/adlist.c:listJoin() 2020-11-09 21:48:56 -08:00
sundb
c09b5941ce optimization src/adlist.c:listJoin() 2020-11-09 21:48:56 -08:00
Meir Shpilraien (Spielrein)
560d2dc008 Moved RMAPI_FUNC_SUPPORTED location such that it will be visible to modules (#8037)
The RMAPI_FUNC_SUPPORTED was defined in the wrong place on redismodule.h
and was not visible to modules.
2020-11-09 10:46:23 +02:00
Meir Shpilraien (Spielrein)
97d647a139
Moved RMAPI_FUNC_SUPPORTED location such that it will be visible to modules (#8037)
The RMAPI_FUNC_SUPPORTED was defined in the wrong place on redismodule.h
and was not visible to modules.
2020-11-09 10:46:23 +02:00
bugwz
855b417d11 Fix the init value for repl_id of rdbSaveInfo struct (#8026)
use 40 zeros rather than 30 zeros to match CONFIG_RUN_ID_SIZE.
this doesn't have any real implications.
2020-11-08 09:26:25 +02:00
bugwz
bcc46a2f25
Fix the init value for repl_id of rdbSaveInfo struct (#8026)
use 40 zeros rather than 30 zeros to match CONFIG_RUN_ID_SIZE.
this doesn't have any real implications.
2020-11-08 09:26:25 +02:00
David CARLIER
8f3f9b6ac5 DragonFlyBSD resident memory amount (almost) similar as FreeBSD. (#8023) 2020-11-08 09:16:14 +02:00
David CARLIER
d428de590f
DragonFlyBSD resident memory amount (almost) similar as FreeBSD. (#8023) 2020-11-08 09:16:14 +02:00
sundb
302482712d Typo fix: entires -> entries (#8031) 2020-11-08 08:32:38 +02:00
sundb
cd1c600548
Typo fix: entires -> entries (#8031) 2020-11-08 08:32:38 +02:00
John Sully
4f7ac9c11f Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
John Sully
e085772d01 Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
Wen Hui
8a412b6e64 Debug Populate: Avoid server crash when passing negative value for key and value size (#8018)
* Debug Populate: Add checks for count and keysize to avoid crash

* provide getRangeLongFromObjectOrReply and getPositiveLongFromObjectOrReply for range check
2020-11-05 19:58:54 +02:00
Wen Hui
254367788a
Debug Populate: Avoid server crash when passing negative value for key and value size (#8018)
* Debug Populate: Add checks for count and keysize to avoid crash

* provide getRangeLongFromObjectOrReply and getPositiveLongFromObjectOrReply for range check
2020-11-05 19:58:54 +02:00
Yossi Gottlieb
43c07a877c Fix crash log output on ARM. (#8020) 2020-11-05 15:43:53 +02:00
Yossi Gottlieb
7e4325cbc9
Fix crash log output on ARM. (#8020) 2020-11-05 15:43:53 +02:00
Oran Agra
21c626ef2d Better INFO fields to track diskless and disk-based replication progress (#7981)
Expose new `loading_rdb_used_mem` showing the used memory of the server
that saved the RDB file we're currently using.
This is useful in diskless replication when the total size of the rdb is
unkown, and can be used as a rought estimation of progres.

Use that new field to calculate the "user friendly"
`loading_loaded_perc` and `loading_eta_seconds`.

Expose `master_sync_total_bytes` and `master_sync_total_bytes` to complement
on the existing `master_sync_total_bytes` (which cannot be used on its own
to calculate progress).

Add "user friendly" field for `master_sync_perc`
2020-11-05 11:46:16 +02:00
Oran Agra
7ace7231c6
Better INFO fields to track diskless and disk-based replication progress (#7981)
Expose new `loading_rdb_used_mem` showing the used memory of the server
that saved the RDB file we're currently using.
This is useful in diskless replication when the total size of the rdb is
unkown, and can be used as a rought estimation of progres.

Use that new field to calculate the "user friendly"
`loading_loaded_perc` and `loading_eta_seconds`.

Expose `master_sync_total_bytes` and `master_sync_total_bytes` to complement
on the existing `master_sync_total_bytes` (which cannot be used on its own
to calculate progress).

Add "user friendly" field for `master_sync_perc`
2020-11-05 11:46:16 +02:00
Yossi Gottlieb
a808e33ab3 Add RESET command. (#7982)
Perform full reset of all client connection states, is if the client was
disconnected and re-connected. This affects:

* MULTI state
* Watched keys
* MONITOR mode
* Pub/Sub subscription
* ACL/Authenticated state
* Client tracking state
* Cluster read-only/asking state
* RESP version (reset to 2)
* Selected database
* CLIENT REPLY state

The response is +RESET to make it easily distinguishable from other
responses.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-11-05 10:51:26 +02:00
Yossi Gottlieb
1fd456f91a
Add RESET command. (#7982)
Perform full reset of all client connection states, is if the client was
disconnected and re-connected. This affects:

* MULTI state
* Watched keys
* MONITOR mode
* Pub/Sub subscription
* ACL/Authenticated state
* Client tracking state
* Cluster read-only/asking state
* RESP version (reset to 2)
* Selected database
* CLIENT REPLY state

The response is +RESET to make it easily distinguishable from other
responses.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-11-05 10:51:26 +02:00
Tomasz Poradowski
d480eb05da rdb: corrected RedisModuleIO initialization point (#8014)
- rdbSaveSingleModuleAux() used RedisModuleIO's "bytes" field for
  tracking written bytes before calling moduleInitIOContext() which sets
  "bytes" to zero
- rdbSaveObject() re-initialized RedisModuleIO too late

This return value is not used at the moment since it's only tested
against -1, and the actual byte count isn't used yet.

Co-authored-by: Tomasz Poradowski <tomasz.poradowski@generiscorp.com>
2020-11-04 16:19:18 +02:00
Tomasz Poradowski
d8fbd3a8ba
rdb: corrected RedisModuleIO initialization point (#8014)
- rdbSaveSingleModuleAux() used RedisModuleIO's "bytes" field for
  tracking written bytes before calling moduleInitIOContext() which sets
  "bytes" to zero
- rdbSaveObject() re-initialized RedisModuleIO too late

This return value is not used at the moment since it's only tested
against -1, and the actual byte count isn't used yet.

Co-authored-by: Tomasz Poradowski <tomasz.poradowski@generiscorp.com>
2020-11-04 16:19:18 +02:00
filipe oliveira
0ce56054c3 Enable specifying TLS ciphers(suites) in redis-cli/redis-benchmark (#8005)
Enable specifying the preferred ciphers and/or ciphersuites for redis-cli/redis-benchmark.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-11-04 14:49:15 +02:00
filipe oliveira
10b5006934
Enable specifying TLS ciphers(suites) in redis-cli/redis-benchmark (#8005)
Enable specifying the preferred ciphers and/or ciphersuites for redis-cli/redis-benchmark.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-11-04 14:49:15 +02:00
Egor Seredin
1ad4e18394 Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260)
This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2");

On large string, the previous code would incrementally retry to double the output buffer.
now it uses the the return value of snprintf and grows to the right size in one step.

and also avoids an excessive strlen in sdscat at the end.
2020-11-04 13:38:46 +02:00
Egor Seredin
f4ca3d8757
Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260)
This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2");

On large string, the previous code would incrementally retry to double the output buffer.
now it uses the the return value of snprintf and grows to the right size in one step.

and also avoids an excessive strlen in sdscat at the end.
2020-11-04 13:38:46 +02:00
Wen Hui
c28d5ca026 redis-cli cluster import support source and target that need auth (#7994)
Make it possible for redis-cli cluster import to work with source and
target that require AUTH.

Adding two different flags --cluster-from-user, --cluster-from-pass
and --cluster-askpass for source node authentication.
Also for target authentication, using existing --user and --pass flag.

Example:

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --pass 1234 --user default --cluster-from-user default --cluster-from-pass 123456

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --askpass --cluster-from-user default --cluster-from-askpass
2020-11-04 10:00:18 +02:00
Wen Hui
639b73cd2a
redis-cli cluster import support source and target that need auth (#7994)
Make it possible for redis-cli cluster import to work with source and
target that require AUTH.

Adding two different flags --cluster-from-user, --cluster-from-pass
and --cluster-askpass for source node authentication.
Also for target authentication, using existing --user and --pass flag.

Example:

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --pass 1234 --user default --cluster-from-user default --cluster-from-pass 123456

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --askpass --cluster-from-user default --cluster-from-askpass
2020-11-04 10:00:18 +02:00
Oran Agra
420cd40c17 Add maxclients and cluster_connections to INFO CLIENTS (#7979)
Few config settings are also reflected by the INFO command.
these are mainly ones that are important for either an instant view of
the server status (to compare a metric to it's limit config),
Important configurations that are necessary in the crash log (which
currently doesn't print the config),
And things that are important for monitoring solutions (such as
Prometheus), which rely on INFO to collect their data.

Add cluster_connections to INFO CLUSTER:
This makes it possible to be combined together with connected_clients
and connected_slaves and be matched against maxclients
2020-11-04 09:53:43 +02:00
Oran Agra
a698a6391a
Add maxclients and cluster_connections to INFO CLIENTS (#7979)
Few config settings are also reflected by the INFO command.
these are mainly ones that are important for either an instant view of
the server status (to compare a metric to it's limit config),
Important configurations that are necessary in the crash log (which
currently doesn't print the config),
And things that are important for monitoring solutions (such as
Prometheus), which rely on INFO to collect their data.

Add cluster_connections to INFO CLUSTER:
This makes it possible to be combined together with connected_clients
and connected_slaves and be matched against maxclients
2020-11-04 09:53:43 +02:00
Wang Yuan
16e3af9d23 Disable rehash when redis has child process (#8007)
In redisFork(), we don't set child pid, so updateDictResizePolicy()
doesn't take effect, that isn't friendly for copy-on-write.

The bug was introduced this in redis 6.0: e70fbad
2020-11-03 17:16:11 +02:00
Wang Yuan
89c78a9808
Disable rehash when redis has child process (#8007)
In redisFork(), we don't set child pid, so updateDictResizePolicy()
doesn't take effect, that isn't friendly for copy-on-write.

The bug was introduced this in redis 6.0: 56258c6
2020-11-03 17:16:11 +02:00
Meir Shpilraien (Spielrein)
ceaffc8d0d Added crash report on SIGABRT (#8004)
The reason that we want to get a full crash report on SIGABRT
is that the jmalloc, when detecting a corruption, calls abort().
This will cause the Redis to exist silently without any report
and without any way to analyze what happened.
2020-11-03 14:59:21 +02:00
Meir Shpilraien (Spielrein)
f210e197f3
Added crash report on SIGABRT (#8004)
The reason that we want to get a full crash report on SIGABRT
is that the jmalloc, when detecting a corruption, calls abort().
This will cause the Redis to exist silently without any report
and without any way to analyze what happened.
2020-11-03 14:59:21 +02:00
Oran Agra
54b8623eeb Propagate GETSET and SET-GET as SET (#7957)
- Generates a more backwards compatible command stream
- Slightly more efficient execution in replica/AOF
- Add a test for coverage
2020-11-03 14:56:57 +02:00
Oran Agra
9122379abc
Propagate GETSET and SET-GET as SET (#7957)
- Generates a more backwards compatible command stream
- Slightly more efficient execution in replica/AOF
- Add a test for coverage
2020-11-03 14:56:57 +02:00
guybe7
9cbdc8dcdb Modules: Improve timer accuracy (#7987)
The bug occurs when 'callback' re-registers itself to a point
in the future and the execution time in non-negligible:
'now' refers to time BEFORE callback was executed and is used
to calculate 'next_period'.
We must get the actual current time when calculating 'next_period'
2020-11-02 18:18:42 +02:00
guybe7
1a91a2700b
Modules: Improve timer accuracy (#7987)
The bug occurs when 'callback' re-registers itself to a point
in the future and the execution time in non-negligible:
'now' refers to time BEFORE callback was executed and is used
to calculate 'next_period'.
We must get the actual current time when calculating 'next_period'
2020-11-02 18:18:42 +02:00
yoav-steinberg
dba5c6c516 Add local address to CLIENT LIST, and a CLIENT KILL filter. (#7913)
Useful when you want to know through which bind address the client connected to
the server in case of multiple bind addresses.

- Adding `laddr` field to CLIENT list showing the local (bind) address.
- Adding `LADDR` option to CLIENT KILL to kill all the clients connected
  to a specific local address.
- Refactoring to share code.
2020-10-28 21:13:44 +02:00
yoav-steinberg
84b3c18f71
Add local address to CLIENT LIST, and a CLIENT KILL filter. (#7913)
Useful when you want to know through which bind address the client connected to
the server in case of multiple bind addresses.

- Adding `laddr` field to CLIENT list showing the local (bind) address.
- Adding `LADDR` option to CLIENT KILL to kill all the clients connected
  to a specific local address.
- Refactoring to share code.
2020-10-28 21:13:44 +02:00
Oran Agra
f21548baaa Optionally (default) fail to start if requested bind address is not available (#7936)
Background:
#3467 (redis 4.0.0), started ignoring ENOPROTOOPT, but did that only for
the default bind (in case bind config wasn't explicitly set).
#5598 (redis 5.0.3), added that for bind addresses explicitly set
(following bug reports in Debian for redis 4.0.9 and 5.0.1), it
also ignored a bunch of other errors like EPROTONOSUPPORT which was
requested in #3894, and also added EADDRNOTAVAIL (wasn't clear why).

This (ignoring EADDRNOTAVAIL) makes redis start successfully, even if a
certain network interface isn't up yet , in which case we rather redis
fail and will be re-tried when the NIC is up, see #7933.

However, it turns out that when IPv6 is disabled (supported but unused),
the error we're getting is EADDRNOTAVAIL. and in many systems the
default config file tries to bind to localhost for both v4 and v6 and
would like to silently ignore the error on v6 if disabled.
This means that we sometimes want to ignore EADDRNOTAVAIL and other times
we wanna fail.

So this commit changes these main things:
1. Ignore all the errors we ignore for both explicitly requested bind
   address and a default implicit one.
2. Add a '-' prefix to allow EADDRNOTAVAIL be ignored (by default that's
   different than the previous behavior).
3. Restructure that function in a more readable and maintainable way see
   below.
4. Make the default behavior of listening to all achievable by setting
  a bind config directive to * (previously only possible by omitting
  it)
5. document everything.

The old structure of this function was that even if there are no bind
addresses requested, the loop that runs though the bind addresses runs
at least once anyway!
In that one iteration of the loop it binds to both v4 and v6 addresses,
handles errors for each of them separately, and then eventually at the
if-else chain, handles the error of the last bind attempt again!
This was very hard to read and very error prone to maintain, instead now
when the bind info is missing we create one with two entries, and run
the simple loop twice.
2020-10-28 21:09:15 +02:00
Oran Agra
441bfa2dfb
Optionally (default) fail to start if requested bind address is not available (#7936)
Background:
#3467 (redis 4.0.0), started ignoring ENOPROTOOPT, but did that only for
the default bind (in case bind config wasn't explicitly set).
#5598 (redis 5.0.3), added that for bind addresses explicitly set
(following bug reports in Debian for redis 4.0.9 and 5.0.1), it
also ignored a bunch of other errors like EPROTONOSUPPORT which was
requested in #3894, and also added EADDRNOTAVAIL (wasn't clear why).

This (ignoring EADDRNOTAVAIL) makes redis start successfully, even if a
certain network interface isn't up yet , in which case we rather redis
fail and will be re-tried when the NIC is up, see #7933.

However, it turns out that when IPv6 is disabled (supported but unused),
the error we're getting is EADDRNOTAVAIL. and in many systems the
default config file tries to bind to localhost for both v4 and v6 and
would like to silently ignore the error on v6 if disabled.
This means that we sometimes want to ignore EADDRNOTAVAIL and other times
we wanna fail.

So this commit changes these main things:
1. Ignore all the errors we ignore for both explicitly requested bind
   address and a default implicit one.
2. Add a '-' prefix to allow EADDRNOTAVAIL be ignored (by default that's
   different than the previous behavior).
3. Restructure that function in a more readable and maintainable way see
   below.
4. Make the default behavior of listening to all achievable by setting
  a bind config directive to * (previously only possible by omitting
  it)
5. document everything.

The old structure of this function was that even if there are no bind
addresses requested, the loop that runs though the bind addresses runs
at least once anyway!
In that one iteration of the loop it binds to both v4 and v6 addresses,
handles errors for each of them separately, and then eventually at the
if-else chain, handles the error of the last bind attempt again!
This was very hard to read and very error prone to maintain, instead now
when the bind info is missing we create one with two entries, and run
the simple loop twice.
2020-10-28 21:09:15 +02:00
Madelyn Olson
b9b8ef1130 White space tweaks and skip categories already applied 2020-10-28 10:01:20 -07:00
Madelyn Olson
d310beb417 White space tweaks and skip categories already applied 2020-10-28 10:01:20 -07:00
Madelyn Olson
2eae7756fd Further improved ACL algorithm for picking categories 2020-10-28 10:01:20 -07:00
Madelyn Olson
411bcf1a41 Further improved ACL algorithm for picking categories 2020-10-28 10:01:20 -07:00
Wen Hui
da02092b66 refactor aof rewrite code to avoid memory leaks in error handling (#7976) 2020-10-28 12:35:28 +02:00
Wen Hui
4342703743
refactor aof rewrite code to avoid memory leaks in error handling (#7976) 2020-10-28 12:35:28 +02:00
sundb
3b8bfde6c4 docs: Fix some typos in comments and log messge (#7975) 2020-10-28 08:51:35 +02:00