27332 Commits

Author SHA1 Message Date
tzongw
d5059ba5e1
Invalidate aeTimer when returning AE_NOMORE (#8022) 2020-11-11 12:10:54 +02:00
Wen Hui
ee7e3a221e add tracking bcast flag and client redirection in client list (#7995) 2020-11-11 08:22:17 +02:00
Wen Hui
dd1f20edc5
add tracking bcast flag and client redirection in client list (#7995) 2020-11-11 08:22:17 +02:00
John Sully
2fdac516a6 Add missing file
Former-commit-id: 97fd73692363e05b80e0c0e84b7b2b8fe7f760b8
2020-11-11 03:29:38 +00:00
John Sully
5de36840ad Add missing file
Former-commit-id: 97fd73692363e05b80e0c0e84b7b2b8fe7f760b8
2020-11-11 03:29:38 +00:00
Madelyn Olson
71766ca059 Rewritten commands are logged as their original command (#8006)
* Rewritten commands are logged as their original command

Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2020-11-10 13:50:03 -08:00
Madelyn Olson
3feff7d78a
Rewritten commands are logged as their original command (#8006)
* Rewritten commands are logged as their original command

Co-authored-by: Madelyn Olson <madelyneolson@gmail.com>
2020-11-10 13:50:03 -08:00
Itamar Haber
ec4e47acab Adds help for '--cluster-yes' (#8035) 2020-11-10 22:02:09 +02:00
Itamar Haber
5d0c6b0868
Adds help for '--cluster-yes' (#8035) 2020-11-10 22:02:09 +02:00
sundb
428b809649 optimization src/adlist.c:listJoin() 2020-11-09 21:48:56 -08:00
sundb
c09b5941ce optimization src/adlist.c:listJoin() 2020-11-09 21:48:56 -08:00
nitaicaro
a13bd9ded3 Extend client tracking tests (#7998)
Test support for the new map, null and push message types. Map objects are parsed as a list of lists of key value pairs.
for instance: user => john password => 123

will be parsed to the following TCL list:

{{user john} {password 123}}

Also added the following tests:

Redirection still works with RESP3

Able to use a RESP3 client as a redirection client

No duplicate invalidation messages when turning BCAST mode on after normal tracking

Server is able to evacuate enough keys when num of keys surpasses limit by more than defined initial effort

Different clients using different protocols can track the same key

OPTOUT tests

OPTIN tests

Clients can redirect to the same connection

tracking-redir-broken test

HELLO 3 checks

Invalidation messages still work when using RESP3, with and without redirection

Switching to RESP3 doesn't disturb previous tracked keys

Tracking info is correct

Flushall and flushdb produce invalidation messages

These tests achieve 100% line coverage for tracking.c using lcov.
2020-11-09 22:54:47 +02:00
nitaicaro
19c29b6007
Extend client tracking tests (#7998)
Test support for the new map, null and push message types. Map objects are parsed as a list of lists of key value pairs.
for instance: user => john password => 123

will be parsed to the following TCL list:

{{user john} {password 123}}

Also added the following tests:

Redirection still works with RESP3

Able to use a RESP3 client as a redirection client

No duplicate invalidation messages when turning BCAST mode on after normal tracking

Server is able to evacuate enough keys when num of keys surpasses limit by more than defined initial effort

Different clients using different protocols can track the same key

OPTOUT tests

OPTIN tests

Clients can redirect to the same connection

tracking-redir-broken test

HELLO 3 checks

Invalidation messages still work when using RESP3, with and without redirection

Switching to RESP3 doesn't disturb previous tracked keys

Tracking info is correct

Flushall and flushdb produce invalidation messages

These tests achieve 100% line coverage for tracking.c using lcov.
2020-11-09 22:54:47 +02:00
Meir Shpilraien (Spielrein)
560d2dc008 Moved RMAPI_FUNC_SUPPORTED location such that it will be visible to modules (#8037)
The RMAPI_FUNC_SUPPORTED was defined in the wrong place on redismodule.h
and was not visible to modules.
2020-11-09 10:46:23 +02:00
Meir Shpilraien (Spielrein)
97d647a139
Moved RMAPI_FUNC_SUPPORTED location such that it will be visible to modules (#8037)
The RMAPI_FUNC_SUPPORTED was defined in the wrong place on redismodule.h
and was not visible to modules.
2020-11-09 10:46:23 +02:00
bugwz
855b417d11 Fix the init value for repl_id of rdbSaveInfo struct (#8026)
use 40 zeros rather than 30 zeros to match CONFIG_RUN_ID_SIZE.
this doesn't have any real implications.
2020-11-08 09:26:25 +02:00
bugwz
bcc46a2f25
Fix the init value for repl_id of rdbSaveInfo struct (#8026)
use 40 zeros rather than 30 zeros to match CONFIG_RUN_ID_SIZE.
this doesn't have any real implications.
2020-11-08 09:26:25 +02:00
David CARLIER
8f3f9b6ac5 DragonFlyBSD resident memory amount (almost) similar as FreeBSD. (#8023) 2020-11-08 09:16:14 +02:00
David CARLIER
d428de590f
DragonFlyBSD resident memory amount (almost) similar as FreeBSD. (#8023) 2020-11-08 09:16:14 +02:00
sundb
302482712d Typo fix: entires -> entries (#8031) 2020-11-08 08:32:38 +02:00
sundb
cd1c600548
Typo fix: entires -> entries (#8031) 2020-11-08 08:32:38 +02:00
John Sully
4f7ac9c11f Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
John Sully
e085772d01 Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
Wen Hui
8a412b6e64 Debug Populate: Avoid server crash when passing negative value for key and value size (#8018)
* Debug Populate: Add checks for count and keysize to avoid crash

* provide getRangeLongFromObjectOrReply and getPositiveLongFromObjectOrReply for range check
2020-11-05 19:58:54 +02:00
Wen Hui
254367788a
Debug Populate: Avoid server crash when passing negative value for key and value size (#8018)
* Debug Populate: Add checks for count and keysize to avoid crash

* provide getRangeLongFromObjectOrReply and getPositiveLongFromObjectOrReply for range check
2020-11-05 19:58:54 +02:00
Yossi Gottlieb
43c07a877c Fix crash log output on ARM. (#8020) 2020-11-05 15:43:53 +02:00
Yossi Gottlieb
7e4325cbc9
Fix crash log output on ARM. (#8020) 2020-11-05 15:43:53 +02:00
Oran Agra
21c626ef2d Better INFO fields to track diskless and disk-based replication progress (#7981)
Expose new `loading_rdb_used_mem` showing the used memory of the server
that saved the RDB file we're currently using.
This is useful in diskless replication when the total size of the rdb is
unkown, and can be used as a rought estimation of progres.

Use that new field to calculate the "user friendly"
`loading_loaded_perc` and `loading_eta_seconds`.

Expose `master_sync_total_bytes` and `master_sync_total_bytes` to complement
on the existing `master_sync_total_bytes` (which cannot be used on its own
to calculate progress).

Add "user friendly" field for `master_sync_perc`
2020-11-05 11:46:16 +02:00
Oran Agra
7ace7231c6
Better INFO fields to track diskless and disk-based replication progress (#7981)
Expose new `loading_rdb_used_mem` showing the used memory of the server
that saved the RDB file we're currently using.
This is useful in diskless replication when the total size of the rdb is
unkown, and can be used as a rought estimation of progres.

Use that new field to calculate the "user friendly"
`loading_loaded_perc` and `loading_eta_seconds`.

Expose `master_sync_total_bytes` and `master_sync_total_bytes` to complement
on the existing `master_sync_total_bytes` (which cannot be used on its own
to calculate progress).

Add "user friendly" field for `master_sync_perc`
2020-11-05 11:46:16 +02:00
Yossi Gottlieb
a808e33ab3 Add RESET command. (#7982)
Perform full reset of all client connection states, is if the client was
disconnected and re-connected. This affects:

* MULTI state
* Watched keys
* MONITOR mode
* Pub/Sub subscription
* ACL/Authenticated state
* Client tracking state
* Cluster read-only/asking state
* RESP version (reset to 2)
* Selected database
* CLIENT REPLY state

The response is +RESET to make it easily distinguishable from other
responses.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-11-05 10:51:26 +02:00
Yossi Gottlieb
1fd456f91a
Add RESET command. (#7982)
Perform full reset of all client connection states, is if the client was
disconnected and re-connected. This affects:

* MULTI state
* Watched keys
* MONITOR mode
* Pub/Sub subscription
* ACL/Authenticated state
* Client tracking state
* Cluster read-only/asking state
* RESP version (reset to 2)
* Selected database
* CLIENT REPLY state

The response is +RESET to make it easily distinguishable from other
responses.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-11-05 10:51:26 +02:00
Yossi Gottlieb
c36f8d4481 Tests: fix filename reported in run_solo tests. 2020-11-04 21:43:55 +02:00
Yossi Gottlieb
f6546eff45 Tests: fix filename reported in run_solo tests. 2020-11-04 21:43:55 +02:00
Yossi Gottlieb
74372c3813 Fix test failure on slower systems.
Not disabling save, slower systems begun background save that did not
complete in time, resulting with SAVE failing with "ERR Background save
already in progress".
2020-11-04 21:43:55 +02:00
Yossi Gottlieb
2faa0f19eb Fix test failure on slower systems.
Not disabling save, slower systems begun background save that did not
complete in time, resulting with SAVE failing with "ERR Background save
already in progress".
2020-11-04 21:43:55 +02:00
Tomasz Poradowski
d480eb05da rdb: corrected RedisModuleIO initialization point (#8014)
- rdbSaveSingleModuleAux() used RedisModuleIO's "bytes" field for
  tracking written bytes before calling moduleInitIOContext() which sets
  "bytes" to zero
- rdbSaveObject() re-initialized RedisModuleIO too late

This return value is not used at the moment since it's only tested
against -1, and the actual byte count isn't used yet.

Co-authored-by: Tomasz Poradowski <tomasz.poradowski@generiscorp.com>
2020-11-04 16:19:18 +02:00
Tomasz Poradowski
d8fbd3a8ba
rdb: corrected RedisModuleIO initialization point (#8014)
- rdbSaveSingleModuleAux() used RedisModuleIO's "bytes" field for
  tracking written bytes before calling moduleInitIOContext() which sets
  "bytes" to zero
- rdbSaveObject() re-initialized RedisModuleIO too late

This return value is not used at the moment since it's only tested
against -1, and the actual byte count isn't used yet.

Co-authored-by: Tomasz Poradowski <tomasz.poradowski@generiscorp.com>
2020-11-04 16:19:18 +02:00
filipe oliveira
0ce56054c3 Enable specifying TLS ciphers(suites) in redis-cli/redis-benchmark (#8005)
Enable specifying the preferred ciphers and/or ciphersuites for redis-cli/redis-benchmark.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-11-04 14:49:15 +02:00
filipe oliveira
10b5006934
Enable specifying TLS ciphers(suites) in redis-cli/redis-benchmark (#8005)
Enable specifying the preferred ciphers and/or ciphersuites for redis-cli/redis-benchmark.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-11-04 14:49:15 +02:00
Egor Seredin
1ad4e18394 Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260)
This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2");

On large string, the previous code would incrementally retry to double the output buffer.
now it uses the the return value of snprintf and grows to the right size in one step.

and also avoids an excessive strlen in sdscat at the end.
2020-11-04 13:38:46 +02:00
Egor Seredin
f4ca3d8757
Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260)
This will allow to use: RedisModule_CreateStringPrintf(ctx, "%s %c %s", "string1", 0, "string2");

On large string, the previous code would incrementally retry to double the output buffer.
now it uses the the return value of snprintf and grows to the right size in one step.

and also avoids an excessive strlen in sdscat at the end.
2020-11-04 13:38:46 +02:00
Meir Shpilraien (Spielrein)
a3b52c76eb Disable new SIGABRT test on valgrind (#8013)
The crash reports cause false-positive warnings when run with valgrind.
2020-11-04 13:13:55 +02:00
Meir Shpilraien (Spielrein)
762be79f0b
Disable new SIGABRT test on valgrind (#8013)
The crash reports cause false-positive warnings when run with valgrind.
2020-11-04 13:13:55 +02:00
Wen Hui
c28d5ca026 redis-cli cluster import support source and target that need auth (#7994)
Make it possible for redis-cli cluster import to work with source and
target that require AUTH.

Adding two different flags --cluster-from-user, --cluster-from-pass
and --cluster-askpass for source node authentication.
Also for target authentication, using existing --user and --pass flag.

Example:

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --pass 1234 --user default --cluster-from-user default --cluster-from-pass 123456

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --askpass --cluster-from-user default --cluster-from-askpass
2020-11-04 10:00:18 +02:00
Wen Hui
639b73cd2a
redis-cli cluster import support source and target that need auth (#7994)
Make it possible for redis-cli cluster import to work with source and
target that require AUTH.

Adding two different flags --cluster-from-user, --cluster-from-pass
and --cluster-askpass for source node authentication.
Also for target authentication, using existing --user and --pass flag.

Example:

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --pass 1234 --user default --cluster-from-user default --cluster-from-pass 123456

./redis-cli --cluster import 127.0.0.1:7000 --cluster-from 127.0.0.1:6379 --askpass --cluster-from-user default --cluster-from-askpass
2020-11-04 10:00:18 +02:00
Oran Agra
420cd40c17 Add maxclients and cluster_connections to INFO CLIENTS (#7979)
Few config settings are also reflected by the INFO command.
these are mainly ones that are important for either an instant view of
the server status (to compare a metric to it's limit config),
Important configurations that are necessary in the crash log (which
currently doesn't print the config),
And things that are important for monitoring solutions (such as
Prometheus), which rely on INFO to collect their data.

Add cluster_connections to INFO CLUSTER:
This makes it possible to be combined together with connected_clients
and connected_slaves and be matched against maxclients
2020-11-04 09:53:43 +02:00
Oran Agra
a698a6391a
Add maxclients and cluster_connections to INFO CLIENTS (#7979)
Few config settings are also reflected by the INFO command.
these are mainly ones that are important for either an instant view of
the server status (to compare a metric to it's limit config),
Important configurations that are necessary in the crash log (which
currently doesn't print the config),
And things that are important for monitoring solutions (such as
Prometheus), which rely on INFO to collect their data.

Add cluster_connections to INFO CLUSTER:
This makes it possible to be combined together with connected_clients
and connected_slaves and be matched against maxclients
2020-11-04 09:53:43 +02:00
Wang Yuan
16e3af9d23 Disable rehash when redis has child process (#8007)
In redisFork(), we don't set child pid, so updateDictResizePolicy()
doesn't take effect, that isn't friendly for copy-on-write.

The bug was introduced this in redis 6.0: e70fbad
2020-11-03 17:16:11 +02:00
Wang Yuan
89c78a9808
Disable rehash when redis has child process (#8007)
In redisFork(), we don't set child pid, so updateDictResizePolicy()
doesn't take effect, that isn't friendly for copy-on-write.

The bug was introduced this in redis 6.0: 56258c6
2020-11-03 17:16:11 +02:00
Meir Shpilraien (Spielrein)
ceaffc8d0d Added crash report on SIGABRT (#8004)
The reason that we want to get a full crash report on SIGABRT
is that the jmalloc, when detecting a corruption, calls abort().
This will cause the Redis to exist silently without any report
and without any way to analyze what happened.
2020-11-03 14:59:21 +02:00