27610 Commits

Author SHA1 Message Date
ShooterIT
7fcf29e615 [Redis-benchmark] Support zset type
(cherry picked from commit e5a50ed3c49c211b413d92b1f45f950c830c58c0)
2020-09-01 09:27:58 +03:00
Arun Ranganathan
57c6b0e718 Show threading configuration in INFO output (#7446)
Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 444b53e6402dabc7c2bf52be6603c4c9bbfb9921)
2020-09-01 09:27:58 +03:00
Arun Ranganathan
e81bac32fd Show threading configuration in INFO output (#7446)
Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit f6cad30bb69b2ad35bb0a870077fac2d4605d727)
2020-09-01 09:27:58 +03:00
Meir Shpilraien (Spielrein)
f63e428e5b This PR introduces a new loaded keyspace event (#7536)
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit 73198c50194cbf0254afd4cc5245f9274a538d13)
2020-09-01 09:27:58 +03:00
Meir Shpilraien (Spielrein)
2257f38b68 This PR introduces a new loaded keyspace event (#7536)
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit 8d826393191399e132bd9e56fb51ed83223cc5ca)
2020-09-01 09:27:58 +03:00
Oran Agra
888cbf6822 Fix rejectCommand trims newline in shared error objects, hung clients (#7714)
fe8d6fe74 (released in 6.0.6) has a side effect, when processCommand
rejects a command with pre-made shared object error string, it trims the
newlines from the end of the string. if that string is later used with
addReply, the newline will be missing, breaking the protocol, and
leaving the client hung.

It seems that the only scenario which this happens is when replying with
-LOADING to some command, and later using that reply from the CONFIG
SET command (still during loading). this will result in hung client.

Refactoring the code in order to avoid trimming these newlines from
shared string objects, and do the newline trimming only in other cases
where it's needed.

Co-authored-by: Guy Benoish <guy.benoish@redislabs.com>
(cherry picked from commit 2640897e3a01fbacb620c12e021c934e48eeccb9)
2020-09-01 09:27:58 +03:00
Oran Agra
2165a78d10 Fix rejectCommand trims newline in shared error objects, hung clients (#7714)
65a3307bc (released in 6.0.6) has a side effect, when processCommand
rejects a command with pre-made shared object error string, it trims the
newlines from the end of the string. if that string is later used with
addReply, the newline will be missing, breaking the protocol, and
leaving the client hung.

It seems that the only scenario which this happens is when replying with
-LOADING to some command, and later using that reply from the CONFIG
SET command (still during loading). this will result in hung client.

Refactoring the code in order to avoid trimming these newlines from
shared string objects, and do the newline trimming only in other cases
where it's needed.

Co-authored-by: Guy Benoish <guy.benoish@redislabs.com>
(cherry picked from commit 9fcd9e191e6f54276688fb7c74e1d5c3c4be9a75)
2020-09-01 09:27:58 +03:00
valentinogeron
3c136a7777 EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag

(cherry picked from commit 0292720ccb0a189d3ed49d7bf912602360a4ecdd)
2020-09-01 09:27:58 +03:00
valentinogeron
e6f6731c66 EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag

(cherry picked from commit b7289e912cbe1a011a5569cd67929e83731b9660)
2020-09-01 09:27:58 +03:00
Itamar Haber
c77e8e2f1c Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.

(cherry picked from commit cb504d7fddd09149655e91496588c610e89ca131)
2020-09-01 09:27:58 +03:00
Itamar Haber
5a9a653f3e Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.

(cherry picked from commit 5b0a06af48997794af60dabb58ce4336ef56f73d)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
fe6da3c36b Add language servers stuff, test/tls to gitignore. (#7698)
(cherry picked from commit 74d9d9544969fed000bde5f8504b5b3f211e473f)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
ff0846cbb7 Add language servers stuff, test/tls to gitignore. (#7698)
(cherry picked from commit 5449a2a8b54db6a57b564e453c9f7075ebb81f79)
2020-09-01 09:27:58 +03:00
Valentino Geron
d727f52723 Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.

(cherry picked from commit 7e6c9ef8819a071679f8dd18035dbbe2455c7b12)
2020-09-01 09:27:58 +03:00
Valentino Geron
3e2ed279fd Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.

(cherry picked from commit 8b428cf0f7ce5489fbdf4640a1dd36357f3dc5f0)
2020-09-01 09:27:58 +03:00
Valentino Geron
34124fff88 Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```

(cherry picked from commit 7a555da64f56a4fb2f300d84a35778bee8f471ca)
2020-09-01 09:27:58 +03:00
Valentino Geron
19ef1f371d Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```

(cherry picked from commit 9204a9b2c2f6eb59767ab0bddcde62c75e8c20b0)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
c5675c66bc Tests: fix redis-cli with remote hosts. (#7693)
(cherry picked from commit 257f9f462f7782dcaecf7bbf35f4701b20b88a45)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
8d79702d8a Tests: fix redis-cli with remote hosts. (#7693)
(cherry picked from commit f80f3f492a0ca56e163899eeca7ad40d67d903be)
2020-09-01 09:27:58 +03:00
huangzhw
907b0124e8 RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.

(cherry picked from commit b980e999293e9214a844712f9c88ca69acd20b1b)
2020-09-01 09:27:58 +03:00
huangzhw
6cfcf76e0b RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.

(cherry picked from commit a3d4d7bf68bf825584b34785ed2b117dd24c1754)
2020-09-01 09:27:58 +03:00
guybe7
d0b8583167 Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.

(cherry picked from commit 4de17eb032160c7ba94c505eab4b776a456e5117)
2020-09-01 09:27:58 +03:00
guybe7
c9478a3b4e Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.

(cherry picked from commit 65c24bd3d436a08a680fa80bf5b3f4f9cf8ef395)
2020-09-01 09:27:58 +03:00
杨博东
b42976bd56 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 5e6212e087c4696abc682b64079202c9ade8666c)
2020-09-01 09:27:58 +03:00
杨博东
113d5ae872 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit cbaf3c5bbafd43e009a2d6b38dd0e9fc450a3e12)
2020-09-01 09:27:58 +03:00
Raghav Muddur
3564aab1a5 Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.

(cherry picked from commit f7cf24b5da539dbf4fafc404355a32bac06aa2f5)
2020-09-01 09:27:58 +03:00
Raghav Muddur
1bd75aafdb Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.

(cherry picked from commit 34c3be365a3a8bc74f7a99b3e35cb211afaa58ca)
2020-09-01 09:27:58 +03:00
Madelyn Olson
f80b756076 Fixed hset error since it's shared with hmset (#7678)
(cherry picked from commit 17c6b3c7eeb486a75d05ba7fb3c4490b7abbc597)
2020-09-01 09:27:58 +03:00
Madelyn Olson
ba48e7e9cd Fixed hset error since it's shared with hmset (#7678)
(cherry picked from commit cbd9af85838a73c9c9654829cf8e511ecc5853de)
2020-09-01 09:27:58 +03:00
guybe7
d74a98eaac PERSIST should signalModifiedKey (Like EXPIRE does) (#7671)
(cherry picked from commit 64cceb12ad5982393f9b93557a2c7d3c77ffc946)
2020-09-01 09:27:58 +03:00
guybe7
4e19358d65 PERSIST should signalModifiedKey (Like EXPIRE does) (#7671)
(cherry picked from commit b87c288016c68716cbb97534b7748045ca58c953)
2020-09-01 09:27:58 +03:00
Oran Agra
299c092077 OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.

(cherry picked from commit 1b5cc94836d24b7b36cb6618644f9e2d60113c59)
2020-09-01 09:27:58 +03:00
Oran Agra
43bc1ea065 OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.

(cherry picked from commit 0f741a9e2de355211667c0f2ad38f7bead686c28)
2020-09-01 09:27:58 +03:00
Wen Hui
3e11f9d967 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.

(cherry picked from commit b7236f0002bcaa15f3a487def9c5069b6c422e65)
2020-09-01 09:27:58 +03:00
Wen Hui
3842bfd899 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.

(cherry picked from commit 93d87d6d4cd2aed9a45c4307b4c7b0b19a47b2e9)
2020-09-01 09:27:58 +03:00
Oran Agra
b091e0059d Trim trailing spaces in error replies coming from rejectCommand (#7668)
fe8d6fe749 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.

(cherry picked from commit 05a4af3464b16e42b31dfb1ea62e2a66dc032fb2)
2020-09-01 09:27:58 +03:00
Oran Agra
6dcd641681 Trim trailing spaces in error replies coming from rejectCommand (#7668)
65a3307bc9 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.

(cherry picked from commit cdd925b2898ac270afdf3d72f065410a96980f80)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
d13c44583c Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 2ec11f941ae41188e517670fc3224b12c7666541)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
1180537589 Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 64c360c5156ca6ee6d1eb52bfeb3fa48f3b25da5)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
3ed7d0af20 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.

(cherry picked from commit 24efd22e894c90f380aa05a5fa77134bb9423ad3)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
2e7ad58918 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.

(cherry picked from commit fb2a94af3fbb3f3cf8b26b8bd89387669cb111a1)
2020-09-01 09:27:58 +03:00
Madelyn Olson
e0f6c0de33 Fixed timer warning (#5953)
(cherry picked from commit 46ce35c55e3c6d5c3b2f43194de7694c27ad1d5d)
2020-09-01 09:27:58 +03:00
Madelyn Olson
39ffc3702f Fixed timer warning (#5953)
(cherry picked from commit 79c506ebf07db32bbcf84ede9d72dd9eb04e9812)
2020-09-01 09:27:58 +03:00
Wagner Francisco Mezaroba
4772370a87 allow --pattern to be used along with --bigkeys (#3586)
Adds --pattern option to cli's --bigkeys, --hotkeys & --scan modes

(cherry picked from commit 3ec7f8e915fb15634ea5babf217b8a8ecb3e3647)
2020-09-01 09:27:58 +03:00
Wagner Francisco Mezaroba
31415cb27a allow --pattern to be used along with --bigkeys (#3586)
Adds --pattern option to cli's --bigkeys, --hotkeys & --scan modes

(cherry picked from commit e2a71338ebfb719cba94742f1c75ce755cd6074d)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
a42e7f6876 redis-benchmark: fix wrong random key for hset (#4895)
(cherry picked from commit b1a242b84a7ff4aadf8c47ea83a658db9cf0e98b)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
840a9951b5 redis-benchmark: fix wrong random key for hset (#4895)
(cherry picked from commit ff1e4a7063c4de96bba3661f5f282ed61b6dfe8e)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
f573b23187 CLIENT_MASTER should ignore server.proto_max_bulk_len
(cherry picked from commit 0061dbba04918b3f62f943f469f41590d4200919)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
3c87d572b9 CLIENT_MASTER should ignore server.proto_max_bulk_len
(cherry picked from commit 589e610ebc62ced9c5392b7ece122c68c9990988)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
f0f4ef4423 config: proto-max-bulk-len must be 1mb or greater
(cherry picked from commit 46323cec56793c66273005b72f1f29146c108840)
2020-09-01 09:27:58 +03:00