27578 Commits

Author SHA1 Message Date
Oran Agra
9fcd9e191e
Fix rejectCommand trims newline in shared error objects, hung clients (#7714)
65a3307bc (released in 6.0.6) has a side effect, when processCommand
rejects a command with pre-made shared object error string, it trims the
newlines from the end of the string. if that string is later used with
addReply, the newline will be missing, breaking the protocol, and
leaving the client hung.

It seems that the only scenario which this happens is when replying with
-LOADING to some command, and later using that reply from the CONFIG
SET command (still during loading). this will result in hung client.

Refactoring the code in order to avoid trimming these newlines from
shared string objects, and do the newline trimming only in other cases
where it's needed.

Co-authored-by: Guy Benoish <guy.benoish@redislabs.com>
2020-08-27 12:54:01 +03:00
Oran Agra
8a4d10a3a5 Update memory metrics for INFO during loading (#7690)
During a long AOF or RDB loading, the memory stats were not updated, and
INFO would return stale data, specifically about fragmentation and RSS.
In the past some of these were sampled directly inside the INFO command,
but were moved to cron as an optimization.

This commit introduces a concept of loadingCron which should take
some of the responsibilities of serverCron.
It attempts to limit it's rate to approximately the server Hz, but may
not be very accurate.

In order to avoid too many system call, we use the cached ustime, and
also make sure to update it in both AOF loading and RDB loading inside
processEventsWhileBlocked (it seems AOF loading was missing it).
2020-08-27 11:09:32 +03:00
Oran Agra
8bdcbbb085
Update memory metrics for INFO during loading (#7690)
During a long AOF or RDB loading, the memory stats were not updated, and
INFO would return stale data, specifically about fragmentation and RSS.
In the past some of these were sampled directly inside the INFO command,
but were moved to cron as an optimization.

This commit introduces a concept of loadingCron which should take
some of the responsibilities of serverCron.
It attempts to limit it's rate to approximately the server Hz, but may
not be very accurate.

In order to avoid too many system call, we use the cached ustime, and
also make sure to update it in both AOF loading and RDB loading inside
processEventsWhileBlocked (it seems AOF loading was missing it).
2020-08-27 11:09:32 +03:00
valentinogeron
0292720ccb EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag
2020-08-27 09:19:24 +03:00
valentinogeron
b7289e912c
EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag
2020-08-27 09:19:24 +03:00
Oran Agra
b01816ca6e Add test coverage for CLIENT UNBLOCK (#7712)
plus minor other fixes to list.tcl
2020-08-27 08:09:39 +03:00
Oran Agra
daef1f00c2
Add test coverage for CLIENT UNBLOCK (#7712)
plus minor other fixes to list.tcl
2020-08-27 08:09:39 +03:00
filipe oliveira
2693e8f245 Extended redis-benchmark instant metrics and overall latency report (#7600)
A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
2020-08-25 21:21:29 +03:00
filipe oliveira
21784def70
Extended redis-benchmark instant metrics and overall latency report (#7600)
A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
2020-08-25 21:21:29 +03:00
Itamar Haber
cb504d7fdd Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.
2020-08-25 15:58:50 +03:00
Itamar Haber
5b0a06af48
Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.
2020-08-25 15:58:50 +03:00
Wang Yuan
48a00e6b99 Fix wrong format specifiers of 'sdscatfmt' for the INFO command (#7706)
unlike printf, sdscatfmt doesn't take %d
2020-08-24 22:59:56 +03:00
Wang Yuan
43af28f5b4
Fix wrong format specifiers of 'sdscatfmt' for the INFO command (#7706)
unlike printf, sdscatfmt doesn't take %d
2020-08-24 22:59:56 +03:00
Wang Yuan
959099a969 Fix data race in bugReportStart (#7700)
The previous fix using _Atomic was insufficient, since we check and set it in
different places.

The implications of this bug are just that a portion of the bug report will be shown
twice, in the race case of two concurrent crashes.
2020-08-24 13:54:33 +03:00
Wang Yuan
6b4ae919e8
Fix data race in bugReportStart (#7700)
The previous fix using _Atomic was insufficient, since we check and set it in
different places.

The implications of this bug are just that a portion of the bug report will be shown
twice, in the race case of two concurrent crashes.
2020-08-24 13:54:33 +03:00
Yossi Gottlieb
74d9d95449 Add language servers stuff, test/tls to gitignore. (#7698) 2020-08-24 12:54:56 +03:00
Yossi Gottlieb
5449a2a8b5
Add language servers stuff, test/tls to gitignore. (#7698) 2020-08-24 12:54:56 +03:00
John Sully
ff9df842d8 Implement use-fork config (fails with diskless repl)
Former-commit-id: f2d5c2bca22e9fd506db123c47b7f60cdded7e2c
2020-08-24 03:17:59 +00:00
John Sully
d1f45b8ebf Implement use-fork config (fails with diskless repl)
Former-commit-id: f2d5c2bca22e9fd506db123c47b7f60cdded7e2c
2020-08-24 03:17:59 +00:00
John Sully
91bd125fb4 Implement use-fork config (fails with diskless repl)
Former-commit-id: 4e941b8d30612c1051a9dde67effaf8c5a86456f
2020-08-24 03:17:31 +00:00
John Sully
994e758924 Implement use-fork config (fails with diskless repl)
Former-commit-id: 4e941b8d30612c1051a9dde67effaf8c5a86456f
2020-08-24 03:17:31 +00:00
Valentino Geron
7e6c9ef881 Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.
2020-08-23 16:03:30 +03:00
Valentino Geron
8b428cf0f7 Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.
2020-08-23 16:03:30 +03:00
Valentino Geron
7a555da64f Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```
2020-08-23 16:03:30 +03:00
Valentino Geron
9204a9b2c2 Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```
2020-08-23 16:03:30 +03:00
Yossi Gottlieb
257f9f462f Tests: fix redis-cli with remote hosts. (#7693) 2020-08-23 10:17:43 +03:00
Yossi Gottlieb
f80f3f492a
Tests: fix redis-cli with remote hosts. (#7693) 2020-08-23 10:17:43 +03:00
Wen Hui
7386b998e8 fix make warnings (#7692) 2020-08-21 23:37:49 +03:00
Wen Hui
e61adc0d89
fix make warnings (#7692) 2020-08-21 23:37:49 +03:00
Wen Hui
86cd4629ae use dictSlots for getting total slots number in dict (#7691) 2020-08-21 00:14:09 +03:00
Wen Hui
89f2bfbb58
use dictSlots for getting total slots number in dict (#7691) 2020-08-21 00:14:09 +03:00
huangzhw
b980e99929 RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.
2020-08-20 23:31:06 +03:00
huangzhw
a3d4d7bf68
RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.
2020-08-20 23:31:06 +03:00
guybe7
4de17eb032 Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.
2020-08-20 19:55:14 +03:00
guybe7
65c24bd3d4
Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.
2020-08-20 19:55:14 +03:00
杨博东
5e6212e087 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-08-20 08:59:02 +03:00
杨博东
cbaf3c5bba
Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-08-20 08:59:02 +03:00
Raghav Muddur
f7cf24b5da Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.
2020-08-19 19:13:32 -07:00
Raghav Muddur
34c3be365a
Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.
2020-08-19 19:13:32 -07:00
Madelyn Olson
17c6b3c7ee Fixed hset error since it's shared with hmset (#7678) 2020-08-19 19:07:43 -07:00
Madelyn Olson
cbd9af8583
Fixed hset error since it's shared with hmset (#7678) 2020-08-19 19:07:43 -07:00
Wang Yuan
77155c4744 Add comments on 'slave.repldboff' when use diskless replication (#7679) 2020-08-19 10:52:53 +03:00
Wang Yuan
89d544d6f2
Add comments on 'slave.repldboff' when use diskless replication (#7679) 2020-08-19 10:52:53 +03:00
guybe7
64cceb12ad PERSIST should signalModifiedKey (Like EXPIRE does) (#7671) 2020-08-18 19:07:59 +03:00
guybe7
b87c288016
PERSIST should signalModifiedKey (Like EXPIRE does) (#7671) 2020-08-18 19:07:59 +03:00
Oran Agra
1b5cc94836 OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.
2020-08-18 09:53:59 +03:00
Oran Agra
0f741a9e2d
OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.
2020-08-18 09:53:59 +03:00
Wen Hui
3962f80bd7 edit auth failed message (#7648)
Edit auth failed message include user disabled case in hello command
2020-08-18 08:59:24 +03:00
Wen Hui
88662c243d
edit auth failed message (#7648)
Edit auth failed message include user disabled case in hello command
2020-08-18 08:59:24 +03:00
Wen Hui
b7236f0002 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.
2020-08-18 08:50:03 +03:00