9278 Commits

Author SHA1 Message Date
valentinogeron
e6f6731c66 EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag

(cherry picked from commit b7289e912cbe1a011a5569cd67929e83731b9660)
2020-09-01 09:27:58 +03:00
Itamar Haber
5a9a653f3e Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.

(cherry picked from commit 5b0a06af48997794af60dabb58ce4336ef56f73d)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
ff0846cbb7 Add language servers stuff, test/tls to gitignore. (#7698)
(cherry picked from commit 5449a2a8b54db6a57b564e453c9f7075ebb81f79)
2020-09-01 09:27:58 +03:00
Valentino Geron
3e2ed279fd Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.

(cherry picked from commit 8b428cf0f7ce5489fbdf4640a1dd36357f3dc5f0)
2020-09-01 09:27:58 +03:00
Valentino Geron
19ef1f371d Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```

(cherry picked from commit 9204a9b2c2f6eb59767ab0bddcde62c75e8c20b0)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
8d79702d8a Tests: fix redis-cli with remote hosts. (#7693)
(cherry picked from commit f80f3f492a0ca56e163899eeca7ad40d67d903be)
2020-09-01 09:27:58 +03:00
huangzhw
6cfcf76e0b RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.

(cherry picked from commit a3d4d7bf68bf825584b34785ed2b117dd24c1754)
2020-09-01 09:27:58 +03:00
guybe7
c9478a3b4e Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.

(cherry picked from commit 65c24bd3d436a08a680fa80bf5b3f4f9cf8ef395)
2020-09-01 09:27:58 +03:00
杨博东
113d5ae872 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit cbaf3c5bbafd43e009a2d6b38dd0e9fc450a3e12)
2020-09-01 09:27:58 +03:00
Raghav Muddur
1bd75aafdb Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.

(cherry picked from commit 34c3be365a3a8bc74f7a99b3e35cb211afaa58ca)
2020-09-01 09:27:58 +03:00
Madelyn Olson
ba48e7e9cd Fixed hset error since it's shared with hmset (#7678)
(cherry picked from commit cbd9af85838a73c9c9654829cf8e511ecc5853de)
2020-09-01 09:27:58 +03:00
guybe7
4e19358d65 PERSIST should signalModifiedKey (Like EXPIRE does) (#7671)
(cherry picked from commit b87c288016c68716cbb97534b7748045ca58c953)
2020-09-01 09:27:58 +03:00
Oran Agra
43bc1ea065 OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.

(cherry picked from commit 0f741a9e2de355211667c0f2ad38f7bead686c28)
2020-09-01 09:27:58 +03:00
Wen Hui
3842bfd899 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.

(cherry picked from commit 93d87d6d4cd2aed9a45c4307b4c7b0b19a47b2e9)
2020-09-01 09:27:58 +03:00
Oran Agra
6dcd641681 Trim trailing spaces in error replies coming from rejectCommand (#7668)
65a3307bc9 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.

(cherry picked from commit cdd925b2898ac270afdf3d72f065410a96980f80)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
1180537589 Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 64c360c5156ca6ee6d1eb52bfeb3fa48f3b25da5)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
2e7ad58918 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.

(cherry picked from commit fb2a94af3fbb3f3cf8b26b8bd89387669cb111a1)
2020-09-01 09:27:58 +03:00
Madelyn Olson
39ffc3702f Fixed timer warning (#5953)
(cherry picked from commit 79c506ebf07db32bbcf84ede9d72dd9eb04e9812)
2020-09-01 09:27:58 +03:00
Wagner Francisco Mezaroba
31415cb27a allow --pattern to be used along with --bigkeys (#3586)
Adds --pattern option to cli's --bigkeys, --hotkeys & --scan modes

(cherry picked from commit e2a71338ebfb719cba94742f1c75ce755cd6074d)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
840a9951b5 redis-benchmark: fix wrong random key for hset (#4895)
(cherry picked from commit ff1e4a7063c4de96bba3661f5f282ed61b6dfe8e)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
3c87d572b9 CLIENT_MASTER should ignore server.proto_max_bulk_len
(cherry picked from commit 589e610ebc62ced9c5392b7ece122c68c9990988)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
d3831559cf config: proto-max-bulk-len must be 1mb or greater
(cherry picked from commit bd4b33d7a2b00fddcaaccee9705d21b92d97311e)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
44ef19743e using proto-max-bulk-len in checkStringLength for SETRANGE and APPEND
(cherry picked from commit 2e69bfe44d154d6739da48ff87f1c66045f68e93)
2020-09-01 09:27:58 +03:00
YoongHM
171f83306e Start redis after network is online (#7639)
The two lines allow systemd to start redis.service after the network is online. Only after the network is online that Redis could bind to IP address other than 127.0.0.1 during initial boot up process.

(cherry picked from commit 8e937ce4cc1462d996bae6a45e8c0a66f71e7ee6)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
0314429c8f Run daily workflow on main repo only (no forks). (#7646)
(cherry picked from commit 944cca98c84efcf0365b0525f88124c53dbe1c53)
2020-09-01 09:27:58 +03:00
WuYunlong
086707315f see #7250, fix signature of RedisModule_DeauthenticateAndCloseClient (#7645)
In redismodule.h, RedisModule_DeauthenticateAndCloseClient returns void
`void REDISMODULE_API_FUNC(RedisModule_DeauthenticateAndCloseClient)(RedisModuleCtx *ctx, uint64_t client_id);`
But in module.c, RM_DeauthenticateAndCloseClient returns int
`int RM_DeauthenticateAndCloseClient(RedisModuleCtx *ctx, uint64_t client_id)`

It it safe to change return value from `void` to `int` from the user's perspective.

(cherry picked from commit d6220f12a9fafeb07ad86fa91e88383ca2a524c8)
2020-09-01 09:27:58 +03:00
Wang Yuan
9fd0c027e7 Print error info if failed opening config file (#6943)
(cherry picked from commit ea7eeb2fd2e7a73d951f394d5d14a5ba5a75d4b2)
2020-09-01 09:27:58 +03:00
Wen Hui
0e581e0044 fix memory leak in ACLLoadFromFile error handling (#7623)
(cherry picked from commit 3f67b0337833cbd2a2746dad7cd8411a31b75f70)
2020-09-01 09:27:58 +03:00
Oran Agra
e8801f7eaa redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`

(cherry picked from commit 1aa31e4da9a2daf9058dd923738e331bc5c91e31)
2020-09-01 09:27:58 +03:00
Frank Meier
3a677b45ac reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db

(cherry picked from commit 51077c821236eba5223b1e624f2462214a0e5062)
2020-09-01 09:27:58 +03:00
Frank Meier
dc46f6f041 add force option to 'create-cluster create' script call (#7612)
(cherry picked from commit 3244fae7d4f2e77f6d0d3f8280e852bd63753462)
2020-09-01 09:27:58 +03:00
Oran Agra
916b215fc5 fix new rdb test failing on timing issues (#7604)
apparenlty on github actions sometimes 500ms is not enough

(cherry picked from commit 824bd2ac11472b7a3fce9fcf3189a8e6c6048115)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
9f0e8f7849 Fix test-centos7-tls daily job. (#7598)
(cherry picked from commit f2da3efc78fe39c52fd46a2b9b5ed05e29647119)
2020-09-01 09:27:58 +03:00
Oran Agra
a5294c4e52 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail

(cherry picked from commit f7e77759902aa19cfa537ed454e6bc987498e8c5)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
e511ab2cc6 CI: Add daily CentOS 7.x jobs. (#7582)
(cherry picked from commit 4ac1f9ac55b841094861e84cdec5da0f4a294a86)
2020-09-01 09:27:58 +03:00
WuYunlong
7200b3aa0f Fix running single test 14-consistency-check.tcl (#7587)
(cherry picked from commit f3352daf4f4826e1cad4c163fd6e35b81a72e21b)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
e8efcdee0a Clarify RM_BlockClient() error condition. (#6093)
(cherry picked from commit 7af05f07ff72a4246ddaf986a1e815eadd0cbfef)
2020-09-01 09:27:58 +03:00
namtsui
2bcc056a41 Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
(cherry picked from commit 63dae5232415d216dfc1acce8b5335e20aa3b178)
2020-09-01 09:27:58 +03:00
Wen Hui
b2bd3c0653 Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562)
(cherry picked from commit f33acb3f029d7f5aa68e22e612036e04417e3e91)
2020-09-01 09:27:58 +03:00
Wen Hui
4238df5436 fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop

(cherry picked from commit c69a9b2f61e0747fabb8120f03c9e2a29b43b472)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
c68b3908a6 Fix TLS cluster tests. (#7578)
Fix consistency test added in af5167b7f without considering TLS
redis-cli configuration.

(cherry picked from commit bedf1b21269dfb8afb584bc24585023af8b2a208)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
1c05b87e30 TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
(cherry picked from commit 784ceeb90d84bbc49fc2f2e2e6c7b9fae2524bd5)
2020-09-01 09:27:58 +03:00
Oran Agra
67750ce3b3 Fix failing tests due to issues with wait_for_log_message (#7572)
- the test now waits for specific set of log messages rather than wait for
  timeout looking for just one message.
- we don't wanna sample the current length of the log after an action, due
  to a race, we need to start the search from the line number of the last
  message we where waiting for.
- when attempting to trigger a full sync, use multi-exec to avoid a race
  where the replica manages to re-connect before we completed the set of
  actions that should force a full sync.
- fix verify_log_message which was broken and unused

(cherry picked from commit 109b5ccdcd6e6b8cecdaeb13a246bc49ce7a61f4)
2020-09-01 09:27:58 +03:00
Jiayuan Chen
096285ab64 Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit f31260b0445f5649449da41555e1272a40ae4af7)
2020-09-01 09:27:58 +03:00
Oran Agra
0b2f7c6407 Daily github action: run cluster and sentinel tests with tls (#7575)
(cherry picked from commit 6d92eee69b8e693b9bf0a34320dd108b5a81a81a)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
8a4240b1fa TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.

(cherry picked from commit c75512d89d1697cf782cdc826acffab5b6adc1c7)
2020-09-01 09:27:58 +03:00
grishaf
395d339234 Fix prepareForShutdown function declaration (#7566)
(cherry picked from commit 4126ca466fb9fbf0d84a66e31c9f59f28e2fbff6)
2020-09-01 09:27:58 +03:00
Oran Agra
6daa8b9adb Stabilize bgsave test that sometimes fails with valgrind (#7559)
on ci.redis.io the test fails a lot, reporting that bgsave didn't end.
increaseing the timeout we wait for that bgsave to get aborted.
in addition to that, i also verify that it indeed got aborted by
checking that the save counter wasn't reset.

add another test to verify that a successful bgsave indeed resets the
change counter.

(cherry picked from commit 8a57969fd75db01b881d438200911d95bdead293)
2020-09-01 09:27:58 +03:00
Madelyn Olson
e00b6248c9 Properly reset errno for rdbLoad (#7542)
(cherry picked from commit 818dc3a0898125d621288df671994eed34f849f2)
2020-09-01 09:27:58 +03:00
Oran Agra
e8aa5583d0 testsuite may leave servers alive on error (#7549)
in cases where you have
test name {
  start_server {
    start_server {
      assert
    }
  }
}

the exception will be thrown to the test proc, and the servers are
supposed to be killed on the way out. but it seems there was always a
bug of not cleaning the server stack, and recently (#7404) we started
relying on that stack in order to kill them, so with that bug sometimes
we would have tried to kill the same server twice, and leave one alive.

luckly, in most cases the pattern is:
start_server {
  test name {
  }
}

(cherry picked from commit 36b949438547eb5bf8555fcac2c5040528fd7854)
2020-09-01 09:27:58 +03:00