9327 Commits

Author SHA1 Message Date
Itamar Haber
c77e8e2f1c Expands lazyfree's effort estimate to include Streams (#5794)
Otherwise, it is treated as a single allocation and freed synchronously. The following logic is used for estimating the effort in constant-ish time complexity:

1. Check the number of nodes.
1. Add an allocation for each consumer group registered inside the stream.
1. Check the number of PELs in the first CG, and then add this count times the number of CGs.
1. Check the number of consumers in the first CG, and then add this count times the number of CGs.

(cherry picked from commit cb504d7fddd09149655e91496588c610e89ca131)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
fe6da3c36b Add language servers stuff, test/tls to gitignore. (#7698)
(cherry picked from commit 74d9d9544969fed000bde5f8504b5b3f211e473f)
2020-09-01 09:27:58 +03:00
Valentino Geron
d727f52723 Assert that setDeferredAggregateLen isn't called with negative value
In case the redis is about to return broken reply we want to crash
with assert so that we are notified about the bug. see #7687.

(cherry picked from commit 7e6c9ef8819a071679f8dd18035dbbe2455c7b12)
2020-09-01 09:27:58 +03:00
Valentino Geron
34124fff88 Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```

(cherry picked from commit 7a555da64f56a4fb2f300d84a35778bee8f471ca)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
c5675c66bc Tests: fix redis-cli with remote hosts. (#7693)
(cherry picked from commit 257f9f462f7782dcaecf7bbf35f4701b20b88a45)
2020-09-01 09:27:58 +03:00
huangzhw
907b0124e8 RedisModuleEvent_LoadingProgress always at 100% progress (#7685)
It was also using the wrong struct, but luckily RedisModuleFlushInfo and RedisModuleLoadingProgress
are identical.

(cherry picked from commit b980e999293e9214a844712f9c88ca69acd20b1b)
2020-09-01 09:27:58 +03:00
guybe7
d0b8583167 Modules: Invalidate saved_oparray after use (#7688)
We wanna avoid a chance of someone using the pointer in it after it'll be freed / realloced.

(cherry picked from commit 4de17eb032160c7ba94c505eab4b776a456e5117)
2020-09-01 09:27:58 +03:00
杨博东
b42976bd56 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 5e6212e087c4696abc682b64079202c9ade8666c)
2020-09-01 09:27:58 +03:00
Raghav Muddur
3564aab1a5 Update clusterMsgDataPublish to clusterMsgModule (#7682)
Correcting the variable to clusterMsgModule.

(cherry picked from commit f7cf24b5da539dbf4fafc404355a32bac06aa2f5)
2020-09-01 09:27:58 +03:00
Madelyn Olson
f80b756076 Fixed hset error since it's shared with hmset (#7678)
(cherry picked from commit 17c6b3c7eeb486a75d05ba7fb3c4490b7abbc597)
2020-09-01 09:27:58 +03:00
guybe7
d74a98eaac PERSIST should signalModifiedKey (Like EXPIRE does) (#7671)
(cherry picked from commit 64cceb12ad5982393f9b93557a2c7d3c77ffc946)
2020-09-01 09:27:58 +03:00
Oran Agra
299c092077 OOM Crash log include size of allocation attempt. (#7670)
Since users often post just the crash log in github issues, the log
print that's above it is missing.
No reason not to include the size in the panic message itself.

(cherry picked from commit 1b5cc94836d24b7b36cb6618644f9e2d60113c59)
2020-09-01 09:27:58 +03:00
Wen Hui
3e11f9d967 [module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.

(cherry picked from commit b7236f0002bcaa15f3a487def9c5069b6c422e65)
2020-09-01 09:27:58 +03:00
Oran Agra
b091e0059d Trim trailing spaces in error replies coming from rejectCommand (#7668)
fe8d6fe749 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.

(cherry picked from commit 05a4af3464b16e42b31dfb1ea62e2a66dc032fb2)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
d13c44583c Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 2ec11f941ae41188e517670fc3224b12c7666541)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
3ed7d0af20 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.

(cherry picked from commit 24efd22e894c90f380aa05a5fa77134bb9423ad3)
2020-09-01 09:27:58 +03:00
Madelyn Olson
e0f6c0de33 Fixed timer warning (#5953)
(cherry picked from commit 46ce35c55e3c6d5c3b2f43194de7694c27ad1d5d)
2020-09-01 09:27:58 +03:00
Wagner Francisco Mezaroba
4772370a87 allow --pattern to be used along with --bigkeys (#3586)
Adds --pattern option to cli's --bigkeys, --hotkeys & --scan modes

(cherry picked from commit 3ec7f8e915fb15634ea5babf217b8a8ecb3e3647)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
a42e7f6876 redis-benchmark: fix wrong random key for hset (#4895)
(cherry picked from commit b1a242b84a7ff4aadf8c47ea83a658db9cf0e98b)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
f573b23187 CLIENT_MASTER should ignore server.proto_max_bulk_len
(cherry picked from commit 0061dbba04918b3f62f943f469f41590d4200919)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
f0f4ef4423 config: proto-max-bulk-len must be 1mb or greater
(cherry picked from commit 46323cec56793c66273005b72f1f29146c108840)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
0d78615532 using proto-max-bulk-len in checkStringLength for SETRANGE and APPEND
(cherry picked from commit 912b48e4b9d9a0738ab3010c4f30e47a3beec3fc)
2020-09-01 09:27:58 +03:00
YoongHM
5e0a3e2626 Start redis after network is online (#7639)
The two lines allow systemd to start redis.service after the network is online. Only after the network is online that Redis could bind to IP address other than 127.0.0.1 during initial boot up process.

(cherry picked from commit 1c6caaaef7d69e05b526626f284bb822fc82dd9f)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
9c7c0c511a Run daily workflow on main repo only (no forks). (#7646)
(cherry picked from commit b8f0c2de4aa76a5ba86a6970b7ab550a35ff12c6)
2020-09-01 09:27:58 +03:00
WuYunlong
5240ad73ef see #7250, fix signature of RedisModule_DeauthenticateAndCloseClient (#7645)
In redismodule.h, RedisModule_DeauthenticateAndCloseClient returns void
`void REDISMODULE_API_FUNC(RedisModule_DeauthenticateAndCloseClient)(RedisModuleCtx *ctx, uint64_t client_id);`
But in module.c, RM_DeauthenticateAndCloseClient returns int
`int RM_DeauthenticateAndCloseClient(RedisModuleCtx *ctx, uint64_t client_id)`

It it safe to change return value from `void` to `int` from the user's perspective.

(cherry picked from commit dd08aec539f8a8b35ccb8b8c953a76c07c739d62)
2020-09-01 09:27:58 +03:00
Wang Yuan
753ff31fa3 Print error info if failed opening config file (#6943)
(cherry picked from commit 514a1e223df5b62cd1fc6aff38126d4a74ebef7e)
2020-09-01 09:27:58 +03:00
Wen Hui
9b7da52c3b fix memory leak in ACLLoadFromFile error handling (#7623)
(cherry picked from commit 30ead1edaee1cea93ee5cdeb975880df12b41f5c)
2020-09-01 09:27:58 +03:00
Oran Agra
6f6658ab60 redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`

(cherry picked from commit f519dcb21626e5fd214960c7d83ee4fab7a3929d)
2020-09-01 09:27:58 +03:00
Frank Meier
d5f7ec90bc reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db

(cherry picked from commit c6ac2588db27de9a71effa6dc21417f7c787ac69)
2020-09-01 09:27:58 +03:00
Frank Meier
05e54ec962 add force option to 'create-cluster create' script call (#7612)
(cherry picked from commit c8f3182f378e069f8be0bbc333ae9c5e8682e804)
2020-09-01 09:27:58 +03:00
Oran Agra
b4a6b4f28d fix new rdb test failing on timing issues (#7604)
apparenlty on github actions sometimes 500ms is not enough

(cherry picked from commit 191b1181023b0860ec60afde7a41bd4f03c55097)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
dd94ca7295 Fix test-centos7-tls daily job. (#7598)
(cherry picked from commit e97cec2f28ca37b2f4d1a56fa4ff0b69c8c8fe62)
2020-09-01 09:27:58 +03:00
Oran Agra
3a4ee4b6d6 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail

(cherry picked from commit c5d85c69c75438f98f84e549877c2999a2e450a8)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
0dbf112caf CI: Add daily CentOS 7.x jobs. (#7582)
(cherry picked from commit 92e089b1ab47897e170aab6e95341ae789ec77d5)
2020-09-01 09:27:58 +03:00
WuYunlong
37fba8f4d8 Fix running single test 14-consistency-check.tcl (#7587)
(cherry picked from commit be11e1b5eaf0d6ab5e68f86c1346570531eee766)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
2428b64d5a Clarify RM_BlockClient() error condition. (#6093)
(cherry picked from commit 342d9a642f2d21fe906bf1be24d841fa6f314601)
2020-09-01 09:27:58 +03:00
namtsui
3d98418f4a Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
(cherry picked from commit 8c03eb90da3951ed92d8d2729fc6d2ce4feb9e45)
2020-09-01 09:27:58 +03:00
Wen Hui
46686af7f5 Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562)
(cherry picked from commit 0a2b019b79831e6c593c2de943af61ceef4327e1)
2020-09-01 09:27:58 +03:00
Wen Hui
9f873c6c2a fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop

(cherry picked from commit 2afa308306fc641204f10a2bbe2fe35e28b6d259)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
2e6563cdcf Fix TLS cluster tests. (#7578)
Fix consistency test added in 0c9916d00 without considering TLS
redis-cli configuration.

(cherry picked from commit 675b00c7e0b7d68bafa11fcc7f66a394c3c3cd36)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
ebdfa71584 TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
(cherry picked from commit bc450c5f63d39d0f0b8c97fa91d15bb8d688b86d)
2020-09-01 09:27:58 +03:00
Oran Agra
10a8407a4f Fix failing tests due to issues with wait_for_log_message (#7572)
- the test now waits for specific set of log messages rather than wait for
  timeout looking for just one message.
- we don't wanna sample the current length of the log after an action, due
  to a race, we need to start the search from the line number of the last
  message we where waiting for.
- when attempting to trigger a full sync, use multi-exec to avoid a race
  where the replica manages to re-connect before we completed the set of
  actions that should force a full sync.
- fix verify_log_message which was broken and unused

(cherry picked from commit 06aaeabaea9d9b248e8a790dde352cd14d66628a)
2020-09-01 09:27:58 +03:00
Jiayuan Chen
7b2af98316 Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit 198770751fdc4c46eb4971ead9b5787fd6ce39fd)
2020-09-01 09:27:58 +03:00
Oran Agra
d2ef4c0347 Daily github action: run cluster and sentinel tests with tls (#7575)
(cherry picked from commit 62e84b42d25a716411b108290f2caaff4b837488)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
ba559dc903 TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.

(cherry picked from commit b76a93c362091daafd8a8d15a45d527b7437d013)
2020-09-01 09:27:58 +03:00
grishaf
53f36dc5b6 Fix prepareForShutdown function declaration (#7566)
(cherry picked from commit f8751d03ba9635064c89844d5915c2ec2dcdc827)
2020-09-01 09:27:58 +03:00
Oran Agra
558a343b3c Stabilize bgsave test that sometimes fails with valgrind (#7559)
on ci.redis.io the test fails a lot, reporting that bgsave didn't end.
increaseing the timeout we wait for that bgsave to get aborted.
in addition to that, i also verify that it indeed got aborted by
checking that the save counter wasn't reset.

add another test to verify that a successful bgsave indeed resets the
change counter.

(cherry picked from commit 49d4aebce0a0b94cd2b302d276be95d1a1ce8610)
2020-09-01 09:27:58 +03:00
Madelyn Olson
32d225c644 Properly reset errno for rdbLoad (#7542)
(cherry picked from commit 9615c7480de53c920baadf6279b527b60de8f0d4)
2020-09-01 09:27:58 +03:00
Oran Agra
2b45c88a6a testsuite may leave servers alive on error (#7549)
in cases where you have
test name {
  start_server {
    start_server {
      assert
    }
  }
}

the exception will be thrown to the test proc, and the servers are
supposed to be killed on the way out. but it seems there was always a
bug of not cleaning the server stack, and recently (#7404) we started
relying on that stack in order to kill them, so with that bug sometimes
we would have tried to kill the same server twice, and leave one alive.

luckly, in most cases the pattern is:
start_server {
  test name {
  }
}

(cherry picked from commit bb170fa06e5909dd816b6530121952d57c8209a0)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
6d80011e73 Tests: drop TCL 8.6 dependency. (#7548)
This re-implements the redis-cli --pipe test so it no longer depends on a close feature available only in TCL 8.6.

Basically what this test does is run redis-cli --pipe, generates a bunch of commands and pipes them through redis-cli, and inspects the result in both Redis and the redis-cli output.

To do that, we need to close stdin for redis-cli to indicate we're done so it can flush its buffers and exit. TCL has bi-directional channels can only offers a way to "one-way close" a channel with TCL 8.6. To work around that, we now generate the commands into a file and feed that file to redis-cli directly.

As we're writing to an actual file, the number of commands is now reduced.

(cherry picked from commit dbc0a64843ccd07515ac41ca80497a9e5ffd107a)
2020-09-01 09:27:58 +03:00