12229 Commits

Author SHA1 Message Date
Wen Hui
0e581e0044 fix memory leak in ACLLoadFromFile error handling (#7623)
(cherry picked from commit 3f67b0337833cbd2a2746dad7cd8411a31b75f70)
2020-09-01 09:27:58 +03:00
Oran Agra
e8801f7eaa redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`

(cherry picked from commit 1aa31e4da9a2daf9058dd923738e331bc5c91e31)
2020-09-01 09:27:58 +03:00
Frank Meier
3a677b45ac reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db

(cherry picked from commit 51077c821236eba5223b1e624f2462214a0e5062)
2020-09-01 09:27:58 +03:00
Frank Meier
dc46f6f041 add force option to 'create-cluster create' script call (#7612)
(cherry picked from commit 3244fae7d4f2e77f6d0d3f8280e852bd63753462)
2020-09-01 09:27:58 +03:00
Oran Agra
916b215fc5 fix new rdb test failing on timing issues (#7604)
apparenlty on github actions sometimes 500ms is not enough

(cherry picked from commit 824bd2ac11472b7a3fce9fcf3189a8e6c6048115)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
9f0e8f7849 Fix test-centos7-tls daily job. (#7598)
(cherry picked from commit f2da3efc78fe39c52fd46a2b9b5ed05e29647119)
2020-09-01 09:27:58 +03:00
Oran Agra
a5294c4e52 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail

(cherry picked from commit f7e77759902aa19cfa537ed454e6bc987498e8c5)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
e511ab2cc6 CI: Add daily CentOS 7.x jobs. (#7582)
(cherry picked from commit 4ac1f9ac55b841094861e84cdec5da0f4a294a86)
2020-09-01 09:27:58 +03:00
WuYunlong
7200b3aa0f Fix running single test 14-consistency-check.tcl (#7587)
(cherry picked from commit f3352daf4f4826e1cad4c163fd6e35b81a72e21b)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
e8efcdee0a Clarify RM_BlockClient() error condition. (#6093)
(cherry picked from commit 7af05f07ff72a4246ddaf986a1e815eadd0cbfef)
2020-09-01 09:27:58 +03:00
namtsui
2bcc056a41 Avoid an out-of-bounds read in the redis-sentinel (#7443)
The Redis sentinel would crash with a segfault after a few minutes
because it tried to read from a page without read permissions. Check up
front whether the sds is long enough to contain redis:slave or
redis:master before memcmp() as is done everywhere else in
sentinelRefreshInstanceInfo().

Bug report and commit message from Theo Buehler. Fix from Nam Nguyen.

Co-authored-by: Nam Nguyen <namn@berkeley.edu>
(cherry picked from commit 63dae5232415d216dfc1acce8b5335e20aa3b178)
2020-09-01 09:27:58 +03:00
Wen Hui
b2bd3c0653 Add SignalModifiedKey hook in XGROUP CREATE with MKSTREAM option (#7562)
(cherry picked from commit f33acb3f029d7f5aa68e22e612036e04417e3e91)
2020-09-01 09:27:58 +03:00
Wen Hui
4238df5436 fix leak in error handling of debug populate command (#7062)
valsize was not modified during the for loop below instead of getting from c->argv[4], therefore there is no need to put inside the for loop.. Moreover, putting the check outside loop will also avoid memory leaking, decrRefCount(key) should be called in the original code if we put the check in for loop

(cherry picked from commit c69a9b2f61e0747fabb8120f03c9e2a29b43b472)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
c68b3908a6 Fix TLS cluster tests. (#7578)
Fix consistency test added in af5167b7f without considering TLS
redis-cli configuration.

(cherry picked from commit bedf1b21269dfb8afb584bc24585023af8b2a208)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
1c05b87e30 TLS: Propagate and handle SSL_new() failures. (#7576)
The connection API may create an accepted connection object in an error
state, and callers are expected to check it before attempting to use it.

Co-authored-by: mrpre <mrpre@163.com>
(cherry picked from commit 784ceeb90d84bbc49fc2f2e2e6c7b9fae2524bd5)
2020-09-01 09:27:58 +03:00
Oran Agra
67750ce3b3 Fix failing tests due to issues with wait_for_log_message (#7572)
- the test now waits for specific set of log messages rather than wait for
  timeout looking for just one message.
- we don't wanna sample the current length of the log after an action, due
  to a race, we need to start the search from the line number of the last
  message we where waiting for.
- when attempting to trigger a full sync, use multi-exec to avoid a race
  where the replica manages to re-connect before we completed the set of
  actions that should force a full sync.
- fix verify_log_message which was broken and unused

(cherry picked from commit 109b5ccdcd6e6b8cecdaeb13a246bc49ce7a61f4)
2020-09-01 09:27:58 +03:00
Jiayuan Chen
096285ab64 Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit f31260b0445f5649449da41555e1272a40ae4af7)
2020-09-01 09:27:58 +03:00
Oran Agra
0b2f7c6407 Daily github action: run cluster and sentinel tests with tls (#7575)
(cherry picked from commit 6d92eee69b8e693b9bf0a34320dd108b5a81a81a)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
8a4240b1fa TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.

(cherry picked from commit c75512d89d1697cf782cdc826acffab5b6adc1c7)
2020-09-01 09:27:58 +03:00
grishaf
395d339234 Fix prepareForShutdown function declaration (#7566)
(cherry picked from commit 4126ca466fb9fbf0d84a66e31c9f59f28e2fbff6)
2020-09-01 09:27:58 +03:00
Oran Agra
6daa8b9adb Stabilize bgsave test that sometimes fails with valgrind (#7559)
on ci.redis.io the test fails a lot, reporting that bgsave didn't end.
increaseing the timeout we wait for that bgsave to get aborted.
in addition to that, i also verify that it indeed got aborted by
checking that the save counter wasn't reset.

add another test to verify that a successful bgsave indeed resets the
change counter.

(cherry picked from commit 8a57969fd75db01b881d438200911d95bdead293)
2020-09-01 09:27:58 +03:00
Madelyn Olson
e00b6248c9 Properly reset errno for rdbLoad (#7542)
(cherry picked from commit 818dc3a0898125d621288df671994eed34f849f2)
2020-09-01 09:27:58 +03:00
Oran Agra
e8aa5583d0 testsuite may leave servers alive on error (#7549)
in cases where you have
test name {
  start_server {
    start_server {
      assert
    }
  }
}

the exception will be thrown to the test proc, and the servers are
supposed to be killed on the way out. but it seems there was always a
bug of not cleaning the server stack, and recently (#7404) we started
relying on that stack in order to kill them, so with that bug sometimes
we would have tried to kill the same server twice, and leave one alive.

luckly, in most cases the pattern is:
start_server {
  test name {
  }
}

(cherry picked from commit 36b949438547eb5bf8555fcac2c5040528fd7854)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
f1d5d5d28e Tests: drop TCL 8.6 dependency. (#7548)
This re-implements the redis-cli --pipe test so it no longer depends on a close feature available only in TCL 8.6.

Basically what this test does is run redis-cli --pipe, generates a bunch of commands and pipes them through redis-cli, and inspects the result in both Redis and the redis-cli output.

To do that, we need to close stdin for redis-cli to indicate we're done so it can flush its buffers and exit. TCL has bi-directional channels can only offers a way to "one-way close" a channel with TCL 8.6. To work around that, we now generate the commands into a file and feed that file to redis-cli directly.

As we're writing to an actual file, the number of commands is now reduced.

(cherry picked from commit f57e844b2edbb86a5df2f3436045814812c0a3ae)
2020-09-01 09:27:58 +03:00
Oran Agra
cc7f4ba67e Fixes to release scripts (#7547)
(cherry picked from commit 343dd9bcce0ce28d131510e28bac411599da90b0)
2020-09-01 09:27:58 +03:00
Remi Collet
af907e4b6d Fix deprecated tail syntax in tests (#7543)
(cherry picked from commit 3f2fbc4c614ff718dce7d55fd971d7ed36062c24)
2020-09-01 09:27:58 +03:00
Wen Hui
d2f09a054c Add missing calls to raxStop (#7532)
Since the dynamic allocations in raxIterator are only used for deep walks, memory
leak due to missing call to raxStop can only happen for rax with key names longer
than 32 bytes.

Out of all the missing calls, the only ones that may lead to a leak are the rax
for consumer groups and consumers, and these were only in AOFRW and rdbSave, which
normally only happen in fork or at shutdown.

(cherry picked from commit 4e8f2d6881a38397bfbf0d7d161959163a5f6e88)
2020-09-01 09:27:58 +03:00
Wen Hui
cef46edebc add missing caching command in client help (#7399)
(cherry picked from commit 2fbd0271f6fc0a93b9323b6478ec1a7a4ca77614)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
9e997cd0d1 replication: need handle -NOPERM error after send ping (#7538)
(cherry picked from commit 13e50935a84c319763a0d3b2be6ce64962092541)
2020-09-01 09:27:58 +03:00
Scott Brenner
171aa22b0b GitHub Actions workflows - use latest version of actions/checkout (#7534)
(cherry picked from commit 2f4e9c3f9f38e196fe4a03232c97782cbf8b3702)
2020-09-01 09:27:58 +03:00
John Sully
e87dee8dc7 Add build flag to disable MVCC tstamps
Former-commit-id: f17d178d03f44abcdaddd851a313dd3f7ec87ed5
2020-08-10 06:10:24 +00:00
John Sully
e06152eb13 Fix lock after free in module API
Former-commit-id: d88fd1588d292bffc0aa53c299cb52e7a4e91015
2020-08-10 06:10:24 +00:00
Oran Agra
7bf665f125 Redis 6.0.6. 2020-07-20 21:08:26 +03:00
Oran Agra
a5696bdf4f Run daily CI on PRs to release a branch 2020-07-20 21:08:26 +03:00
Itamar Haber
e15528bf1d Adds SHA256SUM to redis-stable tarball upload
(cherry picked from commit 5df0a64d30e7815c0a4a75a80f165fdee0bd1db6)
2020-07-20 21:08:26 +03:00
yoav-steinberg
e28aa99af1 Support passing stack allocated module strings to moduleCreateArgvFromUserFormat (#7528)
Specifically, the key passed to the module aof_rewrite callback is a stack allocated robj. When passing it to RedisModule_EmitAOF (with appropriate "s" fmt string) redis used to panic when trying to inc the ref count of the stack allocated robj. Now support such robjs by coying them to a new heap robj. This doesn't affect performance because using the alternative "c" or "b" format strings also copies the input to a new heap robj.

(cherry picked from commit d484b8a04ed67e79030fcb060e88641acb6e4f98)
2020-07-20 21:08:26 +03:00
Luke Palmer
3051430040 Send null for invalidate on flush (#7469)
(cherry picked from commit 5f716ea467d29059a89f90b6ccbdee5a60443200)
2020-07-20 21:08:26 +03:00
dmurnane
29b20fd528 Notify systemd on sentinel startup (#7168)
Co-authored-by: Daniel Murnane <dmurnane@eitccorp.com>
(cherry picked from commit 9242ccf238cbed018eb3a7fa3a437618345dd52b)
2020-07-20 21:08:26 +03:00
Developer-Ecosystem-Engineering
5b36681213 Add registers dump support for Apple silicon (#7453)
Export following environment variables before building on macOS on Apple silicon

export ARCH_FLAGS="-arch arm64"
export SDK_NAME=macosx
export SDK_PATH=$(xcrun --show-sdk-path --sdk $SDK_NAME)
export CFLAGS="$ARCH_FLAGS -isysroot $SDK_PATH -I$SDK_PATH/usr/include"
export CXXFLAGS=$CFLAGS
export LDFLAGS="$ARCH_FLAGS"
export CC="$(xcrun -sdk $SDK_PATH --find clang) $CFLAGS"
export CXX="$(xcrun -sdk $SDK_PATH --find clang++) $CXXFLAGS"
export LD="$(xcrun -sdk $SDK_PATH --find ld) $LDFLAGS"

make
make test
..
All tests passed without errors!

Backtrack logging assumes x86 and required updating

(cherry picked from commit c2b5f1c15bbc5363f92f8e697538759c5d929934)
2020-07-20 21:08:26 +03:00
Wen Hui
0f75036c07 correct error msg for num connections reaching maxclients in cluster mode (#7444)
(cherry picked from commit d85af4d6f5fbe9cb9787b81583627cd74b47f838)
2020-07-20 21:08:26 +03:00
WuYunlong
b1a01fda91 Fix command help for unexpected options (#7476)
(cherry picked from commit 93bdbf5aa4857ede0816cf790f951da8e2fa2ae9)
2020-07-20 21:08:26 +03:00
WuYunlong
83f55f61a6 Refactor RM_KeyType() by using macro. (#7486)
(cherry picked from commit dc690161d5652d86d51bd209821bfb0e9c5f7ec2)
2020-07-20 21:08:26 +03:00
Oran Agra
f89f50dbd0 diskless master disconnect replicas when rdb child failed (#7518)
in case the rdb child failed, crashed or terminated unexpectedly redis
would have marked the replica clients with repl_put_online_on_ack and
then kill them only after a minute when no ack was received.

it would not stream anything to these connections, so the only effect of
this bug is a delay of 1 minute in the replicas attempt to re-connect.

(cherry picked from commit a176cb56a3c0235adddde33fcbaee2369a5af73e)
2020-07-20 21:08:26 +03:00
Oran Agra
05f8975d21 redis-cli tests, fix valgrind timing issue (#7519)
this test when run with valgrind on github actions takes 160 seconds

(cherry picked from commit 254c96255420e950bcad1a46bc4f8617b4373797)
2020-07-20 21:08:26 +03:00
WuYunlong
4780cc5efa Fix out of update help info in tcl tests. (#7516)
Before this commit, the output of "./runtest-cluster --help" is incorrect.
After this commit, the format of the following 3 output is consistent:
./runtest --help
./runtest-cluster --help
./runtest-sentinel --help

(cherry picked from commit 8128d39737adaf7092c9a367f71fbe9e0a2b33a2)
2020-07-20 21:08:26 +03:00
Qu Chen
9bc6af2449 Replica always reports master's config epoch in CLUSTER NODES output. (#7235)
(cherry picked from commit 938c35302fb618ec15e62443ef3bb3d00d10f5b9)
2020-07-20 21:08:26 +03:00
Oran Agra
c4b428a388 RESTORE ABSTTL skip expired keys - leak (#7511)
(cherry picked from commit 6a814501448c5673d05eb2cfe4fe650883da8f0e)
2020-07-20 21:08:26 +03:00
Oran Agra
aea4db2f5a fix recently added time sensitive tests failing with valgrind (#7512)
interestingly the latency monitor test fails because valgrind is slow
enough so that the time inside PEXPIREAT command from the moment of
the first mstime() call to get the basetime until checkAlreadyExpired
calls mstime() again is more than 1ms, and that test was too sensitive.

using this opportunity to speed up the test (unrelated to the failure)
the fix is just the longer time passed to PEXPIRE.

(cherry picked from commit e5227aab899628653285478a9d1083e8e8f51b57)
2020-07-20 21:08:26 +03:00
Oran Agra
b5c5f870a4 runtest --stop pause stops before terminating the redis server (#7513)
in the majority of the cases (on this rarely used feature) we want to
stop and be able to connect to the shard with redis-cli.
since these are two different processes interracting with the tty we
need to stop both, and we'll have to hit enter twice, but it's not that
bad considering it is rarely used.

(cherry picked from commit 02ef355f98691adba4126bbdab0d4d2bfe475701)
2020-07-20 21:08:26 +03:00
Oran Agra
23bf3d1cb9 update release scripts for new hosts, and CI to run more tests (#7480)
* update daily CI to include cluster and sentinel tests
* update daily CI to run when creating a new release
* update release scripts to work on the new redis.io hosts

(cherry picked from commit 7f19a04f0f049720ff5f84f3ab1aa81014f2f4ed)
2020-07-20 21:08:26 +03:00