1394 Commits

Author SHA1 Message Date
John Sully
367067c98a Prevent test crash due to unimplemented save for nested hash
Former-commit-id: 7e0077f293c5f91f240ab401e5f167e487afbfe0
2020-12-11 00:23:44 +00:00
VivekSainiEQ
52561f1729 Added module data type load/save tests
Former-commit-id: cae9924fd9eefcd88cef1c964f0bc8bce7dd4242
2020-11-30 17:47:04 -05:00
VivekSainiEQ
3d15a7b65a Removed use of module datatypes, now should work if tests/modules is not built
Former-commit-id: 089f28e8ba91ed2b875048efa0b28ceb32010541
2020-11-30 17:47:04 -05:00
VivekSainiEQ
54b36868ba Added tests for saving various data types to disk and loading them back, and for loading data types from redis to maintain compatibility
Former-commit-id: dcb44d3a09a4021f05079bedbac690e33ec7f39e
2020-11-30 17:47:04 -05:00
Kajaruban Surendran
fd03008bbf Configurable option for MOTD
Former-commit-id: 49a89d636ba698dbd0858d5059d3d6387c8c1fc7
2020-11-30 10:42:48 -05:00
VivekSainiEQ
86ca4300c8 Added test for issue #257
Former-commit-id: 779c775df9e8be36064689404e3ff51bead8a364
2020-11-25 17:21:37 -05:00
John Sully
c179c98870 Fix issue where active replication doesn't replicate RDB data
Former-commit-id: 527b7eb0742567302e0343e3acbed9814c0cbb95
2020-11-23 02:01:40 +00:00
John Sully
e8753d1b4b Blocking clients should not crash if an active replica loads a remote RDB with a key in the blocklist
Former-commit-id: 1c525e20b10e0a47af687a0d46faf75229a1cbf5
2020-11-19 23:28:01 +00:00
John Sully
5de36840ad Add missing file
Former-commit-id: 97fd73692363e05b80e0c0e84b7b2b8fe7f760b8
2020-11-11 03:29:38 +00:00
John Sully
e085772d01 Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
John Sully
7db922f44b Additional test reliability fixes
Former-commit-id: dad5a902d394719ba722e487879fc283ca148786
2020-10-27 07:25:43 +00:00
John Sully
18c34bbfe2 Active replica test reliability enhancements
Former-commit-id: 444555d3e4ec6e9469dae847dc631f2be263fb5e
2020-10-27 06:23:14 +00:00
John Sully
b226b318ff Prevent crashes on shutdown due to lock being held
Former-commit-id: 6a74f524e558100dfb3e54779020cd3407706e08
2020-10-27 04:53:19 +00:00
John Sully
4f18a247e3 Merge tag '6.0.8' into unstable
Former-commit-id: 4c7e4b91a6bb2034636856b608b8c386d07f5541
2020-09-30 19:47:55 +00:00
John Sully
3c0556093b Test RDB merge on load with active replication
Former-commit-id: 28183f4b66fc4c865048080b61e599eeb1d2293b
2020-09-29 03:26:06 +00:00
John Sully
ff1a43ea18 Reduce test noise
Former-commit-id: 13c9a094d1636d4ea9842746e20096eabbd80cba
2020-09-19 22:00:17 +00:00
John Sully
4eecb1825f Implement replica-quorum config
Former-commit-id: ab4cdd2ed4d3ee9752737d20662178d73d16b4c2
2020-09-16 03:26:31 +00:00
Yossi Gottlieb
bf6c002005 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.


Former-commit-id: d3834c50699bc4f31f381d6d03d4c1b022380895
2020-09-16 02:10:29 +00:00
Oran Agra
18b30467f7 Fix leak in new blockedclient module API test
(cherry picked from commit 0e20ad14a6a857cb168b808f94721df19b23dc0c)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb
0052d5196d Tests: fix oom-score-adj false positives. (#7772)
The key save delay is too short and on certain systems the child process
is gone before we have a chance to inspect it.

(cherry picked from commit b2a73c404bf277bac287c72494a4c4cd2ba02f8c)
2020-09-10 14:09:00 +03:00
杨博东
b55b0ea0f2 Tests: Add aclfile load and save tests (#7765)
improves test coverage

(cherry picked from commit 0666267d2771b1a46cdf36eef27d8a7a393c0c7a)
2020-09-10 14:09:00 +03:00
Roi Lipman
25e211bcf0 RM_ThreadSafeContextTryLock a non-blocking method for acquiring GIL (#7738)
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 042189fd8707544139337b3ddcf38b5c5fea1bf0)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb
635704d424 Tests: validate CONFIG REWRITE for all params. (#7764)
This is a catch-all test to confirm that that rewrite produces a valid
output for all parameters and that this process does not introduce
undesired configuration changes.

(cherry picked from commit a8b7268911803f4ac827ab22e8e5b10134664832)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb
3c8b394511 Tests: clean up stale .cli files. (#7768)
(cherry picked from commit 918abd7276afcb994f2d3f8a86a0708993420e37)
2020-09-10 14:09:00 +03:00
Eran Liberty
84ba2ec82b Allow exec with read commands on readonly replica in cluster (#7766)
There was a bug. Although cluster replicas would allow read commands,
they would not allow a MULTI-EXEC that's composed solely of read commands.
Adds tests for coverage.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Eran Liberty <eranl@amazon.com>
(cherry picked from commit b120366d48d9e488a406965773e64f29ba2946f7)
2020-09-10 14:09:00 +03:00
Oran Agra
643d3e0b79 handle cur_test for nested tests
if there are nested tests and nested servers, we need to restore the
previous value of cur_test when a test exist.

example:
```
test{test 1} {
	start_server {
		test{test 1.1 - master only} {
		}
		start_server {
		    test{test 1.2 - with replication} {
            }
		}
	}
}
```
when `test 1.1 - master only exists`, we're still inside `test 1`

(cherry picked from commit 0a1e7341935dbca4bae582de1a4a26d5ed4c652d)
2020-09-10 14:09:00 +03:00
bodong.ybd
781e50d41f Tests: Some fixes for macOS
1) cur_test: when restart_server, "no such variable" error occurs
  ./runtest --single integration/rdb
  test {client freed during loading}
      SET ::cur_test
      restart_server
        kill_server
          test "Check for memory leaks (pid $pid)"
          SET ::cur_test
          UNSET ::cur_test
      UNSET ::cur_test // This global variable has been unset.

2) `ps --ppid` not available on macOS platform, can be replaced with
`pgrep -P pid`.

(cherry picked from commit f22fa9594d536cb53f83ed8e508c03d4278778b0)
2020-09-10 14:09:00 +03:00
Oran Agra
57dbd6e90d Fix cluster consistency-check test (#7754)
This test was failing from time to time see discussion at the bottom of #7635
This was probably due to timing, the DEBUG SLEEP executed by redis-cli
didn't sleep for enough time.

This commit changes:
1) use SET-ACTIVE-EXPIRE instead of DEBUG SLEEP
2) reduce many `after` sleeps with retry loops to speed up the test.
3) add many comment explaining the different steps of the test and
   it's purpose.
4) config appendonly before populating the volatile keys, so that they'll
   be part of the AOF command stream rather than the preamble RDB portion.

other complications: recently kill_instance switched from SIGKILL to
SIGTERM, and this would sometimes fail since there was an AOFRW running
in the background. now we wait for it to end before attempting the kill.

(cherry picked from commit b491d477c3062d1fb064a794d74643c82d1c4adc)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb
e9fef49e12 Tests: fix unmonitored servers. (#7756)
There is an inherent race condition in port allocation for spawned
servers. If a server fails to start because a port is taken, a new port
is allocated. This fixes a problem where the logs are not truncated and
as a result a large number of unmonitored servers are started.

(cherry picked from commit 2df4cb93acabf10bb0ff39c12030791b0947e719)
2020-09-10 14:09:00 +03:00
Oran Agra
a227bc8053 fix broken cluster/sentinel tests by recent commit (#7752)
2b998de46 added a file for stderr to keep valgrind log but i forgot to
add a similar thing when valgrind isn't being used.
the result is that `glob */err.txt` fails.

(cherry picked from commit 42ba7a1b75aa100ba99d2024420bc658439452fb)
2020-09-10 14:09:00 +03:00
Oran Agra
d410dc3162 Improve valgrind support for cluster tests (#7725)
- redirect valgrind reports to a dedicated file rather than console
- try to avoid killing instances with SIGKILL so that we get the memory
  leak report (killing with SIGTERM before resorting to SIGKILL)
- search for valgrind reports when done, print them and fail the tests
- add --dont-clean option to keep the logs on exit
- fix exit error code when crash is found (would have exited with 0)

changes that affect the normal redis test suite:
- refactor check_valgrind_errors into two functions one to search and
  one to report
- move the search half into util.tcl to serve the cluster tests too
- ignore "address range perms" valgrind warnings which seem non relevant.

(cherry picked from commit 2b998de46078c172c6b19ac3b779318e7992c60a)
2020-09-10 14:09:00 +03:00
Oran Agra
41c7c7919c test infra - add durable mode to work around test suite crashing
in some cases a command that returns an error possibly due to a timing
issue causes the tcl code to crash and thus prevents the rest of the
tests from running. this adds an option to make the test proceed despite
the crash.
maybe it should be the default mode some day.

(cherry picked from commit fe5da2e60d8d6d907062f4789673fbe06fa8773e)
2020-09-10 14:09:00 +03:00
Oran Agra
db6c763d8b test infra - wait_done_loading
reduce code duplication in aof.tcl.
move creation of clients into the test so that it can be skipped

(cherry picked from commit 1b7ba44e7917082ac6d5523666d3b4ab210dfbad)
2020-09-10 14:09:00 +03:00
Oran Agra
72d6f966ac test infra - flushall between tests in external mode
(cherry picked from commit b65e5aca86b9c2d24b96abc8414a45f9907b6f7d)
2020-09-10 14:09:00 +03:00
Oran Agra
28e074608c test infra - improve test skipping ability
- skip full units
- skip a single test (not just a list of tests)
- when skipping tag, skip spinning up servers, not just the tests
- skip tags when running against an external server too
- allow using multiple tags (split them)

(cherry picked from commit 677d14c2137ab50fa25c8163d20b14bc563261c7)
2020-09-10 14:09:00 +03:00
Oran Agra
5b8de5b7f2 test infra - reduce disk space usage
this is important when running a test with --loop

(cherry picked from commit e3e69c25fd05b608f5ea8d612bc0e377922a6115)
2020-09-10 14:09:00 +03:00
Oran Agra
bce350c666 test infra - write test name to logfile
(cherry picked from commit 9d527d076b17851b87bc95aa34cca8fa5a91d41b)
2020-09-10 14:09:00 +03:00
Oran Agra
4bb40a9688 Reduce the probability of failure when start redis in runtest-cluster #7554 (#7635)
When runtest-cluster, at first, we need to create a cluster use spawn_instance,
a port which is not used is choosen, however sometimes we can't run server on
the port. possibley due to a race with another process taking it first.
such as redis/redis/runs/896537490. It may be due to the machine problem or
In order to reduce the probability of failure when start redis in
runtest-cluster, we attemp to use another port when find server do not start up.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: yanhui13 <yanhui13@meituan.com>
(cherry picked from commit e2d64485b8262971776fb1be803c7296c98d1572)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
f38e2802b6 Fix oom-score-adj on older distros. (#7724)
Don't assume `ps` handles `-h` to display output without headers and
manually trim headers line from output.

(cherry picked from commit b61b663895f16d9f559a14c408c225062254a57b)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
f6d04d01b9 Add oom-score-adj configuration option to control Linux OOM killer. (#1690)
Add Linux kernel OOM killer control option.

This adds the ability to control the Linux OOM killer oom_score_adj
parameter for all Redis processes, depending on the process role (i.e.
master, replica, background child).

A oom-score-adj global boolean flag control this feature. In addition,
specific values can be configured using oom-score-adj-values if
additional tuning is required.

(cherry picked from commit 2530dc0ebd8be8d792f4673073401377cd5bdc42)
2020-09-01 09:27:58 +03:00
Meir Shpilraien (Spielrein)
2fc915f509 see #7544, added RedisModule_HoldString api. (#7577)
Added RedisModule_HoldString that either returns a
shallow copy of the given String (by increasing
the String ref count) or a new deep copy of String
in case its not possible to get a shallow copy.

Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit 3f494cc49d25929f27fa75a78d9921a9dee771f2)
2020-09-01 09:27:58 +03:00
Meir Shpilraien (Spielrein)
2257f38b68 This PR introduces a new loaded keyspace event (#7536)
Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Itamar Haber <itamar@redislabs.com>
(cherry picked from commit 8d826393191399e132bd9e56fb51ed83223cc5ca)
2020-09-01 09:27:58 +03:00
valentinogeron
e6f6731c66 EXEC with only read commands should not be rejected when OOM (#7696)
If the server gets MULTI command followed by only read
commands, and right before it gets the EXEC it reaches OOM,
the client will get OOM response.

So, from now on, it will get OOM response only if there was
at least one command that was tagged with `use-memory` flag

(cherry picked from commit b7289e912cbe1a011a5569cd67929e83731b9660)
2020-09-01 09:27:58 +03:00
Valentino Geron
19ef1f371d Fix LPOS command when RANK is greater than matches
When calling to LPOS command when RANK is higher than matches,
the return value is non valid response. For example:
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*-4
```
It may break client-side parser.

Now, we count how many replies were replied in the array.
```
LPUSH l a
:1
LPOS l b RANK 5 COUNT 10
*0
```

(cherry picked from commit 9204a9b2c2f6eb59767ab0bddcde62c75e8c20b0)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
8d79702d8a Tests: fix redis-cli with remote hosts. (#7693)
(cherry picked from commit f80f3f492a0ca56e163899eeca7ad40d67d903be)
2020-09-01 09:27:58 +03:00
杨博东
113d5ae872 Fix flock cluster config may cause failure to restart after kill -9 (#7674)
After fork, the child process(redis-aof-rewrite) will get the fd opened
by the parent process(redis), when redis killed by kill -9, it will not
graceful exit(call prepareForShutdown()), so redis-aof-rewrite thread may still
alive, the fd(lock) will still be held by redis-aof-rewrite thread, and
redis restart will fail to get lock, means fail to start.

This issue was causing failures in the cluster tests in github actions.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit cbaf3c5bbafd43e009a2d6b38dd0e9fc450a3e12)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
1180537589 Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.

(cherry picked from commit 64c360c5156ca6ee6d1eb52bfeb3fa48f3b25da5)
2020-09-01 09:27:58 +03:00
Oran Agra
916b215fc5 fix new rdb test failing on timing issues (#7604)
apparenlty on github actions sometimes 500ms is not enough

(cherry picked from commit 824bd2ac11472b7a3fce9fcf3189a8e6c6048115)
2020-09-01 09:27:58 +03:00
Oran Agra
a5294c4e52 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail

(cherry picked from commit f7e77759902aa19cfa537ed454e6bc987498e8c5)
2020-09-01 09:27:58 +03:00
WuYunlong
7200b3aa0f Fix running single test 14-consistency-check.tcl (#7587)
(cherry picked from commit f3352daf4f4826e1cad4c163fd6e35b81a72e21b)
2020-09-01 09:27:58 +03:00