12920 Commits

Author SHA1 Message Date
Oran Agra
67750ce3b3 Fix failing tests due to issues with wait_for_log_message (#7572)
- the test now waits for specific set of log messages rather than wait for
  timeout looking for just one message.
- we don't wanna sample the current length of the log after an action, due
  to a race, we need to start the search from the line number of the last
  message we where waiting for.
- when attempting to trigger a full sync, use multi-exec to avoid a race
  where the replica manages to re-connect before we completed the set of
  actions that should force a full sync.
- fix verify_log_message which was broken and unused

(cherry picked from commit 109b5ccdcd6e6b8cecdaeb13a246bc49ce7a61f4)
2020-09-01 09:27:58 +03:00
Jiayuan Chen
096285ab64 Add optional tls verification (#7502)
Adds an `optional` value to the previously boolean `tls-auth-clients` configuration keyword.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit f31260b0445f5649449da41555e1272a40ae4af7)
2020-09-01 09:27:58 +03:00
Oran Agra
0b2f7c6407 Daily github action: run cluster and sentinel tests with tls (#7575)
(cherry picked from commit 6d92eee69b8e693b9bf0a34320dd108b5a81a81a)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
8a4240b1fa TLS: support cluster/replication without tls-port.
Initialize and configure OpenSSL even when tls-port is not used, because
we may still have tls-cluster or tls-replication.

Also, make sure to reconfigure OpenSSL when these parameters are changed
as TLS could have been enabled for the first time.

(cherry picked from commit c75512d89d1697cf782cdc826acffab5b6adc1c7)
2020-09-01 09:27:58 +03:00
grishaf
395d339234 Fix prepareForShutdown function declaration (#7566)
(cherry picked from commit 4126ca466fb9fbf0d84a66e31c9f59f28e2fbff6)
2020-09-01 09:27:58 +03:00
Oran Agra
6daa8b9adb Stabilize bgsave test that sometimes fails with valgrind (#7559)
on ci.redis.io the test fails a lot, reporting that bgsave didn't end.
increaseing the timeout we wait for that bgsave to get aborted.
in addition to that, i also verify that it indeed got aborted by
checking that the save counter wasn't reset.

add another test to verify that a successful bgsave indeed resets the
change counter.

(cherry picked from commit 8a57969fd75db01b881d438200911d95bdead293)
2020-09-01 09:27:58 +03:00
Madelyn Olson
e00b6248c9 Properly reset errno for rdbLoad (#7542)
(cherry picked from commit 818dc3a0898125d621288df671994eed34f849f2)
2020-09-01 09:27:58 +03:00
Oran Agra
e8aa5583d0 testsuite may leave servers alive on error (#7549)
in cases where you have
test name {
  start_server {
    start_server {
      assert
    }
  }
}

the exception will be thrown to the test proc, and the servers are
supposed to be killed on the way out. but it seems there was always a
bug of not cleaning the server stack, and recently (#7404) we started
relying on that stack in order to kill them, so with that bug sometimes
we would have tried to kill the same server twice, and leave one alive.

luckly, in most cases the pattern is:
start_server {
  test name {
  }
}

(cherry picked from commit 36b949438547eb5bf8555fcac2c5040528fd7854)
2020-09-01 09:27:58 +03:00
Yossi Gottlieb
f1d5d5d28e Tests: drop TCL 8.6 dependency. (#7548)
This re-implements the redis-cli --pipe test so it no longer depends on a close feature available only in TCL 8.6.

Basically what this test does is run redis-cli --pipe, generates a bunch of commands and pipes them through redis-cli, and inspects the result in both Redis and the redis-cli output.

To do that, we need to close stdin for redis-cli to indicate we're done so it can flush its buffers and exit. TCL has bi-directional channels can only offers a way to "one-way close" a channel with TCL 8.6. To work around that, we now generate the commands into a file and feed that file to redis-cli directly.

As we're writing to an actual file, the number of commands is now reduced.

(cherry picked from commit f57e844b2edbb86a5df2f3436045814812c0a3ae)
2020-09-01 09:27:58 +03:00
Oran Agra
cc7f4ba67e Fixes to release scripts (#7547)
(cherry picked from commit 343dd9bcce0ce28d131510e28bac411599da90b0)
2020-09-01 09:27:58 +03:00
Remi Collet
af907e4b6d Fix deprecated tail syntax in tests (#7543)
(cherry picked from commit 3f2fbc4c614ff718dce7d55fd971d7ed36062c24)
2020-09-01 09:27:58 +03:00
Wen Hui
d2f09a054c Add missing calls to raxStop (#7532)
Since the dynamic allocations in raxIterator are only used for deep walks, memory
leak due to missing call to raxStop can only happen for rax with key names longer
than 32 bytes.

Out of all the missing calls, the only ones that may lead to a leak are the rax
for consumer groups and consumers, and these were only in AOFRW and rdbSave, which
normally only happen in fork or at shutdown.

(cherry picked from commit 4e8f2d6881a38397bfbf0d7d161959163a5f6e88)
2020-09-01 09:27:58 +03:00
Wen Hui
cef46edebc add missing caching command in client help (#7399)
(cherry picked from commit 2fbd0271f6fc0a93b9323b6478ec1a7a4ca77614)
2020-09-01 09:27:58 +03:00
zhaozhao.zz
9e997cd0d1 replication: need handle -NOPERM error after send ping (#7538)
(cherry picked from commit 13e50935a84c319763a0d3b2be6ce64962092541)
2020-09-01 09:27:58 +03:00
Scott Brenner
171aa22b0b GitHub Actions workflows - use latest version of actions/checkout (#7534)
(cherry picked from commit 2f4e9c3f9f38e196fe4a03232c97782cbf8b3702)
2020-09-01 09:27:58 +03:00
John Sully
d1f45b8ebf Implement use-fork config (fails with diskless repl)
Former-commit-id: f2d5c2bca22e9fd506db123c47b7f60cdded7e2c
2020-08-24 03:17:59 +00:00
John Sully
bfa8e2b6e2 Fix ARM build of RocksDB
Former-commit-id: 7301d6d5e76b65115e2610a5fc5afee443782d7c
2020-08-17 02:25:45 +00:00
John Sully
035cde502a Fast cleanup of snapshots without leaving them forever
Former-commit-id: fdd83b2b49244ed2988b080892ee5cffe9fd2684
2020-08-17 00:33:37 +00:00
John Sully
b71f16471b Allow garbage collection of generic data
Former-commit-id: feadb7fb1845027422bcfca43dbcb6097409b8dc
2020-08-17 00:32:48 +00:00
John Sully
e97e4f1a35 Add snapshot depth to info string
Former-commit-id: 66aef678e9d6d7ab6c28622d54ada0ea8cdd2a99
2020-08-16 00:27:10 +00:00
John Sully
6ba6652fcf Don't try and consolidate snapshots with a depth of 1
Former-commit-id: 26c298bd9bc4e2c6981de5c20284120ea54580c3
2020-08-16 00:26:05 +00:00
John Sully
c3365964a0 Only save MVCC tstamps if its an active replica
Former-commit-id: 5aded7dbad055308dc3932f7797fc71684b60966
2020-08-16 00:24:18 +00:00
John Sully
71506d7e0a Don't free snapshot objects in a critical path (under the AE lock)
Former-commit-id: d0da3d3cb74334cc8a2d14f4bdaef7935181700a
2020-08-16 00:13:19 +00:00
John Sully
e946e5f907 BUG: Cannot have an EMBSTR bigger than 255
Former-commit-id: e2d6e2d9d585cb7a73f469a8580f9cb0ec71a429
2020-08-15 23:30:22 +00:00
John Sully
2268a2b761 remove unnecessary allocation in rdb save
Former-commit-id: dfa76f04fd862e0cc7b4d3284b3922e3a5e5549d
2020-08-15 23:27:32 +00:00
John Sully
a14b2097c3 Remove unnecessary key comparisons in perf critical snapshot paths
Former-commit-id: 40f8a8d102fdca9443399ef03a47df609b146d58
2020-08-15 23:25:58 +00:00
John Sully
0311011131 dictMerge perf improvements
Former-commit-id: 48401ec369c5693689ef658cca518dc94ab1402e
2020-08-15 23:14:29 +00:00
John Sully
34937b0ad5 Rehash efficiency
Former-commit-id: fab383156626ec683881101c22eb2f6c2cea4c5d
2020-08-15 23:05:56 +00:00
John Sully
07c019fd3d Prevent unnecessary copy when overwriting a value from a snapshot
Former-commit-id: 654a7bc6ea82f4ac45a1c1a25c794e1c27c0d902
2020-08-15 22:59:01 +00:00
John Sully
db193a1ef1 Merge branch 'unstable' into keydbpro
Former-commit-id: ae482585f0dc470efd73833f74111c2f87a172c5
2020-08-15 21:29:00 +00:00
John Sully
6c83ecbb48 Prehash the tombstone for cleanup
Former-commit-id: c9d97a7c7448fc769486175bea1648589487c87c
2020-08-14 16:05:39 +00:00
John Sully
e87dee8dc7 Add build flag to disable MVCC tstamps
Former-commit-id: f17d178d03f44abcdaddd851a313dd3f7ec87ed5
2020-08-10 06:10:24 +00:00
John Sully
e06152eb13 Fix lock after free in module API
Former-commit-id: d88fd1588d292bffc0aa53c299cb52e7a4e91015
2020-08-10 06:10:24 +00:00
John Sully
85d7a4c1e2 MVCC Perf fixes
Former-commit-id: 5a4afe5fb4231bec34d434f9e3214a7320842091
2020-08-10 05:45:56 +00:00
John Sully
3c1e63fa74 Fix assert caused by freeTombstoneObjects and null check in consolidate_children
Former-commit-id: 8565a02b331cd2bba2a1c7c6693dfb3f6e61c845
2020-08-10 05:01:36 +00:00
John Sully
649745924b RocksDB Read Performance Improvements
Former-commit-id: 80cca4869888e048e10e11f1f20796c482c3e5b3
2020-08-09 23:36:20 +00:00
John Sully
5bda84cbc2 Ensure we use the SSE CRC32 instruction
Former-commit-id: 78ef8976e86d07d4c1fa7c4631a610babf2f4776
2020-08-09 01:47:53 +00:00
John Sully
1610f4211f Start off storage cache with a larger size
Former-commit-id: 5f6fb970a81cc73586ba595b35564e7865e7262d
2020-08-09 00:57:56 +00:00
John Sully
11a8ab1865 Disable compression it destroys read perf
Former-commit-id: d3fffc12ae5339886f54c064127497f277393b00
2020-08-06 22:30:56 +00:00
John Sully
b388da300b keydbpro unstable version should be 0.0.0
Former-commit-id: 4a6e0772f36d5dd721ad2e9b83f9fccce2e38ff0
2020-08-06 02:13:23 +00:00
John Sully
299794b32c Latency normalization with SCAN
Former-commit-id: 237f25d854e70d4d7a3095fdf56aaa80770e492e
2020-08-04 04:37:16 +00:00
Oran Agra
7bf665f125 Redis 6.0.6. 2020-07-20 21:08:26 +03:00
Oran Agra
a5696bdf4f Run daily CI on PRs to release a branch 2020-07-20 21:08:26 +03:00
Itamar Haber
e15528bf1d Adds SHA256SUM to redis-stable tarball upload
(cherry picked from commit 5df0a64d30e7815c0a4a75a80f165fdee0bd1db6)
2020-07-20 21:08:26 +03:00
yoav-steinberg
e28aa99af1 Support passing stack allocated module strings to moduleCreateArgvFromUserFormat (#7528)
Specifically, the key passed to the module aof_rewrite callback is a stack allocated robj. When passing it to RedisModule_EmitAOF (with appropriate "s" fmt string) redis used to panic when trying to inc the ref count of the stack allocated robj. Now support such robjs by coying them to a new heap robj. This doesn't affect performance because using the alternative "c" or "b" format strings also copies the input to a new heap robj.

(cherry picked from commit d484b8a04ed67e79030fcb060e88641acb6e4f98)
2020-07-20 21:08:26 +03:00
Luke Palmer
3051430040 Send null for invalidate on flush (#7469)
(cherry picked from commit 5f716ea467d29059a89f90b6ccbdee5a60443200)
2020-07-20 21:08:26 +03:00
dmurnane
29b20fd528 Notify systemd on sentinel startup (#7168)
Co-authored-by: Daniel Murnane <dmurnane@eitccorp.com>
(cherry picked from commit 9242ccf238cbed018eb3a7fa3a437618345dd52b)
2020-07-20 21:08:26 +03:00
Developer-Ecosystem-Engineering
5b36681213 Add registers dump support for Apple silicon (#7453)
Export following environment variables before building on macOS on Apple silicon

export ARCH_FLAGS="-arch arm64"
export SDK_NAME=macosx
export SDK_PATH=$(xcrun --show-sdk-path --sdk $SDK_NAME)
export CFLAGS="$ARCH_FLAGS -isysroot $SDK_PATH -I$SDK_PATH/usr/include"
export CXXFLAGS=$CFLAGS
export LDFLAGS="$ARCH_FLAGS"
export CC="$(xcrun -sdk $SDK_PATH --find clang) $CFLAGS"
export CXX="$(xcrun -sdk $SDK_PATH --find clang++) $CXXFLAGS"
export LD="$(xcrun -sdk $SDK_PATH --find ld) $LDFLAGS"

make
make test
..
All tests passed without errors!

Backtrack logging assumes x86 and required updating

(cherry picked from commit c2b5f1c15bbc5363f92f8e697538759c5d929934)
2020-07-20 21:08:26 +03:00
Wen Hui
0f75036c07 correct error msg for num connections reaching maxclients in cluster mode (#7444)
(cherry picked from commit d85af4d6f5fbe9cb9787b81583627cd74b47f838)
2020-07-20 21:08:26 +03:00
WuYunlong
b1a01fda91 Fix command help for unexpected options (#7476)
(cherry picked from commit 93bdbf5aa4857ede0816cf790f951da8e2fa2ae9)
2020-07-20 21:08:26 +03:00