11882 Commits

Author SHA1 Message Date
John Sully
d5ed1b9168 Fix CI
Former-commit-id: 6a902b29e16bebdb2da1d0a33ea5a170fc9949a0
2020-05-29 01:01:41 -04:00
John Sully
cfe9f8f3bc Merge tag '6.0.4' into unstable
Redis 6.0.4.


Former-commit-id: 9c31ac7925edba187e527f506e5e992946bd38a6
2020-05-29 00:57:07 -04:00
John Sully
75471778ed Fix TLS tests
Former-commit-id: f0d8d1680f594cc72ccd863eb74b1071368f3052
2020-05-29 00:16:06 -04:00
John Sully
72b02869fd gencert its own step
Former-commit-id: 6dcbcacb104814e6a710a4ffe101668a573ebd4f
2020-05-28 23:49:13 -04:00
John Sully
e65a56d015 CI yml syntax error
Former-commit-id: 3196717ceee0bde8a5e30b25dd99bd7f3776c6ab
2020-05-28 23:18:02 -04:00
John Sully
4d5f5c0924 Fix TLS tests in CI
Former-commit-id: f95578ac0268b3f29139fc715bba0b27b92767c3
2020-05-28 23:16:42 -04:00
John Sully
3b6f82b056 fix TLS test failure
Former-commit-id: 57ca6facc3038e005656912b6378cb7fbe04f55a
2020-05-28 23:16:27 -04:00
antirez
886d287c40 Redis 6.0.4. 2020-05-28 12:18:38 +02:00
antirez
59cd4c9f65 Test: take PSYNC2 test master timeout high during switch.
This will likely avoid false positives due to trailing pings.
2020-05-28 10:56:14 +02:00
antirez
6c1bb7b194 Test: add the tracking unit as default. 2020-05-28 10:22:29 +02:00
Oran Agra
1aee695e52 tests: find_available_port start search from next port
i.e. don't start the search from scratch hitting the used ones again.
this will also reduce the likelihood of collisions (if there are any
left) by increasing the time until we re-use a port we did use in the
past.
2020-05-28 10:09:51 +02:00
Oran Agra
a2ae463520 tests: each test client work on a distinct port range
apparently when running tests in parallel (the default of --clients 16),
there's a chance for two tests to use the same port.
specifically, one test might shutdown a master and still have the
replica up, and then another test will re-use the port number of master
for another master, and then that replica will connect to the master of
the other test.

this can cause a master to count too many full syncs and fail a test if
we run the tests with --single integration/psync2 --loop --stop

see Probmem 2 in #7314
2020-05-28 10:09:51 +02:00
Oran Agra
86e562d691 32bit CI needs to build modules correctly 2020-05-28 10:09:51 +02:00
Oran Agra
ab2984b1e2 adjust revived meaningful offset tests
these tests create several edge cases that are otherwise uncovered (at
least not consistently) by the test suite, so although they're no longer
testing what they were meant to test, it's still a good idea to keep
them in hope that they'll expose some issue in the future.
2020-05-28 10:09:51 +02:00
Oran Agra
1ff5a222de revive meaningful offset tests 2020-05-28 10:09:51 +02:00
antirez
cc549b46a4 Replication: showLatestBacklog() refactored out. 2020-05-28 10:09:51 +02:00
antirez
377dd0515c Drop useless line from replicationCacheMaster(). 2020-05-28 10:09:51 +02:00
antirez
3f8d113f1b Another meaningful offset test removed. 2020-05-28 10:09:51 +02:00
antirez
d4541349dc Remove the PSYNC2 meaningful offset test. 2020-05-28 10:09:51 +02:00
antirez
2112a57028 Remove the meaningful offset feature.
After a closer look, the Redis core devleopers all believe that this was
too fragile, caused many bugs that we didn't expect and that were very
hard to track. Better to find an alternative solution that is simpler.
2020-05-28 10:09:51 +02:00
antirez
d2eb6e0b47 Set a protocol error if master use the inline protocol.
We want to react a bit more aggressively if we sense that the master is
sending us some corrupted stream. By setting the protocol error we both
ensure that the replica will disconnect, and avoid caching the master so
that a full SYNC will be required. This is protective against
replication bugs.
2020-05-28 10:09:51 +02:00
Oran Agra
9c1df3b769 daily CI test with tls 2020-05-28 10:09:51 +02:00
Oran Agra
115ed1911e avoid using sendfile if tls-replication is enabled
this obviously broke the tests, but went unnoticed so far since tls
wasn't often tested.
2020-05-28 10:09:51 +02:00
antirez
11c748aaca Replication: log backlog creation event. 2020-05-28 10:09:51 +02:00
antirez
8f10137227 Test: PSYNC2 test can now show server logs. 2020-05-28 10:09:51 +02:00
John Sully
743c7c6c4a Test TLS as part of CI
Former-commit-id: 561a1f8d8a2ad5a048acbc3a7b17360ce114dec0
2020-05-26 01:39:54 -04:00
John Sully
2bb0be3a96 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 1449c8b9fe647288da7de0ff10cc68ada4be8f92
2020-05-26 01:29:08 -04:00
John Sully
d7fa406a0e Fix client tracking, also ensure tracking tests are enabled
Former-commit-id: 1938af27f50f9686dc98b4839fb439cc03b4a250
2020-05-26 01:28:52 -04:00
antirez
2e591fc4a4 Clarify what is happening in PR #7320. 2020-05-25 12:08:01 +02:00
zhaozhao.zz
cbb51fb8f9 PSYNC2: second_replid_offset should be real meaningful offset
After adjustMeaningfulReplOffset(), all the other related variable
should be updated, including server.second_replid_offset.

Or the old version redis like 5.0 may receive wrong data from
replication stream, cause redis 5.0 can sync with redis 6.0,
but doesn't know meaningful offset.
2020-05-25 12:08:01 +02:00
Oran Agra
e0fc88b4d9 add CI for 32bit build 2020-05-25 12:08:01 +02:00
antirez
e3f864b5fd Make disconnectSlaves() synchronous in the base case.
Otherwise we run into that:

Backtrace:
src/redis-server 127.0.0.1:21322(logStackTrace+0x45)[0x479035]
src/redis-server 127.0.0.1:21322(sigsegvHandler+0xb9)[0x4797f9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd373c5e390]
src/redis-server 127.0.0.1:21322(_serverAssert+0x6a)[0x47660a]
src/redis-server 127.0.0.1:21322(freeReplicationBacklog+0x42)[0x451282]
src/redis-server 127.0.0.1:21322[0x4552d4]
src/redis-server 127.0.0.1:21322[0x4c5593]
src/redis-server 127.0.0.1:21322(aeProcessEvents+0x2e6)[0x42e786]
src/redis-server 127.0.0.1:21322(aeMain+0x1d)[0x42eb0d]
src/redis-server 127.0.0.1:21322(main+0x4c5)[0x42b145]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd3738a3830]
src/redis-server 127.0.0.1:21322(_start+0x29)[0x42b409]

Since we disconnect all the replicas and free the replication backlog in
certain replication paths, and the code that will free the replication
backlog expects that no replica is connected.

However we still need to free the replicas asynchronously in certain
cases, as documented in the top comment of disconnectSlaves().
2020-05-25 12:08:01 +02:00
ShooterIT
8af1e513f6 Implements sendfile for redis. 2020-05-25 12:08:01 +02:00
antirez
3c21418cd8 Fix #7306 less aggressively.
Citing from the issue:

btw I suggest we change this fix to something else:
* We revert the fix.
* We add a call that disconnects chained replicas in the place where we trim the replica (that is a master i this case) offset.
This way we can avoid disconnections when there is no trimming of the backlog.

Note that we now want to disconnect replicas asynchronously in
disconnectSlaves(), because it's in general safer now that we can call
it from freeClient(). Otherwise for instance the command:

    CLIENT KILL TYPE master

May crash: clientCommand() starts running the linked of of clients,
looking for clients to kill. However it finds the master, kills it
calling freeClient(), but this in turn calls replicationCacheMaster()
that may also call disconnectSlaves() now. So the linked list iterator
of the clientCommand() will no longer be valid.
2020-05-25 12:08:01 +02:00
Maxime de Roucy
b9b4b4cfde acl: @replication command group
containing mandatory commands for replication (used by masteruser user)


Former-commit-id: 123bc125cfe7b30fdeef833759a33ccc854b5c51
2020-05-25 02:35:22 -04:00
John Sully
c15802af8b Fix issue #170. Intermittent crash destroying fastlock due to implicit type conversion in assert
Former-commit-id: ecbe168b5421687899a621c995e44f43faec2b71
2020-05-25 02:13:57 -04:00
John Sully
fa671d5193 Fix infinite logs of Error in rreplay
Former-commit-id: 00cef78858cd05fc067fa5002f2995653452c63f
2020-05-24 20:05:22 -04:00
John Sully
fcfff165da Enable TLS connections
Former-commit-id: d05da0fabdfeb7eadce6546c7c1d85739b2794d7
2020-05-24 19:57:16 -04:00
John Sully
0a1d2c99ac don't keep feeding data to clients pending close
Former-commit-id: 4db3f2a443a6f71d1486781005ee37546e89d6e1
2020-05-24 12:46:11 -04:00
John Sully
975ca828d4 Quiet TSAN during exit
Former-commit-id: fd710d590a6297b74e06ce6832dd75bfca0a266f
2020-05-24 12:45:10 -04:00
John Sully
70f3eab882 Install deps for daily workflow
Former-commit-id: 9b98c9edd3e8c9316514b7f9e0efc7a9f8d17851
2020-05-24 11:19:53 -04:00
John Sully
23e1601276 Hold the AE lock until we process the repl backlog after a command
Former-commit-id: 4893eedacbf5f9210d02361ed07b9beee0c9f655
2020-05-24 11:19:23 -04:00
John Sully
b892d3deff Add standardized way to post client dependent functions
Former-commit-id: fb50163b47cf660911dc7f67809c15ba1394613a
2020-05-24 02:46:39 -04:00
John Sully
f517eaf182 remove unnecessary cplusplus guard
Former-commit-id: 3b8463f9d20162369cffd9ed347865c5c8049f40
2020-05-24 02:45:55 -04:00
John Sully
78b647bd6f race in connection state when setting a handler on a different thread
Former-commit-id: 42c7677690c434168135992429f7521239475ef8
2020-05-24 02:45:11 -04:00
John Sully
dc27b95a47 Fix ASAN races
Former-commit-id: 8665f250314c4e3de32f3b31a2c0158908ef982b
2020-05-24 01:58:33 -04:00
John Sully
cc080de221 Incorrect locking in replication error paths
Former-commit-id: 1c6b57314747787958ad215826296627f9050f59
2020-05-24 01:57:49 -04:00
John Sully
51976e2be8 Incorrect memory order on weak ordered architectures
Former-commit-id: 90162da1a6c025d148a0d1761ae4d5fbc9bfc0ef
2020-05-24 01:57:18 -04:00
John Sully
926da1f899 If we request shutdown from a client on a different thread than a blocking lua script we fail to actually shutdown until the script completes
Former-commit-id: 1ca6c1dd56131bc28fe9f831e2b9cab7adb846a3
2020-05-24 01:56:23 -04:00
John Sully
de376acb15 race calculating client overhead
Former-commit-id: 2d27c8ecdb5242284301f135b1d542f9ead5b8f2
2020-05-24 01:53:43 -04:00