9841 Commits

Author SHA1 Message Date
John Sully
bbff81e891 Fix memory test failures with multithreading enabled
Former-commit-id: 58035404227a9ef1c3bd92623a333c915d50eab6
2020-05-30 00:06:15 -04:00
John Sully
5316f656d9 sendFile blocks too long for use with active replication
Former-commit-id: aad6a7ce159a3679633020dc407a2068129bbd49
2020-05-29 20:27:45 -04:00
John Sully
68bf5a04a0 Unify beforeSleep handling
Former-commit-id: 1cb48c7bf6a7e91e728a677902a7bfc64fe80dd6
2020-05-29 17:44:55 -04:00
John Sully
f18ba66242 Cluster crash
Former-commit-id: f25c405ad2a8004b79a816072cb011c1dc2407d5
2020-05-29 03:40:52 -04:00
John Sully
2e0c684324 active replica tests on slow computers
Former-commit-id: c9920849dd6d6d0f6ecfe0d1002cb0edd7f7bfa9
2020-05-29 01:58:15 -04:00
John Sully
688dceb3a8 Fix test issue with TLS
Former-commit-id: 81b240f81d1c52fd331c4e0e89659913380229c4
2020-05-29 01:44:52 -04:00
John Sully
f79cc0011a Fix CI
Former-commit-id: 6a902b29e16bebdb2da1d0a33ea5a170fc9949a0
2020-05-29 01:01:41 -04:00
John Sully
ed2e0e66f6 Merge tag '6.0.4' into unstable
Redis 6.0.4.


Former-commit-id: 9c31ac7925edba187e527f506e5e992946bd38a6
2020-05-29 00:57:07 -04:00
John Sully
a768c260dc Fix TLS tests
Former-commit-id: f0d8d1680f594cc72ccd863eb74b1071368f3052
2020-05-29 00:16:06 -04:00
John Sully
34d3bac791 gencert its own step
Former-commit-id: 6dcbcacb104814e6a710a4ffe101668a573ebd4f
2020-05-28 23:49:13 -04:00
John Sully
8bb3ac40f1 CI yml syntax error
Former-commit-id: 3196717ceee0bde8a5e30b25dd99bd7f3776c6ab
2020-05-28 23:18:02 -04:00
John Sully
f4f9e83ead Fix TLS tests in CI
Former-commit-id: f95578ac0268b3f29139fc715bba0b27b92767c3
2020-05-28 23:16:42 -04:00
John Sully
71fbeb65b6 fix TLS test failure
Former-commit-id: 57ca6facc3038e005656912b6378cb7fbe04f55a
2020-05-28 23:16:27 -04:00
antirez
c512f64428 Redis 6.0.4. 2020-05-28 12:18:38 +02:00
antirez
41bb699867 Test: take PSYNC2 test master timeout high during switch.
This will likely avoid false positives due to trailing pings.
2020-05-28 10:56:14 +02:00
antirez
40578433c7 Test: add the tracking unit as default. 2020-05-28 10:22:29 +02:00
Oran Agra
571b03021a tests: find_available_port start search from next port
i.e. don't start the search from scratch hitting the used ones again.
this will also reduce the likelihood of collisions (if there are any
left) by increasing the time until we re-use a port we did use in the
past.
2020-05-28 10:09:51 +02:00
Oran Agra
4653d796f0 tests: each test client work on a distinct port range
apparently when running tests in parallel (the default of --clients 16),
there's a chance for two tests to use the same port.
specifically, one test might shutdown a master and still have the
replica up, and then another test will re-use the port number of master
for another master, and then that replica will connect to the master of
the other test.

this can cause a master to count too many full syncs and fail a test if
we run the tests with --single integration/psync2 --loop --stop

see Probmem 2 in #7314
2020-05-28 10:09:51 +02:00
Oran Agra
31bd963557 32bit CI needs to build modules correctly 2020-05-28 10:09:51 +02:00
Oran Agra
01039e5964 adjust revived meaningful offset tests
these tests create several edge cases that are otherwise uncovered (at
least not consistently) by the test suite, so although they're no longer
testing what they were meant to test, it's still a good idea to keep
them in hope that they'll expose some issue in the future.
2020-05-28 10:09:51 +02:00
Oran Agra
98e6f2cd5b revive meaningful offset tests 2020-05-28 10:09:51 +02:00
antirez
84117d13b7 Replication: showLatestBacklog() refactored out. 2020-05-28 10:09:51 +02:00
antirez
14d99c183f Drop useless line from replicationCacheMaster(). 2020-05-28 10:09:51 +02:00
antirez
0163e4e495 Another meaningful offset test removed. 2020-05-28 10:09:51 +02:00
antirez
24a0f7bf55 Remove the PSYNC2 meaningful offset test. 2020-05-28 10:09:51 +02:00
antirez
911c579b68 Remove the meaningful offset feature.
After a closer look, the Redis core devleopers all believe that this was
too fragile, caused many bugs that we didn't expect and that were very
hard to track. Better to find an alternative solution that is simpler.
2020-05-28 10:09:51 +02:00
antirez
7e55485b21 Set a protocol error if master use the inline protocol.
We want to react a bit more aggressively if we sense that the master is
sending us some corrupted stream. By setting the protocol error we both
ensure that the replica will disconnect, and avoid caching the master so
that a full SYNC will be required. This is protective against
replication bugs.
2020-05-28 10:09:51 +02:00
Oran Agra
abb9dcd975 daily CI test with tls 2020-05-28 10:09:51 +02:00
Oran Agra
0705a29959 avoid using sendfile if tls-replication is enabled
this obviously broke the tests, but went unnoticed so far since tls
wasn't often tested.
2020-05-28 10:09:51 +02:00
antirez
fee0c76304 Replication: log backlog creation event. 2020-05-28 10:09:51 +02:00
antirez
2411e4e33f Test: PSYNC2 test can now show server logs. 2020-05-28 10:09:51 +02:00
John Sully
d9164a07b5 Test TLS as part of CI
Former-commit-id: 561a1f8d8a2ad5a048acbc3a7b17360ce114dec0
2020-05-26 01:39:54 -04:00
John Sully
ef4284c439 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 1449c8b9fe647288da7de0ff10cc68ada4be8f92
2020-05-26 01:29:08 -04:00
John Sully
5b44b62854 Fix client tracking, also ensure tracking tests are enabled
Former-commit-id: 1938af27f50f9686dc98b4839fb439cc03b4a250
2020-05-26 01:28:52 -04:00
antirez
7a32a8485e Clarify what is happening in PR #7320. 2020-05-25 12:08:01 +02:00
zhaozhao.zz
d089cc8963 PSYNC2: second_replid_offset should be real meaningful offset
After adjustMeaningfulReplOffset(), all the other related variable
should be updated, including server.second_replid_offset.

Or the old version redis like 5.0 may receive wrong data from
replication stream, cause redis 5.0 can sync with redis 6.0,
but doesn't know meaningful offset.
2020-05-25 12:08:01 +02:00
Oran Agra
1f5163f454 add CI for 32bit build 2020-05-25 12:08:01 +02:00
antirez
8a4e01f2bc Make disconnectSlaves() synchronous in the base case.
Otherwise we run into that:

Backtrace:
src/redis-server 127.0.0.1:21322(logStackTrace+0x45)[0x479035]
src/redis-server 127.0.0.1:21322(sigsegvHandler+0xb9)[0x4797f9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd373c5e390]
src/redis-server 127.0.0.1:21322(_serverAssert+0x6a)[0x47660a]
src/redis-server 127.0.0.1:21322(freeReplicationBacklog+0x42)[0x451282]
src/redis-server 127.0.0.1:21322[0x4552d4]
src/redis-server 127.0.0.1:21322[0x4c5593]
src/redis-server 127.0.0.1:21322(aeProcessEvents+0x2e6)[0x42e786]
src/redis-server 127.0.0.1:21322(aeMain+0x1d)[0x42eb0d]
src/redis-server 127.0.0.1:21322(main+0x4c5)[0x42b145]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd3738a3830]
src/redis-server 127.0.0.1:21322(_start+0x29)[0x42b409]

Since we disconnect all the replicas and free the replication backlog in
certain replication paths, and the code that will free the replication
backlog expects that no replica is connected.

However we still need to free the replicas asynchronously in certain
cases, as documented in the top comment of disconnectSlaves().
2020-05-25 12:08:01 +02:00
ShooterIT
fc18f9a798 Implements sendfile for redis. 2020-05-25 12:08:01 +02:00
antirez
ed3fd6c524 Fix #7306 less aggressively.
Citing from the issue:

btw I suggest we change this fix to something else:
* We revert the fix.
* We add a call that disconnects chained replicas in the place where we trim the replica (that is a master i this case) offset.
This way we can avoid disconnections when there is no trimming of the backlog.

Note that we now want to disconnect replicas asynchronously in
disconnectSlaves(), because it's in general safer now that we can call
it from freeClient(). Otherwise for instance the command:

    CLIENT KILL TYPE master

May crash: clientCommand() starts running the linked of of clients,
looking for clients to kill. However it finds the master, kills it
calling freeClient(), but this in turn calls replicationCacheMaster()
that may also call disconnectSlaves() now. So the linked list iterator
of the clientCommand() will no longer be valid.
2020-05-25 12:08:01 +02:00
Maxime de Roucy
c0f60008cb acl: @replication command group
containing mandatory commands for replication (used by masteruser user)


Former-commit-id: 123bc125cfe7b30fdeef833759a33ccc854b5c51
2020-05-25 02:35:22 -04:00
John Sully
4bb5c35548 Fix issue #170. Intermittent crash destroying fastlock due to implicit type conversion in assert
Former-commit-id: ecbe168b5421687899a621c995e44f43faec2b71
2020-05-25 02:13:57 -04:00
John Sully
9a357ec4b5 Fix infinite logs of Error in rreplay
Former-commit-id: 00cef78858cd05fc067fa5002f2995653452c63f
2020-05-24 20:05:22 -04:00
John Sully
845027c291 Enable TLS connections
Former-commit-id: d05da0fabdfeb7eadce6546c7c1d85739b2794d7
2020-05-24 19:57:16 -04:00
John Sully
d176ae50ec don't keep feeding data to clients pending close
Former-commit-id: 4db3f2a443a6f71d1486781005ee37546e89d6e1
2020-05-24 12:46:11 -04:00
John Sully
5c35593937 Quiet TSAN during exit
Former-commit-id: fd710d590a6297b74e06ce6832dd75bfca0a266f
2020-05-24 12:45:10 -04:00
John Sully
6b93792f8f Install deps for daily workflow
Former-commit-id: 9b98c9edd3e8c9316514b7f9e0efc7a9f8d17851
2020-05-24 11:19:53 -04:00
John Sully
7630b63f06 Hold the AE lock until we process the repl backlog after a command
Former-commit-id: 4893eedacbf5f9210d02361ed07b9beee0c9f655
2020-05-24 11:19:23 -04:00
John Sully
0d8c7c7182 Add standardized way to post client dependent functions
Former-commit-id: fb50163b47cf660911dc7f67809c15ba1394613a
2020-05-24 02:46:39 -04:00
John Sully
65e4f82982 remove unnecessary cplusplus guard
Former-commit-id: 3b8463f9d20162369cffd9ed347865c5c8049f40
2020-05-24 02:45:55 -04:00