10948 Commits

Author SHA1 Message Date
John Sully
1ac6e3ecc5 Auto tune lock for high CPU tension scenarios
Former-commit-id: 8edbae2e04538f82a146a6c2b459a6dfcacf99b2
2020-05-30 18:22:27 -04:00
John Sully
2320e632d9 We only want one master connection in progress at a time, ensure that if the connection fails we try a different master
Former-commit-id: 8a0441c14475dc54616337270e092068acaa274a
2020-05-30 15:13:28 -04:00
John Sully
29a45dfabc TLS requires we explicitly marshal connected sockets when using them on a different thread
Former-commit-id: 341a8caef959883ca6fc71f81f8a3fed747b2341
2020-05-30 01:14:53 -04:00
Ben Schermel
8e7765b626 centos build fix
Former-commit-id: f17c492a633ea2be99567dfe2f8a59c0b0136d86
2020-05-29 22:23:48 -04:00
John Sully
27cae3750c sendFile blocks too long for use with active replication
Former-commit-id: aad6a7ce159a3679633020dc407a2068129bbd49
2020-05-29 20:27:45 -04:00
John Sully
e380087cdb Unify beforeSleep handling
Former-commit-id: 1cb48c7bf6a7e91e728a677902a7bfc64fe80dd6
2020-05-29 17:44:55 -04:00
antirez
bdd5411747 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-05-29 11:09:15 +02:00
antirez
1f8ea99b4b Fix handling of special chars in ACL LOAD.
Now it is also possible for ACL SETUSER to accept empty strings
as valid operations (doing nothing), so for instance

    ACL SETUSER myuser ""

Will have just the effect of creating a user in the default state.

This should fix #7329.
2020-05-29 11:07:13 +02:00
Salvatore Sanfilippo
5ef157bae1
Merge pull request #7330 from liuzhen/unstable
fix clusters mixing accidentally by gossip
2020-05-29 10:15:26 +02:00
John Sully
343745f218 Cluster crash
Former-commit-id: f25c405ad2a8004b79a816072cb011c1dc2407d5
2020-05-29 03:40:52 -04:00
John Sully
cfe9f8f3bc Merge tag '6.0.4' into unstable
Redis 6.0.4.


Former-commit-id: 9c31ac7925edba187e527f506e5e992946bd38a6
2020-05-29 00:57:07 -04:00
John Sully
3b6f82b056 fix TLS test failure
Former-commit-id: 57ca6facc3038e005656912b6378cb7fbe04f55a
2020-05-28 23:16:27 -04:00
antirez
886d287c40 Redis 6.0.4. 2020-05-28 12:18:38 +02:00
antirez
cc549b46a4 Replication: showLatestBacklog() refactored out. 2020-05-28 10:09:51 +02:00
antirez
377dd0515c Drop useless line from replicationCacheMaster(). 2020-05-28 10:09:51 +02:00
antirez
2112a57028 Remove the meaningful offset feature.
After a closer look, the Redis core devleopers all believe that this was
too fragile, caused many bugs that we didn't expect and that were very
hard to track. Better to find an alternative solution that is simpler.
2020-05-28 10:09:51 +02:00
antirez
d2eb6e0b47 Set a protocol error if master use the inline protocol.
We want to react a bit more aggressively if we sense that the master is
sending us some corrupted stream. By setting the protocol error we both
ensure that the replica will disconnect, and avoid caching the master so
that a full SYNC will be required. This is protective against
replication bugs.
2020-05-28 10:09:51 +02:00
Oran Agra
115ed1911e avoid using sendfile if tls-replication is enabled
this obviously broke the tests, but went unnoticed so far since tls
wasn't often tested.
2020-05-28 10:09:51 +02:00
antirez
11c748aaca Replication: log backlog creation event. 2020-05-28 10:09:51 +02:00
antirez
6a16a636bf Replication: showLatestBacklog() refactored out. 2020-05-28 10:08:16 +02:00
antirez
484af8ed53 Drop useless line from replicationCacheMaster(). 2020-05-27 17:08:51 +02:00
Kevin Fwu
151b12a80f Fix TLS certificate loading for chained certificates.
This impacts client verification for chained certificates (such as Lets
Encrypt certificates). Client Verify requires the full chain in order to
properly verify the certificate.
2020-05-27 08:53:29 -04:00
antirez
22472fe5a1 Remove the meaningful offset feature.
After a closer look, the Redis core devleopers all believe that this was
too fragile, caused many bugs that we didn't expect and that were very
hard to track. Better to find an alternative solution that is simpler.
2020-05-27 12:06:33 +02:00
antirez
325409a011 Set a protocol error if master use the inline protocol.
We want to react a bit more aggressively if we sense that the master is
sending us some corrupted stream. By setting the protocol error we both
ensure that the replica will disconnect, and avoid caching the master so
that a full SYNC will be required. This is protective against
replication bugs.
2020-05-27 11:45:49 +02:00
Liu Zhen
3984dc6539 fix clusters mixing accidentally by gossip
`clusterStartHandshake` will start hand handshake
and eventually send CLUSTER MEET message, which is strictly prohibited
in the REDIS CLUSTER SPEC.
Only system administrator can initiate CLUSTER MEET message.
Futher, according to the SPEC, rather than IP/PORT pairs, only nodeid
can be trusted.
2020-05-27 12:01:40 +08:00
antirez
94c026cd19 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-05-26 23:55:52 +02:00
antirez
7d35939206 Replication: log backlog creation event. 2020-05-26 23:55:18 +02:00
Salvatore Sanfilippo
caf7c50408
Merge pull request #7328 from oranagra/daily_tls_test
avoid using sendfile if tls-replication is enabled
2020-05-26 13:19:55 +02:00
Oran Agra
b9abecfc4c avoid using sendfile if tls-replication is enabled
this obviously broke the tests, but went unnoticed so far since tls
wasn't often tested.
2020-05-26 13:52:06 +03:00
John Sully
292c891ab1 Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: f98c1999f95ec59f7201056781cb89dc465a1461
2020-05-26 01:38:52 -04:00
John Sully
9ba82f9083 Complete TLS merge
Former-commit-id: 856d23023259d59ce766e9cd00526de0e7d06c2f
2020-05-26 01:38:29 -04:00
John Sully
d50c0c5ba3 Merge branch 'unstable' into keydbpro
Former-commit-id: b0636273806ce323627ce31c1ad7f86ceb39696d
2020-05-26 01:34:19 -04:00
John Sully
2bb0be3a96 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 1449c8b9fe647288da7de0ff10cc68ada4be8f92
2020-05-26 01:29:08 -04:00
John Sully
d7fa406a0e Fix client tracking, also ensure tracking tests are enabled
Former-commit-id: 1938af27f50f9686dc98b4839fb439cc03b4a250
2020-05-26 01:28:52 -04:00
antirez
2e591fc4a4 Clarify what is happening in PR #7320. 2020-05-25 12:08:01 +02:00
zhaozhao.zz
cbb51fb8f9 PSYNC2: second_replid_offset should be real meaningful offset
After adjustMeaningfulReplOffset(), all the other related variable
should be updated, including server.second_replid_offset.

Or the old version redis like 5.0 may receive wrong data from
replication stream, cause redis 5.0 can sync with redis 6.0,
but doesn't know meaningful offset.
2020-05-25 12:08:01 +02:00
antirez
e3f864b5fd Make disconnectSlaves() synchronous in the base case.
Otherwise we run into that:

Backtrace:
src/redis-server 127.0.0.1:21322(logStackTrace+0x45)[0x479035]
src/redis-server 127.0.0.1:21322(sigsegvHandler+0xb9)[0x4797f9]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd373c5e390]
src/redis-server 127.0.0.1:21322(_serverAssert+0x6a)[0x47660a]
src/redis-server 127.0.0.1:21322(freeReplicationBacklog+0x42)[0x451282]
src/redis-server 127.0.0.1:21322[0x4552d4]
src/redis-server 127.0.0.1:21322[0x4c5593]
src/redis-server 127.0.0.1:21322(aeProcessEvents+0x2e6)[0x42e786]
src/redis-server 127.0.0.1:21322(aeMain+0x1d)[0x42eb0d]
src/redis-server 127.0.0.1:21322(main+0x4c5)[0x42b145]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fd3738a3830]
src/redis-server 127.0.0.1:21322(_start+0x29)[0x42b409]

Since we disconnect all the replicas and free the replication backlog in
certain replication paths, and the code that will free the replication
backlog expects that no replica is connected.

However we still need to free the replicas asynchronously in certain
cases, as documented in the top comment of disconnectSlaves().
2020-05-25 12:08:01 +02:00
ShooterIT
8af1e513f6 Implements sendfile for redis. 2020-05-25 12:08:01 +02:00
antirez
3c21418cd8 Fix #7306 less aggressively.
Citing from the issue:

btw I suggest we change this fix to something else:
* We revert the fix.
* We add a call that disconnects chained replicas in the place where we trim the replica (that is a master i this case) offset.
This way we can avoid disconnections when there is no trimming of the backlog.

Note that we now want to disconnect replicas asynchronously in
disconnectSlaves(), because it's in general safer now that we can call
it from freeClient(). Otherwise for instance the command:

    CLIENT KILL TYPE master

May crash: clientCommand() starts running the linked of of clients,
looking for clients to kill. However it finds the master, kills it
calling freeClient(), but this in turn calls replicationCacheMaster()
that may also call disconnectSlaves() now. So the linked list iterator
of the clientCommand() will no longer be valid.
2020-05-25 12:08:01 +02:00
antirez
92a3ff6168 Clarify what is happening in PR #7320. 2020-05-25 11:47:38 +02:00
Maxime de Roucy
b9b4b4cfde acl: @replication command group
containing mandatory commands for replication (used by masteruser user)


Former-commit-id: 123bc125cfe7b30fdeef833759a33ccc854b5c51
2020-05-25 02:35:22 -04:00
John Sully
c15802af8b Fix issue #170. Intermittent crash destroying fastlock due to implicit type conversion in assert
Former-commit-id: ecbe168b5421687899a621c995e44f43faec2b71
2020-05-25 02:13:57 -04:00
zhaozhao.zz
eec769be59 PSYNC2: second_replid_offset should be real meaningful offset
After adjustMeaningfulReplOffset(), all the other related variable
should be updated, including server.second_replid_offset.

Or the old version redis like 5.0 may receive wrong data from
replication stream, cause redis 5.0 can sync with redis 6.0,
but doesn't know meaningful offset.
2020-05-25 11:17:54 +08:00
John Sully
fa671d5193 Fix infinite logs of Error in rreplay
Former-commit-id: 00cef78858cd05fc067fa5002f2995653452c63f
2020-05-24 20:05:22 -04:00
John Sully
fcfff165da Enable TLS connections
Former-commit-id: d05da0fabdfeb7eadce6546c7c1d85739b2794d7
2020-05-24 19:57:16 -04:00
John Sully
5a669218eb bump version
Former-commit-id: 9353b3aac2ab65120f2ad5caa2f8272ae02df380
2020-05-24 18:35:03 -04:00
John Sully
c4d524df9c Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: 056c672cf60a00980cb10e97fc5da93d8eada937
2020-05-24 18:34:15 -04:00
John Sully
15bf66b922 Fix test failures
Former-commit-id: c0b2263478d79f33d04ed409a68caa312884fa4c
2020-05-24 18:33:15 -04:00
John Sully
69b550d91a Fix crash on shutdown with snapshots pending
Former-commit-id: 124e79a81d6995ea7d5ba39d1830b591b705acbf
2020-05-24 17:11:56 -04:00
John Sully
cece963cf3 Merge branch 'unstable' into keydbpro
Former-commit-id: a830cf85df236885558c5571c0bf23cfb23e3655
2020-05-24 14:41:53 -04:00