gvsafronov/futriix
1
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

valkey-cli make source node ignores NOREPLICAS when doing the last CLUSTER SETSLOT (#928)

Browse Source 
This fixes #899. In that issue, the primary is cluster-allow-replica-migration no
and its replica is cluster-allow-replica-migration yes.

And during the slot migration:
1. Primary calling blockClientForReplicaAck, waiting its replica.
2. Its replica reconfiguring itself as a replica of other shards due to
replica migration and disconnect from the old primary.
3. The old primary never got the chance to receive the ack, so it got a
timeout and got a NOREPLICAS error.

In this case, the replicas might automatically migrate to another primary,
resulting in the client being unblocked with the NOREPLICAS error. In this
case, since the configuration will eventually propagate itself, we can safely
ignore this error on the source node.

Signed-off-by: Binbin <binloveplay1314@qq.com>
This commit is contained in:
Binbin 2024-08-23 16:22:30 +08:00 committed by GitHub
parent 5d97f5133c
commit 8045994972
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 43 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
src/valkey-cli.c
View File

@ -5030,11 +5030,18 @@ clusterManagerMoveSlot(clusterManagerNode *source, clusterManagerNode *target, i
* the face of primary failures. However, while our client is blocked on
* the primary awaiting replication, the primary might become a replica
* for the same reason as mentioned above, resulting in the client being
* unblocked with the role change error. */
* unblocked with the role change error.
*
* Another acceptable error can arise now that the primary pre-replicates
* `cluster setslot` commands to replicas while blocking the client on the
* primary. And during the block, the replicas might automatically migrate
* to another primary, resulting in the client being unblocked with the
* NOREPLICAS error. In this case, since the configuration will eventually
* propagate itself, we can safely ignore this error on the source node. */
success = clusterManagerSetSlot(source, target, slot, "node", err);
if (!success && err) {
const char *acceptable[] = {"ERR Please use SETSLOT only with masters.",
"ERR Please use SETSLOT only with primaries.", "UNBLOCKED"};
"ERR Please use SETSLOT only with primaries.", "UNBLOCKED", "NOREPLICAS"};
for (size_t i = 0; i < sizeof(acceptable) / sizeof(acceptable[0]); i++) {
if (!strncmp(*err, acceptable[i], strlen(acceptable[i]))) {
zfree(*err);

34
tests/unit/cluster/replica_migration.tcl → tests/unit/cluster/replica-migration.tcl
View File

@ -157,3 +157,37 @@ start_cluster 4 4 {tags {external:skip cluster} overrides {cluster-node-timeout
}
}
} my_slot_allocation cluster_allocate_replicas ;# start_cluster
start_cluster 4 4 {tags {external:skip cluster} overrides {cluster-node-timeout 1000 cluster-migration-barrier 999}} {
test "valkey-cli make source node ignores NOREPLICAS error when doing the last CLUSTER SETSLOT" {
R 3 config set cluster-allow-replica-migration no
R 7 config set cluster-allow-replica-migration yes
# Record the current primary node, server 7 will be migrated later.
set old_primary_ip [lindex [R 7 role] 1]
set old_primary_port [lindex [R 7 role] 2]
# Move slot 0 from primary 3 to primary 0.
set addr "[srv 0 host]:[srv 0 port]"
set myid [R 3 CLUSTER MYID]
set code [catch {
exec src/valkey-cli {*}[valkeycli_tls_config "./tests"] --cluster rebalance $addr --cluster-weight $myid=0
} result]
if {$code != 0} {
fail "valkey-cli --cluster rebalance returns non-zero exit code, output below:\n$result"
}
wait_for_cluster_propagation
wait_for_cluster_state "ok"
# Make sure server 3 is still a primary and has no replicas.
assert_equal [s -3 role] {master}
assert_equal [lindex [R 3 role] 2] {}
# And server 7 becomes a replica of another primary.
set new_primary_ip [lindex [R 7 role] 1]
set new_primary_port [lindex [R 7 role] 2]
assert_equal [s -7 role] {slave}
assert_not_equal "$old_primary_ip:$old_primary_port" "new_primary_ip:new_primary_port"
}
} my_slot_allocation cluster_allocate_replicas ;# start_cluster