27381 Commits

Author SHA1 Message Date
antirez
74cd3ba381 Cluster: fixed MF condition in clusterHandleSlaveFailover().
For manual failover we need a manual failover in progress, and that
mf_can_start is true (master offset received and matched).
2014-02-05 16:01:56 +01:00
antirez
04fe000bf8 Cluster: fixed MF condition in clusterHandleSlaveFailover().
For manual failover we need a manual failover in progress, and that
mf_can_start is true (master offset received and matched).
2014-02-05 16:01:56 +01:00
antirez
847cfcf06a Cluster: CLUSTER FAILOVER replies with OK and logs the event. 2014-02-05 15:52:38 +01:00
antirez
c6f02fd67a Cluster: CLUSTER FAILOVER replies with OK and logs the event. 2014-02-05 15:52:38 +01:00
antirez
e457826cdc Cluster: check that a MF is in progress in manualFailoverCheckTimeout().
Otherwise it is always detected as a manual failover timed out.
2014-02-05 15:45:24 +01:00
antirez
c72449af30 Cluster: check that a MF is in progress in manualFailoverCheckTimeout().
Otherwise it is always detected as a manual failover timed out.
2014-02-05 15:45:24 +01:00
antirez
6edbc88416 Cluster: force AUTH ACK on manual failover.
When a slave requests masters vote for a manual failover, the
REQUEST_AUTH message is flagged in a special way in order to force the
masters to give the authorization even if the master is not marked as
failing.
2014-02-05 13:10:03 +01:00
antirez
b7402bcad5 Cluster: force AUTH ACK on manual failover.
When a slave requests masters vote for a manual failover, the
REQUEST_AUTH message is flagged in a special way in order to force the
masters to give the authorization even if the master is not marked as
failing.
2014-02-05 13:10:03 +01:00
antirez
45a900e448 Cluster: manual failover initial implementation. 2014-02-05 13:01:24 +01:00
antirez
4cf0cd5719 Cluster: manual failover initial implementation. 2014-02-05 13:01:24 +01:00
antirez
c94826dfdf CLIENT PAUSE and related API implemented.
The API is one of the bulding blocks of CLUSTER FAILOVER command that
executes a manual failover in Redis Cluster. However exposed as a
command that the user can call directly, it makes much simpler to
upgrade a standalone Redis instance using a slave in a safer way.

The commands works like that:

    CLIENT PAUSE <milliesconds>

All the clients that are not slaves and not in MONITOR state are paused
for the specified number of milliesconds. This means that slaves are
normally served in the meantime.

At the end of the specified amount of time all the clients are unblocked
and will continue operations normally. This command has no effects on
the population of the slow log, since clients are not blocked in the
middle of operations but only when there is to process new data.

Note that while the clients are unblocked, still new commands are
accepted and queued in the client buffer, so clients will likely not
block while writing to the server while the pause is active.
2014-02-04 16:16:09 +01:00
antirez
4919a13f50 CLIENT PAUSE and related API implemented.
The API is one of the bulding blocks of CLUSTER FAILOVER command that
executes a manual failover in Redis Cluster. However exposed as a
command that the user can call directly, it makes much simpler to
upgrade a standalone Redis instance using a slave in a safer way.

The commands works like that:

    CLIENT PAUSE <milliesconds>

All the clients that are not slaves and not in MONITOR state are paused
for the specified number of milliesconds. This means that slaves are
normally served in the meantime.

At the end of the specified amount of time all the clients are unblocked
and will continue operations normally. This command has no effects on
the population of the slow log, since clients are not blocked in the
middle of operations but only when there is to process new data.

Note that while the clients are unblocked, still new commands are
accepted and queued in the client buffer, so clients will likely not
block while writing to the server while the pause is active.
2014-02-04 16:16:09 +01:00
antirez
c7426670a0 Scripting: expire keys in scripts only at first access.
Keys expiring in the middle of the execution of Lua scripts are to
create inconsistencies in masters and / or AOF files. See the following
example:

    if redis.call("exists",KEYS[1]) == 1
    then
        redis.call("incr","mycounter")
    end

    if redis.call("exists",KEYS[1]) == 1
    then
        return redis.call("incr","mycounter")
    end

The script executes two times the same *if key exists then incrementcounter*
logic. However the two executions will work differently in the master and
the slaves, provided some unlucky timing happens.

In the master the first time the key may still exist, while the second time
the key may no longer exist. This will result in the key incremented just one
time. However as a side effect the master will generate a synthetic
`DEL` command in the replication channel in order to force the slaves to
expire the key (given that key expiration is master-driven).

When the same script will run in the slave, the key will no longer be
there, so the script will not increment the key.

The key idea used to implement the expire-at-first-lookup semantics was
provided by Marc Gravell.
2014-02-03 16:15:53 +01:00
antirez
b089ba98cc Scripting: expire keys in scripts only at first access.
Keys expiring in the middle of the execution of Lua scripts are to
create inconsistencies in masters and / or AOF files. See the following
example:

    if redis.call("exists",KEYS[1]) == 1
    then
        redis.call("incr","mycounter")
    end

    if redis.call("exists",KEYS[1]) == 1
    then
        return redis.call("incr","mycounter")
    end

The script executes two times the same *if key exists then incrementcounter*
logic. However the two executions will work differently in the master and
the slaves, provided some unlucky timing happens.

In the master the first time the key may still exist, while the second time
the key may no longer exist. This will result in the key incremented just one
time. However as a side effect the master will generate a synthetic
`DEL` command in the replication channel in order to force the slaves to
expire the key (given that key expiration is master-driven).

When the same script will run in the slave, the key will no longer be
there, so the script will not increment the key.

The key idea used to implement the expire-at-first-lookup semantics was
provided by Marc Gravell.
2014-02-03 16:15:53 +01:00
antirez
6ed232ebfd Allow CONFIG and SHUTDOWN while in stale-slave state. 2014-02-03 15:51:03 +01:00
antirez
b770079f2c Allow CONFIG and SHUTDOWN while in stale-slave state. 2014-02-03 15:51:03 +01:00
antirez
d0db6b9fcf Scripting: use mstime() and mstime_t for lua_time_start.
server.lua_time_start is expressed in milliseconds. Use mstime_t instead
of long long, and populate it with mstime() instead of ustime()/1000.

Functionally identical but more natural.
2014-02-03 15:45:40 +01:00
antirez
89884e8f6e Scripting: use mstime() and mstime_t for lua_time_start.
server.lua_time_start is expressed in milliseconds. Use mstime_t instead
of long long, and populate it with mstime() instead of ustime()/1000.

Functionally identical but more natural.
2014-02-03 15:45:40 +01:00
Salvatore Sanfilippo
7751d90b7d Merge pull request #1534 from gdi2290/patch-1
update copyright year
2014-02-03 02:18:24 -08:00
Salvatore Sanfilippo
b0335287ac Merge pull request #1534 from gdi2290/patch-1
update copyright year
2014-02-03 02:18:24 -08:00
PatrickJS
4400b317ab update copyright year 2014-02-03 02:10:54 -08:00
PatrickJS
79270cae64 update copyright year 2014-02-03 02:10:54 -08:00
antirez
3c7a5b29aa Test: fixed osx msg passing issue in testing framework.
The Redis test uses a server-clients model in order to parallelize the
execution of different tests. However in recent versions of osx not
setting the channel to a binary encoding caused issues even if AFAIK no
binary data is really sent via this channel. However now the channels
are deliberately set to a binary encoding and this solves the issue.

The exact issue was the test not terminating and giving the impression
of running forever, since test clients or servers were unable to
exchange the messages to continue.
2014-01-31 16:27:03 +01:00
antirez
331de6930a Test: fixed osx msg passing issue in testing framework.
The Redis test uses a server-clients model in order to parallelize the
execution of different tests. However in recent versions of osx not
setting the channel to a binary encoding caused issues even if AFAIK no
binary data is really sent via this channel. However now the channels
are deliberately set to a binary encoding and this solves the issue.

The exact issue was the test not terminating and giving the impression
of running forever, since test clients or servers were unable to
exchange the messages to continue.
2014-01-31 16:27:03 +01:00
antirez
3d54628d55 Redis.conf comment about tcp-backlog option improved. 2014-01-31 14:59:50 +01:00
antirez
e40d3e28e8 Redis.conf comment about tcp-backlog option improved. 2014-01-31 14:59:50 +01:00
antirez
2d253d1543 Option "backlog" renamed "tcp-backlog".
This is especially important since we already have a concept of backlog
(the replication backlog).
2014-01-31 14:56:10 +01:00
antirez
7be946fde2 Option "backlog" renamed "tcp-backlog".
This is especially important since we already have a concept of backlog
(the replication backlog).
2014-01-31 14:56:10 +01:00
Nenad Merdanovic
ca81272ea4 Add support for listen(2) backlog definition
In high RPS environments, the default listen backlog is not sufficient, so
giving users the power to configure it is the right approach, especially
since it requires only minor modifications to the code.
2014-01-31 14:52:10 +01:00
Nenad Merdanovic
d76aa96d1a Add support for listen(2) backlog definition
In high RPS environments, the default listen backlog is not sufficient, so
giving users the power to configure it is the right approach, especially
since it requires only minor modifications to the code.
2014-01-31 14:52:10 +01:00
antirez
1c93894ace Cluster: fix an error in migration-barrier comment in redis.conf. 2014-01-31 11:31:50 +01:00
antirez
b8bfbf46c5 Cluster: fix an error in migration-barrier comment in redis.conf. 2014-01-31 11:31:50 +01:00
antirez
244b6263db Cluster: barrer -> barrier in redis.conf. 2014-01-31 11:30:49 +01:00
antirez
cbfdd13b3a Cluster: barrer -> barrier in redis.conf. 2014-01-31 11:30:49 +01:00
antirez
bc300b22af Cluster: configurable replicas migration barrier.
It is possible to configure the min number of additional working slaves
a master should be left with, for a slave to migrate to an orphaned
master.
2014-01-31 11:26:36 +01:00
antirez
a7d30681c9 Cluster: configurable replicas migration barrier.
It is possible to configure the min number of additional working slaves
a master should be left with, for a slave to migrate to an orphaned
master.
2014-01-31 11:26:36 +01:00
antirez
b9c51f518b Sentinel: check arity for SENTINEL MASTER command.
This fixes issue #1530.
2014-01-31 10:13:38 +01:00
antirez
3ff1bb4b2e Sentinel: check arity for SENTINEL MASTER command.
This fixes issue #1530.
2014-01-31 10:13:38 +01:00
antirez
1d804400f5 Cluster: perform orphaned masters check before continue statements.
The check was placed in a way that conflicted with the continue
statements used by the node hearth beat code later that needs to skip
the current node sometimes. Moved at the start of the function so that's
always executed.
2014-01-30 18:23:31 +01:00
antirez
6c9359add1 Cluster: perform orphaned masters check before continue statements.
The check was placed in a way that conflicted with the continue
statements used by the node hearth beat code later that needs to skip
the current node sometimes. Moved at the start of the function so that's
always executed.
2014-01-30 18:23:31 +01:00
antirez
4f1fdc2c64 Cluster: replica migration implementation.
This feature allows slaves to migrate to orphaned masters (masters
without working slaves), as long as a set of conditions are met,
including the fact that the migrating slave needs to be in a
master-slaves ring with at least another slave working.
2014-01-30 18:05:11 +01:00
antirez
c2507b0ff6 Cluster: replica migration implementation.
This feature allows slaves to migrate to orphaned masters (masters
without working slaves), as long as a set of conditions are met,
including the fact that the migrating slave needs to be in a
master-slaves ring with at least another slave working.
2014-01-30 18:05:11 +01:00
antirez
ef82c59716 Cluster: swap two code blocks to have a more obvious flow. 2014-01-30 16:34:23 +01:00
antirez
5b4020fb42 Cluster: swap two code blocks to have a more obvious flow. 2014-01-30 16:34:23 +01:00
antirez
b582a91335 Cluster: remove not needed return statement breaking failover. 2014-01-29 17:28:46 +01:00
antirez
4beaaff8ea Cluster: remove not needed return statement breaking failover. 2014-01-29 17:28:46 +01:00
antirez
378e9745db Cluster: broadcast pong to other slaves in the same ring.
When we schedule a failover, broadcast a PONG to the slaves.
The other slaves that plan to get elected will do the same too, this way
it is likely that every slave will have a good picture of its own rank.

Note that this is N*N messages where N is the number of slaves for the
failing master, however usually even large clusters have many master
nodes but a limited number of replicas per node, so this is harmless.
2014-01-29 17:19:55 +01:00
antirez
3582054982 Cluster: broadcast pong to other slaves in the same ring.
When we schedule a failover, broadcast a PONG to the slaves.
The other slaves that plan to get elected will do the same too, this way
it is likely that every slave will have a good picture of its own rank.

Note that this is N*N messages where N is the number of slaves for the
failing master, however usually even large clusters have many master
nodes but a limited number of replicas per node, so this is harmless.
2014-01-29 17:19:55 +01:00
antirez
6546c6ff3b Cluster: log offset when announcing the failover election delay. 2014-01-29 17:16:10 +01:00
antirez
e2b59621a8 Cluster: log offset when announcing the failover election delay. 2014-01-29 17:16:10 +01:00