3644 Commits

Author SHA1 Message Date
antirez
24103bafd0 Scripting: single commands replication mode implemented.
By calling redis.replicate_commands(), the scripting engine of Redis
switches to commands replication instead of replicating whole scripts.
This is useful when the script execution is costly but only results in a
few writes performed to the dataset.

Morover, in this mode, it is possible to call functions with side
effects freely, since the script execution does not need to be
deterministic: anyway we'll capture the outcome from the point of view
of changes to the dataset.

In this mode math.random() returns different sequences at every call.

If redis.replicate_commnads() is not called before any other write, the
command returns false and sticks to whole scripts replication instead.
2015-10-30 12:06:08 +01:00
antirez
52b4464ed6 call(): selective ability to prevent propagation on AOF / slaves. 2015-10-30 12:06:08 +01:00
antirez
78d342ec30 call(): don't inherit CLIENT_PREVENT_PROP + minor refactoring. 2015-10-30 12:06:08 +01:00
antirez
63988c9026 CLIENT REPLY command implemented: ON, OFF and SKIP modes.
Sometimes it can be useful for clients to completely disable replies
from the Redis server. For example when the client sends fire and forget
commands or performs a mass loading of data, or in caching contexts
where new data is streamed constantly. In such contexts to use server
time and bandwidth in order to send back replies to clients, which are
going to be ignored, is a shame.

Multiple mechanisms are possible to implement such a feature. For
example it could be a feature of MULTI/EXEC, or a command prefix
such as "NOREPLY SADD myset foo", or a different mechanism that allows
to switch on/off requests using the CLIENT command.

The MULTI/EXEC approach has the problem that transactions are not
strictly part of the no-reply semantics, and if we want to insert a lot
of data in a bulk way, creating a huge MULTI/EXEC transaction in the
server memory is bad.

The prefix is the best in this specific use case since it does not allow
desynchronizations, and is pretty clear semantically. However Redis
internals and client libraries are not prepared to handle this
currently.

So the implementation uses the CLIENT command, providing a new REPLY
subcommand with three options:

    CLIENT REPLY OFF disables the replies, and does not reply itself.
    CLIENT REPLY ON re-enables the replies, replying +OK.
    CLIENT REPLY SKIP only discards the reply of the next command, and
                      like OFF does not reply anything itself.

The reason to add the SKIP command is that it allows to have an easy
way to send conceptually "single" commands that don't need a reply
as the sum of two pipelined commands:

    CLIENT REPLY SKIP
    SET key value

Note that CLIENT REPLY ON replies with +OK so it should be used when
sending multiple commands that don't need a reply. However since it
replies with +OK the client can check that the connection is still
active and all the previous commands were received.

This is currently just into Redis "unstable" so the proposal can be
modified or abandoned based on users inputs.
2015-10-21 20:43:37 +02:00
Salvatore Sanfilippo
e209b1fde3 Merge pull request #2810 from dwlt/add-copy-replace-parameters-to-redis-trib-import
Add copy & replace parameters to redis-trib import
2015-10-15 13:02:35 +02:00
antirez
0c9c68346c PR 2813 fix ported to unstable. 2015-10-15 10:20:09 +02:00
David Thomson
80d1a5c065 Add back blank line 2015-10-14 06:58:36 +01:00
David Thomson
1361b951cd Update import command to optionally use copy and replace parameters 2015-10-14 06:56:14 +01:00
antirez
9fc48ed151 DEBUG RESTART/CRASH-AND-RECOVER [delay] implemented. 2015-10-13 11:12:25 +02:00
antirez
5b7bd883e6 Server: restartServer() API.
This new function is able to restart the server "in place". The current
Redis process executes the same executable it was executed with, using
the same arguments and configuration file.
2015-10-13 11:02:35 +02:00
antirez
33dedde038 Cluster: redis-trib fix, coverage for migrating=1 case.
Kinda related to #2770.
2015-10-09 16:15:58 +02:00
antirez
246b02eebe Fix extractLongLatOrReply() sanity check conditionals.
the check for lat/long valid ranges were performed inside the for loop,
two times instead of one, and the first time when the second element of
the array, xy[1], was yet not populated. This resulted into issue #2799.

Close issue #2799.
2015-10-07 22:27:18 +02:00
antirez
ea9586b9e0 Fix GEORADIUS COUNT option arity checks. 2015-10-06 09:25:28 +02:00
antirez
6432b8a4d1 Lazyfree options implemented in the configuration. 2015-10-05 12:11:27 +02:00
antirez
27ec5105eb Lazyfree: cond vars to enabled/disable it based on DEL context. 2015-10-02 15:27:57 +02:00
antirez
714b06d5fa Fixed a bug in the emptyDb() new implementation. 2015-10-01 13:02:26 +02:00
antirez
93da3c7874 FLUSHDB and FLUSHALL ASYNC option implemented. 2015-10-01 13:02:26 +02:00
antirez
da103b45ed Lazyfree: pending objects count in INFO output. 2015-10-01 13:02:26 +02:00
antirez
cf03e071ea Lazyfree: ability to free whole DBs in background. 2015-10-01 13:02:26 +02:00
antirez
062f789407 Lazyfree: keep count of objects to free. 2015-10-01 13:02:25 +02:00
antirez
8668aa6196 zmalloc.c converted to use atomicvar.h. 2015-10-01 13:02:25 +02:00
antirez
2ff5fef11b Atomic vars implemented in a more general way.
We have them into zmalloc.c, but this is going to replace that
implementation, so that it's possible to use the same idea everywhere
inside the code base.
2015-10-01 13:02:25 +02:00
antirez
7babd6e184 Lazyfree: incremental removed, only threaded survived. 2015-10-01 13:02:25 +02:00
antirez
fbc46738f4 Threaded lazyfree WIP #1. 2015-10-01 13:02:25 +02:00
antirez
9295b586b4 bio.c: new API bioWaitStepOfType(). 2015-10-01 13:02:25 +02:00
antirez
cc71d239bd Hash new implementation memleaks fixed. 2015-10-01 13:02:25 +02:00
antirez
cd94476e0b Lazyfree: Hash converted to use plain SDS WIP 5. 2015-10-01 13:02:25 +02:00
antirez
5eb85d8fe7 Lazyfree: Hash converted to use plain SDS WIP 4. 2015-10-01 13:02:25 +02:00
antirez
b4da37e32e Lazyfree: Hash converted to use plain SDS WIP 3. 2015-10-01 13:02:24 +02:00
antirez
5ae12b50d1 Lazyfree: Hash converted to use plain SDS WIP 2. 2015-10-01 13:02:24 +02:00
antirez
5fd01e29d9 Lazyfree: Hash converted to use plain SDS WIP 1. 2015-10-01 13:02:24 +02:00
antirez
d3b2c3f2f8 DEBUG DIGEST Set type memory leak fixed. 2015-10-01 13:02:24 +02:00
antirez
4ef4617c44 SORT memory leak fixed. 2015-10-01 13:02:24 +02:00
antirez
c4175281f6 Lazyfree: Sorted sets convereted to plain SDS. (several commits squashed) 2015-10-01 13:02:24 +02:00
antirez
062bf5ce19 Lazyfree: Convert Sets to use plains SDS (several commits squashed). 2015-10-01 13:02:24 +02:00
antirez
ac508699de Lazyfree: client output buffers no longer use Redis Objects. 2015-10-01 13:02:24 +02:00
antirez
ca8d13bd90 Lazyfree: a first implementation of non blocking DEL. 2015-10-01 13:00:19 +02:00
antirez
3b47d3f11d Call writeToClient() directly instead of the write handler. 2015-09-30 17:41:52 +02:00
antirez
0810ed570e Fix processEventsWhileBlocked() to handle PENDING_WRITE clients.
After the introduction of the list with clients with pending writes, to
process clients incrementally outside of the event loop we also need to
process the pending writes list.
2015-09-30 17:23:44 +02:00
antirez
af182a200f Refactoring: unlinkClient() added to lower freeClient() complexity. 2015-09-30 17:10:03 +02:00
antirez
8268ed723a Refactoring: new function to test if client has pending output. 2015-09-30 16:41:48 +02:00
antirez
5fad6cea15 Reverse list of clients with pending writes.
May potentially improve locality... not exactly clear if this makes a
difference or not. But for sure is harmless.
2015-09-30 16:29:42 +02:00
antirez
eaa5b8d9bc writeToClient(): don't remove write handler if not needed. 2015-09-30 16:29:42 +02:00
antirez
2fe94ee816 handleClientsWithPendingWrites(): detect dead clients. 2015-09-30 16:29:42 +02:00
antirez
7a7b8bc244 Move handleClientsWithPendingWrites() in networking.c. 2015-09-30 16:29:42 +02:00
antirez
e5121eba31 Avoid installing the client write handler when possible. 2015-09-30 16:29:41 +02:00
antirez
d638d3de00 redis-cli pipe mode: don't stay in the write loop forever.
The code was broken and resulted in redis-cli --pipe to, most of the
times, writing everything received in the standard input to the Redis
connection socket without ever reading back the replies, until all the
content to write was written.

This means that Redis had to accumulate all the output in the output
buffers of the client, consuming a lot of memory.

Fixed thanks to the original report of anomalies in the behavior
provided by Twitter user @fsaintjacques.
2015-09-30 16:24:21 +02:00
antirez
c0eb960fc1 Mark version of unstable branch in an unique way. 2015-09-29 17:30:24 +02:00
antirez
3ba00aae8e GEORADIUS: Don't report duplicates when radius is huge.
Georadius works by computing the center + neighbors squares covering all
the area of the specified position and radius. Then a distance filter is
used to remove elements which are actually outside the range.

When a huge radius is used, like 5000 km or more, adjacent neighbors may
collide and be the same, leading to the reporting of the same element
multiple times. This only happens in the edge case of huge radius but is
not ideal.

A robust but slow solution would involve qsorting the range to remove
all the duplicates. However since the collisions are only in adjacent
boxes, for the way they are ordered in the code, it is much faster to
just check if the current box is the same as the previous one processed.

This commit adds a regression test for the bug.

Fixes #2767.
2015-09-14 23:10:50 +02:00
antirez
cbb80c6107 MOVE re-add TTL check fixed.
getExpire() returns -1 when no expire exists.

Related to #2765.
2015-09-14 12:34:17 +02:00