7286 Commits

Author SHA1 Message Date
antirez
6df222cbcd Streams: XRANGE fuzz testing. 2017-12-01 10:24:24 +01:00
antirez
aa9cff0400 Streams: more advanced XADD and XRANGE tests. 2017-12-01 10:24:24 +01:00
antirez
d9a50e2d94 Streams: basic XADD tests. 2017-12-01 10:24:24 +01:00
antirez
7118da8954 Streams: AOF rewriting + minor iterator improvements. 2017-12-01 10:24:24 +01:00
antirez
3d15e3f722 Streams: export iteration API. 2017-12-01 10:24:24 +01:00
antirez
1b0b5fa224 Streams: implement streamReplyWithRange() in terms of the iterator. 2017-12-01 10:24:24 +01:00
antirez
e630b17282 Streams: stream iteration refactoring, WIP 2. 2017-12-01 10:24:24 +01:00
antirez
11045c4399 Streams: stream iteration refactoring, WIP 1. 2017-12-01 10:24:24 +01:00
antirez
ba13aba3a0 Streams: fix bug in XREAD last received ID processing. 2017-12-01 10:24:24 +01:00
antirez
110e66b0c2 Streams: fix memory leak in freeStream(). 2017-12-01 10:24:24 +01:00
antirez
91e09d0959 Streams: rewrite XADD ID argument for AOF/slaves. 2017-12-01 10:24:24 +01:00
antirez
eb8d0671f8 Streams: fix XADD API and keyspace notifications.
XADD was suboptimal in the first incarnation of the command, not being
able to accept an ID (very useufl for replication), nor options for
having capped streams.

The keyspace notification for streams was not implemented.
2017-12-01 10:24:24 +01:00
antirez
6410396372 Streams: When XREAD blocks without COUNT, set a default one.
A client may lose a lot of time between invocations of blocking XREAD,
for example because it is processing the messages or for any other
cause. When it returns back, it may provide a low enough message ID that
the server will block to send an unreasonable number of messages in a
single call. For this reason we set a COUNT when the client is blocked
with XREAD calls, even if no COUNT is given. This is arbitrarily set to
1000 because it's enough to avoid slowing down the reception of many
messages, but low enough to avoid to block.
2017-12-01 10:24:24 +01:00
antirez
789f74bba2 Streams: fix handleClientsBlockedOnKeys() access to invalid ID. 2017-12-01 10:24:24 +01:00
antirez
06a30111a8 Streams: fix XREAD ready-key signaling.
With lists we need to signal only on key creation, but streams can
provide data to clients listening at every new item added.
To make this slightly more efficient we now track different classes of
blocked clients to avoid signaling keys when there is nobody listening.
A typical case is when the stream is used as a time series DB and
accessed only by range with XRANGE.
2017-12-01 10:24:24 +01:00
antirez
f3b3ca41f7 Streams: fix XREAD timeout handling, zero is valid. 2017-12-01 10:24:24 +01:00
antirez
ea5ea8da3d Streams: XREAD related code to serve blocked clients. 2017-12-01 10:24:24 +01:00
antirez
6d54ec5e24 Streams: XREAD ability to block fixed. 2017-12-01 10:24:24 +01:00
antirez
b6f1106630 Streams: synchronous xread fixes and improvements. 2017-12-01 10:24:24 +01:00
antirez
bfeeeb9f34 Streams: XREAD get-key method fixed. 2017-12-01 10:24:24 +01:00
antirez
084a871019 Streams: XREAD get-keys method. 2017-12-01 10:24:24 +01:00
antirez
6f79b52dfa Streams: XREAD, first draft. Handling of blocked clients still missing. 2017-12-01 10:24:24 +01:00
antirez
54e221af43 Streams: XREAD arguments parsing. 2017-12-01 10:24:24 +01:00
antirez
428bb7cca0 Streams: augment client.bpop with XREAD specific fields. 2017-12-01 10:24:24 +01:00
antirez
6195349b50 Streams: more internal preparation for blocking XREAD. 2017-12-01 10:24:24 +01:00
antirez
896300be30 Streams: initial work to use blocking lists logic for streams XREAD. 2017-12-01 10:24:24 +01:00
antirez
a168cbef13 Streams: implement stream object release. 2017-12-01 10:24:24 +01:00
antirez
760ad8f65c Streams: XLEN command. 2017-12-01 10:24:24 +01:00
antirez
64a7ad038d Streams: Save stream->length in RDB. 2017-12-01 10:24:24 +01:00
antirez
fac8b716ab Streams: change listpack allocator to zmalloc. 2017-12-01 10:24:24 +01:00
antirez
90c980b105 Streams: RDB loading. RDB saving modified.
After a few attempts it looked quite saner to just add the last item ID
at the end of the serialized listpacks, instead of scanning the last
listpack loaded from head to tail just to fetch it. It's a disk space VS
CPU-and-simplicity tradeoff basically.
2017-12-01 10:24:24 +01:00
antirez
f028e4e68e Streams: RDB saving. 2017-12-01 10:24:24 +01:00
antirez
e4e216f8cc Streams: assign value of 6 to OBJ_STREAM + some refactoring. 2017-12-01 10:24:24 +01:00
antirez
71ad8ff26e Streams: 12 commits squashed into the initial Streams implementation. 2017-12-01 10:24:24 +01:00
antirez
95eaf3aada PSYNC2: Fix off by one buffer size in luaCreateFunction(). 2017-11-30 18:38:29 +01:00
antirez
6051ebea33 PSYNC2: just store script bodies into RDB.
Related to #4483. As suggested by @soloestoy, we can retrieve the SHA1
from the body. Given that in the new implementation using AUX fields we
ended copying around a lot to create new objects and strings, extremize
such concept and trade CPU for space inside the RDB file.
2017-11-30 18:38:26 +01:00
antirez
d14beab331 PSYNC2: luaCreateFunction() should handle NULL client parameter.
See #4483. This is needed because luaCreateFunction() is now called
from RDB loading code outside a client context.
2017-11-30 18:37:52 +01:00
antirez
445d37f24d PSYNC2: Save Lua scripts state into RDB file.
This is currently needed in order to fix #4483, but this can be
useful in other contexts, so maybe later we may want to remove the
conditionals and always save/load scripts.

Note that we are using the "lua" AUX field here, in order to guarantee
backward compatibility of the RDB file. The unknown AUX fields must be
discarded by past versions of Redis.
2017-11-30 18:37:52 +01:00
antirez
fbc1d53416 Regression test: Slave restart with EVALSHA in backlog issue #4483. 2017-11-30 18:37:10 +01:00
antirez
7f04efa1a8 Prevent corruption of server.executable after DEBUG RESTART.
Doing the following ended with a broken server.executable:

1. Start Redis with src/redis-server
2. Send CONFIG SET DIR /tmp/
3. Send DEBUG RESTART

At this point we called execve with an argv[0] that is no longer related
to the new path. So after the restart the absolute path of the
executable is recomputed in the wrong way. With this fix we pass the
absolute path already computed as argv[0].
2017-11-30 18:30:06 +01:00
antirez
f8d42c5ec6 Be more verbose when DEBUG RESTART fails. 2017-11-30 18:08:21 +01:00
zhaozhao.zz
44e3dc083a networking: optimize unlinkClient() in freeClient() 2017-11-30 18:11:05 +08:00
zhaozhao.zz
e40beab163 aof: cast sdslen to ssize_t 2017-11-30 10:27:12 +08:00
zhaozhao.zz
ba78815203 aof: fix the short write 2017-11-30 10:22:12 +08:00
Itamar Haber
22698eef45 Check arity in SLOWLOG before accessing arg 2017-11-30 00:30:30 +02:00
antirez
7435fd6be9 Merge branch 'lfu-fixes' into unstable 2017-11-29 17:16:13 +01:00
Itamar Haber
0bc9d1d6f1 Merge branch 'unstable' into help_subcommands 2017-11-28 21:28:40 +02:00
Itamar Haber
78aabf66ff Standardizes the 'help' subcommand
This adds a new `addReplyHelp` helper that's used by commands
when returning a help text. The following commands have been
touched: DEBUG, OBJECT, COMMAND, PUBSUB, SCRIPT and SLOWLOG.

WIP

Fix entry command table entry for OBJECT for HELP option.

After #4472 the command may have just 2 arguments.

Improve OBJECT HELP descriptions.

See #4472.

WIP 2

WIP 3
2017-11-28 21:15:45 +02:00
Salvatore Sanfilippo
e13b5ea614 Merge pull request #4200 from jeesyn/fix_typo
fix a typo
2017-11-28 18:44:11 +01:00
Salvatore Sanfilippo
5a836dc690 Merge pull request #4165 from zyegfryed/patch-1
Fix some typos
2017-11-28 18:43:45 +01:00