6613 Commits

Author SHA1 Message Date
antirez
431e12a033 CG: fix RDB saving when there are no consumer groups. 2018-03-15 12:54:10 +01:00
antirez
591e3f031d Streams: fix error description for XADD when specified ID is small. 2018-03-15 12:54:10 +01:00
antirez
096350cf33 CG: XCLAIM initial draft. 2018-03-15 12:54:10 +01:00
antirez
bab33460ab Make addReplyError...() family functions able to get error codes.
Now you can use:

    addReplyError("-MYERRORCODE some message");

If the error code is omitted, the behavior is like in the past,
the generic -ERR will be used.
2018-03-15 12:54:10 +01:00
antirez
75ad54c9a2 CG: test XACK remaining features. 2018-03-15 12:54:10 +01:00
antirez
c3396a6c8d CG: test XACK ability to remove items from the PELs. 2018-03-15 12:54:10 +01:00
antirez
08ea1440cf CG: test XPENDING ability to return pending items. 2018-03-15 12:54:10 +01:00
antirez
b1aa4a5c8b CG: test XGROUPREAD abilities. 2018-03-15 12:54:10 +01:00
antirez
e780693add CG: test group creation. 2018-03-15 12:54:10 +01:00
antirez
dfff06af91 CG: More specific duplicated group error. 2018-03-15 12:54:10 +01:00
antirez
81f45896d0 CG: RDB loading, fix inverted conditional. 2018-03-15 12:54:10 +01:00
antirez
84667bff70 CG: RDB loading first implementation. 2018-03-15 12:54:10 +01:00
antirez
76b264dd7e CG: RDB saving part 2, consumers. 2018-03-15 12:54:10 +01:00
antirez
3763d5c164 CG: RDB saving part 1, metadata and PEL. 2018-03-15 12:54:10 +01:00
antirez
ec0faf1bd9 CG: XPENDING should not create consumers and obey to count. 2018-03-15 12:54:10 +01:00
antirez
c0cf16f988 CG: XPENDING with start/stop/count variant implemented. 2018-03-15 12:54:10 +01:00
antirez
af40a25c52 CG: XPENDING without start/stop variant implemented. 2018-03-15 12:54:10 +01:00
antirez
505520ab09 CG: Now XREADGROUP + blocking operations work. 2018-03-15 12:54:10 +01:00
antirez
797d4df6fd CG: XACK should return zero when nothing is processed. 2018-03-15 12:54:10 +01:00
antirez
c6ec6fb82a CG: XACK implementation. 2018-03-15 12:54:10 +01:00
antirez
34d41d8aa5 CG: XREADGROUP can fetch data from the consumer PEL. 2018-03-15 12:54:10 +01:00
antirez
68a8dbb256 CG: first draft of streamReplyWithRangeFromConsumerPEL(). 2018-03-15 12:54:10 +01:00
antirez
9292fee76f CG: Fix order of calls in streamReplyWithRange().
We need to check if we are going to serve the request via the PEL before
inserting a deferred array len in the client output buffer.
2018-03-15 12:54:10 +01:00
antirez
4a5eaaeb76 CG: creation of NACK entries in PELs. 2018-03-15 12:54:10 +01:00
antirez
4de6308f6e CG: fix XREADGROUP ">" special ID parsing due to missing "continue". 2018-03-15 12:54:10 +01:00
antirez
5fa401d531 CG: streamCompareID() + group last_id updating. 2018-03-15 12:54:10 +01:00
antirez
6648d60246 CG: consumer lookup + initial streamReplyWithRange() work to supprot CG. 2018-03-15 12:54:10 +01:00
antirez
f7fbc91be3 CG: add & populate group+consumer in the blocking state. 2018-03-15 12:54:10 +01:00
antirez
2cf40e99f2 CG: fix parsing in XREADGROUP and streamLookupCG() NULL check. 2018-03-15 12:54:10 +01:00
antirez
7d3280143c CG: add XREADGROUP in the command table. 2018-03-15 12:54:10 +01:00
antirez
59eb392351 CG: XGROUPREAD group option parsing and groups lookup. 2018-03-15 12:54:10 +01:00
antirez
4e65528940 CG: fix raxFind() retval check in streamCreateCG(). 2018-03-15 12:54:10 +01:00
antirez
4925f7773c CG: data structures design + XGROUP CREATE implementation. 2018-03-15 12:54:10 +01:00
antirez
8bfc141157 Cluster: add test for the nofailover flag. 2018-03-14 16:30:32 +01:00
antirez
f467e1fce3 Cluster: ability to prevent slaves from failing over their masters.
This commit, in some parts derived from PR #3041 which is no longer
possible to merge (because the user deleted the original branch),
implements the ability of slaves to have a special configuration
preventing that they try to start a failover when the master is failing.

There are multiple reasons for wanting this, and the feautre was
requested in issue #3021 time ago.

The differences between this patch and the original PR are the
following:

1. The flag is saved/loaded on the nodes configuration.
2. The 'myself' node is now flag-aware, the flag is updated as needed
   when the configuration is changed via CONFIG SET.
3. The flag name uses NOFAILOVER instead of NO_FAILOVER to be consistent
   with existing NOADDR.
4. The redis.conf documentation was rewritten.

Thanks to @deep011 for the original patch.
2018-03-14 14:01:38 +01:00
antirez
c277aaf3b4 Stream: update the listpack pointer in streamTrimByLength(). 2018-03-01 17:26:02 +01:00
antirez
8e0fbea741 Remove warning from lpGet snprintf(). 2018-03-01 15:26:27 +01:00
antirez
1db665cda9 redis-cli: fix missed unit in array. Change define name. 2018-03-01 15:06:41 +01:00
Salvatore Sanfilippo
38ecac9dd0 Merge pull request #4714 from charsyam/feature/fix-out-of-index-range
[BugFix] Fix out of array index range for findBigKeys in redis-cli
2018-03-01 03:39:15 -08:00
antirez
99f94354a6 Actually use ae_flags to add AE_BARRIER if needed.
Many thanks to @Plasma that spotted this problem reviewing the code.
2018-02-28 18:03:51 +01:00
Salvatore Sanfilippo
c3934db151 Merge pull request #4715 from charsyam/feature/refactoring-make-condition-clear-for-rdb
[BugFix] fix calculation length in rdbSaveAuxField
2018-02-27 10:15:27 -08:00
antirez
550181a96b expireIfNeeded() needed a top comment documenting the behavior. 2018-02-27 16:44:43 +01:00
antirez
4db08588cc expireIfNeeded() comment: claim -> pretend. 2018-02-27 16:37:37 +01:00
charsyam
7bf2ef9dba refactoring-make-condition-clear-for-rdb 2018-02-27 21:55:20 +09:00
charsyam
aecbdde3c0 fix-out-of-index-range-for-redis-cli-findbigkey 2018-02-27 21:46:19 +09:00
antirez
b745b98f3d ae.c: insetad of not firing, on AE_BARRIER invert the sequence.
AE_BARRIER was implemented like:

    - Fire the readable event.
    - Do not fire the writabel event if the readable fired.

However this may lead to the writable event to never be called if the
readable event is always fired. There is an alterantive, we can just
invert the sequence of the calls in case AE_BARRIER is set. This commit
does that.
2018-02-27 13:06:42 +01:00
antirez
5a57c953c9 AOF: fix a bug that may prevent proper fsyncing when fsync=always.
In case the write handler is already installed, it could happen that we
serve the reply of a query in the same event loop cycle we received it,
preventing beforeSleep() from guaranteeing that we do the AOF fsync
before sending the reply to the client.

The AE_BARRIER mechanism, introduced in a previous commit, prevents this
problem. This commit makes actual use of this new feature to fix the
bug.
2018-02-27 13:06:42 +01:00
antirez
61da5a9da8 Cluster: improve crash-recovery safety after failover auth vote.
Add AE_BARRIER to the writable event loop so that slaves requesting
votes can't be served before we re-enter the event loop in the next
iteration, so clusterBeforeSleep() will fsync to disk in time.
Also add the call to explicitly fsync, given that we modified the last
vote epoch variable.
2018-02-27 13:06:42 +01:00
antirez
6c7d27c711 ae.c: introduce the concept of read->write barrier.
AOF fsync=always, and certain Redis Cluster bus operations, require to
fsync data on disk before replying with an acknowledge.
In such case, in order to implement Group Commits, we want to be sure
that queries that are read in a given cycle of the event loop, are never
served to clients in the same event loop iteration. This way, by using
the event loop "before sleep" callback, we can fsync the information
just one time before returning into the event loop for the next cycle.
This is much more efficient compared to calling fsync() multiple times.

Unfortunately because of a bug, this was not always guaranteed: the
actual way the events are installed was the sole thing that could
control. Normally this problem is hard to trigger when AOF is enabled
with fsync=always, because we try to flush the output buffers to the
socekt directly in the beforeSleep() function of Redis. However if the
output buffers are full, we actually install a write event, and in such
a case, this bug could happen.

This change to ae.c modifies the event loop implementation to make this
concept explicit. Write events that are registered with:

    AE_WRITABLE|AE_BARRIER

Are guaranteed to never fire after the readable event was fired for the
same file descriptor. In this way we are sure that data is persisted to
disk before the client performing the operation receives an
acknowledged.

However note that this semantics does not provide all the guarantees
that one may believe are automatically provided. Take the example of the
blocking list operations in Redis.

With AOF and fsync=always we could have:

    Client A doing: BLPOP myqueue 0
    Client B doing: RPUSH myqueue a b c

In this scenario, Client A will get the "a" elements immediately after
the Client B RPUSH will be executed, even before the operation is persisted.
However when Client B will get the acknowledge, it can be sure that
"b,c" are already safe on disk inside the list.

What to note here is that it cannot be assumed that Client A receiving
the element is a guaranteed that the operation succeeded from the point
of view of Client B.

This is due to the fact that the barrier exists within the same socket,
and not between different sockets. However in the case above, the
element "a" was not going to be persisted regardless, so it is a pretty
synthetic argument.
2018-02-27 13:06:42 +01:00
Salvatore Sanfilippo
3f379e3d70 Merge pull request #3828 from oranagra/sdsnewlen_pr
add SDS_NOINIT option to sdsnewlen to avoid unnecessary memsets.
2018-02-27 04:04:32 -08:00