1240 Commits

Author SHA1 Message Date
antirez
f855db61b0 incrRefCount(): abort on statically allocated object. 2020-04-15 16:03:16 +02:00
antirez
61b153073f RDB: load files faster avoiding useless free+realloc.
Reloading of the RDB generated by

    DEBUG POPULATE 5000000
    SAVE

is now 25% faster.

This commit also prepares the ability to have more flexibility when
loading stuff from the RDB, since we no longer use dbAdd() but can
control exactly how things are added in the database.
2020-04-15 16:03:16 +02:00
John Sully
d3ebe3b6bd Merge tag '6.0-rc3' into redis_6_merge
Redis 6.0 RC3.


Former-commit-id: b2cb10de5f39b4d8e1ee19877c2bdaf19eefd9db
2020-04-14 22:56:19 -04:00
John Sully
68fd7c5c3d Merge commit 'c3b268a0bcded2d730790a772176483efb7c31fb' into redis_6_merge
Former-commit-id: 50768cd242c0360c6e943c57f866789280d30dc0
2020-04-14 22:25:44 -04:00
John Sully
02147b4be0 Merge commit '0f7dfc378ce5ab0524ac28b6142545bc41435386' into redis_6_merge
Former-commit-id: cc3ebbe5194e9744fb84ce490e90ac5fbe7f8716
2020-04-14 22:19:29 -04:00
John Sully
b20093e2f3 Merge commit 'e74e68c84f5eba8013769087c9a46cab811b8417' into redis_6_merge
Former-commit-id: 2825e515504cffcf6000be2e547ab1cbd86441bc
2020-04-14 20:55:29 -04:00
John Sully
f27524674a Merge commit '973297336fc05a601e17be70aba88e5dca6480ae' into redis_6_merge
Former-commit-id: ef1236b6009ebd7b00f6dd2f43df57ad95e51253
2020-04-14 20:19:48 -04:00
John Sully
d57fa7c8f9 Merge commit '349aa24511253b3e323064b95cefb4414f31726c' into redis_6_merge
Former-commit-id: 28cbed1d13961c5568f2bdc50c6a23107d3434d0
2020-04-14 20:09:53 -04:00
John Sully
12999c17dd Merge commit 'df83892760d19b1b7e92d76e72daf4834ad2df6c' into redis_6_merge
Former-commit-id: cde199a7973ad63317b68f581df607321e12bf46
2020-04-14 19:43:04 -04:00
John Sully
947bf289a5 Merge commit 'f42ce57d0f7eee6705fb0a81714cabe97a4f2c0a' into redis_6_merge
Former-commit-id: 5589a0a69ca6f5798b750a6a79f7e9b44d20e136
2020-04-14 19:22:44 -04:00
John Sully
5dbcd55669 Merge commit '51c1a9f8fbc12a9276489178242e498bb6ccbdba' into redis_6_merge
Former-commit-id: 09e8fb17cd0889ad17461e48446221b3955f5a8f
2020-04-14 18:44:42 -04:00
antirez
0fac746f0e PSYNC2: meaningful offset implemented.
A very commonly signaled operational problem with Redis master-replicas
sets is that, once the master becomes unavailable for some reason,
especially because of network problems, many times it wont be able to
perform a partial resynchronization with the new master, once it rejoins
the partition, for the following reason:

1. The master becomes isolated, however it keeps sending PINGs to the
replicas. Such PINGs will never be received since the link connection is
actually already severed.
2. On the other side, one of the replicas will turn into the new master,
setting its secondary replication ID offset to the one of the last
command received from the old master: this offset will not include the
PINGs sent by the master once the link was already disconnected.
3. When the master rejoins the partion and is turned into a replica, its
offset will be too advanced because of the PINGs, so a PSYNC will fail,
and a full synchronization will be required.

Related to issue #7002 and other discussion we had in the past around
this problem.


Former-commit-id: 5d6e8fe3e3e43162f0c57f580b6e8432274fca30
2020-04-14 17:56:09 -04:00
antirez
34f7c653d5 Remove RDB files used for replication in persistence-less instances.
Former-commit-id: b323645227a3e2cc5928e649586221aba508b10d
2020-04-14 17:27:05 -04:00
Guy Benoish
234e6e7b09 Diskless-load emptyDb-related fixes
1. Call emptyDb even in case of diskless-load: We want modules
   to get the same FLUSHDB event as disk-based replication.
2. Do not fire any module events when flushing the backups array.
3. Delete redundant call to signalFlushedDb (Called from emptyDb).


Former-commit-id: aa8a3077a2d20e66e34f72f2860d0cc3daad496e
2020-04-14 17:21:10 -04:00
antirez
6ad44b76b9 Use the special static refcount for stack objects. 2020-04-09 16:25:30 +02:00
antirez
d88f52ee7d RDB: refactor some RDB loading code into dbAddRDBLoad(). 2020-04-09 16:21:48 +02:00
antirez
399a6b2b47 incrRefCount(): abort on statically allocated object. 2020-04-09 16:20:41 +02:00
antirez
30adc62232 RDB: load files faster avoiding useless free+realloc.
Reloading of the RDB generated by

    DEBUG POPULATE 5000000
    SAVE

is now 25% faster.

This commit also prepares the ability to have more flexibility when
loading stuff from the RDB, since we no longer use dbAdd() but can
control exactly how things are added in the database.
2020-04-09 10:24:46 +02:00
antirez
35c64b898c Speedup INFO by counting client memory incrementally.
Related to #5145.

Design note: clients may change type when they turn into replicas or are
moved into the Pub/Sub category and so forth. Moreover the recomputation
of the bytes used is problematic for obvious reasons: it changes
continuously, so as a conservative way to avoid accumulating errors,
each client remembers the contribution it gave to the sum, and removes
it when it is freed or before updating it with the new memory usage.
2020-04-07 16:53:13 +02:00
zhaozhao.zz
abd5156f25 lazyfree: add a new configuration lazyfree-lazy-user-del
Delete keys in async way when executing DEL command, if
lazyfree-lazy-user-del is yes.
2020-04-07 16:53:13 +02:00
antirez
a518a9a766 LCS: initial functionality implemented. 2020-04-07 16:52:57 +02:00
srzhao
026cc11b05 Check OOM at script start to get stable lua OOM state.
Checking OOM by `getMaxMemoryState` inside script might get different result
with `freeMemoryIfNeededAndSafe` at script start, because lua stack and
arguments also consume memory.

This leads to memory `borderline` when memory grows near server.maxmemory:

- `freeMemoryIfNeededAndSafe` at script start detects no OOM, no memory freed
- `getMaxMemoryState` inside script detects OOM, script aborted

We solve this 'borderline' issue by saving OOM state at script start to get
stable lua OOM state.

related to issue #6565 and #5250.
2020-04-07 16:52:28 +02:00
Oran Agra
a38ff404bd modules don't signalModifiedKey in setKey() since that's done (optionally) in RM_CloseKey 2020-04-07 16:52:04 +02:00
antirez
f69876280c Speedup INFO by counting client memory incrementally.
Related to #5145.

Design note: clients may change type when they turn into replicas or are
moved into the Pub/Sub category and so forth. Moreover the recomputation
of the bytes used is problematic for obvious reasons: it changes
continuously, so as a conservative way to avoid accumulating errors,
each client remembers the contribution it gave to the sum, and removes
it when it is freed or before updating it with the new memory usage.
2020-04-07 12:07:54 +02:00
Salvatore Sanfilippo
094b47391d
Merge pull request #6243 from soloestoy/expand-lazy-free-server-del
lazyfree: add a new configuration lazyfree-lazy-user-del
2020-04-06 17:27:39 +02:00
antirez
121c51f4f3 Merge branch 'lcs' into unstable 2020-04-06 13:51:55 +02:00
Salvatore Sanfilippo
af5c11874c
Merge pull request #6797 from patpatbear/issue_#6565_memory_borderline
Check OOM at script start to get stable lua OOM state.
2020-04-06 11:59:01 +02:00
John Sully
6f15ce125c Add the ability to set a starting core # when setting thread affinity
Former-commit-id: 9e2e2067c6df5919f1c6b8b9e6e3457c7edc0755
2020-04-04 22:58:17 -04:00
John Sully
eeca58777d Fix subkey expires not replicating correctly, and AOF issues
Former-commit-id: bd183cdee13081a02efef5df75edf2292b872a16
2020-04-04 21:52:27 -04:00
Salvatore Sanfilippo
eb9ca33414
Merge pull request #6694 from oranagra/signal_modified_key
modules don't signalModifiedKey in setKey() since that's done (optionally) in RM_CloseKey
2020-04-02 19:00:20 +02:00
antirez
1010c1b43e LCS: initial functionality implemented. 2020-04-01 16:13:18 +02:00
Guy Benoish
a509400d58 Modules: Test MULTI/EXEC replication of RM_Replicate
Makse sure call() doesn't wrap replicated commands with
a redundant MULTI/EXEC

Other, unrelated changes:
1. Formatting compiler warning in INFO CLIENTS
2. Use CLIENT_ID_AOF instead of UINT64_MAX
2020-03-31 17:12:19 +02:00
Salvatore Sanfilippo
0c52ce6c8e
Merge pull request #7037 from guybe7/fix_module_replicate_multi
Modules: Test MULTI/EXEC replication of RM_Replicate
2020-03-31 17:00:57 +02:00
antirez
c3b268a0bc timeout.c created: move client timeouts code there. 2020-03-31 16:57:20 +02:00
antirez
616b1cb7ac Fix module commands propagation double MULTI bug.
37a10cef introduced automatic wrapping of MULTI/EXEC for the
alsoPropagate API. However this collides with the built-in mechanism
already present in module.c. To avoid complex changes near Redis 6 GA
this commit introduces the ability to exclude call() MUTLI/EXEC wrapping
for also propagate in order to continue to use the old code paths in
module.c.
2020-03-31 16:57:20 +02:00
antirez
67643ead69 Precise timeouts: cleaup the table on unblock.
Now that this mechanism is the sole one used for blocked clients
timeouts, it is more wise to cleanup the table when the client unblocks
for any reason. We use a flag: CLIENT_IN_TO_TABLE, in order to avoid a
radix tree lookup when the client was already removed from the table
because we processed it by scanning the radix tree.
2020-03-31 16:56:46 +02:00
antirez
ad94066ec8 Precise timeouts: fix comments after functional change. 2020-03-31 16:56:46 +02:00
antirez
a443ec2e9a Precise timeouts: use only radix tree for timeouts. 2020-03-31 16:56:46 +02:00
antirez
7add0f2490 Precise timeouts: working initial implementation. 2020-03-31 16:56:46 +02:00
antirez
9d6d177994 Precise timeouts: refactor unblocking on timeout. 2020-03-31 16:56:46 +02:00
Guy Benoish
d6eb3afd13 Modules: Test MULTI/EXEC replication of RM_Replicate
Makse sure call() doesn't wrap replicated commands with
a redundant MULTI/EXEC

Other, unrelated changes:
1. Formatting compiler warning in INFO CLIENTS
2. Use CLIENT_ID_AOF instead of UINT64_MAX
2020-03-31 13:55:51 +03:00
伯成
11db53f875 Boost up performance for redis PUB-SUB patterns matching
If lots of clients PSUBSCRIBE to same patterns, multiple pattens matching will take place. This commit change it into just one single pattern matching by using a `dict *` to store the unique pattern and which clients subscribe to it.
2020-03-31 12:47:14 +02:00
antirez
1b4bc60999 Merge branch 'pubsub_patterns_boost' of https://github.com/leeyiw/redis into leeyiw-pubsub_patterns_boost 2020-03-31 12:40:08 +02:00
antirez
9dcf878f1b Fix module commands propagation double MULTI bug.
37a10cef introduced automatic wrapping of MULTI/EXEC for the
alsoPropagate API. However this collides with the built-in mechanism
already present in module.c. To avoid complex changes near Redis 6 GA
this commit introduces the ability to exclude call() MUTLI/EXEC wrapping
for also propagate in order to continue to use the old code paths in
module.c.
2020-03-31 11:00:45 +02:00
antirez
dd7e61d77f timeout.c created: move client timeouts code there. 2020-03-27 16:35:03 +01:00
antirez
0e22cb2680 Precise timeouts: cleaup the table on unblock.
Now that this mechanism is the sole one used for blocked clients
timeouts, it is more wise to cleanup the table when the client unblocks
for any reason. We use a flag: CLIENT_IN_TO_TABLE, in order to avoid a
radix tree lookup when the client was already removed from the table
because we processed it by scanning the radix tree.
2020-03-27 16:35:03 +01:00
antirez
13e4c2a9b1 Precise timeouts: fix comments after functional change. 2020-03-27 16:35:03 +01:00
antirez
aa9d92d94a Precise timeouts: use only radix tree for timeouts. 2020-03-27 16:35:03 +01:00
antirez
324a8c91d0 Precise timeouts: working initial implementation. 2020-03-27 16:35:03 +01:00
antirez
8d116cc8a2 Precise timeouts: refactor unblocking on timeout. 2020-03-27 16:35:02 +01:00