10800 Commits

Author SHA1 Message Date
John Sully
6c510bc5cb Repair async rehash code removed by merge
Former-commit-id: 14a3ab8133c474bf8f1ad636c8699d5d59c5394e
2021-05-29 00:24:46 +00:00
John Sully
6b4f686f5f Merge branch 'unstable' into keydbpro
Former-commit-id: 205d8f18d2bb8df5253bab40578b006b7aa73fd5
2021-05-28 23:32:46 +00:00
John Sully
d53454a392 Ensure multi-thread handling of cached master is equivalent to single thread case
Former-commit-id: 7ee766aee13f501923afc127a74c79b29c0b0aa2
2021-05-26 17:29:47 +00:00
John Sully
96622e82f5 Avoid differences between single thread and multi-thread
Former-commit-id: 848c5ad784d6437533f6c86663b81e41e06a11ea
2021-05-26 17:29:01 +00:00
John Sully
551edf9443 Eliminate test issue while clients wait for async free
Former-commit-id: a41fb1b036dd4a933093f1df6dffe16f820153e4
2021-05-26 01:18:47 +00:00
John Sully
7cdbd48e7a Fix failure sending deferred aggregate lengths to clients on other threads (merge bug)
Former-commit-id: 915117ae7bac90fe41c7f36d97bcd654d6e90b62
2021-05-25 20:51:18 +00:00
John Sully
070ea24010 Remove duplicate code
Former-commit-id: de5fa787625ae27ce16737b539a981a0dede091a
2021-05-25 20:50:40 +00:00
John Sully
e75e080e7e Fix TLS test failure due to tls-port not marked as runtime configurable
Former-commit-id: b0c8efd8407168d1d5435f417932323656efe109
2021-05-25 20:50:23 +00:00
John Sully
e69b6d89e6 Fix crash in TLS due to merge
Former-commit-id: 1dcd5edbf953b74c9186672f8f157c6bd56bcdf8
2021-05-25 18:52:27 +00:00
John Sully
01d0145ed9 Build break fix: Remove dependency on future C++ feature
Former-commit-id: 87d7fc8e5d78a996562c477ecc0833c63f6fd8d7
2021-05-25 18:52:08 +00:00
John Sully
87add83a32 More OS X build breaks
Former-commit-id: 100a747eabf42bccbcf2991349be0fb3810799b6
2021-05-25 18:35:45 +00:00
John Sully
ddd0ae9daf OS X build break fix
Former-commit-id: 105fd03ef8cd810e67c01e4d5a3ea7816e215a99
2021-05-25 18:32:31 +00:00
John Sully
209f6ed49c Fix uninitialized variable
Former-commit-id: 9395a9c1d90859f88f3d82d66e50083c8409a3f8
2021-05-25 18:30:07 +00:00
John Sully
98b366af04 Fix duplicate checks (resolve PR comments)
Former-commit-id: 02f7efd9d4804b71765ae11d7fcf7adca46af750
2021-05-25 16:57:49 +00:00
John Sully
caba5639f4 Fix TLS Build Errors
Former-commit-id: aea86c5977c01debb4f4a4340c866aa5c5f20f04
2021-05-25 16:55:59 +00:00
John Sully
5267928381 Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17adec5d406d2e30a5b9ea71c07fce5c
2021-05-21 05:54:39 +00:00
John Sully
fe8efa916b Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0edaa724b9d060a6bb8edc5a6de65fa
2021-05-19 02:59:48 +00:00
John Sully
61e054f826 Fix test hang
Former-commit-id: 23647390e628de07759f8e7d8768a7f638edf01d
2021-05-07 00:28:10 +00:00
VivekSainiEQ
7b982ae4c5 Changed branding in various locations
Former-commit-id: 96bca72bb4528a0789354ee4fd6bf3ab66d82bf7
2021-05-06 20:13:45 -04:00
jsully
fe9f276f95 Merge branch 'cluster_bugfix' into 'keydbpro'
Set repl_down_since in freeClientAsync

See merge request keydb-dev/KeyDB-Pro!12

Former-commit-id: bd7591fe50b1e5565da2fa521438b88058d4da07
2021-05-03 21:41:05 +00:00
christian
12044bab09 Set repl_down_since in freeClientAsync
Former-commit-id: fb4bd2984651837fadd1dbbbb56327e8452b4a2a
2021-05-03 21:41:04 +00:00
Oran Agra
aa730ef1ea Redis 6.2.2 2021-04-19 21:39:40 +03:00
Oran Agra
f5ca1f9ee9 Merge unstable into 6.2 2021-04-19 21:36:00 +03:00
Viktor Söderqvist
61d3fdb474
Fix space included in links in modules API doc (#8818) 2021-04-19 21:35:14 +03:00
Hanna Fadida
53a4d6c3b1
Modules: adding a module type for key space notification (#8759)
Adding a new type mask ​for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported).

Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents,
and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' 
(REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications).

Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
2021-04-19 21:33:26 +03:00
guybe7
f40ca9cb58
Modules: Replicate lazy-expire even if replication is not allowed (#8816)
Before this commit using RM_Call without "!" could cause the master
to lazy-expire a key (delete it) but without replicating to replicas.
This could cause the replica's memory usage to gradually grow and
could also cause consistency issues if the master and replica have
a clock diff.
This bug was introduced in #8617

Added a test which demonstrates that scenario.
2021-04-19 17:16:02 +03:00
Harkrishn Patro
7a3d1487e4
ACL channels permission handling for save/load scenario. (#8794)
In the initial release of Redis 6.2 setting a user to only allow pubsub access to
a specific channel, and doing ACL SAVE, resulted in an assertion when
ACL LOAD was used. This was later changed by #8723 (not yet released),
but still not properly resolved (now it errors instead of crash).

The problem is that the server that generates an ACL file, doesn't know what
would be the setting of the acl-pubsub-default config in the server that will load it.
so ACL SAVE needs to always start with resetchannels directive.

This should still be compatible with old acl files (from redis 6.0), and ones from earlier
versions of 6.2 that didn't mess with channels.

Co-authored-by: Harkrishn Patro <harkrisp@amazon.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-19 13:27:44 +03:00
Wen Hui
0413fbc7d0
fix invalid master_link_down_since_seconds in info repication (#8785)
When replica never successfully connect to master, server.repl_down_since
will be initialized to 0, therefore, the info master_link_down_since_seconds
was showing the current unix timestamp, which does not make much sense.

This commit fixes the issue by showing master_link_down_since_seconds to -1.
means the replica never connect to master before.

This commit also resets this variable back to 0 when a replica is turned into
a master, so that it'll behave the same if the master is later turned into a
replica again.

The implication of this change is that if some app is checking if the value is > 60
do something, like conclude the replica is stale, this could case harm (changing
a big positive number with a small one).
2021-04-19 09:34:21 +03:00
guybe7
a60016e061
ReplicationCron: Prevent invalid access to freed pointer (#8799)
Fixes #8797
2021-04-16 16:56:38 +03:00
Wen Hui
374401d786
Avoid close before logging to preserve errno (#8703) 2021-04-15 21:11:40 +03:00
guybe7
7d749d810b
Fix error reply in case zset command is not the STORE variant (#8793) 2021-04-15 18:36:51 +03:00
guybe7
d63d02601f
Add a timeout mechanism for replicas stuck in fullsync (#8762)
Starting redis 6.0 (part of the TLS feature), diskless master uses pipe from the fork
child so that the parent is the one sending data to the replicas.
This mechanism has an issue in which a hung replica will cause the master to wait
for it to read the data sent to it forever, thus preventing the fork child from terminating
and preventing the creations of any other forks.

This PR adds a timeout mechanism, much like the ACK-based timeout,
we disconnect replicas that aren't reading the RDB file fast enough.
2021-04-15 17:18:51 +03:00
Bonsai
0a2621c673
clean condition and variable store to nwritten that is never read (#8788) 2021-04-14 22:44:08 -07:00
Bonsai
07601b7e07
fix typo, stracture to structure (#8784) 2021-04-14 15:46:54 +03:00
Viktor Söderqvist
d7920ff9b1
Modules API docs: Sections and links (#8442)
* Modules API docs: Link API function names to their definitions

Occurrences of API functions are linked to their definition.

A function index with links to all functions is added on the bottom
of the page.

Comment blocks in module.c starting with a markdown h2 heading are
used as sections. A table of contents is generated from these
headings.

The functions names are changed from h2 to h3, since they are now
rendered as sub-headings within each section.

Existing sections in module.c are used with some minor changes.
Some documentation text is added or sligtly modified.

The markdown renderer will add IDs which may clash with our
generated IDs. By prefixing section IDs with "section-" we make
them different.

Replace double dashes with a unicode long ndash
2021-04-14 00:58:05 +03:00
Huang Zhw
c6cd1e59b1
Remove extra param in role-change event of sentinelEvent. (#8742) 2021-04-13 20:19:42 +03:00
Viktor Söderqvist
4938052f6b
Small doc fix for stream module API (#8757)
In a code example, using RedisModule_FreeString instead of
RedisModule_Free makes it behave correctly regardless of whether
automatic memory is used or not.
2021-04-13 20:14:12 +03:00
Oran Agra
b278e44376
Revert "Fix: server will crash if rdbload or rdbsave method is not provided in module (#8670)" (#8771)
This reverts commit 808f3004f0de8c129b3067d8b2ce5002fa703e77.
2021-04-13 17:41:46 +03:00
Wen Hui
38da8d07d0
Clean up no-conf server warning for sentinel mode (#8769) 2021-04-13 16:28:54 +03:00
Oran Agra
733daef127
fix access to uninitialized var in checkClientPauseTimeoutAndReturnIfPaused (#8765)
server.client_pause_end_time is uninitialized, or actually 0, at startup,
which means this method would think the timeout was reached
and go look for paused clients.

This causes no harm since unpauseClients will not find any paused clients.
2021-04-13 08:41:12 +03:00
Oran Agra
175a9e3199
Fix busy loop in ae.c when timer event is about to fire (#8764)
The code used to decide on the next time to wake on a timer with
microsecond accuracy, but when deciding to go to sleep it used
milliseconds accuracy (with truncation), this means that it would wake
up too early, see that there's no timer to process, and go to sleep
again for 0ms again and again until the right microsecond arrived.

i.e. a timer for 100ms, would sleep for 99ms, but then do a busy loop
through the kernel in the last millisecond, triggering many calls to
beforeSleep.

The fix is to change all the logic in ae.c to work with microseconds,
which is good since most of the ae backends support micro (or even nano)
seconds. however the epoll backend, doesn't support micro, so to avoid
this problem it needs to round upwards, rather than truncate.

Issue created by the monotonic timer PR #7644 (redis 6.2)
Before that, all the timers in ae.c were in milliseconds (using
mstime), so when it requested the backend to sleep till the next timer
event, it would have worked ok.
2021-04-13 07:35:03 +03:00
John Sully
b2ffe82640 Allow prefetch even with a single thread
Former-commit-id: 2e68821b330a6bae8a352e37c3da643d504b2ce3
2021-04-12 04:44:52 +00:00
John Sully
211b0e5f80 Avoid posting unnecessary async tasks
Former-commit-id: 1661d2a05943f96992be195f5dc32dd9a67c0e68
2021-04-12 03:44:29 +00:00
John Sully
ea2bbfb5c1 Fix issue where GC is not free'd until a BGSAVE
Former-commit-id: 38523e6b508cf5f4b40c178dfe98554abea8f6bd
2021-04-12 03:44:15 +00:00
John Sully
14250faddd Make prefetch more aggressive
Former-commit-id: 25a5cfefcf7fa3451e92500f5d310290c4b6bbde
2021-04-12 03:42:05 +00:00
John Sully
1b121723e2 Don't prefetch when lock contention is low, it increases latency
Former-commit-id: 0d21614e0e5aba28acd364231823d51a3073081f
2021-04-12 03:41:53 +00:00
John Sully
e6782d8f1b Reduce P99 latency with async rehash
Former-commit-id: 7ea4c26fd82c0cdfa377183083f38a43336e480b
2021-04-12 03:39:13 +00:00
John Sully
df66c598e9 Make prefetch configurable
Former-commit-id: 3b660347d70cc25d57119080bd43fb4671e36488
2021-04-12 03:38:57 +00:00
Wang Yuan
a0e19e3cf1
Fix wrong check for aof fsync and handle aof fsync errno (#8751)
The bio aof fsync fd may be closed by main thread (AOFRW done handler)
and even possibly reused for another socket, pipe, or file.
This can can an EBADF or EINVAL fsync error, which will lead to -MISCONF errors failing all writes.
We just ignore these errno because aof fsync did not really fail.

We handle errno when fsyncing aof in bio, so we could know the real reason
when users get -MISCONF Errors writing to the AOF file error

Issue created with #8419
2021-04-11 08:14:31 +03:00
yjph
cd03e293c3
Fix the display of make install (#8667) 2021-04-10 21:25:53 +03:00