14537 Commits

Author SHA1 Message Date
John Sully
209f6ed49c Fix uninitialized variable
Former-commit-id: 9395a9c1d90859f88f3d82d66e50083c8409a3f8
2021-05-25 18:30:07 +00:00
John Sully
98b366af04 Fix duplicate checks (resolve PR comments)
Former-commit-id: 02f7efd9d4804b71765ae11d7fcf7adca46af750
2021-05-25 16:57:49 +00:00
John Sully
caba5639f4 Fix TLS Build Errors
Former-commit-id: aea86c5977c01debb4f4a4340c866aa5c5f20f04
2021-05-25 16:55:59 +00:00
John Sully
5267928381 Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17adec5d406d2e30a5b9ea71c07fce5c
2021-05-21 05:54:39 +00:00
John Sully
fe8efa916b Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0edaa724b9d060a6bb8edc5a6de65fa
2021-05-19 02:59:48 +00:00
John Sully
61e054f826 Fix test hang
Former-commit-id: 23647390e628de07759f8e7d8768a7f638edf01d
2021-05-07 00:28:10 +00:00
VivekSainiEQ
7b982ae4c5 Changed branding in various locations
Former-commit-id: 96bca72bb4528a0789354ee4fd6bf3ab66d82bf7
2021-05-06 20:13:45 -04:00
John Sully
a8d177e758 Pause execution during rdbLoadProgressCallback as its too risky to let it run
Former-commit-id: e70c01cb3e756d1e02ed190b76c73b7b7010c0d3
2021-05-06 00:42:49 +00:00
John Sully
10383a8234 Add endurance testing to better detect threading bugs
Former-commit-id: 945e428aa110968479fdcdfc2d5c5308a99eadc3
2021-05-06 00:09:07 +00:00
John Sully
f41b79cea6 Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: 446daf8f4c33005096372ad9e29ae9b3d465c64e
2021-05-03 22:58:56 +00:00
jsully
fe9f276f95 Merge branch 'cluster_bugfix' into 'keydbpro'
Set repl_down_since in freeClientAsync

See merge request keydb-dev/KeyDB-Pro!12

Former-commit-id: bd7591fe50b1e5565da2fa521438b88058d4da07
2021-05-03 21:41:05 +00:00
christian
12044bab09 Set repl_down_since in freeClientAsync
Former-commit-id: fb4bd2984651837fadd1dbbbb56327e8452b4a2a
2021-05-03 21:41:04 +00:00
John Sully
443f3511a9 Ensure multithread load works with FLASH storage
Former-commit-id: 24e2991c7aa2cef90a89b1640f7095235c5d34ed
2021-05-03 20:18:45 +00:00
John Sully
c83c3f1f8e Respect replica output buffer limits when adding large commands to the ring buffer
Former-commit-id: 37ec01cfd8a8da1e895c7cdc358d382d35ad59dd
2021-05-03 16:33:16 +00:00
John Sully
f106327e4e Don't sync too often it hurts perf
Former-commit-id: a20e89a457a0a682483c22f0f1cdb5c93c574d28
2021-05-03 04:15:46 +00:00
John Sully
3469890ac4 Fix remaining test failures
Former-commit-id: 37e607f9b13b6601ff52e74e613fb369cab22b56
2021-05-03 02:37:02 +00:00
John Sully
915ab02c5d Fix issue where async load thread misses work
Former-commit-id: a24a7b093295c5f5d69feee9fbc37c64cfa8aa03
2021-04-30 18:48:23 +00:00
John Sully
f4159146c8 Fix majority of test issues with multithread load
Former-commit-id: 4db88176e33e3615ffb90852b49e76b12d5b4622
2021-04-30 18:45:37 +00:00
John Sully
3023bf4e6e Initial implementation of multithread load
Former-commit-id: 87b0657c3acd7a3c89964afe1702851b44467c9a
2021-04-30 17:32:54 +00:00
benschermel
9c2a85e8c8 update README with community slack info
Former-commit-id: 6163e7b13afcdd4f306e1690c400e93dfa904122
2021-04-29 03:11:54 +00:00
John Sully
0c7589580d Optimize freeMemory by lazy freeing objects
Former-commit-id: cca31ed5ee2f42975f0051cfabf1e88720b3d678
2021-04-21 01:20:13 +00:00
John Sully
0adcb82652 Better RocksDB perf tuning
Former-commit-id: cbe1e0a7c1b5dc49c1adff07b7c32042e673acf4
2021-04-21 01:19:44 +00:00
John Sully
da1360383c Fix issue where we reply a dangling pointer
Former-commit-id: fd11e490c39fe876d979eace5eaec56c645cdfcf
2021-04-21 01:19:29 +00:00
John Sully
0441c4c1fa _dictExpandIfNeeded is called too late to be useful
Former-commit-id: 7f75ca5d3a9ed47465bceb22f5f74fd6f0760008
2021-04-21 01:18:38 +00:00
Oran Agra
959d6035e5
Merge 6.2.2 release
Release 6.2.2
2021-04-20 08:03:58 +03:00
Oran Agra
aa730ef1ea Redis 6.2.2 2021-04-19 21:39:40 +03:00
Oran Agra
f5ca1f9ee9 Merge unstable into 6.2 2021-04-19 21:36:00 +03:00
Viktor Söderqvist
61d3fdb474
Fix space included in links in modules API doc (#8818) 2021-04-19 21:35:14 +03:00
Hanna Fadida
53a4d6c3b1
Modules: adding a module type for key space notification (#8759)
Adding a new type mask ​for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported).

Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents,
and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' 
(REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications).

Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
2021-04-19 21:33:26 +03:00
guybe7
f40ca9cb58
Modules: Replicate lazy-expire even if replication is not allowed (#8816)
Before this commit using RM_Call without "!" could cause the master
to lazy-expire a key (delete it) but without replicating to replicas.
This could cause the replica's memory usage to gradually grow and
could also cause consistency issues if the master and replica have
a clock diff.
This bug was introduced in #8617

Added a test which demonstrates that scenario.
2021-04-19 17:16:02 +03:00
Harkrishn Patro
7a3d1487e4
ACL channels permission handling for save/load scenario. (#8794)
In the initial release of Redis 6.2 setting a user to only allow pubsub access to
a specific channel, and doing ACL SAVE, resulted in an assertion when
ACL LOAD was used. This was later changed by #8723 (not yet released),
but still not properly resolved (now it errors instead of crash).

The problem is that the server that generates an ACL file, doesn't know what
would be the setting of the acl-pubsub-default config in the server that will load it.
so ACL SAVE needs to always start with resetchannels directive.

This should still be compatible with old acl files (from redis 6.0), and ones from earlier
versions of 6.2 that didn't mess with channels.

Co-authored-by: Harkrishn Patro <harkrisp@amazon.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-19 13:27:44 +03:00
sundb
3a955d9ad4
Fix ouput buffer limit test (#8803)
The tail size of c->reply is 16kb, but in the test only publish a
few chars each time, due to a change in #8699, the obuf limit
is now checked a new memory allocation is made, so this test
would have sometimes failed to trigger a soft limit disconnection
in time.

The solution is to write bigger payloads to the output buffer, but
still limit their rate (not more than 100k/s).
2021-04-19 10:08:07 +03:00
Wen Hui
0413fbc7d0
fix invalid master_link_down_since_seconds in info repication (#8785)
When replica never successfully connect to master, server.repl_down_since
will be initialized to 0, therefore, the info master_link_down_since_seconds
was showing the current unix timestamp, which does not make much sense.

This commit fixes the issue by showing master_link_down_since_seconds to -1.
means the replica never connect to master before.

This commit also resets this variable back to 0 when a replica is turned into
a master, so that it'll behave the same if the master is later turned into a
replica again.

The implication of this change is that if some app is checking if the value is > 60
do something, like conclude the replica is stale, this could case harm (changing
a big positive number with a small one).
2021-04-19 09:34:21 +03:00
Yossi Gottlieb
c0f5c678c2
Revert cluster slot migration tests. (#8806)
Disables #8649 and subsequent attempts to stabilize the test.
2021-04-18 20:51:08 +03:00
Oran Agra
a9897b0084
Fix timing of new replication test (#8807)
In github actions CI with valgrind, i saw that even the fast replica
(one that wasn't paused), didn't get to complete the replication fast
enough, and ended up getting disconnected by timeout.

Additionally, due to a typo in uname, we didn't get to actually run the
CPU efficiency part of the test.
2021-04-18 15:12:34 +03:00
Oran Agra
f4b5a4d869
Improve testsuite print of log file (#8805)
1. the `dump_logs` option would have printed only logs of servers that were
   spawn before the test proc started, and not ones that the test proc
   started inside it.
2. when a server proc catches an exception it should normally forward the
   exception upwards, specifically when it's an assertion that should be
   caught by a test proc above. however, in `durable` mode, we caught all
   exceptions printed them to stdout and let the code continue,
   this was wrong to do for assertions, which should have still been
   propagated to the test function.
3. don't bother to search for crash log to print if we printed the the
   entire log anyway
4. if no crash log was found, no need to print anything (i.e. the fact it
   wasn't found)
5. rename warnings_from_file to crashlog_from_file
2021-04-18 11:55:54 +03:00
guybe7
a60016e061
ReplicationCron: Prevent invalid access to freed pointer (#8799)
Fixes #8797
2021-04-16 16:56:38 +03:00
Wen Hui
374401d786
Avoid close before logging to preserve errno (#8703) 2021-04-15 21:11:40 +03:00
guybe7
7d749d810b
Fix error reply in case zset command is not the STORE variant (#8793) 2021-04-15 18:36:51 +03:00
guybe7
d63d02601f
Add a timeout mechanism for replicas stuck in fullsync (#8762)
Starting redis 6.0 (part of the TLS feature), diskless master uses pipe from the fork
child so that the parent is the one sending data to the replicas.
This mechanism has an issue in which a hung replica will cause the master to wait
for it to read the data sent to it forever, thus preventing the fork child from terminating
and preventing the creations of any other forks.

This PR adds a timeout mechanism, much like the ACK-based timeout,
we disconnect replicas that aren't reading the RDB file fast enough.
2021-04-15 17:18:51 +03:00
Bonsai
0a2621c673
clean condition and variable store to nwritten that is never read (#8788) 2021-04-14 22:44:08 -07:00
Bonsai
07601b7e07
fix typo, stracture to structure (#8784) 2021-04-14 15:46:54 +03:00
YaacovHazan
645c664fbb
stabilized and improve pendingquerybuf test suit (#8780)
replace the hardcoded after 2000, with waiting for the sync and
wait for condition
2021-04-14 11:49:00 +03:00
Viktor Söderqvist
d7920ff9b1
Modules API docs: Sections and links (#8442)
* Modules API docs: Link API function names to their definitions

Occurrences of API functions are linked to their definition.

A function index with links to all functions is added on the bottom
of the page.

Comment blocks in module.c starting with a markdown h2 heading are
used as sections. A table of contents is generated from these
headings.

The functions names are changed from h2 to h3, since they are now
rendered as sub-headings within each section.

Existing sections in module.c are used with some minor changes.
Some documentation text is added or sligtly modified.

The markdown renderer will add IDs which may clash with our
generated IDs. By prefixing section IDs with "section-" we make
them different.

Replace double dashes with a unicode long ndash
2021-04-14 00:58:05 +03:00
Huang Zhw
c6cd1e59b1
Remove extra param in role-change event of sentinelEvent. (#8742) 2021-04-13 20:19:42 +03:00
Viktor Söderqvist
4938052f6b
Small doc fix for stream module API (#8757)
In a code example, using RedisModule_FreeString instead of
RedisModule_Free makes it behave correctly regardless of whether
automatic memory is used or not.
2021-04-13 20:14:12 +03:00
Oran Agra
b278e44376
Revert "Fix: server will crash if rdbload or rdbsave method is not provided in module (#8670)" (#8771)
This reverts commit 808f3004f0de8c129b3067d8b2ce5002fa703e77.
2021-04-13 17:41:46 +03:00
Oran Agra
c07e16fadd
Add more attempts to a timing sensitive test (#8770) 2021-04-13 17:35:10 +03:00
Wen Hui
38da8d07d0
Clean up no-conf server warning for sentinel mode (#8769) 2021-04-13 16:28:54 +03:00
Oran Agra
733daef127
fix access to uninitialized var in checkClientPauseTimeoutAndReturnIfPaused (#8765)
server.client_pause_end_time is uninitialized, or actually 0, at startup,
which means this method would think the timeout was reached
and go look for paused clients.

This causes no harm since unpauseClients will not find any paused clients.
2021-04-13 08:41:12 +03:00