27289 Commits

Author SHA1 Message Date
John Sully
eaaff16cca Don't sync too often it hurts perf
Former-commit-id: a20e89a457a0a682483c22f0f1cdb5c93c574d28
2021-05-03 04:15:46 +00:00
John Sully
f106327e4e Don't sync too often it hurts perf
Former-commit-id: a20e89a457a0a682483c22f0f1cdb5c93c574d28
2021-05-03 04:15:46 +00:00
John Sully
5583fbb838 Fix remaining test failures
Former-commit-id: 37e607f9b13b6601ff52e74e613fb369cab22b56
2021-05-03 02:37:02 +00:00
John Sully
3469890ac4 Fix remaining test failures
Former-commit-id: 37e607f9b13b6601ff52e74e613fb369cab22b56
2021-05-03 02:37:02 +00:00
John Sully
4677c62428 Fix issue where async load thread misses work
Former-commit-id: a24a7b093295c5f5d69feee9fbc37c64cfa8aa03
2021-04-30 18:48:23 +00:00
John Sully
915ab02c5d Fix issue where async load thread misses work
Former-commit-id: a24a7b093295c5f5d69feee9fbc37c64cfa8aa03
2021-04-30 18:48:23 +00:00
John Sully
7352e4a45f Fix majority of test issues with multithread load
Former-commit-id: 4db88176e33e3615ffb90852b49e76b12d5b4622
2021-04-30 18:45:37 +00:00
John Sully
f4159146c8 Fix majority of test issues with multithread load
Former-commit-id: 4db88176e33e3615ffb90852b49e76b12d5b4622
2021-04-30 18:45:37 +00:00
John Sully
556f394ed9 Initial implementation of multithread load
Former-commit-id: 87b0657c3acd7a3c89964afe1702851b44467c9a
2021-04-30 17:32:54 +00:00
John Sully
3023bf4e6e Initial implementation of multithread load
Former-commit-id: 87b0657c3acd7a3c89964afe1702851b44467c9a
2021-04-30 17:32:54 +00:00
VivekSainiEQ
7ef58a333f Performance optimizations
Former-commit-id: 7fd83d467784d293f7da78b74f9b9763ce387238
2021-04-29 18:51:30 +00:00
VivekSainiEQ
f0728a7ead Performance optimizations
Former-commit-id: 7fd83d467784d293f7da78b74f9b9763ce387238
2021-04-29 18:51:30 +00:00
VivekSainiEQ
d8367a92b2 Updated resize logic
Former-commit-id: e6d892ef21b7fc6f51433f32b01198038e555419
2021-04-29 17:01:06 +00:00
VivekSainiEQ
089cbfa580 Updated resize logic
Former-commit-id: e6d892ef21b7fc6f51433f32b01198038e555419
2021-04-29 17:01:06 +00:00
benschermel
f8ddcf1e8e update README with community slack info
Former-commit-id: 6163e7b13afcdd4f306e1690c400e93dfa904122
2021-04-29 03:11:54 +00:00
benschermel
9c2a85e8c8 update README with community slack info
Former-commit-id: 6163e7b13afcdd4f306e1690c400e93dfa904122
2021-04-29 03:11:54 +00:00
christianEQ
861e54942f fixed diagnostic tool to use correct obj files
Former-commit-id: 66547bd28ab025c4d118e6b8d35e2aa0b1f42f10
2021-04-28 16:02:44 +00:00
christianEQ
060b919250 fixed diagnostic tool to use correct obj files
Former-commit-id: 66547bd28ab025c4d118e6b8d35e2aa0b1f42f10
2021-04-28 16:02:44 +00:00
christianEQ
d1330899a8 updated makefile vars to keydb names
Former-commit-id: e6ab823473f7f215dcd61c3101b7c9ad310a0483
2021-04-27 21:14:05 +00:00
christianEQ
57836424e7 updated makefile vars to keydb names
Former-commit-id: e6ab823473f7f215dcd61c3101b7c9ad310a0483
2021-04-27 21:14:05 +00:00
christianEQ
0bf752a543 added keydb-diagnostic-tool binary (copy of benchmark)
Former-commit-id: a2c0bce4cc1403e01e70508b4297cfe5e76643cc
2021-04-27 20:56:37 +00:00
christianEQ
927c08aca5 added keydb-diagnostic-tool binary (copy of benchmark)
Former-commit-id: a2c0bce4cc1403e01e70508b4297cfe5e76643cc
2021-04-27 20:56:37 +00:00
VivekSainiEQ
05fe41b33a Primitive implementation of bypassing client buffer, stats are all messed up and print statements everywhere
Former-commit-id: 8ae310fb0f7b53add826f76891da333b63860001
2021-04-26 22:13:32 +00:00
VivekSainiEQ
5dfac7172f Primitive implementation of bypassing client buffer, stats are all messed up and print statements everywhere
Former-commit-id: 8ae310fb0f7b53add826f76891da333b63860001
2021-04-26 22:13:32 +00:00
John Sully
95ae92a691 Optimize freeMemory by lazy freeing objects
Former-commit-id: cca31ed5ee2f42975f0051cfabf1e88720b3d678
2021-04-21 01:20:13 +00:00
John Sully
0c7589580d Optimize freeMemory by lazy freeing objects
Former-commit-id: cca31ed5ee2f42975f0051cfabf1e88720b3d678
2021-04-21 01:20:13 +00:00
John Sully
3966f84862 Better RocksDB perf tuning
Former-commit-id: cbe1e0a7c1b5dc49c1adff07b7c32042e673acf4
2021-04-21 01:19:44 +00:00
John Sully
0adcb82652 Better RocksDB perf tuning
Former-commit-id: cbe1e0a7c1b5dc49c1adff07b7c32042e673acf4
2021-04-21 01:19:44 +00:00
John Sully
6e546b3c44 Fix issue where we reply a dangling pointer
Former-commit-id: fd11e490c39fe876d979eace5eaec56c645cdfcf
2021-04-21 01:19:29 +00:00
John Sully
da1360383c Fix issue where we reply a dangling pointer
Former-commit-id: fd11e490c39fe876d979eace5eaec56c645cdfcf
2021-04-21 01:19:29 +00:00
John Sully
9c1cf39b70 _dictExpandIfNeeded is called too late to be useful
Former-commit-id: 7f75ca5d3a9ed47465bceb22f5f74fd6f0760008
2021-04-21 01:18:38 +00:00
John Sully
0441c4c1fa _dictExpandIfNeeded is called too late to be useful
Former-commit-id: 7f75ca5d3a9ed47465bceb22f5f74fd6f0760008
2021-04-21 01:18:38 +00:00
Oran Agra
87c5cf52df Merge 6.2.2 release
Release 6.2.2
2021-04-20 08:03:58 +03:00
Oran Agra
959d6035e5
Merge 6.2.2 release
Release 6.2.2
2021-04-20 08:03:58 +03:00
Oran Agra
e0049d881d Redis 6.2.2 2021-04-19 21:39:40 +03:00
Oran Agra
aa730ef1ea Redis 6.2.2 2021-04-19 21:39:40 +03:00
Oran Agra
9b13be9b59 Merge unstable into 6.2 2021-04-19 21:36:00 +03:00
Oran Agra
f5ca1f9ee9 Merge unstable into 6.2 2021-04-19 21:36:00 +03:00
Viktor Söderqvist
9e70361abe Fix space included in links in modules API doc (#8818) 2021-04-19 21:35:14 +03:00
Viktor Söderqvist
61d3fdb474
Fix space included in links in modules API doc (#8818) 2021-04-19 21:35:14 +03:00
Hanna Fadida
6dc47733d0 Modules: adding a module type for key space notification (#8759)
Adding a new type mask ​for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported).

Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents,
and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' 
(REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications).

Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
2021-04-19 21:33:26 +03:00
Hanna Fadida
53a4d6c3b1
Modules: adding a module type for key space notification (#8759)
Adding a new type mask ​for key space notification, REDISMODULE_NOTIFY_MODULE, to enable unique notifications from commands on REDISMODULE_KEYTYPE_MODULE type keys (which is currently unsupported).

Modules can subscribe to a module key keyspace notification by RM_SubscribeToKeyspaceEvents,
and clients by notify-keyspace-events of redis.conf or via the CONFIG SET, with the characters 'd' or 'A' 
(REDISMODULE_NOTIFY_MODULE type mask is part of the '**A**ll' notation for key space notifications).

Refactor: move some pubsub test infra from pubsub.tcl to util.tcl to be re-used by other tests.
2021-04-19 21:33:26 +03:00
guybe7
8b33aa6661 Modules: Replicate lazy-expire even if replication is not allowed (#8816)
Before this commit using RM_Call without "!" could cause the master
to lazy-expire a key (delete it) but without replicating to replicas.
This could cause the replica's memory usage to gradually grow and
could also cause consistency issues if the master and replica have
a clock diff.
This bug was introduced in #8617

Added a test which demonstrates that scenario.
2021-04-19 17:16:02 +03:00
guybe7
f40ca9cb58
Modules: Replicate lazy-expire even if replication is not allowed (#8816)
Before this commit using RM_Call without "!" could cause the master
to lazy-expire a key (delete it) but without replicating to replicas.
This could cause the replica's memory usage to gradually grow and
could also cause consistency issues if the master and replica have
a clock diff.
This bug was introduced in #8617

Added a test which demonstrates that scenario.
2021-04-19 17:16:02 +03:00
Harkrishn Patro
b5b604aebc ACL channels permission handling for save/load scenario. (#8794)
In the initial release of Redis 6.2 setting a user to only allow pubsub access to
a specific channel, and doing ACL SAVE, resulted in an assertion when
ACL LOAD was used. This was later changed by #8723 (not yet released),
but still not properly resolved (now it errors instead of crash).

The problem is that the server that generates an ACL file, doesn't know what
would be the setting of the acl-pubsub-default config in the server that will load it.
so ACL SAVE needs to always start with resetchannels directive.

This should still be compatible with old acl files (from redis 6.0), and ones from earlier
versions of 6.2 that didn't mess with channels.

Co-authored-by: Harkrishn Patro <harkrisp@amazon.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-19 13:27:44 +03:00
Harkrishn Patro
7a3d1487e4
ACL channels permission handling for save/load scenario. (#8794)
In the initial release of Redis 6.2 setting a user to only allow pubsub access to
a specific channel, and doing ACL SAVE, resulted in an assertion when
ACL LOAD was used. This was later changed by #8723 (not yet released),
but still not properly resolved (now it errors instead of crash).

The problem is that the server that generates an ACL file, doesn't know what
would be the setting of the acl-pubsub-default config in the server that will load it.
so ACL SAVE needs to always start with resetchannels directive.

This should still be compatible with old acl files (from redis 6.0), and ones from earlier
versions of 6.2 that didn't mess with channels.

Co-authored-by: Harkrishn Patro <harkrisp@amazon.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-04-19 13:27:44 +03:00
sundb
cb73d3a084 Fix ouput buffer limit test (#8803)
The tail size of c->reply is 16kb, but in the test only publish a
few chars each time, due to a change in #8699, the obuf limit
is now checked a new memory allocation is made, so this test
would have sometimes failed to trigger a soft limit disconnection
in time.

The solution is to write bigger payloads to the output buffer, but
still limit their rate (not more than 100k/s).
2021-04-19 10:08:07 +03:00
sundb
3a955d9ad4
Fix ouput buffer limit test (#8803)
The tail size of c->reply is 16kb, but in the test only publish a
few chars each time, due to a change in #8699, the obuf limit
is now checked a new memory allocation is made, so this test
would have sometimes failed to trigger a soft limit disconnection
in time.

The solution is to write bigger payloads to the output buffer, but
still limit their rate (not more than 100k/s).
2021-04-19 10:08:07 +03:00
Wen Hui
06c087e5ce fix invalid master_link_down_since_seconds in info repication (#8785)
When replica never successfully connect to master, server.repl_down_since
will be initialized to 0, therefore, the info master_link_down_since_seconds
was showing the current unix timestamp, which does not make much sense.

This commit fixes the issue by showing master_link_down_since_seconds to -1.
means the replica never connect to master before.

This commit also resets this variable back to 0 when a replica is turned into
a master, so that it'll behave the same if the master is later turned into a
replica again.

The implication of this change is that if some app is checking if the value is > 60
do something, like conclude the replica is stale, this could case harm (changing
a big positive number with a small one).
2021-04-19 09:34:21 +03:00
Wen Hui
0413fbc7d0
fix invalid master_link_down_since_seconds in info repication (#8785)
When replica never successfully connect to master, server.repl_down_since
will be initialized to 0, therefore, the info master_link_down_since_seconds
was showing the current unix timestamp, which does not make much sense.

This commit fixes the issue by showing master_link_down_since_seconds to -1.
means the replica never connect to master before.

This commit also resets this variable back to 0 when a replica is turned into
a master, so that it'll behave the same if the master is later turned into a
replica again.

The implication of this change is that if some app is checking if the value is > 60
do something, like conclude the replica is stale, this could case harm (changing
a big positive number with a small one).
2021-04-19 09:34:21 +03:00