252 Commits

Author SHA1 Message Date
antirez
581c9133e5 Modules hooks: implement the FLUSHDB event. 2019-10-23 18:39:53 +02:00
Oran Agra
7737e5814a Merge remote-tracking branch 'antirez/unstable' into jemalloc_purge_bg 2019-10-04 13:53:40 +03:00
antirez
1ef0c2f630 Function renamed hasForkChild() -> hasActiveChildProcess(). 2019-09-27 12:03:09 +02:00
Salvatore Sanfilippo
5c5761bcd6 Merge branch 'unstable' into modules_fork 2019-09-27 11:24:06 +02:00
antirez
fe6d9e4c95 emptyDbGeneric(): call signalFlushDb() before deleting the keys.
This was broken since a refactoring performed recently by myself.
2019-07-30 11:20:54 +02:00
antirez
ac05029790 Move signalFlushedDb() into a better place.
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
2019-07-22 12:37:47 +02:00
antirez
df2e7cf4fc Client side caching: call the invalidation functions always.
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
2019-07-22 12:29:54 +02:00
Oran Agra
e70fbad802 Module API for Forking
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
2019-07-17 16:40:24 +03:00
zhaozhao.zz
8d5e3b7893 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
Salvatore Sanfilippo
3829c2c768 Merge pull request #5738 from bmerry/fix-swapdb-int-overflow
Make dbSwapDatabases take args as long
2019-07-12 12:36:33 +02:00
Oran Agra
29754ebe22 diskless replication on slave side (don't store rdb to file), plus some other related fixes
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
2019-07-08 15:37:48 +03:00
Salvatore Sanfilippo
9c1932beeb Merge pull request #6116 from AngusP/scan-types
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
2019-07-08 12:53:34 +02:00
Angus Pearson
00c9b4f15d Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
antirez
4b5027845e Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
Angus Pearson
5ca48db2e2 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands,
and to keep OBJ_* enum to string canonicalization in one place.
2019-06-10 17:41:44 +01:00
Oran Agra
f7833d560d make redis purge jemalloc after flush, and enable background purging thread
jemalloc 5 doesn't immediately release memory back to the OS, instead there's a decaying
mechanism, which doesn't work when there's no traffic (no allocations).
this is most evident if there's no traffic after flushdb, the RSS will remain high.

1) enable jemalloc background purging
2) explicitly purge in flushdb
2019-06-02 15:33:14 +03:00
Angus Pearson
4bbaf621a1 Implement SCAN cursor [TYPE type] modifier suggested in issue #6107.
Add tests to check basic functionality of this optional keyword, and also tested with
a module (redisgraph). Checked quickly with valgrind, no issues.

Copies name the type name canonicalisation code from `typeCommand`, perhaps this would
be better factored out to prevent the two diverging and both needing to be edited to
add new `OBJ_*` types, but this is a little fiddly with C strings.

The [redis-doc](https://github.com/antirez/redis-doc/blob/master/commands.json) repo
will need to be updated with this new arg if accepted.

A quirk to be aware of here is that the GEO commands are backed by zsets not their own
type, so they're not distinguishable from other zsets.

Additionally, for sparse types this has the same behaviour as `MATCH` in that it may
return many empty results before giving something, even for large `COUNT`s.
2019-05-22 16:39:04 +01:00
Dvir Volk
1a24f23a50 Renamed event name from "miss" to "keymiss" 2019-03-21 20:33:11 +02:00
Dvir Volk
50befc42ad added special flag for keyspace miss notifications 2019-03-21 11:47:14 +02:00
Dvir Volk
278c7a6b6d Added keyspace miss notifications support 2019-03-19 13:11:37 +02:00
antirez
94a9e27bb8 Use dictGetFairRandomKey() for RANDOMKEY as well. 2019-02-19 17:29:51 +01:00
antirez
da54f1fd3f Refactoring: always kill AOF/RDB child via helper functions. 2019-01-21 11:28:44 +01:00
Salvatore Sanfilippo
081ef93364 Merge branch 'unstable' into fixChildInfoPipeFdLeak 2019-01-21 11:20:56 +01:00
Salvatore Sanfilippo
3faf82d14b Merge pull request #5797 from trevor211/fixUpdateDictResizePolicy
Fix update dict resize policy
2019-01-21 11:14:48 +01:00
WuYunlong
1e1a5ceb65 Fix child info pipe fd leak when child process gets killed. 2019-01-21 17:48:45 +08:00
WuYunlong
f02ae4788b Update dict resize policy when rdb child process gets killed. 2019-01-21 17:33:18 +08:00
antirez
c8304b099d RESP3: most null replies converted. 2019-01-09 17:00:29 +01:00
antirez
293659ae04 RESP3: Use new deferred len API in dict.c. 2019-01-09 17:00:29 +01:00
Bruce Merry
26b109ae23 Make dbSwapDatabases take args as long
This prevents an integer overflow bug. Closes #5737.
2018-12-31 11:51:03 +02:00
lsytj0413
2538516df1 fix a typo: craeted -> created 2018-12-06 11:08:59 +08:00
Salvatore Sanfilippo
6f79cc62e6 Merge pull request #5469 from soloestoy/stat-key-miss-if-expired
if we read a expired key, misses++
2018-10-24 12:31:28 +02:00
antirez
5f8d96699d Simplify part of the #5470 patch. 2018-10-24 12:26:27 +02:00
Salvatore Sanfilippo
0f4e72fb8d Merge pull request #5470 from soloestoy/keys-no-trigger-expire
do not delete expired keys in KEYS command
2018-10-24 12:23:59 +02:00
hujie
2cf0ab76fd fix typo 2018-10-20 12:18:56 +08:00
zhaozhao.zz
2906ba5b5c if we read a expired key, misses++ 2018-10-19 18:16:29 +08:00
zhaozhao.zz
357909d7dd do not delete expired keys in KEYS command 2018-10-19 18:00:57 +08:00
zhaozhao.zz
b880a352d6 refactor dbOverwrite to make lazyfree work 2018-07-31 12:07:57 +08:00
antirez
8acaa863c8 Remove useless conditional from emptyDb().
Related to #4852.
2018-07-25 16:34:57 +02:00
antirez
b2aaf15b7b Make emptyDb() change introduced in #4852 simpler to read. 2018-07-25 16:32:52 +02:00
zhaozhao.zz
391a1cda58 optimize flushdb, avoid useless loops 2018-07-25 18:13:34 +08:00
Jack Drogon
bae1d36e5d Fix typo 2018-07-03 18:19:46 +02:00
zhaozhao.zz
07f01cc4cd fix some compile warnings 2018-06-28 17:22:59 +08:00
shenlongxing
dbd1e254bd fix typo 2018-06-21 22:08:09 +08:00
antirez
7f72840c36 Streams: fix xreadGetKeys() for correctness.
The old version could not handle the fact that "STREAMS" is a valid key
name for streams. Now we really try to parse the command like the
command implementation would do.

Related to #5028 and 4857.
2018-06-18 14:06:06 +02:00
Salvatore Sanfilippo
780c3bef6a Merge pull request #4857 from youjiali1995/fix-command-getkeys
Fix core dump when using some commands with wrong arguments.
2018-06-18 13:54:38 +02:00
antirez
102106fc61 Streams: fix xreadGetKeys() buffer overflow.
The loop allocated a buffer for the right number of keys positions, then
overflowed it going past the limit.

Related to #4857 and cause of the memory violation seen in #5028.
2018-06-18 13:51:19 +02:00
antirez
ea5d760ed5 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2018-06-14 18:11:04 +02:00
antirez
063dd636ab Fix infinite loop in dbRandomKey().
Thanks to @kevinmcgehee for signaling the issue and reasoning about the
consequences and potential fixes.

Issue #5015.
2018-06-14 18:08:21 +02:00
zhaozhao.zz
6786bcba94 fix exists command on slave 2018-06-14 01:30:07 +08:00
antirez
778b846ca9 In scanDatabaseForReadyLists() now we need to handle ZSETs as well.
Since the introduction of ZPOP makes this needed. Thanks to @oranagra
for reporting.
2018-06-12 17:28:40 +02:00