715 Commits

Author SHA1 Message Date
John Sully
b98eed9945 Fix race condition in PUB/SUB and other async reply commands where the client can be freed before our handler is executed on the client thread. When this occurs the client pointer is dangling
Former-commit-id: 46a78c69e718f0aeb5e62f33c59458b15b8d9bc6
2019-08-26 20:18:52 -04:00
Oran Agra
78bbb9b58d Modlue fork is killed when the parent exists 2019-08-25 10:11:48 +03:00
antirez
0e9b5adbd3 Replication: clarify why repl_put_online_on_ack exists at all. 2019-08-05 17:38:15 +02:00
Salvatore Sanfilippo
fc5c2052b5
Merge pull request #6266 from madolson/dev-unstable-hide-auth-and-hello
Hide HELLO and AUTH from slowlog and monitor
2019-07-31 11:12:46 +02:00
Oran Agra
e5187ad2ae Merge remote-tracking branch 'oss/unstable' into module_rdb_load_errors 2019-07-30 15:14:08 +03:00
Oran Agra
4339706e07 Avoid diskelss-load if modules did not declare they handle read errors 2019-07-30 15:11:57 +03:00
Salvatore Sanfilippo
f42846e8c7
Merge pull request #6264 from oranagra/modules_api_aux_rdb
Implement module api for aux data in rdb
2019-07-30 11:50:44 +02:00
John Sully
2181eca5de Fix crash setting expire on a key with an existing subkey expiration
Former-commit-id: 4858fd893c8e638b0efdcd3ab2c6dc188a6dc6bd
2019-07-29 17:09:51 -04:00
John Sully
f3aa422a3f RREPLAY failures should be logged
Former-commit-id: 08b6a04055e950e53f043391ec9f9a09f654b1ee
2019-07-24 22:49:30 -04:00
Oran Agra
e91d9a6fff Extend modules API to allow modules report to redis INFO
this implements #6012
2019-07-24 12:58:15 +03:00
antirez
32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
John Sully
f88592451d Plumb support for sub expires to all expire related code
Former-commit-id: 184abac6942a9a6aa8783741b50b23210afddcc5
2019-07-23 18:53:59 -04:00
John Sully
2a2225d150 Initial prototype of EXPIREMEMBER command
Former-commit-id: 0b3d74ea67d616a6869cbd66198c8dd7ffa72eb7
2019-07-23 18:53:58 -04:00
John Sully
d58b2575e5 Support TTL stats with the new expire datastructure
Former-commit-id: 271df3dad4f55f20177a8a9a065778f4943835f1
2019-07-23 18:30:10 -04:00
John Sully
27cb5dba75 New expire datastructure and algorithm. Allows us to expire in sublinear time
Former-commit-id: 3880d2616c882e19169180dc10268564347b0279
2019-07-23 18:30:10 -04:00
antirez
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
Madelyn Olson
7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Oran Agra
3b6aeea44c Implement module api for aux data in rdb
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
2019-07-22 21:15:33 +03:00
antirez
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
Oran Agra
d7d028a7a7 Allow modules to handle RDB loading errors.
This is especially needed in diskless loading, were a short read could have
caused redis to exit. now the module can handle the error and return to the
caller gracefully.

this fixes #5326
2019-07-21 18:19:32 +03:00
John Sully
15f5c7ccf7 Merge commit '722446510faf0debf0d309708b2ed4fe4d939319' into unstable
Former-commit-id: 821c12c482d20b15dfb5a6eeab52e167997627d8
2019-07-19 00:32:43 -04:00
John Sully
4088de0ed2 Merge commit '677e95e2f5ee903682ce44a7bfd9558a22577926' into unstable
Former-commit-id: 4602f8c391409e9dd59f1fbee6a5ef011b219ca1
2019-07-19 00:01:56 -04:00
John Sully
e642441354 Fix compile errors from merges
Former-commit-id: 27a927fe0011536c6539d7c2a79ccfdaf78cee22
2019-07-18 23:35:51 -04:00
Angus Pearson
7b615e7274 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands,
and to keep OBJ_* enum to string canonicalization in one place.


Former-commit-id: 3cdc6e8d846e88cf4e250b2643662bde2a9317c5
2019-07-18 23:31:31 -04:00
John Sully
e41ab66722 Resolve race on neterr variable
Former-commit-id: 1512274f4c51dc4e8f4e3d11df2f35e051c0c079
2019-07-18 17:11:28 -04:00
John Sully
aa560582b4 Make object refcounts atomic, technically it doesn't need to be currently but the semantics are complicated and error prone. This is much safer with little performance impact
Former-commit-id: e310d33c121550f69b1c06d101db0c3f944a7fb0
2019-07-18 15:49:12 -04:00
Oran Agra
56258c6b7d Module API for Forking
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
2019-07-17 16:40:24 +03:00
zhaozhao.zz
6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
John Sully
bdc901f01c Implement test mode to make finding bugs easier
Former-commit-id: 971f69fe1ff9cce50492a47f306b312457e50b9f
2019-07-15 14:55:18 -04:00
John Sully
e78ed8cc64 Fix more locking deadlocks
Former-commit-id: 3081b6f98b5e7a9f3ef7cfe040236070398b081c
2019-07-12 23:51:45 -04:00
John Sully
8f1e7d9d5b Fix deadlock with client and ae locks
Former-commit-id: 8630339e43c1de1cd723bdfdca8ab5924e2cb8b0
2019-07-12 20:46:50 -04:00
John Sully
2a1d7890f6 Fix compile errors on GCC v5
Former-commit-id: 1f08a0efb33511ddc75c2acc62199bfcd0860137
2019-07-11 20:20:01 -04:00
John Sully
361b7655ef Fix active replication offset synchronization accounting, and enable the wait command
Former-commit-id: a5784ef09e71a9a45780a8f3dbab875b1f1fe1a5
2019-07-11 17:00:23 -04:00
Oran Agra
2de544cfcc diskless replication on slave side (don't store rdb to file), plus some other related fixes
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
2019-07-08 15:37:48 +03:00
Salvatore Sanfilippo
722446510f
Merge pull request #6116 from AngusP/scan-types
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
2019-07-08 12:53:34 +02:00
Angus Pearson
6eb52e200c Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
antirez
6b29f2d83d Client side caching: RESP2 support. 2019-07-05 12:24:28 +02:00
antirez
46edb55de9 Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
antirez
506764b3f8 Client side caching: hook inside call() for tracking. 2019-07-03 12:42:16 +02:00
antirez
c29f3bcf2e Client side caching: enable tracking mode. 2019-06-30 06:19:08 -04:00
antirez
45d64f229e Client side caching: fields and flags for tracking mode. 2019-06-29 20:08:41 -04:00
John Sully
a9837e9d95 Fix Travis CI issue
Former-commit-id: 8238d3207e484e5c9515d00ee7d2247168c03612
2019-06-17 23:56:23 -04:00
John Sully
9ca83cc9fd convert the pending write list to a vector for better performance and less cache thrashing
Former-commit-id: ae33ff3a6ef6d47bab63cf8362055c1ed210dbad
2019-06-17 23:00:12 -04:00
John Sully
913fe8eb78 Move more code out of the GIL
Former-commit-id: 59dbd625c143c1a890d4387f7a32c997f0d64f5f
2019-06-17 23:00:12 -04:00
Angus Pearson
38cd5fd9f6 Spelling cannonical -> canonical 2019-06-13 17:49:33 +01:00
Angus Pearson
e2adea2188 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands,
and to keep OBJ_* enum to string canonicalization in one place.
2019-06-10 17:41:44 +01:00
Oran Agra
09f99c2a92 make redis purge jemalloc after flush, and enable background purging thread
jemalloc 5 doesn't immediately release memory back to the OS, instead there's a decaying
mechanism, which doesn't work when there's no traffic (no allocations).
this is most evident if there's no traffic after flushdb, the RSS will remain high.

1) enable jemalloc background purging
2) explicitly purge in flushdb
2019-06-02 15:33:14 +03:00
John Sully
a7afb4c3f7 Fix regressions from RESP3 changes in commit 317f8b9d383f1b7f171aef7ea29f9e05abf0ba83
Former-commit-id: c5cbb77fc7e3b2b5124b9432e144d8c74190c755
2019-05-20 23:39:44 -04:00
John Sully
e2b3627360 Fix build breaks on some older compilers
Former-commit-id: 39a60c8120ac4a7c2aad83df00a4fbea07a801cc
2019-05-18 20:58:48 -04:00