108 Commits

Author SHA1 Message Date
John Sully
af459476ea Merge branch 'unstable' into redis_6_merge
Former-commit-id: 718aee242dd75abd16a5a6a89353d2a35f37b010
2020-03-25 15:47:12 -04:00
John Sully
7b2b437539 Fix failure to load subkey expires
Former-commit-id: 528a43bd6c80f073d928dd18c4f67f37cfd0977a
2020-03-25 01:38:58 -04:00
John Sully
e12170c724 Evict on load if we have a storage provider
Former-commit-id: bb091796c3da7282e040c7b72a28ec1c5f5ecfb7
2020-03-24 14:49:43 -04:00
John Sully
79f48a214e Prevent issue where count can be out of sync temporarily, causing crashes where we expect the count to be perfect
Former-commit-id: 77c9f36413c6f0cbb0b13a7ec746746c97faadcd
2020-03-24 00:21:12 -04:00
John Sully
3de971b10c Fix memory leak in RDB load
Former-commit-id: 4e9d2f08b11cc76bb2716514fa6d55f76160fd13
2020-02-16 18:42:14 -05:00
John Sully
2f62f3fbcd Fix memory leak in RDB load
Former-commit-id: 06ad1c15d719a34fed36244b12a593f749bbb8a6
2020-02-16 18:41:50 -05:00
John Sully
ed1b2ce2c8 Fix memory leak in RDB load
Former-commit-id: a424194f42e61a324489464a0fed14837b8191e4
2020-02-16 18:41:29 -05:00
John Sully
2ee97b5bb5 Fix failure to save RDB on shutdown
Former-commit-id: 1a8331c88a17deaa5faf34c7ea65957c57530546
2020-02-03 16:00:01 -05:00
John Sully
70994b5a07 RDB thread cleanup fixes
Former-commit-id: 3e30cf6de930c40c3a1c63a761b2018836d4ae52
2020-02-02 23:29:20 -05:00
John Sully
d8dcb94e37 endSnapshotAsync implementation (fails tests w/ multithreading)
Former-commit-id: b401e9fcae40cf1d4e4c1584f57e760d9adf36a9
2020-02-01 22:28:24 -05:00
John Sully
6be6785b31 Merge branch 'unstable' into keydbpro
Former-commit-id: aca1a6605685811833729d746df85781177d5a78
2020-01-30 21:15:42 -05:00
John Sully
958b86ddbb RDB memory leaks
Former-commit-id: 6208118b133c7f4209fd0a55d2a75341407e3e2c
2020-01-30 17:57:10 -05:00
John Sully
14cf73deaa Merge KeyDB 6
Former-commit-id: 9129b98c12b0da856bbda3eb68b4ddc7530693c9
2020-01-29 12:55:23 -05:00
John Sully
9440c60954 Yet more multithreading fixes
Former-commit-id: 9f23062ebdf389f0c95e1f4ab22c36ca96060e1f
2020-01-28 21:42:55 -05:00
John Sully
8e5fe97525 Merge remote-tracking branch 'redis/6.0' into redis_merge
Former-commit-id: ef9a3cadcf94326bf2f163db7698aad9a3c01690
2020-01-27 02:55:48 -05:00
John Sully
262b9a5d92 Initial implementation of the CRON command
Former-commit-id: 3204a39ada15ec33ac7926dc8b8f0e1875b99acb
2020-01-21 19:50:28 -05:00
John Sully
f5b428d6e1 Merge branch 'unstable' into keydbpro
Former-commit-id: 7ddcbb8ec0c842bd4a79b6e14ebd33616174e793
2019-12-27 18:17:36 -05:00
John Sully
da17594170 Fix some static analysis warnings
Former-commit-id: 42a8f22c21706f9ddcaa63ceafc5ad817c1fe876
2019-12-27 18:17:22 -05:00
John Sully
94ea48978d Add more test code, and fix bugs uncovered
Former-commit-id: 5362fa4b62f89cbc1e92e01c73a45c4e3718708b
2019-12-23 17:17:41 -05:00
John Sully
4466b2dba4 Perf fixes for expire snapshots
Former-commit-id: da805e4442815c89e85ca7b9fc855dd11ef2a6c8
2019-12-17 17:39:04 -05:00
John Sully
d95ae909a9 Threading fixes
Former-commit-id: 4f1cec6abd72d6e215880f2ce7812e88509cd218
2019-12-16 22:08:18 -05:00
John Sully
e7d8d1eb78 Extra expire validation on save
Former-commit-id: 037290cd2ec1b237df5236a86915d63d53fb48ce
2019-12-16 20:16:07 -05:00
John Sully
8851ddfd4a modify how the database array is stored so swaps are easier
Former-commit-id: 48fd6ec5ca69dd505d0e757862c09a898c19cf22
2019-12-08 16:49:59 -05:00
John Sully
995539a7f5 Bug: only update global flags for sync save
Former-commit-id: f01a7cf59bf965b35e5bb23e3ecbb7265ef8942a
2019-11-29 22:21:17 -05:00
John Sully
5012176262 Nested snapshot garbage collection. Works but huge fragmentation
Former-commit-id: 82b2a3f3dbf8f864d9157655b5422c69845c4019
2019-11-28 19:00:51 -05:00
John Sully
4346bcd7dc Restrict the snapshot API
Former-commit-id: cbafb88f267a9480cdbde6a88e9e1992e8a85971
2019-11-24 17:53:06 -05:00
John Sully
dc47a20da3 Fix multithreading data races
Former-commit-id: 80f6e5818fd575cb08a5f620c35eed1cd862eb57
2019-11-24 13:44:43 -05:00
John Sully
252db8939b Add support for nested snapshots
Former-commit-id: 43456c7807152062d59b2a90597b6204f637f5cd
2019-11-22 20:53:36 -05:00
John Sully
5e692ff15a Merge branch 'unstable' into advanced_db
Former-commit-id: 00ad497fb3f409dbfcaa62fa9ea5d793263eb13d
2019-11-21 20:21:48 -05:00
John Sully
3668715ba3 Fix issue #83
Former-commit-id: 3028a890ef11cd99b2c7538de0f480d2466eb150
2019-11-21 20:05:16 -05:00
John Sully
e25a4e80e7 Multiple threads should be able to get the same snapshot provided its not too old
Former-commit-id: 054331098ee18dfb1887fd2b0a67688ef894823e
2019-11-19 17:40:35 -05:00
John Sully
b51ece810f Final design of forkless background save. expires NYI
Former-commit-id: e2dc24b441bf52b181c820c853e0bc7524254f3f
2019-11-18 19:47:12 -05:00
John Sully
9df7197221 Forkless background save. NOT OPTIMIZED
Former-commit-id: bd9d8e05b0430efd226be773c0530828f1f6b428
2019-10-20 23:54:05 -04:00
John Sully
764a05fedd Factor out dependencies on rdb_child_pid which will be going away
Former-commit-id: 2ab4c6b99e3f3cb9f216bd81e78914a6fcd394f6
2019-10-17 23:10:45 -04:00
John Sully
5fb7d4a325 Dependencies for forkless RDB saving
Former-commit-id: c12be594754ac8079f467e0fa9f8580bd65e3a55
2019-10-17 22:53:50 -04:00
John Sully
7de19770cb Ensure RDB saving only uses data from the persistent data struct, and cleanup inheritance
Former-commit-id: fcdda67d87b3076023f459532c75392b72768a51
2019-10-17 17:20:47 -04:00
John Sully
63f2b3a987 Isolate the persistent parts of the database from the runtime parts
Former-commit-id: 8d63f5b560d067a634f2ef671962be8815c9e23c
2019-10-13 15:24:08 -04:00
John Sully
e4482cb695 Hide the database dict
Former-commit-id: 024365521c2b74275839562ba9326b6ea26d3836
2019-10-13 15:13:01 -04:00
John Sully
c61b6cc8fd Update more slave instances to use replica (Issue #75)
Former-commit-id: 252725d50fc9d4ff2b6e9246a36c38176d61beae
2019-10-13 12:29:20 -04:00
John Sully
d4e4b4f4d1 Update a few uses of slave to use the new replica terminology (Issue #75)
Former-commit-id: c15633ea9e784ef0e12e4015a75245661a8cf1b7
2019-10-13 12:12:05 -04:00
John Sully
08b353cd55 Fix double free in RDB load
Former-commit-id: 070c4818715b56645855abb72af47c846fc63027
2019-09-28 17:38:26 -04:00
John Sully
9f3cbd580e Fix multi master bugs: 1. we fail to create the temp file. 2. We use a master RDB as our backup even though we merged databases (and therefore it is not representitive)
Former-commit-id: e776474f68a2824bb7d4082c41991a9a9f3a9c9d
2019-09-26 20:35:51 -04:00
John Sully
9711895721 Subexpire entries should load/save
Former-commit-id: a55d98043655473ecdd53db2927381635eefc0b8
2019-07-23 18:53:59 -04:00
John Sully
f88592451d Plumb support for sub expires to all expire related code
Former-commit-id: 184abac6942a9a6aa8783741b50b23210afddcc5
2019-07-23 18:53:59 -04:00
John Sully
2a2225d150 Initial prototype of EXPIREMEMBER command
Former-commit-id: 0b3d74ea67d616a6869cbd66198c8dd7ffa72eb7
2019-07-23 18:53:58 -04:00
John Sully
27cb5dba75 New expire datastructure and algorithm. Allows us to expire in sublinear time
Former-commit-id: 3880d2616c882e19169180dc10268564347b0279
2019-07-23 18:30:10 -04:00
antirez
1aa5638b93 RDB: make sure to abort on LZF encoding error.
Former-commit-id: 27fe1658a2019bcd5d880e844bac21ccef8303f2
2019-07-19 01:03:16 -04:00
antirez
3e75e77dcd RDB: handle encoding errors with rdbExitReportCorruptRDB().
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.


Former-commit-id: 47feb2719ca7fd04e7e108ec1af0f777e536bf8a
2019-07-19 01:02:49 -04:00
John Sully
47e7652611 Active Replicas should always use their own slaveseldb variable when sending/saving rdbs - never those of their masters. This is because the Active Replica is also simultaneously a master
Former-commit-id: bb090b6d20bfc3d5d3ca07270f64bf15fad42681
2019-07-18 20:50:20 -04:00
John Sully
2a1d7890f6 Fix compile errors on GCC v5
Former-commit-id: 1f08a0efb33511ddc75c2acc62199bfcd0860137
2019-07-11 20:20:01 -04:00