345 Commits

Author SHA1 Message Date
John Sully
7e62ed49af Implement database merging for Active Replicas
Former-commit-id: 91e6368de0f0ecb7e4db497ce286a15336d4ec34
2019-03-28 15:12:43 -04:00
John Sully
8a9e9ff3f4 Merge branch 'unstable' of https://github.com/antirez/redis into unstable
Former-commit-id: 9322d604eea7b48df3feff47ce2c04f82291228f
2019-03-21 20:15:59 -04:00
Yossi Gottlieb
a88264d934 Add RedisModule_GetKeyNameFromIO(). 2019-03-15 10:23:27 +02:00
John Sully
25c335488e Implement load database dumps from S3. We already save.
Former-commit-id: a45f212693956a6fb1aacf465d88e940bbbfd56f
2019-03-13 16:53:37 -04:00
John Sully
02b030bc8c Module threading fixes
Former-commit-id: 2785a8b4d40b09caea5e209ab49fc5f1484981a8
2019-03-07 19:13:01 -05:00
zhaozhao.zz
fb81d1b3f8 Fix compile warning when log aux field 2019-03-04 19:43:00 +08:00
John Sully
15c301798f Merge branch 'unstable' of https://github.com/antirez/redis into unstable
We want that awesome multithreaded benchmark


Former-commit-id: 07594d4f958892f9270a18bc182728e4dfbf8144
2019-03-02 18:07:18 -05:00
antirez
b9b140e2e2 Use the RDB info fields to provide info to users.
Fix #5790 and 5878.

Maybe a better option was to have such fields named with the first
byte '%' as those are info fields for specification, however now to
break it in a backward incompatible way is not an option, so let's use
the fields actively to provide info when sensible, otherwise ignore
when they are not really helpful.
2019-03-02 21:17:42 +01:00
John Sully
f4b060e0bd Prevent mixed up client replies, and deadlocks 2019-02-22 01:24:16 -05:00
John Sully
29c1105132 Multithreading works! 2019-02-20 01:20:26 -05:00
John Sully
d62178ec8c Initial work of multithreaded key-db. Note: Fails tests 2019-02-11 03:36:18 -05:00
John Sully
2f9d958e96 Reduce memory usage for in place strings by 8 bytes 2019-02-09 13:04:18 -05:00
John Sully
d3fc46b3a8 complete rebranding with tests passing 2019-02-09 10:11:46 -05:00
John Sully
e38d1e6c7f Support AWS S3 saving via the s3 cli tools 2019-02-06 01:06:48 -05:00
John Sully
41a24e689e Make main headers C++ safe, and change rdb to use file descriptor instead of FILE pointer 2019-02-05 23:36:40 -05:00
John Sully
2f753a3539 complete malloc memory class work, and pass tests 2019-02-04 16:56:13 -05:00
John Sully
5e6b9b9c54 crash fixes 2019-01-30 01:15:10 -05:00
John Sully
0ffcf355fe Custom flash heap 2019-01-29 18:10:46 -05:00
antirez
23abe9612a Refactoring: always kill AOF/RDB child via helper functions. 2019-01-21 11:28:44 +01:00
antirez
4dc69497f5 Refactoring: always kill AOF/RDB child via helper functions. 2019-01-21 11:28:44 +01:00
antirez
4c65479744 Abort instead of crashing when loading bad stream master key.
See #5612.
2018-11-28 16:24:53 +01:00
antirez
edd3939bef Abort instead of crashing when loading bad stream master key.
See #5612.
2018-11-28 16:24:53 +01:00
Jack Drogon
df7bafeb44 Fix typo 2018-07-03 18:19:46 +02:00
Jack Drogon
93238575f7 Fix typo 2018-07-03 18:19:46 +02:00
Salvatore Sanfilippo
578583e31b Merge pull request #5049 from youjiali1995/fix-load-rdb
Fix rdbLoadIntegerObject() to create shared objects when needed.
2018-06-22 15:47:45 +02:00
Salvatore Sanfilippo
70b7fa2c52
Merge pull request #5049 from youjiali1995/fix-load-rdb
Fix rdbLoadIntegerObject() to create shared objects when needed.
2018-06-22 15:47:45 +02:00
youjiali1995
12cd55a62e Fix rdbLoadIntegerObject() to create shared objects when needed. 2018-06-21 19:23:16 +08:00
youjiali1995
df6644fe0e Fix rdbLoadIntegerObject() to create shared objects when needed. 2018-06-21 19:23:16 +08:00
Salvatore Sanfilippo
f0d8e44f7c Merge pull request #5039 from oranagra/rdb_dbsize_hint
64 bit RDB_OPCODE_RESIZEDB in rdb saving
2018-06-20 11:18:31 +02:00
Salvatore Sanfilippo
c1e8240565
Merge pull request #5039 from oranagra/rdb_dbsize_hint
64 bit RDB_OPCODE_RESIZEDB in rdb saving
2018-06-20 11:18:31 +02:00
Guy Benoish
56163a78a4 Enhance RESTORE with RDBv9 new features
RESTORE now supports:
1. Setting LRU/LFU
2. Absolute-time TTL

Other related changes:
1. RDB loading will not override LRU bits when RDB file
   does not contain the LRU opcode.
2. RDB loading will not set LRU/LFU bits if the server's
   maxmemory-policy does not match.
2018-06-20 15:11:08 +07:00
Guy Benoish
b5197f1fc9 Enhance RESTORE with RDBv9 new features
RESTORE now supports:
1. Setting LRU/LFU
2. Absolute-time TTL

Other related changes:
1. RDB loading will not override LRU bits when RDB file
   does not contain the LRU opcode.
2. RDB loading will not set LRU/LFU bits if the server's
   maxmemory-policy does not match.
2018-06-20 15:11:08 +07:00
Oran Agra
5dc8ef5b47 64 bit RDB_OPCODE_RESIZEDB in rdb saving
this complication in the code is from times were rdbSaveLen didn't support 64 bits.
2018-06-19 16:43:12 +03:00
Oran Agra
5cd3c9529d 64 bit RDB_OPCODE_RESIZEDB in rdb saving
this complication in the code is from times were rdbSaveLen didn't support 64 bits.
2018-06-19 16:43:12 +03:00
Salvatore Sanfilippo
a5c30009e3 Merge pull request #4758 from soloestoy/rdb-save-incremental-fsync
Rdb save incremental fsync
2018-06-16 10:59:37 +02:00
Salvatore Sanfilippo
94658303e9
Merge pull request #4758 from soloestoy/rdb-save-incremental-fsync
Rdb save incremental fsync
2018-06-16 10:59:37 +02:00
antirez
6dbc7987d7 RDB: Apply fix to rdbLoadMillisecondTime() only for new RDB versions.
This way we let big endian systems to still load old RDB versions.
However newver versions will be saved and loaded in a way that make RDB
expires cross-endian again. Thanks to @oranagra for the reporting and
the discussion about this problem, leading to this fix.
2018-06-12 18:21:39 +02:00
antirez
032ea657d7 RDB: Apply fix to rdbLoadMillisecondTime() only for new RDB versions.
This way we let big endian systems to still load old RDB versions.
However newver versions will be saved and loaded in a way that make RDB
expires cross-endian again. Thanks to @oranagra for the reporting and
the discussion about this problem, leading to this fix.
2018-06-12 18:21:39 +02:00
antirez
cfeffb1afe Fix rdbSaveKeyValuePair() integer overflow.
Again thanks to @oranagra. The object idle time does not fit into an int
sometimes: use the native type that the serialization function will get
as argument, which is uint64_t.
2018-06-12 17:31:04 +02:00
antirez
b38682199b Fix rdbSaveKeyValuePair() integer overflow.
Again thanks to @oranagra. The object idle time does not fit into an int
sometimes: use the native type that the serialization function will get
as argument, which is uint64_t.
2018-06-12 17:31:04 +02:00
antirez
49afd7d4c5 RDB: store times consistently in little endian.
I'm not sure how this escaped the attention of Redis users for years,
but finally @oranagra reported this issue... Thanks to Oran.
2018-06-12 17:22:03 +02:00
antirez
f70e88c1f6 RDB: store times consistently in little endian.
I'm not sure how this escaped the attention of Redis users for years,
but finally @oranagra reported this issue... Thanks to Oran.
2018-06-12 17:22:03 +02:00
Salvatore Sanfilippo
5ff23db284 Merge pull request #4861 from soloestoy/rdb-dict-expand
RDB: expand dict if needed when rdb load object
2018-06-08 12:12:34 +02:00
Salvatore Sanfilippo
9fa2e7020e
Merge pull request #4861 from soloestoy/rdb-dict-expand
RDB: expand dict if needed when rdb load object
2018-06-08 12:12:34 +02:00
antirez
9e81213746 Don't expire keys while loading RDB from AOF preamble.
The AOF tail of a combined RDB+AOF is based on the premise of applying
the AOF commands to the exact state that there was in the server while
the RDB was persisted. By expiring keys while loading the RDB file, we
change the state, so applying the AOF tail later may change the state.

Test case:

* Time1: SET a 10
* Time2: EXPIREAT a $time5
* Time3: INCR a
* Time4: PERSIT A. Start bgrewiteaof with RDB preamble. The value of a is 11 without expire time.
* Time5: Restart redis from the RDB+AOF: consistency violation.

Thanks to @soloestoy for providing the patch.
Thanks to @trevor211 for the original issue report and the initial fix.

Check issue #4950 for more info.
2018-05-29 12:37:42 +02:00
antirez
49147f36e9 Don't expire keys while loading RDB from AOF preamble.
The AOF tail of a combined RDB+AOF is based on the premise of applying
the AOF commands to the exact state that there was in the server while
the RDB was persisted. By expiring keys while loading the RDB file, we
change the state, so applying the AOF tail later may change the state.

Test case:

* Time1: SET a 10
* Time2: EXPIREAT a $time5
* Time3: INCR a
* Time4: PERSIT A. Start bgrewiteaof with RDB preamble. The value of a is 11 without expire time.
* Time5: Restart redis from the RDB+AOF: consistency violation.

Thanks to @soloestoy for providing the patch.
Thanks to @trevor211 for the original issue report and the initial fix.

Check issue #4950 for more info.
2018-05-29 12:37:42 +02:00
WuYunlong
09ae1a46d6 Fix rdb save by allowing dumping of expire keys, so that when
we add a new slave, and do a failover, eighter by manual or
not, other local slaves will delete the expired keys properly.
2018-05-29 12:35:15 +02:00
WuYunlong
2a887bd53f Fix rdb save by allowing dumping of expire keys, so that when
we add a new slave, and do a failover, eighter by manual or
not, other local slaves will delete the expired keys properly.
2018-05-29 12:35:15 +02:00
Salvatore Sanfilippo
7f225a69a7 Merge pull request #4908 from soloestoy/aof-rdb-preamble-compatible-checksum-no
AOF & RDB: be compatible with rdbchecksum no
2018-05-23 17:11:00 +02:00
Salvatore Sanfilippo
1ab3c82375
Merge pull request #4908 from soloestoy/aof-rdb-preamble-compatible-checksum-no
AOF & RDB: be compatible with rdbchecksum no
2018-05-23 17:11:00 +02:00