10340 Commits

Author SHA1 Message Date
zhaozhao.zz
db1d277847 PSYNC2: clarify the scenario when repl_stream_db can be -1 2017-11-02 10:45:33 +08:00
zhaozhao.zz
885c4f856e PSYNC2 & RDB: fix the missing rdbSaveInfo for BGSAVE 2017-11-01 17:52:43 +08:00
zhaozhao.zz
015a0f42b0 PSYNC2 & RDB: fix the missing rdbSaveInfo for BGSAVE 2017-11-01 17:52:43 +08:00
zhaozhao.zz
6ddf0ea293 PSYNC2: safe free backlog when reach the time limit
When we free the backlog, we should use a new
replication ID and clear the ID2. Since without
backlog we can not increment master_repl_offset
even do write commands, that may lead to inconsistency
when we try to connect a "slave-before" master
(if this master is our slave before, our replid
equals the master's replid2). As the master have our
history, so we can match the master's replid2 and
second_replid_offset, that make partial sync work,
but the data is inconsistent.
2017-11-01 17:32:27 +08:00
zhaozhao.zz
39acc4df0e PSYNC2: safe free backlog when reach the time limit
When we free the backlog, we should use a new
replication ID and clear the ID2. Since without
backlog we can not increment master_repl_offset
even do write commands, that may lead to inconsistency
when we try to connect a "slave-before" master
(if this master is our slave before, our replid
equals the master's replid2). As the master have our
history, so we can match the master's replid2 and
second_replid_offset, that make partial sync work,
but the data is inconsistent.
2017-11-01 17:32:27 +08:00
antirez
ffcf7d5ab1 Fix buffer overflows occurring reading redis.conf.
There was not enough sanity checking in the code loading the slots of
Redis Cluster from the nodes.conf file, this resulted into the
attacker's ability to write data at random addresses in the process
memory, by manipulating the index of the array. The bug seems
exploitable using the following techique: the config file may be altered so
that one of the nodes gets, as node ID (which is the first field inside the
structure) some data that is actually executable: then by writing this
address in selected places, this node ID part can be executed after a
jump. So it is mostly just a matter of effort in order to exploit the
bug. In practice however the issue is not very critical because the
bug requires an unprivileged user to be able to modify the Redis cluster
nodes configuration, and at the same time this should result in some
gain. However Redis normally is unprivileged as well. Yet much better to
have this fixed indeed.

Fix #4278.
2017-10-31 09:41:22 +01:00
antirez
401cfa554f Fix buffer overflows occurring reading redis.conf.
There was not enough sanity checking in the code loading the slots of
Redis Cluster from the nodes.conf file, this resulted into the
attacker's ability to write data at random addresses in the process
memory, by manipulating the index of the array. The bug seems
exploitable using the following techique: the config file may be altered so
that one of the nodes gets, as node ID (which is the first field inside the
structure) some data that is actually executable: then by writing this
address in selected places, this node ID part can be executed after a
jump. So it is mostly just a matter of effort in order to exploit the
bug. In practice however the issue is not very critical because the
bug requires an unprivileged user to be able to modify the Redis cluster
nodes configuration, and at the same time this should result in some
gain. However Redis normally is unprivileged as well. Yet much better to
have this fixed indeed.

Fix #4278.
2017-10-31 09:41:22 +01:00
antirez
b2e295971f Regression test for issue #4391. 2017-10-30 13:45:46 +01:00
antirez
a07c0db3af Regression test for issue #4391. 2017-10-30 13:45:46 +01:00
antirez
de474186bd More robust object -> double conversion.
Certain checks were useless, at the same time certain malformed inputs
were accepted without problems (emtpy strings parsed as zero).
Cases where strtod() returns ERANGE but we still want to parse the input
where ok in getDoubleFromObject() but not in the long variant.

As a side effect of these fixes, this commit fixes #4391.
2017-10-30 13:39:58 +01:00
antirez
fa36221bf6 More robust object -> double conversion.
Certain checks were useless, at the same time certain malformed inputs
were accepted without problems (emtpy strings parsed as zero).
Cases where strtod() returns ERANGE but we still want to parse the input
where ok in getDoubleFromObject() but not in the long variant.

As a side effect of these fixes, this commit fixes #4391.
2017-10-30 13:39:58 +01:00
rouzier
6eb996540c Fix file descriptor leak and error handling 2017-10-13 13:20:45 -04:00
rouzier
8eb5823f43 Fix file descriptor leak and error handling 2017-10-13 13:20:45 -04:00
antirez
2bf8c2c130 Limit statement in RM_BlockClient() to 80 cols. 2017-09-28 23:15:34 +02:00
antirez
ba954302f6 Limit statement in RM_BlockClient() to 80 cols. 2017-09-28 23:15:34 +02:00
Salvatore Sanfilippo
54ad626cce Merge pull request #4337 from dvirsky/module_ctx_flags
Module Context flags
2017-09-28 14:40:55 +02:00
Salvatore Sanfilippo
32c0b27765 Merge pull request #4337 from dvirsky/module_ctx_flags
Module Context flags
2017-09-28 14:40:55 +02:00
zhaozhao.zz
6dffc1b7a3 Modules: handle the busy module name 2017-09-28 17:38:40 +08:00
zhaozhao.zz
f1333eb992 Modules: handle the busy module name 2017-09-28 17:38:40 +08:00
zhaozhao.zz
cb9dde3280 Modules: handle the conflict of registering commands 2017-09-28 16:21:21 +08:00
zhaozhao.zz
7fb3864e95 Modules: handle the conflict of registering commands 2017-09-28 16:21:21 +08:00
Dvir Volk
7393fd814e Added safety net preventing redis from crashing if a module decide to block in MULTI 2017-09-27 15:17:53 +03:00
Dvir Volk
6f37695620 Added safety net preventing redis from crashing if a module decide to block in MULTI 2017-09-27 15:17:53 +03:00
Dvir Volk
b246635d6d Renamed GetCtxFlags to GetContextFlags 2017-09-27 11:58:16 +03:00
Dvir Volk
c3777932f6 Renamed GetCtxFlags to GetContextFlags 2017-09-27 11:58:16 +03:00
Dvir Volk
616c546b01 Added support for module context flags with RM_GetCtxFlags 2017-09-27 11:58:07 +03:00
Dvir Volk
f5909f2c18 Added support for module context flags with RM_GetCtxFlags 2017-09-27 11:58:07 +03:00
antirez
474adba9fa Clarify comment in change fixing #4323. 2017-09-21 12:35:04 +02:00
antirez
ab601f284f Clarify comment in change fixing #4323. 2017-09-21 12:35:04 +02:00
Salvatore Sanfilippo
fa2c0582cf Merge pull request #4324 from soloestoy/lazyfree-fix-memory-leak
Lazyfree: avoid memory leak when free slowlog entry
2017-09-21 12:32:02 +02:00
Salvatore Sanfilippo
52b92bc6ab Merge pull request #4324 from soloestoy/lazyfree-fix-memory-leak
Lazyfree: avoid memory leak when free slowlog entry
2017-09-21 12:32:02 +02:00
zhaozhao.zz
269760edbb Lazyfree: avoid memory leak when free slowlog entry 2017-09-21 14:19:21 +08:00
zhaozhao.zz
36252140aa Lazyfree: avoid memory leak when free slowlog entry 2017-09-21 14:19:21 +08:00
antirez
bb3b5ddd19 PSYNC2: More refinements related to #4316. 2017-09-20 11:28:13 +02:00
antirez
4e01e51338 PSYNC2: More refinements related to #4316. 2017-09-20 11:28:13 +02:00
zhaozhao.zz
b541ccef25 PSYNC2: make persisiting replication info more solid
This commit is a reinforcement of commit c1c99e9.

1. Replication information can be stored when the RDB file is
generated by a mater using server.slaveseldb when server.repl_backlog
is not NULL, or set repl_stream_db be -1. That's safe, because
NULL server.repl_backlog will trigger full synchronization,
then master will send SELECT command to replicaiton stream.
2. Only do rdbSave* when rsiptr is not NULL,
if we do rdbSave* without rdbSaveInfo, slave will miss repl-stream-db.
3. Save the replication informations also in the case of
SAVE command, FLUSHALL command and DEBUG reload.
2017-09-20 11:18:10 +02:00
zhaozhao.zz
a45d0fc482 PSYNC2: make persisiting replication info more solid
This commit is a reinforcement of commit c1c99e9.

1. Replication information can be stored when the RDB file is
generated by a mater using server.slaveseldb when server.repl_backlog
is not NULL, or set repl_stream_db be -1. That's safe, because
NULL server.repl_backlog will trigger full synchronization,
then master will send SELECT command to replicaiton stream.
2. Only do rdbSave* when rsiptr is not NULL,
if we do rdbSave* without rdbSaveInfo, slave will miss repl-stream-db.
3. Save the replication informations also in the case of
SAVE command, FLUSHALL command and DEBUG reload.
2017-09-20 11:18:10 +02:00
antirez
c1c99e9f4e PSYNC2: Fix the way replication info is saved/loaded from RDB.
This commit attempts to fix a number of bugs reported in #4316.
They are related to the way replication info like replication ID,
offsets, and currently selected DB in the master client, are stored
and loaded by Redis. In order to avoid inconsistencies the changes in
this commit try to enforce that:

1. Replication information are only stored when the RDB file is
generated by a slave that has a valid 'master' client, so that we can
always extract the currently selected DB.
2. When replication informations are persisted in the RDB file, all the
info for a successful PSYNC or nothing is persisted.
3. The RDB replication informations are only loaded if the instance is
configured as a slave, otherwise a master can start with IDs that relate
to a different history of the data set, and stil retain such IDs in the
future while receiving unrelated writes.
2017-09-19 23:03:39 +02:00
antirez
bc6b7c949d PSYNC2: Fix the way replication info is saved/loaded from RDB.
This commit attempts to fix a number of bugs reported in #4316.
They are related to the way replication info like replication ID,
offsets, and currently selected DB in the master client, are stored
and loaded by Redis. In order to avoid inconsistencies the changes in
this commit try to enforce that:

1. Replication information are only stored when the RDB file is
generated by a slave that has a valid 'master' client, so that we can
always extract the currently selected DB.
2. When replication informations are persisted in the RDB file, all the
info for a successful PSYNC or nothing is persisted.
3. The RDB replication informations are only loaded if the instance is
configured as a slave, otherwise a master can start with IDs that relate
to a different history of the data set, and stil retain such IDs in the
future while receiving unrelated writes.
2017-09-19 23:03:39 +02:00
antirez
a4152119c6 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2017-09-19 10:35:49 +02:00
antirez
9290786e73 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2017-09-19 10:35:49 +02:00
antirez
b75ae0bbea PSYNC2: Create backlog on slave partial sync as well.
A slave may be started with an RDB file able to provide enough slave to
perform a successful partial SYNC with its master. However in such a
case, how outlined in issue #4268, the slave backlog will not be
started, since it was only initialized on full syncs attempts. This
creates different problems with successive PSYNC attempts that will
always result in full synchronizations.

Thanks to @fdingiit for discovering the issue.
2017-09-19 10:33:14 +02:00
antirez
d0c77132d8 PSYNC2: Create backlog on slave partial sync as well.
A slave may be started with an RDB file able to provide enough slave to
perform a successful partial SYNC with its master. However in such a
case, how outlined in issue #4268, the slave backlog will not be
started, since it was only initialized on full syncs attempts. This
creates different problems with successive PSYNC attempts that will
always result in full synchronizations.

Thanks to @fdingiit for discovering the issue.
2017-09-19 10:33:14 +02:00
Salvatore Sanfilippo
2d13bf4c59 Merge pull request #3785 from GitHubMota/unstable
redis-benchmark: default value size usage update.
2017-09-18 12:18:57 +02:00
Salvatore Sanfilippo
a85314ecd3 Merge pull request #3785 from GitHubMota/unstable
redis-benchmark: default value size usage update.
2017-09-18 12:18:57 +02:00
Salvatore Sanfilippo
9b4cb4addc Merge pull request #3554 from jybaek/Delete_duplicate
Remove Duplicate Processing
2017-09-18 12:18:15 +02:00
Salvatore Sanfilippo
ef7c5887b8 Merge pull request #3554 from jybaek/Delete_duplicate
Remove Duplicate Processing
2017-09-18 12:18:15 +02:00
Salvatore Sanfilippo
8885921dfb Merge pull request #4311 from oranagra/aof_shutdown_flush
Flush append only buffers before existing.
2017-09-18 12:03:41 +02:00
Salvatore Sanfilippo
42e1112109 Merge pull request #4311 from oranagra/aof_shutdown_flush
Flush append only buffers before existing.
2017-09-18 12:03:41 +02:00
Oran Agra
b122cadc66 Flush append only buffers before existing.
when SHUTDOWN command is recived it is possible that some of the recent
command were not yet flushed from the AOF buffer, and the server
experiences data loss at shutdown.
2017-09-17 07:22:16 +03:00