9295 Commits

Author SHA1 Message Date
Yossi Gottlieb
b057ff81ee TLS: Add missing redis-cli options. (#7456)
* Tests: fix and reintroduce redis-cli tests.

These tests have been broken and disabled for 10 years now!

* TLS: add remaining redis-cli support.

This adds support for the redis-cli --pipe, --rdb and --replica options
previously unsupported in --tls mode.

* Fix writeConn().

(cherry picked from commit d9f970d8d3f0b694f1e8915cab6d4eab9cfb2ef1)
2020-07-20 21:08:26 +03:00
Oran Agra
a9d1014fb2 redis-cli --hotkeys fixed to handle non-printable key names
(cherry picked from commit b23e2510364bd13d9988921235c400c5657a8438)
2020-07-20 21:08:26 +03:00
Oran Agra
183bdc11c0 redis-cli --bigkeys fixed to handle non-printable key names
(cherry picked from commit 6f8a8647de5b16ff937dc25046151619a2eafabb)
2020-07-20 21:08:26 +03:00
Oran Agra
95ba01b538 RESTORE ABSTTL won't store expired keys into the db (#7472)
Similarly to EXPIREAT with TTL in the past, which implicitly deletes the
key and return success, RESTORE should not store key that are already
expired into the db.
When used together with REPLACE it should emit a DEL to keyspace
notification and replication stream.

(cherry picked from commit 5977a94842a25140520297fe4bfda15e0e4de711)
2020-07-20 21:08:26 +03:00
huangzhw
7b21b8c3fb defrag.c activeDefragSdsListAndDict when defrag sdsele, We can't use (#7492)
it to calculate hash, we should use newsds.

(cherry picked from commit d6180c8c8674ffdae3d6efa5f946d85fe9163464)
2020-07-20 21:08:26 +03:00
Oran Agra
33ca884cc5 skip a test that uses +inf on valgrind (#7440)
On some platforms strtold("+inf") with valgrind returns a non-inf result

[err]: INCRBYFLOAT does not allow NaN or Infinity in tests/unit/type/incr.tcl
Expected 'ERR*would produce*' to equal or match '1189731495357231765085759.....'

(cherry picked from commit 909bc97c526db757a3d022b29911ff6d08eba59c)
2020-07-20 21:08:26 +03:00
Oran Agra
2b5f23197c stabilize tests that look for log lines (#7367)
tests were sensitive to additional log lines appearing in the log
causing the search to come empty handed.

instead of just looking for the n last log lines, capture the log lines
before performing the action, and then search from that offset.

(cherry picked from commit 8e76e13472b7d277af78691775c2cf845f68ab90)
2020-07-20 21:08:26 +03:00
Oran Agra
1104113c07 tests/valgrind: don't use debug restart (#7404)
* tests/valgrind: don't use debug restart

DEBUG REATART causes two issues:
1. it uses execve which replaces the original process and valgrind doesn't
   have a chance to check for errors, so leaks go unreported.
2. valgrind report invalid calls to close() which we're unable to resolve.

So now the tests use restart_server mechanism in the tests, that terminates
the old server and starts a new one, new PID, but same stdout, stderr.

since the stderr can contain two or more valgrind report, it is not enough
to just check for the absence of leaks, we also need to check for some known
errors, we do both, and fail if we either find an error, or can't find a
report saying there are no leaks.

other changes:
- when killing a server that was already terminated we check for leaks too.
- adding DEBUG LEAK which was used to test it.
- adding --trace-children to valgrind, although no longer needed.
- since the stdout contains two or more runs, we need slightly different way
  of checking if the new process is up (explicitly looking for the new PID)
- move the code that handles --wait-server to happen earlier (before
  watching the startup message in the log), and serve the restarted server too.

* squashme - CR fixes

(cherry picked from commit 69ade87325eedebdb44760af9a8c28e15381888e)
2020-07-20 21:08:26 +03:00
Oran Agra
62f8583445 change references to the github repo location (#7479)
(cherry picked from commit 9bbf768d3ceaa882c7dcc0033fc3cb4be0973248)
2020-07-20 21:08:26 +03:00
zhaozhao.zz
3155adb299 BITOP: propagate only when it really SET or DEL targetkey (#5783)
For example:
    BITOP not targetkey sourcekey

If targetkey and sourcekey doesn't exist, BITOP has no effect,
we do not propagate it, thus can save aof and replica flow.

(cherry picked from commit 1978f996d8b13db112d5d2fdf4a4ce2baf636729)
2020-07-20 21:08:26 +03:00
antirez
14a59d4ce7 Update comment to clarify change in #7398.
(cherry picked from commit ad0a9df77a2ccf3fdf309dcdd1b54cf350fcbe3c)
2020-07-20 21:08:26 +03:00
antirez
17aaf5ec97 LPOS: option FIRST renamed RANK.
(cherry picked from commit a5a3a7bbc61203398ecc1d5b52c76214f5672776)
2020-07-20 21:08:26 +03:00
Dave Nielsen
3f5cf24da9 updated copyright year
Changed "2015" to "2020"

(cherry picked from commit 2d6d9f75524f280b9bc02613ebe9727b92c1429e)
2020-07-20 21:08:26 +03:00
Oran Agra
05e483cbb3 EXEC always fails with EXECABORT and multi-state is cleared
In order to support the use of multi-exec in pipeline, it is important that
MULTI and EXEC are never rejected and it is easy for the client to know if the
connection is still in multi state.

It was easy to make sure MULTI and DISCARD never fail (done by previous
commits) since these only change the client state and don't do any actual
change in the server, but EXEC is a different story.

Since in the past, it was possible for clients to handle some EXEC errors and
retry the EXEC, we now can't affort to return any error on EXEC other than
EXECABORT, which now carries with it the real reason for the abort too.

Other fixes in this commit:
- Some checks that where performed at the time of queuing need to be re-
  validated when EXEC runs, for instance if the transaction contains writes
  commands, it needs to be aborted. there was one check that was already done
  in execCommand (-READONLY), but other checks where missing: -OOM, -MISCONF,
  -NOREPLICAS, -MASTERDOWN
- When a command is rejected by processCommand it was rejected with addReply,
  which was not recognized as an error in case the bad command came from the
  master. this will enable to count or MONITOR these errors in the future.
- make it easier for tests to create additional (non deferred) clients.
- add tests for the fixes of this commit.

(cherry picked from commit 65a3307bc95aadbc91d85cdf9dfbe1b3493222ca)
2020-07-20 21:08:26 +03:00
antirez
c8f250f8dd Include cluster.h for getClusterConnectionsCount().
(cherry picked from commit 21f62c3346ab22ac37f81110e59dae9372c9e4d1)
2020-07-20 21:08:26 +03:00
antirez
9412094f25 Fix BITFIELD i64 type handling, see #7417.
(cherry picked from commit 746297314f3c198a90ceab8462a40abd9fc69f26)
2020-07-20 21:08:26 +03:00
antirez
8312aa27d4 Clarify maxclients and cluster in conf. Remove myself too.
(cherry picked from commit 59fd178014c7cca1b0c668b30ab0d991dd3030f3)
2020-07-20 21:08:26 +03:00
hwware
d3aa3791fe fix memory leak in sentinel connection sharing
(cherry picked from commit 1bfa2d27a637119226ee3244d2d219c7e5a7ff33)
2020-07-20 21:08:26 +03:00
chenhui0212
2b189f098e Fix comments in function raxLowWalk of listpack.c
(cherry picked from commit f800b52172622b84aa8bdf09aa6cd522aade93be)
2020-07-20 21:08:26 +03:00
Tomasz Poradowski
3328b7a514 ensure SHUTDOWN_NOSAVE in Sentinel mode
- enforcing of SHUTDOWN_NOSAVE flag in one place to make it consitent
  when running in Sentinel mode

(cherry picked from commit 4ee011adb5f0d56085ecd3e5d643ab192ef77ce6)
2020-07-20 21:08:26 +03:00
chenhui0212
88e1f80eee fix comments in listpack.c
(cherry picked from commit 71fafd761ad5f939f485259eb3856ed3766c98be)
2020-07-20 21:08:26 +03:00
antirez
0ebbc36059 Use cluster connections too, to limit maxclients.
See #7401.

(cherry picked from commit 4b8d8826afa0f240b26977e9d128144ebf8d5d7a)
2020-07-20 21:08:26 +03:00
antirez
43ed3c3589 Tracking: fix enableBcastTrackingForPrefix() invalid sdslen() call.
Related to #7387.

(cherry picked from commit 784479939d9e560835a9eb7a410304b46047d5f5)
2020-07-20 21:08:26 +03:00
root
8095daea4a cluster.c remove if of clusterSendFail in markNodeAsFailingIfNeeded
(cherry picked from commit c92464db694172dac8b0f9eeedd366c494d6db8a)
2020-07-20 21:08:26 +03:00
meir@redislabs.com
51e178454d Fix RM_ScanKey module api not to return int encoded strings
The scan key module API provides the scan callback with the current
field name and value (if it exists). Those arguments are RedisModuleString*
which means it supposes to point to robj which is encoded as a string.
Using createStringObjectFromLongLong function might return robj that
points to an integer and so break a module that tries for example to
use RedisModule_StringPtrLen on the given field/value.

The PR introduces a fix that uses the createObject function and sdsfromlonglong function.
Using those function promise that the field and value pass to the to the
scan callback will be Strings.

The PR also changes the Scan test module to use RedisModule_StringPtrLen
to catch the issue. without this, the issue is hidden because
RedisModule_ReplyWithString knows to handle integer encoding of the
given robj (RedisModuleString).

The PR also introduces a new test to verify the issue is solved.

(cherry picked from commit a89bf734a933e45b9dd3ae85ef4c3b62bd6891d8)
2020-07-20 21:08:26 +03:00
antirez
82b2bfd20b Fix LCS object type checking. Related to #7379.
(cherry picked from commit 10553988498acea1d772af69092b67fd5b56d529)
2020-07-20 21:08:26 +03:00
hwware
ec1faeec72 fix memory leak
(cherry picked from commit 7008a0ba66fe13af0d584071eaa5fe3f34c56512)
2020-07-20 21:08:26 +03:00
hwware
c1326d7b10 fix server crash in STRALGO command
(cherry picked from commit 2a05fa0d481d12d3747465c4f14470bdca100c5d)
2020-07-20 21:08:26 +03:00
Benjamin Sergeant
845fb2d1c1 Update redis-cli.c
(cherry picked from commit 93021da221f71cf71fe874fd881ea59f325b82f2)
2020-07-20 21:08:26 +03:00
Jamie Scott
930fac8058 minor fix
(cherry picked from commit 4f3b15e6ea2d72debeb2f0533e2ba11884257021)
2020-07-20 21:08:26 +03:00
antirez
0cae0900cf help.h updated. 2020-06-12 12:16:24 +02:00
antirez
e0022d8cfe LPOS: tests + crash fix. 2020-06-12 12:08:06 +02:00
antirez
495327c020 LPOS: update to latest proposal.
See https://gist.github.com/antirez/3591c5096bc79cad8b5a992e08304f48
2020-06-12 12:08:06 +02:00
antirez
afb8b3c030 LPOS: implement the final design. 2020-06-12 12:08:06 +02:00
Paul Spooren
31c9cd8604 LRANK: Add command (the command will be renamed LPOS).
The `LRANK` command returns the index (position) of a given element
within a list. Using the `direction` argument it is possible to specify
going from head to tail (acending, 1) or from tail to head (decending,
-1). Only the first found index is returend. The complexity is O(N).

When using lists as a queue it can be of interest at what position a
given element is, for instance to monitor a job processing through a
work queue. This came up within the Python `rq` project which is based
on Redis[0].

[0]: https://github.com/rq/rq/issues/1197

Signed-off-by: Paul Spooren <mail@aparcar.org>
2020-06-12 12:08:06 +02:00
antirez
51efb7fe25 Redis 6.0.5. 2020-06-09 12:19:30 +02:00
antirez
a7d3670da5 Adapt EVAL+busy script test to new behavior. 2020-06-09 12:19:30 +02:00
antirez
ee8dd01bbc Temporary fix for #7353 issue about EVAL during -BUSY. 2020-06-09 11:53:01 +02:00
xhe
a4a856d532 return the correct proto version
HELLO should return the current proto version, while the code hardcoded
3
2020-06-09 11:53:01 +02:00
Oran Agra
e2046b3003 Don't queue commands in an already aborted MULTI state 2020-06-09 11:53:01 +02:00
Oran Agra
b35fdf1de1 Avoid rejecting WATCH / UNWATCH, like MULTI/EXEC/DISCARD
Much like MULTI/EXEC/DISCARD, the WATCH and UNWATCH are not actually
operating on the database or server state, but instead operate on the
client state. the client may send them all in one long pipeline and check
all the responses only at the end, so failing them may lead to a
mismatch between the client state on the server and the one on the
client end, and execute the wrong commands (ones that were meant to be
discarded)

the watched keys are not actually stored in the client struct, but they
are in fact part of the client state. for instance, they're not cleared
or moved in SWAPDB or FLUSHDB.
2020-06-09 11:53:01 +02:00
zhaozhao.zz
1d7bf208ce AOF: append origin SET if no expire option 2020-06-09 11:53:01 +02:00
Oran Agra
676445ad95 fix disconnectSlaves, to try to free each slave.
the recent change in that loop (iteration rather than waiting for it to
be empty) was intended to avoid an endless loop in case some slave would
refuse to be freed.

but the lookup of the first client remained, which would have caused it
to try the first one again and again instead of moving on.
2020-06-09 11:53:01 +02:00
zhaozhao.zz
4846c0c8af donot free protected client in freeClientsInAsyncFreeQueue
related #7234
2020-06-09 11:53:01 +02:00
Oran Agra
f33de403ed fix pingoff test race 2020-06-06 11:44:21 +02:00
Kevin Fwu
49af4d07e4 Fix TLS certificate loading for chained certificates.
This impacts client verification for chained certificates (such as Lets
Encrypt certificates). Client Verify requires the full chain in order to
properly verify the certificate.
2020-06-06 11:44:21 +02:00
antirez
329fddbda1 Revert "Implements sendfile for redis."
This reverts commit 9cf500a3f67e4e2ce51414c354e3472faf095d5b.
2020-06-06 11:43:29 +02:00
antirez
925a2cd5aa Revert "avoid using sendfile if tls-replication is enabled"
This reverts commit b9abecfc4c97db01fa6f09180c68a92ea5974ac1.
2020-06-06 11:43:29 +02:00
Liu Zhen
84a7a90586 fix clusters mixing accidentally by gossip
`clusterStartHandshake` will start hand handshake
and eventually send CLUSTER MEET message, which is strictly prohibited
in the REDIS CLUSTER SPEC.
Only system administrator can initiate CLUSTER MEET message.
Futher, according to the SPEC, rather than IP/PORT pairs, only nodeid
can be trusted.
2020-06-06 11:43:29 +02:00
antirez
cd63359a17 Fix handling of special chars in ACL LOAD.
Now it is also possible for ACL SETUSER to accept empty strings
as valid operations (doing nothing), so for instance

    ACL SETUSER myuser ""

Will have just the effect of creating a user in the default state.

This should fix #7329.
2020-06-06 11:43:29 +02:00