9954 Commits

Author SHA1 Message Date
Itamar Haber
aea14c800c Merge pull request #6777 from pponnuvel/typos_in_readme
Fix typos in README.md
2020-08-11 21:10:58 +03:00
zhaozhao.zz
b1a242b84a redis-benchmark: fix wrong random key for hset (#4895) 2020-08-11 20:51:27 +08:00
Itamar Haber
a9a1e4925f Merge pull request #7618 from ShooterIT/benchmark-zset
[Redis-benchmark] Support zset type
2020-08-11 14:31:11 +03:00
Muhammad Zahalqa
7589560731 Fix unidentical function declaration in bio.c. lazyfree.c: lazyfreeFreeSlotsMapFromBioThread (#7228) 2020-08-11 19:16:10 +08:00
zhaozhao.zz
0061dbba04 CLIENT_MASTER should ignore server.proto_max_bulk_len 2020-08-11 18:59:29 +08:00
zhaozhao.zz
46323cec56 config: proto-max-bulk-len must be 1mb or greater 2020-08-11 18:59:29 +08:00
zhaozhao.zz
912b48e4b9 using proto-max-bulk-len in checkStringLength for SETRANGE and APPEND 2020-08-11 18:59:29 +08:00
pingfan108
b082164834 Correct error message of runtest-cluster and runtest-moduleapi (#7647) 2020-08-11 13:35:20 +03:00
YoongHM
1c6caaaef7 Start redis after network is online (#7639)
The two lines allow systemd to start redis.service after the network is online. Only after the network is online that Redis could bind to IP address other than 127.0.0.1 during initial boot up process.
2020-08-11 12:30:32 +03:00
Tyson Andre
6e17afa80a Implement SMISMEMBER key member [member ...] (#7615)
This is a rebased version of #3078 originally by shaharmor
with the following patches by TysonAndre made after rebasing
to work with the updated C API:

1. Add 2 more unit tests
   (wrong argument count error message, integer over 64 bits)
2. Use addReplyArrayLen instead of addReplyMultiBulkLen.
3. Undo changes to src/help.h - for the ZMSCORE PR,
   I heard those should instead be automatically
   generated from the redis-doc repo if it gets updated

Motivations:

- Example use case: Client code to efficiently check if each element of a set
  of 1000 items is a member of a set of 10 million items.
  (Similar to reasons for working on #7593)
- HMGET and ZMSCORE already exist. This may lead to developers deciding
  to implement functionality that's best suited to a regular set with a
  data type of sorted set or hash map instead, for the multi-get support.

Currently, multi commands or lua scripting to call sismember multiple times
would almost definitely be less efficient than a native smismember
for the following reasons:

- Need to fetch the set from the string every time
  instead of reusing the C pointer.
- Using pipelining or multi-commands would result in more bytes sent
  and received by the client for the repeated SISMEMBER KEY sections.
- Need to specially encode the data and decode it from the client
  for lua-based solutions.
- Proposed solutions using Lua or SADD/SDIFF could trigger writes to
  memory, which is undesirable on a redis replica server
  or when commands get replicated to replicas.

Co-Authored-By: Shahar Mor <shahar@peer5.com>
Co-Authored-By: Tyson Andre <tysonandre775@hotmail.com>
2020-08-11 11:55:06 +03:00
Yossi Gottlieb
b8f0c2de4a Run daily workflow on main repo only (no forks). (#7646) 2020-08-11 10:59:47 +03:00
Rajat Pawar
8644c26487 Fix comment about ACLGetCommandPerm() 2020-08-10 23:11:26 -07:00
Jim Brunner
88e97fe2bd Prevent dictRehashMilliseconds from rehashing while a safe iterator is present (#5948) 2020-08-10 22:36:34 -07:00
杨博东
51aa08d26d Avoid redundant calls to signalKeyAsReady (#7625)
signalKeyAsReady has some overhead (namely dictFind) so we should
only call it when there are clients blocked on the relevant type (BLOCKED_*)
2020-08-11 08:18:09 +03:00
Itamar Haber
594691fb37 Removes dead code (#7642)
Appears to be handled by server.stream_node_max_bytes in reality.
2020-08-11 08:11:47 +03:00
WuYunlong
dd08aec539 see #7250, fix signature of RedisModule_DeauthenticateAndCloseClient (#7645)
In redismodule.h, RedisModule_DeauthenticateAndCloseClient returns void
`void REDISMODULE_API_FUNC(RedisModule_DeauthenticateAndCloseClient)(RedisModuleCtx *ctx, uint64_t client_id);`
But in module.c, RM_DeauthenticateAndCloseClient returns int
`int RM_DeauthenticateAndCloseClient(RedisModuleCtx *ctx, uint64_t client_id)`

It it safe to change return value from `void` to `int` from the user's perspective.
2020-08-10 19:18:21 -07:00
Hamed Momeni
ea4d58a822 fixed a typo in redis.conf (#7636) 2020-08-09 12:02:29 +03:00
Yossi Gottlieb
c4c8e09f21 Merge pull request #7609 from michael-grunder/redis-cli-resp3-push
Add redis-cli RESP3 Push support
2020-08-09 11:19:04 +03:00
Meir Shpilraien (Spielrein)
4f99b22118 see #7544, added RedisModule_HoldString api. (#7577)
Added RedisModule_HoldString that either returns a
shallow copy of the given String (by increasing
the String ref count) or a new deep copy of String
in case its not possible to get a shallow copy.

Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-08-09 06:11:47 +03:00
Oran Agra
1deaad884c Reduce the probability of failure when start redis in runtest-cluster #7554 (#7635)
When runtest-cluster, at first, we need to create a cluster use spawn_instance,
a port which is not used is choosen, however sometimes we can't run server on
the port. possibley due to a race with another process taking it first.
such as redis/redis/runs/896537490. It may be due to the machine problem or
In order to reduce the probability of failure when start redis in
runtest-cluster, we attemp to use another port when find server do not start up.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: yanhui13 <yanhui13@meituan.com>
2020-08-09 06:08:00 +03:00
Wen Hui
31c079cc71 add sentinel command help (#7579) 2020-08-08 12:28:44 -07:00
WuYunlong
96655bd231 Optimize calls to mstime in trackInstantaneousMetric() (#6472) 2020-08-08 22:11:14 +03:00
Wang Yuan
514a1e223d Print error info if failed opening config file (#6943) 2020-08-08 22:03:56 +03:00
michael-grunder
1f0482942a Create PUSH handlers in redis-cli
Add logic to redis-cli to display RESP3 PUSH messages when we detect
STDOUT is a tty, with an optional command-line argument to override
the default behavior.

The new argument:  --show-pushes <yn>

Examples:

$ redis-cli -3 --show-pushes no
$ echo "client tracking on\nget k1\nset k1 v1"| redis-cli -3 --show-pushes y
2020-08-08 11:59:17 -07:00
ShooterIT
8925fac395 [Redis-benchmark] Remove zrem test, add zpopmin test 2020-08-08 23:08:27 +08:00
Wen Hui
30ead1edae fix memory leak in ACLLoadFromFile error handling (#7623) 2020-08-08 14:42:32 +03:00
Wang Yuan
921c633df9 Fix applying zero offset to null pointer when creating moduleFreeContextReusedClient (#7323)
Before this fix we where attempting to select a db before creating db the DB, see: #7323

This issue doesn't seem to have any implications, since the selected DB index is 0,
the db pointer remains NULL, and will later be correctly set before using this dummy
client for the first time.

As we know, we call 'moduleInitModulesSystem()' before 'initServer()'. We will allocate
memory for server.db in 'initServer', but we call 'createClient()' that will call 'selectDb()'
in 'moduleInitModulesSystem()', before the databases where created. Instead, we should call
'createClient()' for moduleFreeContextReusedClient after 'initServer()'.
2020-08-08 14:36:41 +03:00
xuannianz
d478dc1c0c remove superfluous else block (#7620)
The else block would be executed when newlen == 0 and in the case memmove won't be called, so there's no need to set start.
2020-08-08 00:19:18 +03:00
fayadexinqing
0054ba666f fix migration's broadcast PONG message, after the slot modification (#7590) 2020-08-07 13:01:14 -07:00
michael-grunder
7ee5a41aac Squashed 'deps/hiredis/' content from commit 39de5267c
git-subtree-dir: deps/hiredis
git-subtree-split: 39de5267c092859b4cab4bdf79081e9634b70e39
2020-08-06 12:41:58 -07:00
michael-grunder
ea43542cc5 Merge commit '7ee5a41aac7e5abc90d050fa509fa953ca7f1da1' as 'deps/hiredis' 2020-08-06 12:41:58 -07:00
michael-grunder
b110dc9bbb Remove hiredis so we can add it as a subtree 2020-08-06 12:41:46 -07:00
Oran Agra
cad93ed273 Accelerate diskless master connections, and general re-connections (#6271)
Diskless master has some inherent latencies.
1) fork starts with delay from cron rather than immediately
2) replica is put online only after an ACK. but the ACK
   was sent only once a second.
3) but even if it would arrive immediately, it will not
   register in case cron didn't yet detect that the fork is done.

Besides that, when a replica disconnects, it doesn't immediately
attempts to re-connect, it waits for replication cron (one per second).
in case it was already online, it may be important to try to re-connect
as soon as possible, so that the backlog at the master doesn't vanish.

In case it disconnected during rdb transfer, one can argue that it's
not very important to re-connect immediately, but this is needed for the
"diskless loading short read" test to be able to run 100 iterations in 5
seconds, rather than 3 (waiting for replication cron re-connection)

changes in this commit:
1) sync command starts a fork immediately if no sync_delay is configured
2) replica sends REPLCONF ACK when done reading the rdb (rather than on 1s cron)
3) when a replica unexpectedly disconnets, it immediately tries to
   re-connect rather than waiting 1s
4) when when a child exits, if there is another replica waiting, we spawn a new
   one right away, instead of waiting for 1s replicationCron.
5) added a call to connectWithMaster from replicationSetMaster. which is called
   from the REPLICAOF command but also in 3 places in cluster.c, in all of
   these the connection attempt will now be immediate instead of delayed by 1
   second.

side note:
we can add a call to rdbPipeReadHandler in replconfCommand when getting
a REPLCONF ACK from the replica to solve a race where the replica got
the entire rdb and EOF marker before we detected that the pipe was
closed.
in the test i did see this race happens in one about of some 300 runs,
but i concluded that this race is unlikely in real life (where the
replica is on another host and we're more likely to first detect the
pipe was closed.
the test runs 100 iterations in 3 seconds, so in some cases it'll take 4
seconds instead (waiting for another REPLCONF ACK).

Removing unneeded startBgsaveForReplication from updateSlavesWaitingForBgsave
Now that CheckChildrenDone is calling the new replicationStartPendingFork
(extracted from serverCron) there's actually no need to call
startBgsaveForReplication from updateSlavesWaitingForBgsave anymore,
since as soon as updateSlavesWaitingForBgsave returns, CheckChildrenDone is
calling replicationStartPendingFork that handles that anyway.
The code in updateSlavesWaitingForBgsave had a bug in which it ignored
repl-diskless-sync-delay, but removing that code shows that this bug was
hiding another bug, which is that the max_idle should have used >= and
not >, this one second delay has a big impact on my new test.
2020-08-06 16:53:06 +03:00
Oran Agra
10bfd2fc0d Fix potential race in bugReportStart
this race would only happen when two threads paniced at the same time,
and even then the only consequence is some extra log lines.

race reported in #7391
2020-08-06 16:47:27 +03:00
Oran Agra
58ca0b0be2 Assertion and panic, print crash log without generating SIGSEGV
This makes it possible to add tests that generate assertions, and run
them with valgrind, making sure that there are no memory violations
prior to the assertion.

New config options:
- crash-log-enabled - can be disabled for cleaner core dumps
- crash-memcheck-enabled - useful for faster termination after a crash
- use-exit-on-panic - to be used by the test suite so that valgrind can
  detect leaks and memory corruptions

Other changes:
- Crash log is printed even on system that dont HAVE_BACKTRACE, i.e. in
  both SIGSEGV and assert / panic
- Assertion and panic won't print registers and code around EIP (which
  was useless), but will do fast memory test (which may still indicate
  that the assertion was due to memory corrpution)

I had to reshuffle code in order to re-use it, so i extracted come code
into function without actually doing any changes to the code:
- logServerInfo
- logModulesInfo
- doFastMemoryTest (with the exception of it being conditional)
- dumpCodeAroundEIP

changes to the crash report on segfault:
- logRegisters is called right after the stack trace (before info) done
  just in order to have more re-usable code
- stack trace skips the first two items on the stack (the crash log and
  signal handler functions)
2020-08-06 16:47:27 +03:00
ShooterIT
e6c811cd85 [Redis-benchmark] Support zset type 2020-08-06 15:36:28 +08:00
Itamar Haber
04db7fd8e9 Merge pull request #7092 from itamarhaber/fix-5629
Prevents default save configuration being reset...
2020-08-05 21:16:38 +03:00
Oran Agra
f519dcb216 redis-cli --cluster-yes - negate force flag for clarity
this internal flag is there so that some commands do not comply to `--cluster-yes`
2020-08-05 18:30:43 +03:00
Frank Meier
c6ac2588db reintroduce REDISCLI_CLUSTER_YES env variable in redis-cli
the variable was introduced only in the 5.0 branch in #5879 bc6c1c40db
2020-08-05 18:30:43 +03:00
Frank Meier
c8f3182f37 add force option to 'create-cluster create' script call (#7612) 2020-08-05 12:06:33 +03:00
WuYunlong
2c68d2d5c7 Fix tests/cluster/cluster.tcl about wrong usage of lrange. (#6702) 2020-08-04 18:00:58 +03:00
Tyson Andre
486e39e86e Add a ZMSCORE command returning an array of scores. (#7593)
Syntax: `ZMSCORE KEY MEMBER [MEMBER ...]`

This is an extension of #2359
amended by Tyson Andre to work with the changed unstable API,
add more tests, and consistently return an array.

- It seemed as if it would be more likely to get reviewed
  after updating the implementation.

Currently, multi commands or lua scripting to call zscore multiple times
would almost definitely be less efficient than a native ZMSCORE
for the following reasons:

- Need to fetch the set from the string every time instead of reusing the C
  pointer.
- Using pipelining or multi-commands would result in more bytes sent by
  the client for the repeated `ZMSCORE KEY` sections.
- Need to specially encode the data and decode it from the client
  for lua-based solutions.
- The fastest solution I've seen for large sets(thousands or millions)
  involves lua and a variadic ZADD, then a ZINTERSECT, then a ZRANGE 0 -1,
  then UNLINK of a temporary set (or lua). This is still inefficient.

Co-authored-by: Tyson Andre <tysonandre775@hotmail.com>
2020-08-04 17:49:33 +03:00
Oran Agra
191b118102 fix new rdb test failing on timing issues (#7604)
apparenlty on github actions sometimes 500ms is not enough
2020-08-04 08:53:50 +03:00
hujie
46bff240bf remove duplicate semicolon (#7438) 2020-08-02 13:59:51 +03:00
Yossi Gottlieb
e97cec2f28 Fix test-centos7-tls daily job. (#7598) 2020-07-31 13:55:57 +03:00
Oran Agra
c5d85c69c7 module hook for master link up missing on successful psync (#7584)
besides, hooks test was time sensitive. when the replica managed to
reconnect quickly after the client kill, the test would fail
2020-07-31 13:14:29 +03:00
Oran Agra
27a8ded3db Remove dead code from update_zmalloc_stat_alloc (#7589)
this seems like leftover from before 9d8ceca
2020-07-31 13:01:39 +03:00
Yossi Gottlieb
92e089b1ab CI: Add daily CentOS 7.x jobs. (#7582) 2020-07-30 13:25:10 +03:00
WuYunlong
be11e1b5ea Fix running single test 14-consistency-check.tcl (#7587) 2020-07-30 08:56:21 +03:00
fayadexinqing
323d92b689 broadcast a PONG message when slot's migration is over, which may reduce the moved request of clients (#7571) 2020-07-29 18:05:27 -07:00