11906 Commits

Author SHA1 Message Date
zhenwei pi
2c853869bf Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.

There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863

So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.

Examples of cpulist in redis.conf:
server_cpulist 0-7:2      means cpu affinity 0,2,4,6
bio_cpulist 1,3           means cpu affinity 1,3
aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11    means cpu affinity 1,10,11

Test on linux/freebsd, both work fine.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-02 21:19:47 +08:00
Oran Agra
94943ad44b optimize memory usage of deferred replies - fixed
When deffered reply is added the previous reply node cannot be used so
all the extra space we allocated in it is wasted. in case someone uses
deffered replies in a loop, each time adding a small reply, each of
these reply nodes (the small string reply) would have consumed a 16k
block.
now when we add anther diferred reply node, we trim the unused portion
of the previous reply block.

see #7123

cherry picked from commit 4ed5b7cb74caf5bef6606909603e371af0da4f9b
with fix to handle a crash with LIBC allocator, which apparently can
return the same pointer despite changing it's size.
i.e. shrinking an allocation of 16k into 56 bytes without changing the
pointer.
2020-05-02 07:52:03 +03:00
Benjamin Sergeant
52a477c661 Update redis-cli.c 2020-05-01 20:57:51 -07:00
antirez
d36292e2b8 Redis 6.0.1. 2020-05-02 00:10:20 +02:00
antirez
1812dc1644 Cast printf() argument to the format specifier.
We could use uint64_t specific macros, but after all it's simpler to
just use an obvious equivalent type plus casting: this will be a no op
and is simpler than fixed size types printf macros.
2020-05-02 00:04:19 +02:00
antirez
410e48ffb4 Revert "optimize memory usage of deferred replies"
This reverts commit 4ed5b7cb74caf5bef6606909603e371af0da4f9b.
2020-05-02 00:04:19 +02:00
antirez
ef0c2a9268 Save a call to stopThreadedIOIfNeeded() for the base case.
Probably no performance changes, but the code should be trivial to
read as in "No threading? Use the normal function and return".
2020-05-02 00:04:19 +02:00
antirez
d3e39184c4 Cast printf() argument to the format specifier.
We could use uint64_t specific macros, but after all it's simpler to
just use an obvious equivalent type plus casting: this will be a no op
and is simpler than fixed size types printf macros.
2020-05-02 00:03:14 +02:00
antirez
a3c7ee6fa3 Revert "optimize memory usage of deferred replies"
This reverts commit 4ed5b7cb74caf5bef6606909603e371af0da4f9b.
2020-05-01 23:57:11 +02:00
antirez
4637c88845 Save a call to stopThreadedIOIfNeeded() for the base case.
Probably no performance changes, but the code should be trivial to
read as in "No threading? Use the normal function and return".
2020-05-01 18:45:51 +02:00
Diab Neiroukh
4e0d22e919 Clang requires libatomic to be linked via LDFLAGS
Clang requires the libatomic to be linked via LDFLAGS or else the linker
will fail to recognise __atomic macros.

An error as a result of not passing -latomic to the linker can be seen
below:

ld.lld: error: undefined symbol: __atomic_fetch_add_2
>>> referenced by sds.c:185
>>>               lto.tmp:(sdsdupshared)
clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [Makefile:276: keydb-server] Error 1
Former-commit-id: dcd7b684bbb7b4719a0e44d5764b01c48eb619dd
2020-04-30 15:06:10 -04:00
antirez
1367cb7b6d Redis 6.0.0 GA. 2020-04-30 15:04:41 +02:00
antirez
c7e3501d25 Update help.h again before Redis 6 GA. 2020-04-30 13:43:58 +02:00
antirez
9d5ba2e7be redis-cli: fix hints with subcommands. 2020-04-30 13:43:58 +02:00
antirez
8bafad8621 Update help.h again before Redis 6 GA. 2020-04-30 13:43:05 +02:00
antirez
185884ec52 redis-cli: fix hints with subcommands. 2020-04-30 13:42:25 +02:00
antirez
6ffa739c13 redis-cli command help updated. 2020-04-30 13:02:59 +02:00
zhaozhao.zz
b5dc95d4cc lazyfree & eviction: record latency generated by lazyfree eviction
1. add eviction-lazyfree monitor
2. put eviction-del & eviction-lazyfree into eviction-cycle
   that means eviction-cycle contains all the latency in
   the eviction cycle including del and lazyfree
3. use getMaxmemoryState to check if we can break in lazyfree-evict
2020-04-30 13:02:59 +02:00
antirez
c3aa13a46a MIGRATE AUTH2 for ACL support. 2020-04-30 13:02:59 +02:00
antirez
4f61650c3c CLIENT KILL USER <username>. 2020-04-30 13:02:59 +02:00
antirez
aec7e4a836 Fix tracking table max keys option in redis.conf. 2020-04-30 13:02:58 +02:00
antirez
99569af4aa redis-cli: safer cluster fix with unreachalbe masters. 2020-04-30 13:02:58 +02:00
antirez
9a9953d331 redis-cli: simplify cluster nodes coverage display. 2020-04-30 13:02:58 +02:00
antirez
dbf803bf9c redis-cli: try to make clusterManagerFixOpenSlot() more readable.
Also improve the message to make clear that there is no *clear* owner,
not that there is no owner at all.
2020-04-30 13:02:58 +02:00
Guy Benoish
20a9fe531c XINFO STREAM FULL should have a default COUNT of 10 2020-04-30 13:02:58 +02:00
antirez
78b9c097c9 Comment clearly why we moved some code in #6623. 2020-04-30 13:02:58 +02:00
srzhao
e9811c3b12 fix pipelined WAIT performance issue.
If client gets blocked again in `processUnblockedClients`, redis will not send
`REPLCONF GETACK *` to slaves untill next eventloop, so the client will be
blocked for 100ms by default(10hz) if no other file event fired.

move server.get_ack_from_slaves sinppet after `processUnblockedClients`, so
that both the first WAIT command that puts client in blocked context and the
following WAIT command processed in processUnblockedClients would trigger
redis-sever to send `REPLCONF GETACK *`, so that the eventloop would get
`REPLCONG ACK <reploffset>` from slaves and unblocked ASAP.
2020-04-30 13:02:58 +02:00
antirez
d66ac30fd4 Fix create-cluster BIN_PATH. 2020-04-30 13:02:58 +02:00
Guy Benoish
dc3d865edc Extend XINFO STREAM output
Introducing XINFO STREAM <key> FULL
2020-04-30 13:02:58 +02:00
hwware
12bb6b0f08 Fix not used marco in cluster.c 2020-04-30 13:02:58 +02:00
Itamar Haber
1541e3e522 Update create-cluster 2020-04-30 13:02:58 +02:00
Itamar Haber
c028751ef0 Adds BIN_PATH to create-cluster
Allows for setting the binaries path if used outside the upstream repo.

Also documents `call` in usage clause (TODO: port to
`redis-cli --cluster call` or just deprecate it).
2020-04-30 13:02:58 +02:00
antirez
3475f119e8 redis-cli command help updated. 2020-04-30 13:00:37 +02:00
Salvatore Sanfilippo
fb0dbb3783 Merge pull request #4838 from soloestoy/lazyfree-eviction
lazyfree & eviction: record latency generated by lazyfree eviction
2020-04-30 10:43:12 +02:00
antirez
80c5dc1247 MIGRATE AUTH2 for ACL support. 2020-04-30 10:14:15 +02:00
antirez
ccb24b9e16 CLIENT KILL USER <username>. 2020-04-30 09:58:06 +02:00
zhaozhao.zz
f55fc50dde lazyfree & eviction: record latency generated by lazyfree eviction
1. add eviction-lazyfree monitor
2. put eviction-del & eviction-lazyfree into eviction-cycle
   that means eviction-cycle contains all the latency in
   the eviction cycle including del and lazyfree
3. use getMaxmemoryState to check if we can break in lazyfree-evict
2020-04-30 15:54:14 +08:00
John Sully
a3e0c42dc4 cluster tests should be a named task
Former-commit-id: f715d0b860816165a2748ebf21876df87756a25a
2020-04-29 18:43:55 -04:00
John Sully
4a5aa68074 Run cluster tests as part of CI
Former-commit-id: 98d690b8499d0c3085ce56021dac499349898850
2020-04-29 18:41:51 -04:00
antirez
077dfaad72 Fix tracking table max keys option in redis.conf. 2020-04-29 18:49:42 +02:00
antirez
0e450be7cf redis-cli: safer cluster fix with unreachalbe masters. 2020-04-29 16:57:06 +02:00
antirez
ad291c381c redis-cli: simplify cluster nodes coverage display. 2020-04-29 16:33:47 +02:00
antirez
a9cc31e32e redis-cli: try to make clusterManagerFixOpenSlot() more readable.
Also improve the message to make clear that there is no *clear* owner,
not that there is no owner at all.
2020-04-29 12:37:47 +02:00
antirez
bfe8f56210 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-04-29 11:16:43 +02:00
antirez
e978e71402 Comment clearly why we moved some code in #6623. 2020-04-29 11:16:30 +02:00
srzhao
e9d4c24e01 fix pipelined WAIT performance issue.
If client gets blocked again in `processUnblockedClients`, redis will not send
`REPLCONF GETACK *` to slaves untill next eventloop, so the client will be
blocked for 100ms by default(10hz) if no other file event fired.

move server.get_ack_from_slaves sinppet after `processUnblockedClients`, so
that both the first WAIT command that puts client in blocked context and the
following WAIT command processed in processUnblockedClients would trigger
redis-sever to send `REPLCONF GETACK *`, so that the eventloop would get
`REPLCONG ACK <reploffset>` from slaves and unblocked ASAP.
2020-04-29 11:00:35 +02:00
John Sully
ad3de92c57 use serverAssert() instead of assert() to get callstacks in fastlock
Former-commit-id: 45535e8a6377963dce5b158a9a6e448c5c22a0a8
2020-04-28 22:41:07 -04:00
John Sully
d9409d3614 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: c78895aaea4bbe058f49e28151238f4a4c8bae60
2020-04-28 22:24:45 -04:00
Salvatore Sanfilippo
270c1f4d80 Merge pull request #7161 from guybe7/xinfo_full_count
XINFO STREAM FULL should have a default COUNT of 10
2020-04-28 17:19:31 +02:00
Guy Benoish
6544cf0f48 XINFO STREAM FULL should have a default COUNT of 10 2020-04-28 18:09:03 +03:00