9295 Commits

Author SHA1 Message Date
Yossi Gottlieb
16ba33c05b TLS: Fix test failures on recent Debian/Ubuntu.
Seems like on some systems choosing specific TLS v1/v1.1 versions no
longer works as expected. Test is reduced for v1.2 now which is still
good enough to test the mechansim, and matters most anyway.
2020-05-15 22:23:24 +02:00
Yossi Gottlieb
77ae66930c TLS: Add crypto locks for older OpenSSL support.
This is really required only for older OpenSSL versions.

Also, at the moment Redis does not use OpenSSL from multiple threads so
this will only be useful if modules end up doing that.
2020-05-15 22:23:24 +02:00
David Carlier
389697988a NetBSD build update.
This platform supports CPU affinity (but not OpenBSD).
2020-05-15 22:23:24 +02:00
Madelyn Olson
2435341d74 Added a refcount on timer events to prevent deletion of recursive timer calls 2020-05-15 22:23:24 +02:00
antirez
80c906bd30 Cache master without checking of deferred close flags.
The context is issue #7205: since the introduction of threaded I/O we close
clients asynchronously by default from readQueryFromClient(). So we
should no longer prevent the caching of the master client, to later
PSYNC incrementally, if such flags are set. However we also don't want
the master client to be cached with such flags (would be closed
immediately after being restored). And yet we want a way to understand
if a master was closed because of a protocol error, and in that case
prevent the caching.
2020-05-15 22:23:24 +02:00
antirez
74249be4a2 Track events processed while blocked globally.
Related to #7234.
2020-05-14 11:29:43 +02:00
antirez
8bf660af90 Some rework of #7234. 2020-05-14 11:29:43 +02:00
Oran Agra
9da134cd88 fix redis 6.0 not freeing closed connections during loading.
This bug was introduced by a recent change in which readQueryFromClient
is using freeClientAsync, and despite the fact that now
freeClientsInAsyncFreeQueue is in beforeSleep, that's not enough since
it's not called during loading in processEventsWhileBlocked.
furthermore, afterSleep was called in that case but beforeSleep wasn't.

This bug also caused slowness sine the level-triggered mode of epoll
kept signaling these connections as readable causing us to keep doing
connRead again and again for ll of these, which keep accumulating.

now both before and after sleep are called, but not all of their actions
are performed during loading, some are only reserved for the main loop.

fixes issue #7215
2020-05-14 11:29:43 +02:00
antirez
f7f219a137 Regression test for #7249. 2020-05-14 11:29:43 +02:00
antirez
693629585b rax.c updated from upstream antirez/rax. 2020-05-14 11:29:43 +02:00
antirez
e3b5648df0 Tracking: send eviction messages when evicting entries.
A fix for #7249.
2020-05-14 11:29:43 +02:00
Oran Agra
5c41802d55 fix unstable replication test
this test which has coverage for varoius flows of diskless master was
failing randomly from time to time.

the failure was:
[err]: diskless all replicas drop during rdb pipe in tests/integration/replication.tcl
log message of '*Diskless rdb transfer, last replica dropped, killing fork child*' not found

what seemed to have happened is that the master didn't detect that all
replicas dropped by the time the replication ended, it thought that one
replica is still connected.

now the test takes a few seconds longer but it seems stable.
2020-05-14 11:29:43 +02:00
ShooterIT
a23cdbb94b Redis Benchmark: Fix coredump because of double free 2020-05-09 11:14:58 +02:00
antirez
1276058ea8 Cluster: clarify we always resolve the sender. 2020-05-09 11:14:58 +02:00
antirez
002fcde3da Cluster: refactor ping/data delay handling. 2020-05-09 11:14:58 +02:00
antirez
960186a71f Cluster: introduce data_received field.
We want to send pings and pongs at specific intervals, since our packets
also contain information about the configuration of the cluster and are
used for gossip. However since our cluster bus is used in a mixed way
for data (such as Pub/Sub or modules cluster messages) and metadata,
sometimes a very busy channel may delay the reception of pong packets.
So after discussing it in #7216, this commit introduces a new field that
is not exposed in the cluster, is only an internal information about
the last time we received any data from a given node: we use this field
in order to avoid detecting failures, claiming data reception of new
data from the node is a proof of liveness.
2020-05-09 11:14:58 +02:00
antirez
3672875b4c stringmatchlen() should not expect null terminated strings. 2020-05-08 10:37:36 +02:00
Brad Dunbar
24e12641d5 Remove unreachable branch. 2020-05-08 10:37:36 +02:00
hwware
c7edffbd56 add jemalloc-bg-thread config in redis conf 2020-05-08 10:37:36 +02:00
hwware
8a9c84f4a5 add include guard for lolwut.h 2020-05-08 10:37:36 +02:00
antirez
cb683a84f7 Don't propagate spurious MULTI on DEBUG LOADAOF. 2020-05-08 10:37:36 +02:00
antirez
84d9766d6a Dump recent backlog on master query generating errors. 2020-05-08 10:37:36 +02:00
Titouan Christophe
ec1e106ec5 make struct user anonymous (only typedefed)
This works because this struct is never referenced by its name,
but always by its type.

This prevents a conflict with struct user from <sys/user.h>
when compiling against uclibc.

Signed-off-by: Titouan Christophe <titouan.christophe@railnova.eu>
2020-05-08 10:37:36 +02:00
antirez
e48c37316e Test: --dont-clean should do first cleanup. 2020-05-08 10:37:36 +02:00
Benjamin Sergeant
1e561cfaaf Add --user argument to redis-benchmark.c (ACL) 2020-05-08 10:37:36 +02:00
antirez
d1af82a883 Drop not needed part from #7194. 2020-05-08 10:37:36 +02:00
Muhammad Zahalqa
897a360d06 Fix compiler warnings on function rev(unsigned long) 2020-05-08 10:37:36 +02:00
antirez
ac316d8cc5 Move CRC64 initialization in main(). 2020-05-08 10:37:35 +02:00
antirez
fc7bc32045 Fix CRC64 initialization outside the Redis server itself. 2020-05-08 10:37:35 +02:00
hwware
a6e55c096d Client Side Caching: Add Tracking Prefix Number Stats in Server Info 2020-05-08 10:37:35 +02:00
antirez
b062fd5235 Fix NetBSD build by fixing redis_set_thread_title() support.
See #7188.
2020-05-08 10:37:35 +02:00
antirez
4efb25d9c0 Rework a bit the documentation for CPU pinning. 2020-05-08 10:37:35 +02:00
zhenwei pi
d6436eb7cf Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.

There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863

So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.

Examples of cpulist in redis.conf:
server_cpulist 0-7:2      means cpu affinity 0,2,4,6
bio_cpulist 1,3           means cpu affinity 1,3
aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11    means cpu affinity 1,10,11

Test on linux/freebsd, both work fine.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-08 10:37:35 +02:00
Guy Benoish
3a441c7d95 XPENDING should not update consumer's seen-time
Same goes for XGROUP DELCONSUMER (But in this case, it doesn't
have any visible effect)
2020-05-08 10:37:35 +02:00
Oran Agra
75addb4fe2 optimize memory usage of deferred replies - fixed
When deffered reply is added the previous reply node cannot be used so
all the extra space we allocated in it is wasted. in case someone uses
deffered replies in a loop, each time adding a small reply, each of
these reply nodes (the small string reply) would have consumed a 16k
block.
now when we add anther diferred reply node, we trim the unused portion
of the previous reply block.

see #7123

cherry picked from commit fb732f7a944a4d4c90bb7375cb6030e88211f5aa
with fix to handle a crash with LIBC allocator, which apparently can
return the same pointer despite changing it's size.
i.e. shrinking an allocation of 16k into 56 bytes without changing the
pointer.
2020-05-08 10:37:35 +02:00
Deliang Yang
c57d9146f4 reformat code 2020-05-08 10:37:35 +02:00
Oran Agra
3d3861dd88 add daily github actions with libc malloc and valgrind
* fix memlry leaks with diskless replica short read.
* fix a few timing issues with valgrind runs
* fix issue with valgrind and watchdog schedule signal

about the valgrind WD issue:
the stack trace test in logging.tcl, has issues with valgrind:
==28808== Can't extend stack to 0x1ffeffdb38 during signal delivery for thread 1:
==28808==   too small or bad protection modes

it seems to be some valgrind bug with SA_ONSTACK.
SA_ONSTACK seems unneeded since WD is not recursive (SA_NODEFER was removed),
also, not sure if it's even valid without a call to sigaltstack()
2020-05-08 10:37:35 +02:00
antirez
47ff13637e Redis 6.0.1. 2020-05-02 00:10:20 +02:00
antirez
db73d0998f Cast printf() argument to the format specifier.
We could use uint64_t specific macros, but after all it's simpler to
just use an obvious equivalent type plus casting: this will be a no op
and is simpler than fixed size types printf macros.
2020-05-02 00:04:19 +02:00
antirez
7c0fe72714 Revert "optimize memory usage of deferred replies"
This reverts commit fb732f7a944a4d4c90bb7375cb6030e88211f5aa.
2020-05-02 00:04:19 +02:00
antirez
8fe25edc75 Save a call to stopThreadedIOIfNeeded() for the base case.
Probably no performance changes, but the code should be trivial to
read as in "No threading? Use the normal function and return".
2020-05-02 00:04:19 +02:00
antirez
17dfd7cabb Redis 6.0.0 GA. 2020-04-30 15:04:41 +02:00
antirez
1f9b82bd56 Update help.h again before Redis 6 GA. 2020-04-30 13:43:58 +02:00
antirez
3fcffe7d04 redis-cli: fix hints with subcommands. 2020-04-30 13:43:58 +02:00
antirez
455d8a05c1 redis-cli command help updated. 2020-04-30 13:02:59 +02:00
zhaozhao.zz
70287bbc94 lazyfree & eviction: record latency generated by lazyfree eviction
1. add eviction-lazyfree monitor
2. put eviction-del & eviction-lazyfree into eviction-cycle
   that means eviction-cycle contains all the latency in
   the eviction cycle including del and lazyfree
3. use getMaxmemoryState to check if we can break in lazyfree-evict
2020-04-30 13:02:59 +02:00
antirez
7be21139aa MIGRATE AUTH2 for ACL support. 2020-04-30 13:02:59 +02:00
antirez
e1ee1a49d6 CLIENT KILL USER <username>. 2020-04-30 13:02:59 +02:00
antirez
d56f058c08 Fix tracking table max keys option in redis.conf. 2020-04-30 13:02:58 +02:00
antirez
96dd5fc932 redis-cli: safer cluster fix with unreachalbe masters. 2020-04-30 13:02:58 +02:00