449 Commits

Author SHA1 Message Date
antirez
249d933215 Tracking: NOLOOP further implementation and fixes. 2020-04-24 10:14:48 +02:00
zhenwei pi
ff889d2c8a Threaded IO: set thread name for redis-server
Set thread name for each thread of redis-server, this helps us to
monitor the utilization and optimise the performance.

And suggested-by Salvatore, implement this feature for multi
platforms. Currently support linux and bsd, ignore other OS.

An exmaple on Linux:
 # top -d 5 -p `pidof redis-server ` -H

    PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
3682671 root      20   0  227744   8248   3836 R 99.2  0.0   0:19.53 redis-server
3682677 root      20   0  227744   8248   3836 S 26.4  0.0   0:04.15 io_thd_3
3682675 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.98 io_thd_1
3682676 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.97 io_thd_2
3682672 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_close_file
3682673 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_aof_fsync
3682674 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 bio_lazy_free
3682678 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682682 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682683 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682684 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682685 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682687 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd

Another exmaple on FreeBSD-12.1:
  PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
 5212 root        100    0    48M  7280K CPU2     2   0:26  99.52% redis-server{redis-server}
 5212 root         38    0    48M  7280K umtxn    4   0:06  26.94% redis-server{io_thd_3}
 5212 root         36    0    48M  7280K umtxn    6   0:06  26.84% redis-server{io_thd_1}
 5212 root         39    0    48M  7280K umtxn    1   0:06  25.30% redis-server{io_thd_2}
 5212 root         20    0    48M  7280K uwait    3   0:00   0.00% redis-server{redis-server}
 5212 root         21    0    48M  7280K uwait    2   0:00   0.00% redis-server{bio_close_file}
 5212 root         21    0    48M  7280K uwait    3   0:00   0.00% redis-server{bio_aof_fsync}
 5212 root         21    0    48M  7280K uwait    0   0:00   0.00% redis-server{bio_lazy_free}

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-04-24 10:14:48 +02:00
antirez
2dac74ffb4 RESP3: fix HELLO map len in Sentinel mode.
See #6160.
2020-04-16 11:22:18 +02:00
antirez
d1a7b69981 Fix HELLO reply in Sentinel mode, see #6160. 2020-04-15 16:03:16 +02:00
antirez
fdd6da647d Speedup INFO by counting client memory incrementally.
Related to #5145.

Design note: clients may change type when they turn into replicas or are
moved into the Pub/Sub category and so forth. Moreover the recomputation
of the bytes used is problematic for obvious reasons: it changes
continuously, so as a conservative way to avoid accumulating errors,
each client remembers the contribution it gave to the sum, and removes
it when it is freed or before updating it with the new memory usage.
2020-04-07 16:53:13 +02:00
hwware
74778c8a01 add check for not switching between optin optout mode directly 2020-04-07 16:52:04 +02:00
hwware
02cfb2cb1d add check for not providing both optin optout flag 2020-04-07 16:52:04 +02:00
antirez
c202c54ff7 Minor changes to #7037. 2020-03-31 17:12:19 +02:00
Guy Benoish
5c23cd55d4 Modules: Test MULTI/EXEC replication of RM_Replicate
Makse sure call() doesn't wrap replicated commands with
a redundant MULTI/EXEC

Other, unrelated changes:
1. Formatting compiler warning in INFO CLIENTS
2. Use CLIENT_ID_AOF instead of UINT64_MAX
2020-03-31 17:12:19 +02:00
Yossi Gottlieb
700126e9cf Fix crashes related to failed/rejected accepts. 2020-03-25 15:55:24 +01:00
antirez
21c22bcd9a ACL: default user off should not allow automatic authentication.
This fixes issue #7011.
2020-03-25 15:54:34 +01:00
zhaozhao.zz
714ae38ee1 Threaded IO: bugfix client kill may crash redis 2020-03-25 15:54:34 +01:00
antirez
352ea2365f Aesthetic changes in PR #6989. 2020-03-25 15:54:34 +01:00
zhaozhao.zz
d3491c5182 Threaded IO: bugfix #6988 process events while blocked 2020-03-25 15:54:34 +01:00
antirez
75a7615ce2 Tracking: optin/out implemented. 2020-02-27 18:02:30 +01:00
Madelyn Olson
fa54ce9036 Give an error message if you specify redirect twice 2020-02-27 18:00:47 +01:00
Madelyn Olson
188f478f43 Minor CSC fixes and fixed documentation 2020-02-27 18:00:47 +01:00
antirez
5173b1042e Tracking: fix operators precedence error in bcast check. 2020-02-27 17:59:57 +01:00
antirez
d5517def76 Tracking: fix sending messages bug + tracking off bug. 2020-02-27 17:59:57 +01:00
antirez
7841d0234f Tracking: BCAST: basic feature now works. 2020-02-27 17:59:57 +01:00
antirez
abdaebc6c4 Tracking: BCAST: registration in the prefix table. 2020-02-27 17:59:57 +01:00
antirez
ed36f3c76f Tracking: BCAST: parsing of the options + skeleton. 2020-02-27 17:59:57 +01:00
Oran Agra
5e3b4c978b freeClientAsync don't lock mutex if there's just one thread 2020-02-12 14:15:56 +01:00
Guy Benoish
084dc68128 Some refactroing using getClientType instead of CLIENT_SLAVE 2020-02-12 14:15:56 +01:00
Guy Benoish
f4c30fd74b Fix small bugs related to replica and monitor ambiguity
1. server.repl_no_slaves_since can be set when a MONITOR client disconnects
2. c->repl_ack_time can be set by a newline from a MONITOR client
3. Improved comments
2020-02-12 14:15:56 +01:00
antirez
598c324bc1 Setting N I/O threads should mean N-1 additional + 1 main thread. 2020-01-17 10:47:38 +01:00
antirez
0b02b27705 Port PR #6110 to new connection object code. 2020-01-17 10:47:38 +01:00
antirez
a6eb3f8bfb A few comments about main thread serving I/O as well.
Related to #6110.
2020-01-17 10:47:38 +01:00
zhaozhao.zz
1f76ff2fb9 Threaded IO: use main thread to handle read work 2020-01-17 10:47:38 +01:00
zhaozhao.zz
0a3410db70 Threaded IO: use main thread to handle write work 2020-01-17 10:47:38 +01:00
antirez
c2461ba531 Fix duplicated CLIENT SETNAME reply.
Happened when we set the name to "" to cancel the name.
Was introduced during the RESP3 refactoring.

See #6036.
2019-12-29 15:46:31 +01:00
antirez
1b56a873d5 Inline protocol: handle empty strings well.
This bug is from the first version of Redis. Probably the problem here
is that before we used an SDS split function that created empty strings
for additional spaces, like in "SET    foo          bar".
AFAIK later we replaced it with the curretn sdssplitarg() API that has
no such a problem. As a result, we introduced a bug, where it is no
longer possible to do something like:

    SET foo ""

Using the inline protocol. Now it is fixed.
2019-12-29 15:46:31 +01:00
Madelyn Olson
05b7628687 Add module APIs for custom authentication 2019-12-17 06:59:59 +00:00
Salvatore Sanfilippo
ab520ef80f Merge pull request #6531 from oranagra/rm_save_long_double
Module API for loading and saving long double
2019-11-04 17:33:35 +01:00
Oran Agra
04df0cc328 Module API for loading and saving long double
looks like each platform implements long double differently (different bit count)
so we can't save them as binary, and we also want to avoid creating a new RDB
format version, so we save these are hex strings using "%La".

This commit includes a change in the arguments of ld2string to support this.
as well as tests for coverage and short reads.

coded by @guybe7
2019-11-03 16:42:31 +02:00
Oran Agra
68c6aacf3b Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
antirez
261a66b791 Modules hooks: don't call the hook for fake clients. 2019-10-23 18:39:53 +02:00
antirez
d5bd45a368 Modules hooks: fix sub event in calls from client change. 2019-10-23 18:39:53 +02:00
antirez
f3e4f24f68 Modules hooks: fix define / linker issues. Implement one test event. 2019-10-23 18:39:53 +02:00
Yossi Gottlieb
85d7f38136 Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
Oran Agra
8704be6947 TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
6fd5ff8c98 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
10ffeb03e4 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
antirez
90cf427a5c RESP3: Use verbatim in CLIENT LIST. 2019-09-18 18:52:13 +02:00
antirez
3b40a88587 More strict checks and better comments in flushSlaveOutputBuffers().
Related to #6296.
2019-08-31 14:46:24 +02:00
antirez
d5e2fe1532 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-08-31 14:41:28 +02:00
antirez
11228bd0fb Improve comment in flushSlavesOutputBuffers(). 2019-08-31 14:40:09 +02:00
zhaozhao.zz
89dc16f01d networking: flushSlavesOutputBuffers bugfix 2019-08-02 17:17:19 +08:00
Salvatore Sanfilippo
ee8b42cb27 Merge pull request #6196 from AngusP/resp3
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
2019-07-12 13:08:46 +02:00
antirez
c3d91f5d8b Client side caching: implement CLIENT GETREDIR.
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
2019-07-10 18:17:10 +02:00