55 Commits

Author SHA1 Message Date
John Sully
521d4cd817 Fix test failure 2022-03-31 17:48:37 -04:00
VivekSainiEQ
bfcea943ea Merge remote-tracking branch 'mainpro/PRO_RELEASE_6' into keydbpro
Former-commit-id: 5a32d66ee382b6d227a67073afc81ca058d605ed
2021-12-06 20:43:23 +00:00
John Sully
2ed6d3f03a Implement the disk backed backlog functionality
Former-commit-id: 759cc01c6ba05f9a865c11580cc4975b5f1bd1d6
2021-11-26 03:05:23 +00:00
VivekSainiEQ
f9a355900a Merge remote-tracking branch 'opensource/unstable' into PRO_RELEASE_6
Former-commit-id: 60da86471f68b13e2456e113ecd4aa117d95b134
2021-11-12 21:12:53 +00:00
John Sully
bdff0121eb Make the replica weighting configurable
Former-commit-id: be6a8a7e68acb5cfbe950f13b903e6f7b98c5a39
2021-10-29 17:59:46 +00:00
VivekSainiEQ
1d882b5ddd Merge tag '6.2.6' into Redis_626_Merge
Former-commit-id: e6d7e01be6965110d487e12f40511fe0b3497695
2021-10-21 22:33:55 +00:00
John Sully
075ce8db9e Merge branch 'merge_6.2.2' into keydbpro_collab
Former-commit-id: 54fe037e4e215b1e5bdb42b567c6df04a69ba150
2021-06-02 02:05:44 +00:00
Madelyn Olson
875a1f07d8 Hide migrate command from slowlog if they include auth (#8859)
Redact commands that include sensitive data from slowlog and monitor

(cherry picked from commit a59e75a475782d86d7ce2b5b9c6f5bb4a5ef0bd6)
2021-06-01 17:03:36 +03:00
John Sully
6b4f686f5f Merge branch 'unstable' into keydbpro
Former-commit-id: 205d8f18d2bb8df5253bab40578b006b7aa73fd5
2021-05-28 23:32:46 +00:00
John Sully
5267928381 Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17adec5d406d2e30a5b9ea71c07fce5c
2021-05-21 05:54:39 +00:00
John Sully
fe8efa916b Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0edaa724b9d060a6bb8edc5a6de65fa
2021-05-19 02:59:48 +00:00
Cloud User
6684f150d6 Make high pri time thread configurable
Former-commit-id: 7e94207765d2166f46792aea0919786f3f30d7b3
2021-04-06 22:39:20 +00:00
JunhuaY
28375ff63e
re-fix config rewrite for empty save directive (#8722)
the bug was also discussed in #8716, and was solved in #8719, but incompletely:
when the server is started, and the save option is default, if you issue the " config set save "" "
to change the save option, and then issue the “config rewrite” command, the " save "" " won't be saved.
2021-03-30 22:49:06 +03:00
Yossi Gottlieb
65311a3360
Fix config rewrite with an empty "save" parameter. (#8719) 2021-03-29 18:53:20 +03:00
YaacovHazan
a031d268b1
Make port, tls-port and bind configurations modifiable (#8510)
Add ability to modify port, tls-port and bind configurations by CONFIG SET command.

To simplify the code and make it cleaner, a new structure
added, socketFds, which contains the file descriptors array and its counter,
and used for TCP, TLS and Cluster sockets file descriptors.
2021-03-01 16:04:44 +02:00
guybe7
f745c0181a
Fix race in CONFIG REWRITE sanity (#8536)
server may still be LOADING the RDB when receiving the ping
2021-02-23 20:28:03 +02:00
John Sully
bb767ad61d Enable prefetch with storage providers
Former-commit-id: 21f9c52c446ad1b460a340c3d873839b2355728f
2021-02-11 00:50:21 +00:00
christianEQ
cca8d186aa Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: 5bad058733de2c217340bb9ee48f02b07d754808
2021-02-03 18:10:27 +00:00
Z. Liu
17b34c7309
Add 'set-proc-title' config so that this mechanism can be disabled (#3623)
if option `set-proc-title' is no, then do nothing for proc title.

The reason has been explained long ago, see following:

We update redis to 2.8.8, then found there are some side effect when
redis always change the process title.

We run several slave instance on one computer, and all these salves
listen on unix socket only, then ps will show:

  1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server *:0
  1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server *:0

for redis 2.6 the output of ps is like following:

  1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server /etc/redis/a.conf
  1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server /etc/redis/b.conf

Later is more informational in our case. The situation
is worse when we manage the config and process running
state by salt. Salt check the process by running "ps |
grep SIG" (for Gentoo System) to check the running
state, where SIG is the string to search for when
looking for the service process with ps. Previously, we
define sig as "/usr/sbin/redis-server
/etc/redis/a.conf". Since the ps output is identical for
our case, so we have no way to check the state of
specified redis instance.

So, for our case, we prefer the old behavior, i.e, do
not change the process title for the main redis process.
Or add an option such as "set-proc-title [yes|no]" to
control this behavior.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-28 11:12:39 +02:00
christianEQ
358debebfa Merge tag 'tags/6.0.10' into redismerge_2021-01-20
Former-commit-id: dadce055f897cee83946c2d3e5cbb76341b94230
2021-01-26 21:43:09 +00:00
Yossi Gottlieb
8c291b97b9
TLS: Add different client cert support. (#8076)
This adds a new `tls-client-cert-file` and `tls-client-key-file`
configuration directives which make it possible to use different
certificates for the TLS-server and TLS-client functions of Redis.

This is an optional directive. If it is not specified the `tls-cert-file`
and `tls-key-file` directives are used for TLS client functions as well.

Also, `utils/gen-test-certs.sh` now creates additional server-only and client-only certs and will skip intensive operations if target files already exist.
2020-12-11 18:31:40 +02:00
John Sully
706ba4e1dc Merge branch 'unstable' into keydbpro
Former-commit-id: b7a1e16c0f04e8aeb3764e3681fa14fc0f97f6a3
2020-12-10 02:37:28 +00:00
Yossi Gottlieb
bccbc5509a
Add CLIENT INFO and CLIENT LIST [id]. (#8113)
* Add CLIENT INFO subcommand.

The output is identical to CLIENT LIST but provides a single line for
the current client only.

* Add CLIENT LIST ID [id...].

Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-12-07 14:24:05 +02:00
Oran Agra
c31055db61 Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed
The test creates keys with various encodings, DUMP them, corrupt the payload
and RESTORES it.
It utilizes the recently added use-exit-on-panic config to distinguish between
 asserts and segfaults.
If the restore succeeds, it runs random commands on the key to attempt to
trigger a crash.

It runs in two modes, one with deep sanitation enabled and one without.
In the first one we don't expect any assertions or segfaults, in the second one
we expect assertions, but no segfaults.
We also check for leaks and invalid reads using valgrind, and if we find them
we print the commands that lead to that issue.

Changes in the code (other than the test):
- Replace a few NPD (null pointer deference) flows and division by zero with an
  assertion, so that it doesn't fail the test. (since we set the server to use
  `exit` rather than `abort` on assertion).
- Fix quite a lot of flows in rdb.c that could have lead to memory leaks in
  RESTORE command (since it now responds with an error rather than panic)
- Add a DEBUG flag for SET-SKIP-CHECKSUM-VALIDATION so that the test don't need
  to bother with faking a valid checksum
- Remove a pile of code in serverLogObjectDebugInfo which is actually unsafe to
  run in the crash report (see comments in the code)
- fix a missing boundary check in lzf_decompress

test suite infra improvements:
- be able to run valgrind checks before the process terminates
- rotate log files when restarting servers
2020-12-06 14:54:34 +02:00
Kajaruban Surendran
fd03008bbf Configurable option for MOTD
Former-commit-id: 49a89d636ba698dbd0858d5059d3d6387c8c1fc7
2020-11-30 10:42:48 -05:00
Oran Agra
76f3a63de1 memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.

(cherry picked from commit bea40e6a41e31a52e9e6efee77ea5a4bd873b759)
2020-10-27 09:12:01 +02:00
Oran Agra
bea40e6a41
memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.
2020-10-05 11:15:36 +03:00
John Sully
4f18a247e3 Merge tag '6.0.8' into unstable
Former-commit-id: 4c7e4b91a6bb2034636856b608b8c386d07f5541
2020-09-30 19:47:55 +00:00
John Sully
ad4c2390a8 Merge branch 'unstable' into keydbpro
Former-commit-id: d1986d9fcdfd56f2c30ee95edef6d5260e0aa777
2020-09-24 22:02:30 +00:00
Yossi Gottlieb
bf6c002005 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.


Former-commit-id: d3834c50699bc4f31f381d6d03d4c1b022380895
2020-09-16 02:10:29 +00:00
Yossi Gottlieb
635704d424 Tests: validate CONFIG REWRITE for all params. (#7764)
This is a catch-all test to confirm that that rewrite produces a valid
output for all parameters and that this process does not introduce
undesired configuration changes.

(cherry picked from commit a8b7268911803f4ac827ab22e8e5b10134664832)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb
a8b7268911
Tests: validate CONFIG REWRITE for all params. (#7764)
This is a catch-all test to confirm that that rewrite produces a valid
output for all parameters and that this process does not introduce
undesired configuration changes.
2020-09-09 15:43:11 +03:00
Yossi Gottlieb
818a746e32
Fix default/explicit "save" parameter loading. (#7767)
Save parameters should either be default or whatever specified in the
config file. This fixes an issue introduced in #7092 which causes
configuration file settings to be applied on top of the defaults.
2020-09-09 15:12:57 +03:00
John Sully
d1f45b8ebf Implement use-fork config (fails with diskless repl)
Former-commit-id: f2d5c2bca22e9fd506db123c47b7f60cdded7e2c
2020-08-24 03:17:59 +00:00
Yossi Gottlieb
7a536c2912 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.

(cherry picked from commit 3e6f2b1a45176ac3d81b95cb6025f30d7aaa1393)
2020-07-20 21:08:26 +03:00
John Sully
a709cf8250 Fix test failure
Former-commit-id: 05d84f5fc184676d32c4bb61ac5957f1787cd4f8
2020-07-14 04:34:40 +00:00
John Sully
17661f2382 Implement storage key cache, and writeback memory model
Former-commit-id: 732bd9c153459f1174475ad67de36c399ddbe359
2020-07-11 21:23:48 +00:00
Yossi Gottlieb
3e6f2b1a45
TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.
2020-07-10 11:33:47 +03:00
John Sully
67b7d512cb Merge commit 'cb683a84f7a9d530bec629f2b656e7d0842a0f75' into unstable
Former-commit-id: 817a11ff110772893eda0675912bbb0cfc1fca74
2020-05-22 15:56:35 -04:00
zhenwei pi
d6436eb7cf Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.

There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863

So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.

Examples of cpulist in redis.conf:
server_cpulist 0-7:2      means cpu affinity 0,2,4,6
bio_cpulist 1,3           means cpu affinity 1,3
aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11    means cpu affinity 1,10,11

Test on linux/freebsd, both work fine.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-08 10:37:35 +02:00
zhenwei pi
1a0deab2a5 Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.

There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863

So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.

Examples of cpulist in redis.conf:
server_cpulist 0-7:2      means cpu affinity 0,2,4,6
bio_cpulist 1,3           means cpu affinity 1,3
aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11    means cpu affinity 1,10,11

Test on linux/freebsd, both work fine.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-02 21:19:47 +08:00
John Sully
39378f982e Fix most tests (still some failures)
Former-commit-id: da83e841255487efe0e4b13d42b2dcc55a369838
2020-01-27 18:16:19 -05:00
Oran Agra
52ea44e53b config.c adjust config limits and mutable
- make lua-replicate-commands mutable (it never was, but i don't see why)
- make tcp-backlog immutable (fix a recent refactory mistake)
- increase the max limit of a few configs to match what they were before
the recent refactory
2019-12-29 15:46:31 +01:00
Oran Agra
0c3fe52ef7 config.c adjust config limits and mutable
- make lua-replicate-commands mutable (it never was, but i don't see why)
- make tcp-backlog immutable (fix a recent refactory mistake)
- increase the max limit of a few configs to match what they were before
the recent refactory
2019-12-26 15:16:15 +02:00
Oran Agra
18e72c5cc7 Converting more configs to use generic infra, and moving defaults to config.c
Changes in behavior:
- Change server.stream_node_max_entries from int64_t to long long, so that it can be used by the generic infra
- standard error reply instead of "repl-backlog-size must be 1 or greater" and such
- tls-port and a few TLS booleans were readable (config get) even when USE_OPENSSL was off (now they aren't)
- syslog-enabled, syslog-ident, cluster-enabled, appendfilename, and supervised didn't have a get (now they do)
- pidfile was initialized to NULL in InitServerConfig but had CONFIG_DEFAULT_PID_FILE in rewriteConfig (so the real default was "", but rewrite would cause it to be set), fixed the rewrite.
- TLS config in server.h was uninitialized (if no tls config args were provided)

Adding test for sanity and coverage
2019-11-28 11:24:57 +02:00
zhaozhao.zz
b89302c462 adjust qbuf to 26 in test case for client list 2018-08-14 00:57:22 +08:00
Sebastian Waisbrot
97a2248309 Fix race condition in unit/introspection
Make sure monitor is attached in one connection before issuing commands to be monitored in another one
2015-08-11 22:56:17 -07:00
Matt Stancliff
1cedebb799 Remove trailing spaces from tests 2014-09-29 06:49:08 -04:00
antirez
ac8c89cb20 Test: avoid false positives in CLIENT SETNAME closed connection test. 2013-02-12 13:27:24 +01:00
antirez
ea1f503cfe Tests for CLIENT GETNAME/SETNAME. 2013-01-15 13:34:17 +01:00