49 Commits

Author SHA1 Message Date
VivekSainiEQ
7992428583 Merge remote-tracking branch 'opensource/unstable' into PRO_RELEASE_6
Former-commit-id: 60da86471f68b13e2456e113ecd4aa117d95b134
2021-11-12 21:12:53 +00:00
VivekSainiEQ
b84ee93b5d Merge tag '6.2.6' into Redis_626_Merge
Former-commit-id: e6d7e01be6965110d487e12f40511fe0b3497695
2021-10-21 22:33:55 +00:00
Madelyn Olson
f8938d868b Hide migrate command from slowlog if they include auth (#8859)
Redact commands that include sensitive data from slowlog and monitor

(cherry picked from commit a59e75a475782d86d7ce2b5b9c6f5bb4a5ef0bd6)
2021-06-01 17:03:36 +03:00
John Sully
f4151f0d6b Merge branch 'unstable' into keydbpro
Former-commit-id: 205d8f18d2bb8df5253bab40578b006b7aa73fd5
2021-05-28 23:32:46 +00:00
John Sully
ea6a0f214b Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17adec5d406d2e30a5b9ea71c07fce5c
2021-05-21 05:54:39 +00:00
John Sully
f49d8f9adb Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0edaa724b9d060a6bb8edc5a6de65fa
2021-05-19 02:59:48 +00:00
JunhuaY
265f82d08f re-fix config rewrite for empty save directive (#8722)
the bug was also discussed in #8716, and was solved in #8719, but incompletely:
when the server is started, and the save option is default, if you issue the " config set save "" "
to change the save option, and then issue the “config rewrite” command, the " save "" " won't be saved.
2021-03-30 22:49:06 +03:00
Yossi Gottlieb
394ce70f1d Fix config rewrite with an empty "save" parameter. (#8719) 2021-03-29 18:53:20 +03:00
YaacovHazan
c0de6fbd29 Make port, tls-port and bind configurations modifiable (#8510)
Add ability to modify port, tls-port and bind configurations by CONFIG SET command.

To simplify the code and make it cleaner, a new structure
added, socketFds, which contains the file descriptors array and its counter,
and used for TCP, TLS and Cluster sockets file descriptors.
2021-03-01 16:04:44 +02:00
guybe7
e209b6c1c9 Fix race in CONFIG REWRITE sanity (#8536)
server may still be LOADING the RDB when receiving the ping
2021-02-23 20:28:03 +02:00
John Sully
ef471e6e53 Enable prefetch with storage providers
Former-commit-id: 21f9c52c446ad1b460a340c3d873839b2355728f
2021-02-11 00:50:21 +00:00
christianEQ
c69b2845fd Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: 5bad058733de2c217340bb9ee48f02b07d754808
2021-02-03 18:10:27 +00:00
Z. Liu
69bdefde5f Add 'set-proc-title' config so that this mechanism can be disabled (#3623)
if option `set-proc-title' is no, then do nothing for proc title.

The reason has been explained long ago, see following:

We update redis to 2.8.8, then found there are some side effect when
redis always change the process title.

We run several slave instance on one computer, and all these salves
listen on unix socket only, then ps will show:

  1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server *:0
  1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server *:0

for redis 2.6 the output of ps is like following:

  1 S redis 18036 1 0 80 0 - 56130 ep_pol 14:02 ? 00:00:31 /usr/sbin/redis-server /etc/redis/a.conf
  1 S redis 23949 1 0 80 0 - 11074 ep_pol 15:41 ? 00:00:00 /usr/sbin/redis-server /etc/redis/b.conf

Later is more informational in our case. The situation
is worse when we manage the config and process running
state by salt. Salt check the process by running "ps |
grep SIG" (for Gentoo System) to check the running
state, where SIG is the string to search for when
looking for the service process with ps. Previously, we
define sig as "/usr/sbin/redis-server
/etc/redis/a.conf". Since the ps output is identical for
our case, so we have no way to check the state of
specified redis instance.

So, for our case, we prefer the old behavior, i.e, do
not change the process title for the main redis process.
Or add an option such as "set-proc-title [yes|no]" to
control this behavior.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2021-01-28 11:12:39 +02:00
christianEQ
c068f2cd3d Merge tag 'tags/6.0.10' into redismerge_2021-01-20
Former-commit-id: dadce055f897cee83946c2d3e5cbb76341b94230
2021-01-26 21:43:09 +00:00
Yossi Gottlieb
11b9e4092e TLS: Add different client cert support. (#8076)
This adds a new `tls-client-cert-file` and `tls-client-key-file`
configuration directives which make it possible to use different
certificates for the TLS-server and TLS-client functions of Redis.

This is an optional directive. If it is not specified the `tls-cert-file`
and `tls-key-file` directives are used for TLS client functions as well.

Also, `utils/gen-test-certs.sh` now creates additional server-only and client-only certs and will skip intensive operations if target files already exist.
2020-12-11 18:31:40 +02:00
John Sully
5c832c5ad0 Merge branch 'unstable' into keydbpro
Former-commit-id: b7a1e16c0f04e8aeb3764e3681fa14fc0f97f6a3
2020-12-10 02:37:28 +00:00
Yossi Gottlieb
ed3a6c70dc Add CLIENT INFO and CLIENT LIST [id]. (#8113)
* Add CLIENT INFO subcommand.

The output is identical to CLIENT LIST but provides a single line for
the current client only.

* Add CLIENT LIST ID [id...].

Co-authored-by: Itamar Haber <itamar@redislabs.com>
2020-12-07 14:24:05 +02:00
Oran Agra
912e22b4f9 Sanitize dump payload: fuzz tester and fixes for segfaults and leaks it exposed
The test creates keys with various encodings, DUMP them, corrupt the payload
and RESTORES it.
It utilizes the recently added use-exit-on-panic config to distinguish between
 asserts and segfaults.
If the restore succeeds, it runs random commands on the key to attempt to
trigger a crash.

It runs in two modes, one with deep sanitation enabled and one without.
In the first one we don't expect any assertions or segfaults, in the second one
we expect assertions, but no segfaults.
We also check for leaks and invalid reads using valgrind, and if we find them
we print the commands that lead to that issue.

Changes in the code (other than the test):
- Replace a few NPD (null pointer deference) flows and division by zero with an
  assertion, so that it doesn't fail the test. (since we set the server to use
  `exit` rather than `abort` on assertion).
- Fix quite a lot of flows in rdb.c that could have lead to memory leaks in
  RESTORE command (since it now responds with an error rather than panic)
- Add a DEBUG flag for SET-SKIP-CHECKSUM-VALIDATION so that the test don't need
  to bother with faking a valid checksum
- Remove a pile of code in serverLogObjectDebugInfo which is actually unsafe to
  run in the crash report (see comments in the code)
- fix a missing boundary check in lzf_decompress

test suite infra improvements:
- be able to run valgrind checks before the process terminates
- rotate log files when restarting servers
2020-12-06 14:54:34 +02:00
Kajaruban Surendran
188e7a5693 Configurable option for MOTD
Former-commit-id: 49a89d636ba698dbd0858d5059d3d6387c8c1fc7
2020-11-30 10:42:48 -05:00
Oran Agra
a5302a8c21 memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.

(cherry picked from commit 7481e513f0507d01381a87046d8d1366c718f94e)
2020-10-27 09:12:01 +02:00
Oran Agra
7481e513f0 memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.
2020-10-05 11:15:36 +03:00
John Sully
14daf6f909 Merge tag '6.0.8' into unstable
Former-commit-id: 4c7e4b91a6bb2034636856b608b8c386d07f5541
2020-09-30 19:47:55 +00:00
John Sully
cb6bc67a87 Merge branch 'unstable' into keydbpro
Former-commit-id: d1986d9fcdfd56f2c30ee95edef6d5260e0aa777
2020-09-24 22:02:30 +00:00
Yossi Gottlieb
3d111508a1 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.


Former-commit-id: d3834c50699bc4f31f381d6d03d4c1b022380895
2020-09-16 02:10:29 +00:00
Yossi Gottlieb
8fb8c23746 Tests: validate CONFIG REWRITE for all params. (#7764)
This is a catch-all test to confirm that that rewrite produces a valid
output for all parameters and that this process does not introduce
undesired configuration changes.

(cherry picked from commit 995f1fc53f7daf3d289d5d70d7b45cdd486dc6cc)
2020-09-10 14:09:00 +03:00
Yossi Gottlieb
995f1fc53f Tests: validate CONFIG REWRITE for all params. (#7764)
This is a catch-all test to confirm that that rewrite produces a valid
output for all parameters and that this process does not introduce
undesired configuration changes.
2020-09-09 15:43:11 +03:00
Yossi Gottlieb
74bac9610e Fix default/explicit "save" parameter loading. (#7767)
Save parameters should either be default or whatever specified in the
config file. This fixes an issue introduced in #7092 which causes
configuration file settings to be applied on top of the defaults.
2020-09-09 15:12:57 +03:00
John Sully
ff9df842d8 Implement use-fork config (fails with diskless repl)
Former-commit-id: f2d5c2bca22e9fd506db123c47b7f60cdded7e2c
2020-08-24 03:17:59 +00:00
Yossi Gottlieb
27d44fbf73 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.

(cherry picked from commit c611a836f630ecf358b5cfb0d3c5e21c9f0bc105)
2020-07-20 21:08:26 +03:00
John Sully
658ebaa454 Fix test failure
Former-commit-id: 05d84f5fc184676d32c4bb61ac5957f1787cd4f8
2020-07-14 04:34:40 +00:00
John Sully
d67265de66 Implement storage key cache, and writeback memory model
Former-commit-id: 732bd9c153459f1174475ad67de36c399ddbe359
2020-07-11 21:23:48 +00:00
Yossi Gottlieb
c611a836f6 TLS: Session caching configuration support. (#7420)
* TLS: Session caching configuration support.
* TLS: Remove redundant config initialization.
2020-07-10 11:33:47 +03:00
John Sully
c03e9a3e2c Merge commit 'e834acb91c89d083d0075e320dea6ad7188f2d0d' into unstable
Former-commit-id: 817a11ff110772893eda0675912bbb0cfc1fca74
2020-05-22 15:56:35 -04:00
zhenwei pi
1a632b6964 Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.

There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863

So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.

Examples of cpulist in redis.conf:
server_cpulist 0-7:2      means cpu affinity 0,2,4,6
bio_cpulist 1,3           means cpu affinity 1,3
aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11    means cpu affinity 1,10,11

Test on linux/freebsd, both work fine.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-08 10:37:35 +02:00
zhenwei pi
2c853869bf Support setcpuaffinity on linux/bsd
Currently, there are several types of threads/child processes of a
redis server. Sometimes we need deeply optimise the performance of
redis, so we would like to isolate threads/processes.

There were some discussion about cpu affinity cases in the issue:
https://github.com/antirez/redis/issues/2863

So implement cpu affinity setting by redis.conf in this patch, then
we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/
bgsave_cpulist by cpu list.

Examples of cpulist in redis.conf:
server_cpulist 0-7:2      means cpu affinity 0,2,4,6
bio_cpulist 1,3           means cpu affinity 1,3
aof_rewrite_cpulist 8-11  means cpu affinity 8,9,10,11
bgsave_cpulist 1,10-11    means cpu affinity 1,10,11

Test on linux/freebsd, both work fine.

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-05-02 21:19:47 +08:00
John Sully
14188ef92d Fix most tests (still some failures)
Former-commit-id: da83e841255487efe0e4b13d42b2dcc55a369838
2020-01-27 18:16:19 -05:00
Oran Agra
f866b31a74 config.c adjust config limits and mutable
- make lua-replicate-commands mutable (it never was, but i don't see why)
- make tcp-backlog immutable (fix a recent refactory mistake)
- increase the max limit of a few configs to match what they were before
the recent refactory
2019-12-29 15:46:31 +01:00
Oran Agra
f10051c5fd config.c adjust config limits and mutable
- make lua-replicate-commands mutable (it never was, but i don't see why)
- make tcp-backlog immutable (fix a recent refactory mistake)
- increase the max limit of a few configs to match what they were before
the recent refactory
2019-12-26 15:16:15 +02:00
Oran Agra
493cc493b6 Converting more configs to use generic infra, and moving defaults to config.c
Changes in behavior:
- Change server.stream_node_max_entries from int64_t to long long, so that it can be used by the generic infra
- standard error reply instead of "repl-backlog-size must be 1 or greater" and such
- tls-port and a few TLS booleans were readable (config get) even when USE_OPENSSL was off (now they aren't)
- syslog-enabled, syslog-ident, cluster-enabled, appendfilename, and supervised didn't have a get (now they do)
- pidfile was initialized to NULL in InitServerConfig but had CONFIG_DEFAULT_PID_FILE in rewriteConfig (so the real default was "", but rewrite would cause it to be set), fixed the rewrite.
- TLS config in server.h was uninitialized (if no tls config args were provided)

Adding test for sanity and coverage
2019-11-28 11:24:57 +02:00
zhaozhao.zz
b250f9d710 adjust qbuf to 26 in test case for client list 2018-08-14 00:57:22 +08:00
Sebastian Waisbrot
7be8687677 Fix race condition in unit/introspection
Make sure monitor is attached in one connection before issuing commands to be monitored in another one
2015-08-11 22:56:17 -07:00
Matt Stancliff
5fab7e5bf2 Remove trailing spaces from tests 2014-09-29 06:49:08 -04:00
antirez
3f59d66ff0 Test: avoid false positives in CLIENT SETNAME closed connection test. 2013-02-12 13:27:24 +01:00
antirez
eb113b97f9 Tests for CLIENT GETNAME/SETNAME. 2013-01-15 13:34:17 +01:00
antirez
06612a970f Tests for MONITOR. 2012-04-07 11:26:24 +02:00
antirez
8f49d6f257 New client info field added to CLIENT LIST output: multi, containing the length of the current pipeline. Test modified accordingly. 2012-04-07 11:14:52 +02:00
antirez
9704f5a005 CLIENT LIST test modified to reflect the new output. 2012-03-13 18:06:29 +01:00
antirez
d64a700b36 CLIENT LIST test fixed (there is a new omem field in the output) 2012-01-23 16:19:49 +01:00
antirez
73cb81d541 unit/introspection.tcl added 2011-12-19 10:21:50 +01:00