27350 Commits

Author SHA1 Message Date
Madelyn Olson
2192a91d62 Made crc64 test consistent 2020-04-24 17:05:52 -07:00
Madelyn Olson
0978a60b3b Implemented CRC64 based on slice by 4 2020-04-24 17:00:03 -07:00
Madelyn Olson
486e45ffaf Implemented CRC64 based on slice by 4 2020-04-24 17:00:03 -07:00
antirez
75cf725568 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-04-24 16:59:56 +02:00
antirez
022f09447b Merge branch 'unstable' of github.com:/antirez/redis into unstable 2020-04-24 16:59:56 +02:00
antirez
ae3cf7911a LCS -> STRALGO LCS.
STRALGO should be a container for mostly read-only string
algorithms in Redis. The algorithms should have two main
characteristics:

1. They should be non trivial to compute, and often not part of
programming language standard libraries.
2. They should be fast enough that it is a good idea to have optimized C
implementations.

Next thing I would love to see? A small strings compression algorithm.
2020-04-24 16:54:32 +02:00
antirez
8a7f255cd0 LCS -> STRALGO LCS.
STRALGO should be a container for mostly read-only string
algorithms in Redis. The algorithms should have two main
characteristics:

1. They should be non trivial to compute, and often not part of
programming language standard libraries.
2. They should be fast enough that it is a good idea to have optimized C
implementations.

Next thing I would love to see? A small strings compression algorithm.
2020-04-24 16:54:32 +02:00
antirez
262da0ba78 LCS -> STRALGO LCS.
STRALGO should be a container for mostly read-only string
algorithms in Redis. The algorithms should have two main
characteristics:

1. They should be non trivial to compute, and often not part of
programming language standard libraries.
2. They should be fast enough that it is a good idea to have optimized C
implementations.

Next thing I would love to see? A small strings compression algorithm.
2020-04-24 16:49:27 +02:00
antirez
3722f89f49 LCS -> STRALGO LCS.
STRALGO should be a container for mostly read-only string
algorithms in Redis. The algorithms should have two main
characteristics:

1. They should be non trivial to compute, and often not part of
programming language standard libraries.
2. They should be fast enough that it is a good idea to have optimized C
implementations.

Next thing I would love to see? A small strings compression algorithm.
2020-04-24 16:49:27 +02:00
Oran Agra
4ed5b7cb74 optimize memory usage of deferred replies
When deffered reply is added the previous reply node cannot be used so
all the extra space we allocated in it is wasted. in case someone uses
deffered replies in a loop, each time adding a small reply, each of
these reply nodes (the small string reply) would have consumed a 16k
block.
now when we add anther diferred reply node, we trim the unused portion
of the previous reply block.

see #7123
2020-04-24 17:20:28 +03:00
Oran Agra
fb732f7a94 optimize memory usage of deferred replies
When deffered reply is added the previous reply node cannot be used so
all the extra space we allocated in it is wasted. in case someone uses
deffered replies in a loop, each time adding a small reply, each of
these reply nodes (the small string reply) would have consumed a 16k
block.
now when we add anther diferred reply node, we trim the unused portion
of the previous reply block.

see #7123
2020-04-24 17:20:28 +03:00
antirez
e772b55a9c Also use propagate() in streamPropagateGroupID(). 2020-04-24 10:15:04 +02:00
antirez
373ae6061a Also use propagate() in streamPropagateGroupID(). 2020-04-24 10:15:04 +02:00
yanhui13
e0add7e0f1 add tcl test for cluster slots 2020-04-24 10:15:04 +02:00
yanhui13
f03f1fad67 add tcl test for cluster slots 2020-04-24 10:15:04 +02:00
yanhui13
782d9f2ff9 optimize the output of cluster slots 2020-04-24 10:15:04 +02:00
yanhui13
374ffdf1c1 optimize the output of cluster slots 2020-04-24 10:15:04 +02:00
antirez
6d3bd2ed5a Minor aesthetic changes to #7135. 2020-04-24 10:15:04 +02:00
antirez
4db38d2efd Minor aesthetic changes to #7135. 2020-04-24 10:15:04 +02:00
Valentino Geron
a2a5b1d6ae XREADGROUP with NOACK should propagate only one XGROUP SETID command 2020-04-24 10:15:04 +02:00
Valentino Geron
f0a261448c XREADGROUP with NOACK should propagate only one XGROUP SETID command 2020-04-24 10:15:04 +02:00
antirez
408d4fb35d ACL: re-enable command execution of disabled users.
After all I changed idea again: enabled/disabled should have a more
clear meaning, and it only means: you can't authenticate with such user
with new connections, however old connections continue to work as
expected.
2020-04-24 10:15:04 +02:00
antirez
fbdef6a9bd ACL: re-enable command execution of disabled users.
After all I changed idea again: enabled/disabled should have a more
clear meaning, and it only means: you can't authenticate with such user
with new connections, however old connections continue to work as
expected.
2020-04-24 10:15:04 +02:00
antirez
76aa8a43ab getRandomBytes(): use HMAC-SHA256.
Now that we have an interface to use this API directly, via ACL GENPASS,
we are no longer sure what people could do with it. So why don't make it
a strong primitive exported by Redis in order to create unique IDs and
so forth?

The implementation was tested against the test vectors that can
be found in RFC4231.
2020-04-24 10:14:48 +02:00
antirez
05a41da75b getRandomBytes(): use HMAC-SHA256.
Now that we have an interface to use this API directly, via ACL GENPASS,
we are no longer sure what people could do with it. So why don't make it
a strong primitive exported by Redis in order to create unique IDs and
so forth?

The implementation was tested against the test vectors that can
be found in RFC4231.
2020-04-24 10:14:48 +02:00
antirez
32c6699847 ACL GENPASS: take number of bits as argument. 2020-04-24 10:14:48 +02:00
antirez
345c3768de ACL GENPASS: take number of bits as argument. 2020-04-24 10:14:48 +02:00
antirez
9651156a7b ACL GENPASS: emit 256 bits instead of 128. 2020-04-24 10:14:48 +02:00
antirez
639c8a1d91 ACL GENPASS: emit 256 bits instead of 128. 2020-04-24 10:14:48 +02:00
antirez
de38fa2b65 ACL: deny commands execution of disabled users. 2020-04-24 10:14:48 +02:00
antirez
321acea038 ACL: deny commands execution of disabled users. 2020-04-24 10:14:48 +02:00
Theo Buehler
6a8f6ac1a3 TLS: Fix build with SSL_OP_NO_CLIENT_RENEGOTIATION
There is no ssl in this scope, so the build breaks.
All the other options are set directly on the ctx.
2020-04-24 10:14:48 +02:00
Theo Buehler
b0920e6e86 TLS: Fix build with SSL_OP_NO_CLIENT_RENEGOTIATION
There is no ssl in this scope, so the build breaks.
All the other options are set directly on the ctx.
2020-04-24 10:14:48 +02:00
Yossi Gottlieb
48ff17e1de TLS: Fix build on older verisons of OpenSSL. 2020-04-24 10:14:48 +02:00
Yossi Gottlieb
149b658b56 TLS: Fix build on older verisons of OpenSSL. 2020-04-24 10:14:48 +02:00
antirez
6666d30f6a Tracking: test expired keys notifications. 2020-04-24 10:14:48 +02:00
antirez
06917e581c Tracking: test expired keys notifications. 2020-04-24 10:14:48 +02:00
antirez
1b28c0cbab Tracking: NOLOOP tests. 2020-04-24 10:14:48 +02:00
antirez
e434b2ce4f Tracking: NOLOOP tests. 2020-04-24 10:14:48 +02:00
antirez
a05f6b064b Tracking: signal key as modified when evicting. 2020-04-24 10:14:48 +02:00
antirez
f3a1728873 Tracking: signal key as modified when evicting. 2020-04-24 10:14:48 +02:00
antirez
249d933215 Tracking: NOLOOP further implementation and fixes. 2020-04-24 10:14:48 +02:00
antirez
e63bb7ec8c Tracking: NOLOOP further implementation and fixes. 2020-04-24 10:14:48 +02:00
antirez
f066273907 Tracking: NOLOOP internals implementation. 2020-04-24 10:14:48 +02:00
antirez
6791ff0525 Tracking: NOLOOP internals implementation. 2020-04-24 10:14:48 +02:00
antirez
14758cdc63 Implement redis_set_thread_title for MacOS.
Strange enough, pthread_setname_np() produces a warning for not defined
function even if pthread is included. Moreover the MacOS documentation
claims the return value for the function is void, but actually is int.

Related to #7089.
2020-04-24 10:14:48 +02:00
antirez
725b8cc680 Implement redis_set_thread_title for MacOS.
Strange enough, pthread_setname_np() produces a warning for not defined
function even if pthread is included. Moreover the MacOS documentation
claims the return value for the function is void, but actually is int.

Related to #7089.
2020-04-24 10:14:48 +02:00
zhenwei pi
ff889d2c8a Threaded IO: set thread name for redis-server
Set thread name for each thread of redis-server, this helps us to
monitor the utilization and optimise the performance.

And suggested-by Salvatore, implement this feature for multi
platforms. Currently support linux and bsd, ignore other OS.

An exmaple on Linux:
 # top -d 5 -p `pidof redis-server ` -H

    PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
3682671 root      20   0  227744   8248   3836 R 99.2  0.0   0:19.53 redis-server
3682677 root      20   0  227744   8248   3836 S 26.4  0.0   0:04.15 io_thd_3
3682675 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.98 io_thd_1
3682676 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.97 io_thd_2
3682672 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_close_file
3682673 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_aof_fsync
3682674 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 bio_lazy_free
3682678 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682682 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682683 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682684 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682685 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682687 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd

Another exmaple on FreeBSD-12.1:
  PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
 5212 root        100    0    48M  7280K CPU2     2   0:26  99.52% redis-server{redis-server}
 5212 root         38    0    48M  7280K umtxn    4   0:06  26.94% redis-server{io_thd_3}
 5212 root         36    0    48M  7280K umtxn    6   0:06  26.84% redis-server{io_thd_1}
 5212 root         39    0    48M  7280K umtxn    1   0:06  25.30% redis-server{io_thd_2}
 5212 root         20    0    48M  7280K uwait    3   0:00   0.00% redis-server{redis-server}
 5212 root         21    0    48M  7280K uwait    2   0:00   0.00% redis-server{bio_close_file}
 5212 root         21    0    48M  7280K uwait    3   0:00   0.00% redis-server{bio_aof_fsync}
 5212 root         21    0    48M  7280K uwait    0   0:00   0.00% redis-server{bio_lazy_free}

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-04-24 10:14:48 +02:00
zhenwei pi
3575b8706d Threaded IO: set thread name for redis-server
Set thread name for each thread of redis-server, this helps us to
monitor the utilization and optimise the performance.

And suggested-by Salvatore, implement this feature for multi
platforms. Currently support linux and bsd, ignore other OS.

An exmaple on Linux:
 # top -d 5 -p `pidof redis-server ` -H

    PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
3682671 root      20   0  227744   8248   3836 R 99.2  0.0   0:19.53 redis-server
3682677 root      20   0  227744   8248   3836 S 26.4  0.0   0:04.15 io_thd_3
3682675 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.98 io_thd_1
3682676 root      20   0  227744   8248   3836 S 23.6  0.0   0:03.97 io_thd_2
3682672 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_close_file
3682673 root      20   0  227744   8248   3836 S  0.2  0.0   0:00.02 bio_aof_fsync
3682674 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 bio_lazy_free
3682678 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682682 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682683 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682684 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682685 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd
3682687 root      20   0  227744   8248   3836 S  0.0  0.0   0:00.00 jemalloc_bg_thd

Another exmaple on FreeBSD-12.1:
  PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
 5212 root        100    0    48M  7280K CPU2     2   0:26  99.52% redis-server{redis-server}
 5212 root         38    0    48M  7280K umtxn    4   0:06  26.94% redis-server{io_thd_3}
 5212 root         36    0    48M  7280K umtxn    6   0:06  26.84% redis-server{io_thd_1}
 5212 root         39    0    48M  7280K umtxn    1   0:06  25.30% redis-server{io_thd_2}
 5212 root         20    0    48M  7280K uwait    3   0:00   0.00% redis-server{redis-server}
 5212 root         21    0    48M  7280K uwait    2   0:00   0.00% redis-server{bio_close_file}
 5212 root         21    0    48M  7280K uwait    3   0:00   0.00% redis-server{bio_aof_fsync}
 5212 root         21    0    48M  7280K uwait    0   0:00   0.00% redis-server{bio_lazy_free}

Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
2020-04-24 10:14:48 +02:00
antirez
ac512f2928 Sentinel: small refactoring of sentinelCollectTerminatedScripts().
Related to #7113.
2020-04-24 10:14:48 +02:00