27332 Commits

Author SHA1 Message Date
valentinogeron
795c454db1
Stream: Inconsistency between master and replica some XREADGROUP case (#7526)
XREADGROUP auto-creates the consumer inside the consumer group the
first time it saw it.
When XREADGROUP is being used with NOACK option, the message will not
be added into the client's PEL and XGROUP SETID would be propagated.
When the replica gets the XGROUP SETID it will only update the last delivered
id of the group, but will not create the consumer.

So, in this commit XGROUP CREATECONSUMER is being added.
Command pattern: XGROUP CREATECONSUMER <key> <group> <consumer>.

When NOACK option is being used, createconsumer command would be
propagated as well.

In case of AOFREWRITE, consumer with an empty PEL would be saved with
XGROUP CREATECONSUMER whereas consumer with pending entries would be
saved with XCLAIM
2020-09-24 12:02:40 +03:00
bodong.ybd
b298950745 Refactor multi-key command get keys proc 2020-09-24 08:59:14 +03:00
bodong.ybd
b7ce583a5e Refactor multi-key command get keys proc 2020-09-24 08:59:14 +03:00
bodong.ybd
a84156ff15 Add ZINTER/ZUNION command
Syntax: ZINTER/ZUNION numkeys key [key ...] [WEIGHTS weight [weight ...]]
[AGGREGATE SUM|MIN|MAX] [WITHSCORES]

see #7624
2020-09-24 08:59:14 +03:00
bodong.ybd
e08bf16637 Add ZINTER/ZUNION command
Syntax: ZINTER/ZUNION numkeys key [key ...] [WEIGHTS weight [weight ...]]
[AGGREGATE SUM|MIN|MAX] [WITHSCORES]

see #7624
2020-09-24 08:59:14 +03:00
alexronke-channeladvisor
67704e67ab Add GT and LT options to ZADD for conditional score updates (#7818)
Co-authored-by: Alex Ronke <w.alex.ronke@gmail.com>
2020-09-23 21:56:16 +03:00
alexronke-channeladvisor
66a13ccbdf
Add GT and LT options to ZADD for conditional score updates (#7818)
Co-authored-by: Alex Ronke <w.alex.ronke@gmail.com>
2020-09-23 21:56:16 +03:00
Oran Agra
270fcb80bf fix recently broken TLS build error, and add coverage for CI (#7833) 2020-09-23 11:30:24 +03:00
Oran Agra
a735bf5c2a
fix recently broken TLS build error, and add coverage for CI (#7833) 2020-09-23 11:30:24 +03:00
David CARLIER
6bc28d99a3 Further NetBSD update and build fixes. (#7831)
mainly backtrace and register dump support.
2020-09-23 10:00:31 +03:00
David CARLIER
c3edaa7941
Further NetBSD update and build fixes. (#7831)
mainly backtrace and register dump support.
2020-09-23 10:00:31 +03:00
WuYunlong
00668f782f Fix redundancy use of semicolon in do-while macros in ziplist.c. (#7832)
this is very dangerous bug, but it looks like it didn't cause any harm.
2020-09-23 09:09:48 +03:00
WuYunlong
63cd4d4e20
Fix redundancy use of semicolon in do-while macros in ziplist.c. (#7832)
this is very dangerous bug, but it looks like it didn't cause any harm.
2020-09-23 09:09:48 +03:00
John Sully
c373c2b652 Support read only replicas attaching to active replicas (Bug #229)
Former-commit-id: 676644f2db1634f97dd557a6e321234914a5e5d3
2020-09-23 04:16:08 +00:00
John Sully
d0885f945e Support read only replicas attaching to active replicas (Bug #229)
Former-commit-id: 676644f2db1634f97dd557a6e321234914a5e5d3
2020-09-23 04:16:08 +00:00
John Sully
28fdf8d7d4 Active replicas memory leak
Former-commit-id: 22ac56d62d982ef512b05ecd55d9d51d05ea2958
2020-09-22 21:38:35 +00:00
John Sully
1321170cdc Active replicas memory leak
Former-commit-id: 22ac56d62d982ef512b05ecd55d9d51d05ea2958
2020-09-22 21:38:35 +00:00
yixiang
4e70e49d2b Fix connGetSocketError usage (#7811) 2020-09-22 12:53:36 +03:00
yixiang
b96c3595af
Fix connGetSocketError usage (#7811) 2020-09-22 12:53:36 +03:00
Oran Agra
78c80b3f8c RM_GetContextFlags - document missing flags (#7821) 2020-09-22 12:11:19 +03:00
Oran Agra
26ca04a8a2
RM_GetContextFlags - document missing flags (#7821) 2020-09-22 12:11:19 +03:00
Yossi Gottlieb
ecd86283ec Fix occasional hangs on replication reconnection. (#7830)
This happens only on diskless replicas when attempting to reconnect after 
failing to load an RDB file. It is more likely to occur with larger datasets.

After reconnection is initiated, replicationEmptyDbCallback() may get called 
and try to write to an unconnected socket. This triggered another issue where
the connection is put into an error state and the connect handler never gets
called. The problem is a regression introduced by commit cad93ed.
2020-09-22 11:38:52 +03:00
Yossi Gottlieb
1980f639b1
Fix occasional hangs on replication reconnection. (#7830)
This happens only on diskless replicas when attempting to reconnect after 
failing to load an RDB file. It is more likely to occur with larger datasets.

After reconnection is initiated, replicationEmptyDbCallback() may get called 
and try to write to an unconnected socket. This triggered another issue where
the connection is put into an error state and the connect handler never gets
called. The problem is a regression introduced by commit c17e597.
2020-09-22 11:38:52 +03:00
Ariel Shtul
b914d4fc48 Fix redis-check-rdb support for modules aux data (#7826)
redis-check-rdb was unable to parse rdb files containing module aux data.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-22 10:18:07 +03:00
Ariel Shtul
63a05dde46
Fix redis-check-rdb support for modules aux data (#7826)
redis-check-rdb was unable to parse rdb files containing module aux data.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-22 10:18:07 +03:00
Wang Yuan
b551c8fdb7 Kill disk-based fork child when all replicas drop and 'save' is not enabled (#7819)
When all replicas waiting for a bgsave get disconnected (possibly due to output buffer limit),
It may be good to kill the bgsave child. in diskless replication it already happens, but in
disk-based, the child may still serve some purpose (for persistence).

By killing the child, we prevent it from eating COW memory in vain, and we also allow a new child fork sooner for the next full synchronization or bgsave.
We do that only if rdb persistence wasn't enabled in the configuration.

Btw, now, rdbRemoveTempFile in killRDBChild won't block server, so we can killRDBChild safely.
2020-09-22 09:47:58 +03:00
Wang Yuan
1bb5794a1f
Kill disk-based fork child when all replicas drop and 'save' is not enabled (#7819)
When all replicas waiting for a bgsave get disconnected (possibly due to output buffer limit),
It may be good to kill the bgsave child. in diskless replication it already happens, but in
disk-based, the child may still serve some purpose (for persistence).

By killing the child, we prevent it from eating COW memory in vain, and we also allow a new child fork sooner for the next full synchronization or bgsave.
We do that only if rdb persistence wasn't enabled in the configuration.

Btw, now, rdbRemoveTempFile in killRDBChild won't block server, so we can killRDBChild safely.
2020-09-22 09:47:58 +03:00
Wen Hui
7934f163b4 refactor rewriteStreamObject code for adding missing streamIteratorStop call (#7829)
This commit adds streamIteratorStop call in rewriteStreamObject function in some of the return statement. Although currently this will not cause memory leak since stream id is only 16 bytes long.
2020-09-22 09:05:47 +03:00
Wen Hui
23b50bcccc
refactor rewriteStreamObject code for adding missing streamIteratorStop call (#7829)
This commit adds streamIteratorStop call in rewriteStreamObject function in some of the return statement. Although currently this will not cause memory leak since stream id is only 16 bytes long.
2020-09-22 09:05:47 +03:00
YoongHM
539712df55 Fix compilation warning in jemalloc's malloc_vsnprintf (#7789)
Change `val` to `unsigned char` before being tested.
The fix is identical to the one that's been made in upstream jemalloc.

warning is:
src/malloc_io.c: In function ‘malloc_vsnprintf’:
src/malloc_io.c:369:2: warning: case label value exceeds maximum value for type
  369 |  case '?' | 0x80:      \
      |  ^~~~
src/malloc_io.c:581:5: note: in expansion of macro ‘GET_ARG_NUMERIC’
  581 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
2020-09-21 17:04:34 +03:00
YoongHM
9216b96b41
Fix compilation warning in jemalloc's malloc_vsnprintf (#7789)
Change `val` to `unsigned char` before being tested.
The fix is identical to the one that's been made in upstream jemalloc.

warning is:
src/malloc_io.c: In function ‘malloc_vsnprintf’:
src/malloc_io.c:369:2: warning: case label value exceeds maximum value for type
  369 |  case '?' | 0x80:      \
      |  ^~~~
src/malloc_io.c:581:5: note: in expansion of macro ‘GET_ARG_NUMERIC’
  581 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
2020-09-21 17:04:34 +03:00
YoongHM
3359a3a658 Fix warning from jemalloc configure script (#7790)
jemalloc configure shows this:
    configure: WARNING: unrecognized options: --enable-cc-silence

The changelog of jemalloc 4.0 has:
  - Replace --enable-cc-silence with --disable-cc-silence to suppress spurious
    warnings by default.
2020-09-21 16:57:15 +03:00
YoongHM
d3faed875d
Fix warning from jemalloc configure script (#7790)
jemalloc configure shows this:
    configure: WARNING: unrecognized options: --enable-cc-silence

The changelog of jemalloc 4.0 has:
  - Replace --enable-cc-silence with --disable-cc-silence to suppress spurious
    warnings by default.
2020-09-21 16:57:15 +03:00
WuYunlong
6c6ab16e5a Make IO threads killable so that they can be canceled at any time.
This commit can be cherry picked to 6.0 only if we also cherry pick e9b6077.
2020-09-21 12:10:19 +03:00
WuYunlong
c37acb82aa Make IO threads killable so that they can be canceled at any time.
This commit can be cherry picked to 6.0 only if we also cherry pick f866023.
2020-09-21 12:10:19 +03:00
WuYunlong
d229162730 Make main thread killable so that it can be canceled at any time.
Refine comment of makeThreadKillable().

This commit can be backported to 5.0, only if we also backport cf8a6e3.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-21 12:10:19 +03:00
WuYunlong
647cac5bb4 Make main thread killable so that it can be canceled at any time.
Refine comment of makeThreadKillable().

This commit can be backported to 5.0, only if we also backport 8b70cb0.

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-21 12:10:19 +03:00
Oran Agra
0bc59b46aa Fix C11 detection in the makefile (#7822)
6918ebc introudced a makefile script that detects if the toolchain
supports c11, and it looked that it was passing on MacOS and fails on
Ubuntu, looks like Ubuntu's Dash was spawning a background process,
deleted foo.c before gcc tried to compile it.
2020-09-21 11:17:48 +03:00
Oran Agra
f11a0c8f30
Fix C11 detection in the makefile (#7822)
445a4b6 introudced a makefile script that detects if the toolchain
supports c11, and it looked that it was passing on MacOS and fails on
Ubuntu, looks like Ubuntu's Dash was spawning a background process,
deleted foo.c before gcc tried to compile it.
2020-09-21 11:17:48 +03:00
John Sully
18a3e0ba35 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 143f7f634db8adb1f5386d28832a17f7382b750b
2020-09-21 04:42:46 +00:00
John Sully
b243fa876f Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 143f7f634db8adb1f5386d28832a17f7382b750b
2020-09-21 04:42:46 +00:00
Dmitry Dorofeev
a830b2fdbf Fix to support FreeBSD pthread_getthreadid_np call
Former-commit-id: 955faef23a6c080f40896cb9d68391b5586812e9
2020-09-21 00:41:41 -04:00
Dmitry Dorofeev
823de49e83 Fix to support FreeBSD pthread_getthreadid_np call
Former-commit-id: 955faef23a6c080f40896cb9d68391b5586812e9
2020-09-21 00:41:41 -04:00
John Sully
b81dee212a Fix TSAN errors in fastlock
Former-commit-id: 2e159e6ea3fa8bb26c2f471f1493ab6593b85cff
2020-09-21 03:26:50 +00:00
John Sully
7b0a5d1e40 Fix TSAN errors in fastlock
Former-commit-id: 2e159e6ea3fa8bb26c2f471f1493ab6593b85cff
2020-09-21 03:26:50 +00:00
John Sully
3b66147997 Fix issue #222
Former-commit-id: 02b533fec3473f67377237f195004ec1d438d84a
2020-09-21 01:09:58 +00:00
John Sully
c3a0665f2a Fix issue #222
Former-commit-id: 02b533fec3473f67377237f195004ec1d438d84a
2020-09-21 01:09:58 +00:00
John Sully
d38583454c Make multimaster replication info statelessly paresable
Former-commit-id: de3099e6d880b7a544fd184e0f4c172f4d371c77
2020-09-20 23:57:06 +00:00
John Sully
b0379b9655 Make multimaster replication info statelessly paresable
Former-commit-id: de3099e6d880b7a544fd184e0f4c172f4d371c77
2020-09-20 23:57:06 +00:00
John Sully
17648d9705 Dramatically improve the performance of subkey expires
Former-commit-id: 368f67f42217c5fd2cfb3cb3643984917793e994
2020-09-20 23:30:21 +00:00