14176 Commits

Author SHA1 Message Date
Oran Agra
b1149a49b2 Fix Integer overflow issue with intsets (CVE-2021-32687)
The vulnerability involves changing the default set-max-intset-entries
configuration parameter to a very large value and constructing specially
crafted commands to manipulate sets
2021-10-04 13:59:40 +03:00
Maxime de Roucy
edbf9cf918 syslog: enforce syslog identity and facility as soon as possible
Former-commit-id: a9eaf0a130d3685c000d5f9cb500bb5f8cb8b263
2021-09-16 17:29:55 -04:00
Maxime de Roucy
b834e66e2d use LL_… log level instead of syslog LOG_… log level
Else there is an out of bound access in syslogLevelMap.

For example if we use `serverLog(LOG_INFO,…`, later in the code
it tries to access `syslogLevelMap[LOG_INFO]`.
LOG_INFO == 6 but syslogLevelMap only have 4 elements.


Former-commit-id: a1680fa612bdf5f521ea2c06b83994bf0797015a
2021-09-16 17:29:07 -04:00
benschermel
ae6e4a3f1d update deb packaging to build with systemd flag
Former-commit-id: d7bd4f7a1c8d6195b60ccf6417ebb77fba097398
2021-09-08 02:28:17 +00:00
benschermel
d80942e0e8 update pkg conf file
Former-commit-id: 0be40b9fd43ec4e7802e09fa627ad91b2e387707
2021-08-22 22:21:34 +00:00
MalavanEQAlpha
e0af0bcb95 Merge pull request #313 from MalavanEQAlpha/fixissue295
Resolve Issue #295 by speeding up keyIsExpired and setting timeout on subkey expires.

Former-commit-id: 9e5e6c2f86548b18ae27b4e1ac20c72517392b07
2021-08-18 17:34:18 -04:00
malavan
778142bb15 should use LLONG_MAX for INVALID_EXPIRE not LLONG_MIN and add better comments
Former-commit-id: 1ab19b445f00a9ccdab13bf6b96363d068e2fa02
2021-08-18 20:59:01 +00:00
VivekSainiEQ
09b89cf3f2 Include gperftools/tcmalloc.h instead of google/tcmalloc.h
Former-commit-id: ee32ed045cb94334773fc12d7b78a6a4ae29f184
2021-08-18 15:40:09 -04:00
John Sully
e9aadd35c4 Solo tests should work with loopn
Former-commit-id: 82ba449dd0f8e8c81d7391241a6e75ada1bb314c
2021-08-18 15:08:10 -04:00
malavan
1780a8918c Remove duplicate code
Former-commit-id: 9b7febb538f8a9f9aaa8e62732f7f85430b7fcd7
2021-08-16 22:55:46 +00:00
malavan
eabf119fde don't use flags that aren't supported by certain versions of OPENSSL
Former-commit-id: fae6bfb80bc66351e278f8c5c4ce91bdcef64afb
2021-08-16 22:55:46 +00:00
benschermel
00a51c5b60 add keydb-diagnostic-tool to packaging
Former-commit-id: 45bb6e801f5249f11edd93423c5fd0d3d92d53b2
2021-08-16 22:10:31 +00:00
christianEQ
fe50d680e1 install 32bit libraries in github action
Former-commit-id: b08aa26a765ddca4a398458ba6a3b91e0358aaf7
2021-08-13 11:05:12 -04:00
christianEQ
0b1212aaa1 github runner for 32bit build
Former-commit-id: 8dacbed556e858e1ccaa567373bb1466ddabe91e
2021-08-13 11:05:12 -04:00
christianEQ
314cc7d6de changed keys counter variable to unsigned to fix signedness comparison warning
Former-commit-id: b44df43fcbc6392796a9a2ec92b3aab1f43f5503
2021-08-13 11:05:12 -04:00
christianEQ
e0ff0ee45c %zu for printing type size_t
Former-commit-id: 84af829053eaf70439aaf0b7fdebfe2d9d2f3b6f
2021-08-13 11:05:12 -04:00
christianEQ
c48ae9ace7 fixed 32bit compatibility in Makefile
Former-commit-id: f5ee64f9aaeb3f3773de2435191bcff0855154e4
2021-08-13 11:05:12 -04:00
christianEQ
8bccf9bf12 fixed overly strict assert for 32bit
Former-commit-id: ce4891b33d65038bb0543eb2d3526c5310fee59b
2021-08-13 11:05:12 -04:00
MalavanEQAlpha
0750f0f2e6 Merge pull request #318 from VivekSainiEQ/redisearch_fixes
Initialized serverTL in more places in module.cpp

Former-commit-id: ffa46b8f0bd84c3f854a76e375be2108705fa8cc
2021-08-11 15:47:30 -04:00
Vivek Saini
c5b7ff7492 Merge pull request #296 from VivekSainiEQ/systemd_fix
Fixed bug where make USE_SYSTEMD=yes would not pass in correct flags

Former-commit-id: bf05c261d2689683ce89da2d062e1819ed6d772c
2021-08-11 15:39:50 -04:00
VivekSainiEQ
23c0c8171e Merge remote-tracking branch 'upstream/unstable' into redisearch_fixes
Former-commit-id: 952ae9fbe38b99593ec175da6718a5d7fae49837
2021-08-11 19:28:04 +00:00
VivekSainiEQ
2598493a62 Prevent invalid mvcc timestamps from causing critical errors
Former-commit-id: 6f2dbb00119b1d0a1f5a2543d2c6af05f83ef5de
2021-08-11 15:06:14 -04:00
Paweł Sacawa
eb5a1a5e19 Add: zsh completions with descriptions
This commit add zsh completions for the keydb `client`. They have contextual host completion and full argument descriptions.
Vendor-distributed completions for zsh should end up in `/usr/share/zsh/vendor-completions`, but unfortunatly I'm not familiar with the packaging method for *.deb archives, so these completions will need to be moved to the appropriate directory.


Former-commit-id: f77980fce87f22b59677e374e0d5c113775cc08a
2021-08-11 15:05:11 -04:00
MalavanEQAlpha
442f1075d5 Merge pull request #340 from EQ-Alpha/update_keydb.conf
update keydb.conf with new configs from redis unstable

Former-commit-id: d90c1e6eae9b0bb07fb67512e87339d948db5f72
2021-08-11 15:04:22 -04:00
MalavanEQAlpha
dda92b67c8 Merge pull request #341 from EQ-Alpha/pkg_config
update packaging config files

Former-commit-id: d987f463ebeb19ed19869ebbed4e8681bcb6dbcb
2021-08-11 15:01:37 -04:00
benschermel
3745daf938 add script to compare all active config parameters of any 2 config files
Former-commit-id: 89beaf2460e969c3ee18e03fe539de5833dba50a
2021-08-10 20:10:28 -04:00
benschermel
a7b3ab45f2 update packaging config files
Former-commit-id: 0c5d12b91455db4acad57665db8aac79180b42ab
2021-08-10 20:08:39 -04:00
malavan
6879564b48 update keydb.conf with new configs from redis unstable
Former-commit-id: 32ca2e8a7335ee7bc4889a9369c97177f34739ee
2021-08-10 21:34:52 +00:00
Christian Legge
9e562f1fe5 Add REPLPING command for use during replication (#329)
* added replping command for initiating replication

* backwards compatibility for replping (retry if not recognized)

* don't allow ping during loading (load balancer fix)

* changed replping warning to notice

Former-commit-id: d7f6bc16145206e96ffeb9941398d564c3dba6a9
2021-07-29 15:50:30 -04:00
MalavanEQAlpha
31e2a22b6f Merge pull request #328 from EQ-Alpha/backtrace_for_alpine
Backtrace for alpine using libunwind

Former-commit-id: 83d878f11183d8cd54fb34bceb7f839f73599ab1
2021-07-28 17:17:06 -04:00
malavan
c3a6be36e3 add some comments
Former-commit-id: 0fc6441c9608f0650003a7ab29dd704166345272
2021-07-28 20:32:18 +00:00
Oran Agra
db09f6eb2e Redis 6.2.5 2021-07-21 21:06:49 +03:00
Huang Zhw
835d15b536 On 32 bit platform, the bit position of GETBIT/SETBIT/BITFIELD/BITCOUNT,BITPOS may overflow (see CVE-2021-32761) (#9191)
GETBIT, SETBIT may access wrong address because of wrap.
BITCOUNT and BITPOS may return wrapped results.
BITFIELD may access the wrong address but also allocate insufficient memory and segfault (see CVE-2021-32761).

This commit uses `uint64_t` or `long long` instead of `size_t`.
related https://github.com/redis/redis/pull/8096

At 32bit platform:
> setbit bit 4294967295 1
(integer) 0
> config set proto-max-bulk-len 536870913
OK
> append bit "\xFF"
(integer) 536870913
> getbit bit 4294967296
(integer) 0

When the bit index is larger than 4294967295, size_t can't hold bit index. In the past,  `proto-max-bulk-len` is limit to 536870912, so there is no problem.

After this commit, bit position is stored in `uint64_t` or `long long`. So when `proto-max-bulk-len > 536870912`, 32bit platforms can still be correct.

For 64bit platform, this problem still exists. The major reason is bit pos 8 times of byte pos. When proto-max-bulk-len is very larger, bit pos may overflow.
But at 64bit platform, we don't have so long string. So this bug may never happen.

Additionally this commit add a test cost `512MB` memory which is tag as `large-memory`. Make freebsd ci and valgrind ci ignore this test.

(cherry picked from commit 71d452876ebf8456afaadd6b3c27988abadd1148)
2021-07-21 21:06:49 +03:00
Oran Agra
bae0512c8a longer timeout in replication test (#8963)
the test normally passes. but we saw one failure in a valgrind run in github actions

(cherry picked from commit 8458baf6a96fa6c6050bac24160f82d32a0b9ed4)
2021-07-21 21:06:49 +03:00
Huang Zhw
8d6134952a Remove testmodule in src/modules/Makefile. (#9250)
src/modules make failed. As in #3718 testmodule.c was removed. But the makefile was not updated

(cherry picked from commit d54c9086c267d20bb6981f5a60f589e93b662d62)
2021-07-21 21:06:49 +03:00
Oran Agra
1d7c0e5949 Fix failing basics moduleapi test on 32bit CI (#9140)
(cherry picked from commit 5ffdbae1f64bb66b6e2470779540fb1051dcbff1)
2021-07-21 21:06:49 +03:00
Oran Agra
37b0f3617d Adjustments to recent RM_StringTruncate fix (#3718) (#9125)
- Introduce a new sdssubstr api as a building block for sdsrange.
  The API of sdsrange is many times hard to work with and also has
  corner case that cause bugs. sdsrange is easy to work with and also
  simplifies the implementation of sdsrange.
- Revert the fix to RM_StringTruncate and just use sdssubstr instead of
  sdsrange.
- Solve valgrind warnings from the new tests introduced by the previous
  PR.

(cherry picked from commit ae418eca24ba53a7dca07b0e7065f856e625469b)
2021-07-21 21:06:49 +03:00
Huang Zhw
6866117194 Fix missing separator in module info line (usedby and using lists) (#9241)
Fix module info genModulesInfoStringRenderModulesList lack separator when there's more than one module in the list.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 1895e134a77efd789b1a6daee76a6ba5ec90e516)
2021-07-21 21:06:49 +03:00
Binbin
b622537199 SMOVE only notify dstset when the addition is successful. (#9244)
in case dest key already contains the member, the dest key isn't modified, so the command shouldn't invalidate watch.

(cherry picked from commit 11dc4e59b365d6cd8699604d7d1c1025b6bb6259)
2021-07-21 21:06:49 +03:00
qetu3790
355b1b6a57 Set TCP keepalive on inbound clusterbus connections (#9230)
Set TCP keepalive on inbound clusterbus connections to prevent memory leak

(cherry picked from commit f03af47a34ec672a7d9b18150a5be3a83681c19b)
2021-07-21 21:06:49 +03:00
Yossi Gottlieb
d6f4273241 Fix compatibility with OpenSSL 1.1.0. (#9233)
(cherry picked from commit 277e4dc2032356c7712b539e89f7e9154e0a1a86)
2021-07-21 21:06:49 +03:00
Oran Agra
de1b19ea88 fix valgrind issues with recently added test in modules/blockonbackground (#9192)
fixes test issue introduced in #9167

1. invalid reads due to accessing non-retained string (passed as unblock context).
2. leaking module blocked client context, see #6922 for info.

(cherry picked from commit a8518cce951629eaccde40fd0e51b36a5dc6321c)
2021-07-21 21:06:49 +03:00
Yossi Gottlieb
79fa5618f1 Fix CLIENT UNBLOCK crashing modules. (#9167)
Modules that use background threads with thread safe contexts are likely
to use RM_BlockClient() without a timeout function, because they do not
set up a timeout.

Before this commit, `CLIENT UNBLOCK` would result with a crash as the
`NULL` timeout callback is called. Beyond just crashing, this is also
logically wrong as it may throw the module into an unexpected client
state.

This commits makes `CLIENT UNBLOCK` on such clients behave the same as
any other client that is not in a blocked state and therefore cannot be
unblocked.

(cherry picked from commit aa139e2f02292d668370afde8c91575363c2d611)
2021-07-21 21:06:49 +03:00
Huang Zhw
91bf2ab86d redis-cli cluster import command may issue wrong MIGRATE command. (#8945)
In clusterManagerCommandImport strcat was used to concat COPY and
REPLACE, the space maybe not enough.
If we use --cluster-replace but not --cluster-copy, the MIGRATE
command contained COPY instead of REPLACE.

(cherry picked from commit a049f6295a28a20b11eff89083e91dab0738413b)
2021-07-21 21:06:49 +03:00
Binbin
88655019cc Fix accidental deletion of sinterstore command when we meet wrong type error. (#9032)
SINTERSTORE would have deleted the dest key right away,
even when later on it is bound to fail on an (WRONGTYPE) error.

With this change it first picks up all the input keys, and only later
delete the dest key if one is empty.

Also add more tests for some commands.
Mainly focus on
- `wrong type error`:
	expand test case (base on sinter bug) in non-store variant
	add tests for store variant (although it exists in non-store variant, i think it would be better to have same tests)
- the dstkey result when we meet `non-exist key (empty set)` in *store

sdiff:
- improve test case about wrong type error (the one we found in sinter, although it is safe in sdiff)
- add test about using non-exist key (treat it like an empty set)
sdiffstore:
- according to sdiff test case, also add some tests about `wrong type error` and `non-exist key`
- the different is that in sdiffstore, we will consider the `dstkey` result

sunion/sunionstore add more tests (same as above)

sinter/sinterstore also same as above ...

(cherry picked from commit b8a5da80c49501773f8778aaf5cbf595cef615e4)
2021-07-21 21:06:49 +03:00
Jason Elbaum
fad44611dc Change return value type for ZPOPMAX/MIN in RESP3 (#8981)
When using RESP3, ZPOPMAX/ZPOPMIN should return nested arrays for consistency
with other commands (e.g. ZRANGE).

We do that only when COUNT argument is present (similarly to how LPOP behaves).
for reasoning see https://github.com/redis/redis/issues/8824#issuecomment-855427955

This is a breaking change only when RESP3 is used, and COUNT argument is present!

(cherry picked from commit 7f342020dcbdf9abe754d6b666efdeded7063870)
2021-07-21 21:06:49 +03:00
Mikhail Fesenko
8884971223 Direct redis-cli repl prints to stderr, because --rdb can print to stdout. fflush stdout after responses (#9136)
1. redis-cli can output --rdb data to stdout
   but redis-cli also write some messages to stdout which will mess up the rdb.

2. Make redis-cli flush stdout when printing a reply
  This was needed in order to fix a hung in redis-cli test that uses
  --replica.
   Note that printf does flush when there's a newline, but fwrite does not.

3. fix the redis-cli --replica test which used to pass previously
   because it didn't really care what it read, and because redis-cli
   used printf to print these other things to stdout.

4. improve redis-cli --replica test to run with both diskless and disk-based.

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Viktor Söderqvist <viktor@zuiderkwast.se>
(cherry picked from commit 1eb4baa5b8e76adc337ae9fab49acc2585a0cdd0)
2021-07-21 21:06:49 +03:00
Rob Snyder
07e1248686 Fix ziplist length updates on bigendian platforms (#2080)
Adds call to intrev16ifbe to ensure ZIPLIST_LENGTH is compared correctly

(cherry picked from commit eaa52719a355c4467d0383c1c9f5184c9c14fe5a)
2021-07-21 21:06:49 +03:00
Oran Agra
6cd84b64f0 Test infra, handle RESP3 attributes and big-numbers and bools (#9235)
- promote the code in DEBUG PROTOCOL to addReplyBigNum
- DEBUG PROTOCOL ATTRIB skips the attribute when client is RESP2
- networking.c addReply for push and attributes generate assertion when
  called on a RESP2 client, anything else would produce a broken
  protocol that clients can't handle.

(cherry picked from commit 6a5bac309e868deef749c36949723b415de2496f)
2021-07-21 21:06:49 +03:00
Binbin
c6b3966d02 hrandfield and zrandmember with count should return emptyarray when key does not exist. (#9178)
due to a copy-paste bug, it used to reply with null response rather than empty array.
this commit includes new tests that are looking at the RESP response directly in
order to be able to tell the difference between them.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit a418a2d3fc0250c094802d7e8ea64d96eedfda07)
2021-07-21 21:06:49 +03:00