12866 Commits

Author SHA1 Message Date
John Sully
ecad9741bc code coverage reports should test multithreaded
Former-commit-id: 5d016fdb6b11404d0eddbab651ed17180526a991
2020-11-30 22:59:00 +00:00
VivekSainiEQ
52561f1729 Added module data type load/save tests
Former-commit-id: cae9924fd9eefcd88cef1c964f0bc8bce7dd4242
2020-11-30 17:47:04 -05:00
VivekSainiEQ
3d15a7b65a Removed use of module datatypes, now should work if tests/modules is not built
Former-commit-id: 089f28e8ba91ed2b875048efa0b28ceb32010541
2020-11-30 17:47:04 -05:00
VivekSainiEQ
54b36868ba Added tests for saving various data types to disk and loading them back, and for loading data types from redis to maintain compatibility
Former-commit-id: dcb44d3a09a4021f05079bedbac690e33ec7f39e
2020-11-30 17:47:04 -05:00
VivekSainiEQ
6c16fe69e7 Added remainder of runtest scripts to make lcov command. Code coverage goes from 57.9% to 72.1%
Former-commit-id: f99cba0b92d921b1a2d350f2c7545138485c2a74
2020-11-30 17:47:04 -05:00
VivekSainiEQ
6632bafced Added KeyDB.info to .gitignore file
Former-commit-id: e10dc0dace6b152418aad2bd3ee212a471bca9e8
2020-11-30 17:47:04 -05:00
Kajaruban Surendran
fd03008bbf Configurable option for MOTD
Former-commit-id: 49a89d636ba698dbd0858d5059d3d6387c8c1fc7
2020-11-30 10:42:48 -05:00
VivekSainiEQ
f99824faf4 Added references to KeyDB.info instead of redis.info
Former-commit-id: b94f91b9b25b5476d62bdbe2f054cc61691de9f5
2020-11-25 18:49:20 -05:00
VivekSainiEQ
7ad88d0bac Properly escaped $ in lcov command
Former-commit-id: 06a17c8511cfa8e07962401cc01045878525dc27
2020-11-25 18:49:20 -05:00
VivekSainiEQ
2ab3385733 Updated 'make lcov' command to exclude external dependancies and generate final code coverage percentage
Former-commit-id: eebd70aaa6bec0362ea90148174263fa6ffcc169
2020-11-25 18:49:20 -05:00
VivekSainiEQ
86ca4300c8 Added test for issue #257
Former-commit-id: 779c775df9e8be36064689404e3ff51bead8a364
2020-11-25 17:21:37 -05:00
VivekSainiEQ
47a18f5131 Made NULL pointer check explicit, issue #257
Former-commit-id: 1d4d2925c2a11bb7fdffe92c17a315c2df011a54
2020-11-25 17:21:37 -05:00
VivekSainiEQ
d4a2ccd50b Added NULL check to function objFromAllocPtr, now NULL input will map to NULL output even if active replica is enabled
Former-commit-id: b67db7c2d5396892cc0c91c3c822a5b9a547badf
2020-11-25 17:21:37 -05:00
John Sully
3d75ddeb87 Merge branch 'unstable' of https://github.com/JohnSully/KeyDB into unstable
Former-commit-id: 18953b2998e5527260d161ce882b55a9684efab5
2020-11-23 02:01:54 +00:00
John Sully
c179c98870 Fix issue where active replication doesn't replicate RDB data
Former-commit-id: 527b7eb0742567302e0343e3acbed9814c0cbb95
2020-11-23 02:01:40 +00:00
Kajaruban Surendran
3ee8edc8b0 fix for the server crash when the maxclients increased via config set
Former-commit-id: 34a97c447f8cb5753e0747b435c7ce1a5d381c47
2020-11-20 15:20:46 -05:00
John Sully
e8753d1b4b Blocking clients should not crash if an active replica loads a remote RDB with a key in the blocklist
Former-commit-id: 1c525e20b10e0a47af687a0d46faf75229a1cbf5
2020-11-19 23:28:01 +00:00
John Sully
8d8f2a3b9f Merge branch 'RELEASE_6' into keydbpro
Former-commit-id: 2ee33cb050b26cbe34f0b390b8807cdc1b908ded
2020-11-19 20:38:47 +00:00
John Sully
4694161550 Ensure multithreaded tests pass
Former-commit-id: 0bcce502859647529b8835a4a101b64ddf0d625c
2020-11-13 21:08:10 +00:00
John Sully
9120d36bfc Improve replication performance in multithreaded scenarios
Former-commit-id: 96e0b2a2b19df220975e61131cbc535b0c34a828
2020-11-13 21:08:10 +00:00
John Sully
da2a3dbf99 Add ability to compile KeyDB without MOTD and curl
Former-commit-id: f3794d2c420423ca7dcc51688c478f8ada290786
2020-11-13 20:51:30 +00:00
John Sully
5de36840ad Add missing file
Former-commit-id: 97fd73692363e05b80e0c0e84b7b2b8fe7f760b8
2020-11-11 03:29:38 +00:00
John Sully
e085772d01 Second implementation of nested hashes
Former-commit-id: ba950a3d1f5708ed986f9b348eafcace6a7c22b9
2020-11-06 19:24:48 +00:00
John Sully
7db922f44b Additional test reliability fixes
Former-commit-id: dad5a902d394719ba722e487879fc283ca148786
2020-10-27 07:25:43 +00:00
Oran Agra
25214bd7dc Redis 6.0.9. 2020-10-27 09:12:01 +02:00
Yossi Gottlieb
3cc673bdd8 Fix wrong zmalloc_size() assumption. (#7963)
When using a system with no malloc_usable_size(), zmalloc_size() assumed
that the heap allocator always returns blocks that are long-padded.

This may not always be the case, and will result with zmalloc_size()
returning a size that is bigger than allocated. At least in one case
this leads to out of bound write, process crash and a potential security
vulnerability.

Effectively this does not affect the vast majority of users, who use
jemalloc or glibc.

This problem along with a (different) fix was reported by Drew DeVault.

(cherry picked from commit 9824fe3e392caa04dc1b4071886e9ac402dd6d95)
2020-10-27 09:12:01 +02:00
Oran Agra
de0919cd62 Attempt to fix sporadic test failures due to wait_for_log_messages (#7955)
The tests sometimes fail to find a log message.
Recently i added a print that shows the log files that are searched
and it shows that the message was in deed there.
The only reason i can't think of for this seach to fail, is we we
happened to read an incomplete line, which didn't match our pattern and
then on the next iteration we would continue reading from the line after
it.

The fix is to always re-evaluation the previous line.

(cherry picked from commit 4e2e5be201439cae4c0a03cfc8b6a60be4bff625)
2020-10-27 09:12:01 +02:00
David CARLIER
5aa0fef2ce cpu affinity: DragonFlyBSD support (#7956)
(cherry picked from commit 27f4c212f377dfbe1a60fb96635ae16bc405b959)
2020-10-27 09:12:01 +02:00
Zach Fewtrell
3988e32f41 fix invalid 'failover' identifier in cluster slave selection test (#7942)
(cherry picked from commit ebfa7699252b858af57cdd5388c5bd6d7439a021)
2020-10-27 09:12:01 +02:00
WuYunlong
6f16e52a6d Update rdb_last_bgsave_time_sec in INFO on diskless replication (#7917)
`info Persistence` will include correct (updated) rdb_last_bgsave_time_sec
For diskless bgsave (sockets) too (like a few other persistence info fields).

Refactor code to reduce duplicate code.

(cherry picked from commit e05a7df7f942a2e17dee3e0e3d67d80e8d1d2e6e)
2020-10-27 09:12:01 +02:00
Wen Hui
c6c2de9567 do not add save parameter during config rewrite in sentinel mode (#7945)
Previous code would have added default redis save parameters
to the config file on rewrite, which would have been silently ignored
when the config file is loaded.

The new code avoids adding this, and also actively removes these lines
If added by a previous config rewrite. 

(cherry picked from commit 0f370f9b667fab4bede084f1b8b47c93f0d56069)
2020-10-27 09:12:01 +02:00
Qu Chen
cebc1f2696 WATCH no longer ignores keys which have expired for MULTI/EXEC. (#7920)
This wrong behavior was backed by a test, and also documentation, and dates back to 2010.
But it makes no sense to anyone involved so it was decided to change that.

Note that 20eeddf (invalidate watch on expire on access) was released in 6.0 RC2
and 2d1968f released in in 6.0.0 GA (invalidate watch when key is evicted).
both of which do similar changes.

(cherry picked from commit 556acefe7556443b6d1741d804add92047bf4a8b)
2020-10-27 09:12:01 +02:00
Oran Agra
56d8ad932d improve verbose logging on failed test. print log file lines (#7938)
(cherry picked from commit c96ece9f5e7b80d65ca4d1a2b801effe68425c90)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
0103dc09d2 Add a --no-latency tests flag. (#7939)
Useful for running tests on systems which may be way slower than usual.

(cherry picked from commit 843a13e88f3544e936ead06a9d05af28eeefd5a1)
2020-10-27 09:12:01 +02:00
filipe oliveira
71c1427a1b Fixed bug concerning redis-benchmark non clustered benchmark forcing always the same hash tag {tag} (#7931)
Adding the ":{tag}" only if --cluster is used, so that when used against
a proxy it generates traffic to all shards.

Co-authored-by: Oran Agra <oran@redislabs.com>
(cherry picked from commit 6cf23d6610677d38e4892e02649fc2de7095274b)
2020-10-27 09:12:01 +02:00
Oran Agra
30bb0dd595 fix 32bit build warnings (#7926)
(cherry picked from commit a425e1d26df02ac5ce0fc9a0c679b99d72550009)
2020-10-27 09:12:01 +02:00
Wen Hui
99f39d32ab fix double fclose in aofrewrite (#7919)
minor fix for a bug which happen on error handling code
and doesn't look like it could have caused any real harm
(fd number wouldn't have been reused yet)

(cherry picked from commit 04a0af9085ed91eace8af94f71a07b651a1af454)
2020-10-27 09:12:01 +02:00
Oran Agra
851ec530b4 INFO report peak memory before eviction (#7894)
In some cases one command added a very big bulk of memory, and this
would be "resolved" by the eviction before the next command.

Seeing an unexplained mass eviction we would wish to
know the highest momentary usage too.

Tracking it in call() and beforeSleep() adds some hooks in AOF and RDB
loading.

The fix in clientsCronTrackExpansiveClients is related to #7874

(cherry picked from commit 457b7073b5d512c10aaeeb0dbf1763f0cb10462c)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
79e6ab31d8 Fix tests failure on busybox systems. (#7916)
(cherry picked from commit ef92f507dd0c402a916c16435e7f3f92598b7242)
2020-10-27 09:12:01 +02:00
Oran Agra
b20af834ff Allow requirepass config to clear the password (#7899)
This is a compatibility issue with redis 5.0 that was introduced by ACL.
Before this commit, setting requirepass to an empty string will result
in a server that needs an empty AUTH, unlike redis 5.0 which would
accept connections without an AUTH.

(cherry picked from commit 19418b6b28b70818e67c50bbdfad8568da7c3458)
2020-10-27 09:12:01 +02:00
Wang Yuan
1453277294 Remove temporary aof and rdb files in a background thread (#7905)
If we fail or stop to rewrite aof, we need to remove temporary aof.
We also remove temporary rdb when replicas abort to receive rdb.
But currently we delete them in main thread, to avoid blocking,
we should use bg_unlink to remove them in a background thread.

Btw, we have already used this way to removed child process temporary rdb.

(cherry picked from commit aaacb8c955779a76faaa15b7097eef4787a4339f)
2020-10-27 09:12:01 +02:00
guybe7
e4e24460f9 Minor improvements to module blocked on keys (#7903)
- Clarify some documentation comments
- Make sure blocked-on-keys client privdata is accessible
  from withing the timeout callback
- Handle blocked clients in beforeSleep - In case a key
  becomes "ready" outside of processCommand

See #7879 #7880

(cherry picked from commit addf47dcacec596eb7f6e1500846331d3e13fff4)
2020-10-27 09:12:01 +02:00
Andreas Lind
db9cffe7c2 Support redis-cli -u rediss://... (#7900)
(cherry picked from commit 8b497881f2eda5b028cc2ba0aa17ef204f6a083b)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
9c475272da Modules: fix RM_GetCommandKeys API. (#7901)
This cleans up and simplifies the API by passing the command name as the
first argument. Previously the command name was specified explicitly,
but was still included in the argv.

(cherry picked from commit 056a43e1a615080cf9ba405be0538a6a979c821e)
2020-10-27 09:12:01 +02:00
Meir Shpilraien (Spielrein)
6f2c894d69 Add Module API for version and compatibility checks (#7865)
* Introduce a new API's: RM_GetContextFlagsAll, and
RM_GetKeyspaceNotificationFlagsAll that will return the
full flags mask of each feature. The module writer can
check base on this value if the Flags he needs are
supported or not.

* For each flag, introduce a new value on redismodule.h,
this value represents the LAST value and should be there
as a reminder to update it when a new value is added,
also it will be used in the code to calculate the full
flags mask (assuming flags are incrementally increasing).
In addition, stated that the module writer should not use
the LAST flag directly and he should use the GetFlagAll API's.

* Introduce a new API: RM_IsSubEventSupported, that returns for a given
event and subevent, whether or not the subevent supported.

* Introduce a new macro RMAPI_FUNC_SUPPORTED(func) that returns whether
or not a function API is supported by comparing it to NULL.

* Introduce a new API: int RM_GetServerVersion();, that will return the
current Redis version in the format 0x00MMmmpp; e.g. 0x00060008;

* Changed unstable version from 999.999.999 to 255.255.255

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
(cherry picked from commit adc3183cd2b54238424895e4298548df4526f8c3)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
4147a2202e Module API: Add RM_GetClientCertificate(). (#7866)
This API function makes it possible to retrieve the X.509 certificate
used by clients to authenticate TLS connections.

(cherry picked from commit 0aec98dce2acbd280ad8ff4feac631e8afa833b1)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
28d1fe6718 Modules: Add RM_GetDetachedThreadSafeContext(). (#7886)
The main motivation here is to provide a way for modules to create a
single, global context that can be used for logging.

Currently, it is possible to obtain a thread-safe context that is not
attached to any blocked client by using `RM_GetThreadSafeContext`.
However, the attached context is not linked to the module identity so
log messages produced are not tagged with the module name.

Ideally we'd fix this in `RM_GetThreadSafeContext` itself but as it
doesn't accept the current context as an argument there's no way to do
that in a backwards compatible manner.

(cherry picked from commit 907da0580b57013c9e5c38b27f351597c72e4e25)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
116a2042a7 Modules: add RM_GetCommandKeys().
This is essentially the same as calling COMMAND GETKEYS but provides a
more efficient interface that can be used in every context (i.e. not a
Redis command).

(cherry picked from commit 7d117d7591656e947f526f5d5f8a022b88b38ad9)
2020-10-27 09:12:01 +02:00
Yossi Gottlieb
945983ac52 Introduce getKeysResult for getKeysFromCommand.
Avoid using a static buffer for short key index responses, and make it
caller's responsibility to stack-allocate a result type. Responses that
don't fit are still allocated on the heap.

(cherry picked from commit 9b7f8ba84b9de1ec846badd45e4004e3cd23f178)
2020-10-27 09:12:01 +02:00
Madelyn Olson
e7c8002b56 Fixed excessive categories being displayed from acls (#7889)
(cherry picked from commit 2127f7c8ebc1d8638eb0c99764f698cebcdb6513)
2020-10-27 09:12:01 +02:00