1121 Commits

Author SHA1 Message Date
Oran Agra
68c6aacf3b Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
Oran Agra
5752b1718b test infra: improve prints on failed assertions
sometimes we have several assertions with the same condition in the same test
at different stages, and when these fail (the ones that print the condition
text) you don't know which one it was. other assertions didn't print the
condition text (variable names), just the expected and unexpected values.

So now, all assertions print context line, and conditin text.

besides, one of the major differences between 'assert' and 'assert_equal',
is that the later is able to print the value that doesn't match the expected.
if there is a rare non-reproducible failure, it is helpful to know what was
the value the test encountered and how far it was from the threshold.

So now, adding assert_lessthan and assert_range that can be used in some places.
were we used just 'assert { a > b }' so far.
2019-10-29 17:38:12 +02:00
Oran Agra
d725788873 Module api tests for RM_Call
Adding a test for coverage for RM_Call in a new "misc" unit
to be used for various short simple tests

also solves compilation warnings in redismodule.h and fork.c
2019-10-28 12:39:57 +02:00
Salvatore Sanfilippo
d4a63e69b7 Merge pull request #6496 from oranagra/module_tests_valgrind
Make module tests pass with valgrind, and fix a leak in diskless load
2019-10-28 10:05:52 +01:00
John Sully
949270801e Signals can happen on any thread, so look for the signal handler not the command on the callstack
Former-commit-id: f1d2b2945007f8811528b197480e255c6b35559c
2019-10-24 23:30:08 -04:00
Yossi Gottlieb
a6d42ebfd9 Modules: improve tests Makefile.
Removes some boilerplate per module, add clean target.
2019-10-24 13:21:35 +03:00
antirez
3bd7f715c8 Modules hooks: test flush event. 2019-10-24 10:51:03 +02:00
antirez
51b680be42 Modules hooks: initial Tcl test file. 2019-10-24 10:41:25 +02:00
Oran Agra
70815145a7 Make module tests pass with valgrind, and fix a leak in diskless load 2019-10-24 09:45:25 +03:00
antirez
e9f6688a68 Modules hooks: test module draft. 2019-10-23 18:42:02 +02:00
Yossi Gottlieb
85d7f38136 Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
antirez
0ec3cbdb8d Test: fix implementation-dependent test after code change. 2019-10-10 15:22:42 +02:00
Daniel Dai
46c1377c83 update typo 2019-10-09 14:15:31 -04:00
antirez
bdca9e882d Geo: output 10 chars of geohash, not 11.
This does not limit the actual precision, because the last digit bits were
garbage, and the shift value became even negative in the last iteration.
2019-10-08 17:09:37 +02:00
Yossi Gottlieb
df08b624bd TLS: Configuration options.
Add configuration options for TLS protocol versions, ciphers/cipher
suites selection, etc.
2019-10-07 21:07:27 +03:00
Oran Agra
8704be6947 TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
6fd5ff8c98 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
10ffeb03e4 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
antirez
59982e85eb Modules: add RM_Replicate() Tcl test file & fix the module. 2019-10-03 18:44:50 +02:00
antirez
d6d4fea080 Modules: RM_Replicate() test with threads. 2019-10-03 13:23:53 +02:00
antirez
d859b42899 Modules: RM_Replicate() test module: initial implementation. 2019-10-03 13:06:13 +02:00
Salvatore Sanfilippo
b5f075f8ae Merge pull request #6270 from oranagra/modules_info
Extend modules API to allow modules report to redis INFO
2019-10-01 18:02:33 +02:00
Oran Agra
bd49c4679b Merge remote-tracking branch 'antirez/unstable' into modules_info 2019-09-30 20:47:35 +03:00
antirez
6118989790 ACLs: change hashed passwords opcode to also remove them.
Related to PR #6405
2019-09-30 18:28:45 +02:00
Madelyn Olson
364c8601e3 Allowed passing in of password hash and fixed config rewrite 2019-09-30 17:57:49 +02:00
John Sully
4db6193052 RREPLAY command now takes a DB argument
Former-commit-id: 6e1e5bd08b59f8ad4653621a6c01fcf3a76f0692
2019-09-28 14:59:44 -04:00
John Sully
4cac0ca35a Implement left and right shift BITOP operators
Former-commit-id: ba365298ed37a76f0a8630e0ec6c86393293aebe
2019-09-28 00:10:46 -04:00
Salvatore Sanfilippo
5c5761bcd6 Merge branch 'unstable' into modules_fork 2019-09-27 11:24:06 +02:00
Oran Agra
2add4524c5 Fix lastbgsave_status, when new child signal handler get intended kill
And add a test for that.
2019-09-26 15:16:34 +03:00
Salvatore Sanfilippo
f11b71b9b8 Merge pull request #6235 from oranagra/module_rdb_load_errors
Allow modules to handle RDB loading errors.
2019-09-26 11:52:42 +02:00
valentino
bd32b8b2a8 DISCARD should not fail during OOM
discard command should not fail during OOM, otherwise client MULTI state
will not be cleared.
2019-09-22 09:22:53 +03:00
Oran Agra
8319ffb34d Module INFO, support default section for simple modules 2019-08-18 10:01:57 +03:00
Oran Agra
d4c24a3067 Module INFO, add support for dict fields, rename API to have common prefix 2019-08-18 09:41:45 +03:00
Oran Agra
037ac27e41 Add test for module diskless short reads 2019-07-30 15:20:03 +03:00
John Sully
4c49370efe Issue #64 RREPLAY isn't binary safe. Add fix and test.
Former-commit-id: afe66288fe9df6d8247d459e57858430f1ec7a25
2019-07-24 22:31:02 -04:00
Oran Agra
b2fce80695 Extend modules API to allow modules report to redis INFO
this implements #6012
2019-07-24 12:58:15 +03:00
Oran Agra
ff0780e8e6 Implement module api for aux data in rdb
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
2019-07-22 21:15:33 +03:00
John Sully
1c8c4a5db2 Fix bad merge in SCAN KEYS command
Former-commit-id: c21af6b351328ffbdb1d1e2a7eed44f8f929f8b2
2019-07-19 00:43:23 -04:00
John Sully
13fa9bb2be Merge commit '4bbaf621a12f6ec22ec7dad6a2282b7908660497' into unstable
Former-commit-id: 10986ae3a801c13013ec74880dd1f7969c284d01
2019-07-18 23:22:36 -04:00
antirez
12325871b4 Test: fix slowlog test false positive.
In fast systems "SLOWLOG RESET" is fast enough to don't be logged even
when the time limit is "1" sometimes. Leading to false positives such
as:

[err]: SLOWLOG - can be disabled in tests/unit/slowlog.tcl
Expected '1' to be equal to '0'


Former-commit-id: 8198a697fd4455c88712099f20632e554fb564d4
2019-07-18 23:12:52 -04:00
antirez
271db42742 Fix test false positive introduced by threaded I/O.
Now clients that are ready to be terminated asynchronously are processed
more often in beforeSleep() instead of being processed in serverCron().
This means that the test will not be able to catch the moment the client
was terminated, also note that the 'omem' figure now changes in big
steps, because of the new client output buffers layout.

So we have to change the test range in order to accomodate for that.
Yet the test is useful enough to be worth taking, even if its precision
is reduced by this commit. Probably if we get more problems, a thing
that makes sense is just to check that the limit is < 200k. That's more
than enough actually.


Former-commit-id: 8aaa8b0b116dc86473b6a94bf2ff330dd4163ca1
2019-07-18 23:12:05 -04:00
John Sully
133650ac7d remove unnecessary variables
Former-commit-id: a9c4e3970fd0d00896cb5500b2cc926ec3177bc4
2019-07-18 19:37:49 -04:00
John Sully
594559b45e Fix issue #59 - Test replicaof config variable
Former-commit-id: d5ecc2347a717f6e6f938d81325476a9c3dff9a6
2019-07-18 19:22:49 -04:00
Oran Agra
73a945c73c prevent diskless replica from terminating on short read
now that replica can read rdb directly from the socket, it should avoid exiting
on short read and instead try to re-sync.

this commit tries to have minimal effects on non-diskless rdb reading.
and includes a test that tries to trigger this scenario on various read cases.
2019-07-17 16:46:22 +02:00
Oran Agra
e70fbad802 Module API for Forking
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
2019-07-17 16:40:24 +03:00
Salvatore Sanfilippo
89e00762e7 Merge pull request #5779 from madolson/dev-unstable-geo-bugfix
Fixed a rounding bug in geo.tcl
2019-07-15 23:41:57 +02:00
John Sully
0f5b59241f Give the active-rep more time to sync for test reliability
Former-commit-id: 620c2ec1412a3bcea5fecf21238caa065e73f3e9
2019-07-12 23:52:07 -04:00
John Sully
3ffe5b5dde Add Active Replication tests
Former-commit-id: 528d10091fda0d2c56674e825c4f70467587955f
2019-07-12 03:54:41 -04:00
John Sully
e29c6d0ac9 FIX: The dabase count configuration is not respected
Former-commit-id: 149c3cf3dd75e8058c7c560d6a7ffbbb95da3898
2019-07-12 02:13:37 -04:00
Oran Agra
29754ebe22 diskless replication on slave side (don't store rdb to file), plus some other related fixes
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
2019-07-08 15:37:48 +03:00