505 Commits

Author SHA1 Message Date
Oran Agra
9cd3997851 Module API for PUBLISH, FLUSHALL, RANDOMKEY, DBSIZE 2019-11-04 07:40:52 +02:00
Oran Agra
2bbd305a33 Add module api for looking into INFO fields
- Add RM_GetServerInfo and friends
- Add auto memory for new opaque struct
- Add tests for new APIs

other minor fixes:
- add const in various char pointers
- requested_section in modulesCollectInfo was actually not sds but char*
- extract new string2d out of getDoubleFromObject for code reuse

Add module API for
2019-11-03 15:02:25 +02:00
antirez
4af1df6b9f Modules: block on keys: use a better interface.
Using the is_key_ready() callback plus the reply callback later, creates
different issues AFAIK:

1. More complex API.
2. We need to call the reply callback() ASAP if the is_key_ready()
interface returned success, however the internals do not work in that
way, so when the reply callback is called the setup could be different.
To fix that, there is to break the current design that handles the
unblocked clients asyncrhonously, and run the list ASAP.
2019-10-31 11:35:07 +01:00
antirez
c4becaebaa Modules: block on keys: implement the internals. 2019-10-30 10:57:44 +01:00
Oran Agra
68c6aacf3b Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
Oran Agra
57649ce051 Module API for controlling LRU and LFU, and OpenKey without TOUCH
Some commands would want to open a key without touching it's LRU/LFU
similarly to the OBJECT or DEBUG command do.

Other commands may want to implement logic similar to what RESTORE
does (and in the future MIGRATE) and get/set the LRU or LFU.
2019-10-29 15:03:05 +02:00
John Sully
7bdc24a86d Remove the DB lock, its unnecessary
Former-commit-id: 631f863dd89cd642e2023beabf8b31cdc84bbdff
2019-10-24 20:18:48 -04:00
antirez
f3e4f24f68 Modules hooks: fix define / linker issues. Implement one test event. 2019-10-23 18:39:53 +02:00
antirez
98a0d95965 Modules hooks: handle module unloading + API export. 2019-10-23 18:39:53 +02:00
John Sully
60c35d6ea7 Remove race conditions
Former-commit-id: 5a8cb77d0df7f319809ff965a72fe46925f49289
2019-10-22 23:26:37 -04:00
John Sully
b1190c24a8 Optimize deadlock detection, fix callstack for ASM, and annotate locks
Note: This change moves our assembly code to use the GNU Assembler because NASM seems to be incapable of emitting the necessary debug information for callstack unwinding to work.

Former-commit-id: 600fc241cfe79b9b32ac6010c6ea0c66747f0f15
2019-10-22 21:34:51 -04:00
antirez
ea85cda320 Modules: allow to check for AOF loading client. 2019-10-17 18:10:50 +02:00
John Sully
4dc728c1ce Fix crash adding expire to key with a subkey expire already set
Former-commit-id: 18a3768e07eb8c4feb4a666d95a19d97267ac381
2019-10-16 12:26:48 -04:00
Yossi Gottlieb
85d7f38136 Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
John Sully
2980e06c9f Implement the lastmodified query for the OBJECT command
Former-commit-id: 3a3b59a0543148956797ce4ad9d08095051517b9
2019-10-13 14:12:09 -04:00
John Sully
ac243ae4bf Implement the EXPIREMEMBERAT command
Former-commit-id: 203e341bf4427723903d8d3de03af2f0ef043a3f
2019-10-13 13:15:13 -04:00
John Sully
e57d2e24c9 Update more slave instances to use replica (Issue #75)
Former-commit-id: 252725d50fc9d4ff2b6e9246a36c38176d61beae
2019-10-13 12:29:20 -04:00
John Sully
05d7536b02 Update a few uses of slave to use the new replica terminology (Issue #75)
Former-commit-id: c15633ea9e784ef0e12e4015a75245661a8cf1b7
2019-10-13 12:12:05 -04:00
John Sully
0a00341a80 Implement TTL and PERSIST commands for subkeys. Also ensure expiremember replaces any old subkey expiries
Former-commit-id: 16f96efbe7c6c27f2a79f5b472447407f905df15
2019-10-13 12:03:25 -04:00
John Sully
dba52de8ca Fix issue #88 - don't let seperate KeyDB processes listen on the same port
Former-commit-id: 4631119179df5cff638e9b8c51b030cef89a6c02
2019-10-13 10:02:21 -04:00
Yossi Gottlieb
514e5aa8ef TLS: Improve CA certifiate configuration options.
This adds support for explicit configuration of a CA certs directory (in
addition to the previously supported bundle file).  For redis-cli, if no
explicit CA configuration is supplied the system-wide default
configuration will be adopted.
2019-10-08 17:58:50 +03:00
Yossi Gottlieb
df08b624bd TLS: Configuration options.
Add configuration options for TLS protocol versions, ciphers/cipher
suites selection, etc.
2019-10-07 21:07:27 +03:00
Oran Agra
8704be6947 TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
6fd5ff8c98 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
10ffeb03e4 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
Oran Agra
7737e5814a Merge remote-tracking branch 'antirez/unstable' into jemalloc_purge_bg 2019-10-04 13:53:40 +03:00
antirez
46da87cdbb Minor aesthetic changes to #6419. 2019-10-04 12:00:41 +02:00
Salvatore Sanfilippo
2ee9207f33 Merge pull request #6419 from oranagra/lua_timeout_sha
On LUA script timeout, print the script SHA to the log
2019-10-04 11:58:05 +02:00
antirez
67cdd03efa Modules: implement RM_Replicate() from async callbacks. 2019-10-03 10:56:37 +02:00
antirez
c1855c1053 Speedup INFO server section. 2019-10-02 11:30:20 +02:00
Oran Agra
497cfb61d6 On LUA script timeout, print the script SHA to the log
since the slowlog and other means that can help you detect the bad script
are only exposed after the script is done. it might be a good idea to at least
print the script name (sha) to the log when it timeouts.
2019-10-02 08:40:35 +03:00
Oran Agra
bd49c4679b Merge remote-tracking branch 'antirez/unstable' into modules_info 2019-09-30 20:47:35 +03:00
John Sully
4cac0ca35a Implement left and right shift BITOP operators
Former-commit-id: ba365298ed37a76f0a8630e0ec6c86393293aebe
2019-09-28 00:10:46 -04:00
antirez
48aae0d0db TerminateModuleForkChild(): fix function prototype. 2019-09-27 12:23:07 +02:00
antirez
7aaedf9192 TerminateModuleForkChild(): move safety checks there.
We don't want that the API could be used directly in an unsafe way,
without checking if there is an active child. Now the safety checks are
moved directly in the function performing the operations.
2019-09-27 12:17:47 +02:00
antirez
1ef0c2f630 Function renamed hasForkChild() -> hasActiveChildProcess(). 2019-09-27 12:03:09 +02:00
antirez
e885d74018 Modules fork: improve SIGUSR1 handling, fix include.
We can't expect SIGUSR1 to have any specific value range, so let's
define an exit code that we can handle in a special way.
This also fixes an #include <wait.h> that is not standard.
2019-09-27 11:39:45 +02:00
Salvatore Sanfilippo
5c5761bcd6 Merge branch 'unstable' into modules_fork 2019-09-27 11:24:06 +02:00
John Sully
4f19c5de9f Fix multi master bugs: 1. we fail to create the temp file. 2. We use a master RDB as our backup even though we merged databases (and therefore it is not representitive)
Former-commit-id: e776474f68a2824bb7d4082c41991a9a9f3a9c9d
2019-09-26 20:35:51 -04:00
Salvatore Sanfilippo
4d5fc6b46d Merge pull request #6024 from itamarhaber/info_modules
Adds a "Modules" section to `INFO`
2019-09-26 11:58:52 +02:00
Salvatore Sanfilippo
f11b71b9b8 Merge pull request #6235 from oranagra/module_rdb_load_errors
Allow modules to handle RDB loading errors.
2019-09-26 11:52:42 +02:00
John Sully
bd4ed482c4 KEYS now only blocks one database
Former-commit-id: 18d42a5c353f76533a0ccc4ded24ed089cedacc8
2019-09-12 18:51:10 -04:00
antirez
2e1e170aad RESP3: fix cases of NULL reported instead of empty aggregate. 2019-09-02 12:50:47 +02:00
John Sully
a21a7a8ee9 Fix race condition in PUB/SUB and other async reply commands where the client can be freed before our handler is executed on the client thread. When this occurs the client pointer is dangling
Former-commit-id: 46a78c69e718f0aeb5e62f33c59458b15b8d9bc6
2019-08-26 20:18:52 -04:00
Oran Agra
8bf7c6d789 Modlue fork is killed when the parent exists 2019-08-25 10:11:48 +03:00
antirez
e87768e8ca Replication: clarify why repl_put_online_on_ack exists at all. 2019-08-05 17:38:15 +02:00
Salvatore Sanfilippo
3e66c43e84 Merge pull request #6266 from madolson/dev-unstable-hide-auth-and-hello
Hide HELLO and AUTH from slowlog and monitor
2019-07-31 11:12:46 +02:00
Oran Agra
7d18a61559 Merge remote-tracking branch 'oss/unstable' into module_rdb_load_errors 2019-07-30 15:14:08 +03:00
Oran Agra
9fa285523c Avoid diskelss-load if modules did not declare they handle read errors 2019-07-30 15:11:57 +03:00
Salvatore Sanfilippo
8e2a99eb4b Merge pull request #6264 from oranagra/modules_api_aux_rdb
Implement module api for aux data in rdb
2019-07-30 11:50:44 +02:00