473 Commits

Author SHA1 Message Date
antirez
0b02b27705 Port PR #6110 to new connection object code. 2020-01-17 10:47:38 +01:00
antirez
a6eb3f8bfb A few comments about main thread serving I/O as well.
Related to #6110.
2020-01-17 10:47:38 +01:00
zhaozhao.zz
1f76ff2fb9 Threaded IO: use main thread to handle read work 2020-01-17 10:47:38 +01:00
zhaozhao.zz
0a3410db70 Threaded IO: use main thread to handle write work 2020-01-17 10:47:38 +01:00
antirez
c2461ba531 Fix duplicated CLIENT SETNAME reply.
Happened when we set the name to "" to cancel the name.
Was introduced during the RESP3 refactoring.

See #6036.
2019-12-29 15:46:31 +01:00
antirez
1b56a873d5 Inline protocol: handle empty strings well.
This bug is from the first version of Redis. Probably the problem here
is that before we used an SDS split function that created empty strings
for additional spaces, like in "SET    foo          bar".
AFAIK later we replaced it with the curretn sdssplitarg() API that has
no such a problem. As a result, we introduced a bug, where it is no
longer possible to do something like:

    SET foo ""

Using the inline protocol. Now it is fixed.
2019-12-29 15:46:31 +01:00
Madelyn Olson
05b7628687 Add module APIs for custom authentication 2019-12-17 06:59:59 +00:00
Salvatore Sanfilippo
ab520ef80f Merge pull request #6531 from oranagra/rm_save_long_double
Module API for loading and saving long double
2019-11-04 17:33:35 +01:00
Oran Agra
04df0cc328 Module API for loading and saving long double
looks like each platform implements long double differently (different bit count)
so we can't save them as binary, and we also want to avoid creating a new RDB
format version, so we save these are hex strings using "%La".

This commit includes a change in the arguments of ld2string to support this.
as well as tests for coverage and short reads.

coded by @guybe7
2019-11-03 16:42:31 +02:00
Oran Agra
68c6aacf3b Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
antirez
261a66b791 Modules hooks: don't call the hook for fake clients. 2019-10-23 18:39:53 +02:00
antirez
d5bd45a368 Modules hooks: fix sub event in calls from client change. 2019-10-23 18:39:53 +02:00
antirez
f3e4f24f68 Modules hooks: fix define / linker issues. Implement one test event. 2019-10-23 18:39:53 +02:00
Yossi Gottlieb
85d7f38136 Merge remote-tracking branch 'upstream/unstable' into tls 2019-10-16 17:08:07 +03:00
Oran Agra
8704be6947 TLS: Implement support for write barrier. 2019-10-07 21:06:30 +03:00
Oran Agra
6fd5ff8c98 diskless replication rdb transfer uses pipe, and writes to sockets form the parent process.
misc:
- handle SSL_has_pending by iterating though these in beforeSleep, and setting timeout of 0 to aeProcessEvents
- fix issue with epoll signaling EPOLLHUP and EPOLLERR only to the write handlers. (needed to detect the rdb pipe was closed)
- add key-load-delay config for testing
- trim connShutdown which is no longer needed
- rioFdsetWrite -> rioFdWrite - simplified since there's no longer need to write to multiple FDs
- don't detect rdb child exited (don't call wait3) until we detect the pipe is closed
- Cleanup bad optimization from rio.c, add another one
2019-10-07 21:06:30 +03:00
Yossi Gottlieb
10ffeb03e4 TLS: Connections refactoring and TLS support.
* Introduce a connection abstraction layer for all socket operations and
integrate it across the code base.
* Provide an optional TLS connections implementation based on OpenSSL.
* Pull a newer version of hiredis with TLS support.
* Tests, redis-cli updates for TLS support.
2019-10-07 21:06:13 +03:00
antirez
90cf427a5c RESP3: Use verbatim in CLIENT LIST. 2019-09-18 18:52:13 +02:00
antirez
3b40a88587 More strict checks and better comments in flushSlaveOutputBuffers().
Related to #6296.
2019-08-31 14:46:24 +02:00
antirez
d5e2fe1532 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-08-31 14:41:28 +02:00
antirez
11228bd0fb Improve comment in flushSlavesOutputBuffers(). 2019-08-31 14:40:09 +02:00
zhaozhao.zz
89dc16f01d networking: flushSlavesOutputBuffers bugfix 2019-08-02 17:17:19 +08:00
Salvatore Sanfilippo
ee8b42cb27 Merge pull request #6196 from AngusP/resp3
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
2019-07-12 13:08:46 +02:00
antirez
c3d91f5d8b Client side caching: implement CLIENT GETREDIR.
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
2019-07-10 18:17:10 +02:00
antirez
4b5027845e Client side caching: implement trackingInvalidateKey(). 2019-07-03 19:16:20 +02:00
antirez
b826cf89f1 Client side caching: CLIENT TRACKING subcommand. 2019-07-03 11:58:20 +02:00
Angus Pearson
17e98d8851 RESP3 double representation for -infinity is ,-inf\r\n, not -inf\r\n 2019-07-02 14:28:48 +01:00
antirez
7f9de752de Client side caching: fields and flags for tracking mode. 2019-06-29 20:08:41 -04:00
antirez
d0d1cbbf5c Make comment in getClientOutputBufferMemoryUsage() describing the present. 2019-05-13 17:30:06 +02:00
antirez
cd4845c478 Threaded IO: handleClientsWithPendingReadsUsingThreads top comment. 2019-05-06 18:02:51 +02:00
antirez
50b09bc408 Threaded IO: ability to disable reads from threaded path. 2019-05-06 18:02:51 +02:00
antirez
6e29b364a8 Threaded IO: put fflush() inside tio_debug conditional. 2019-05-06 18:02:51 +02:00
antirez
1347c13551 Threaded IO: parsing WIP 2: refactoring to parse from thread. 2019-05-06 18:02:51 +02:00
antirez
67d79b9b10 Threaded IO: parsing WIP 1: set current_client in a better scoped way. 2019-05-06 18:02:51 +02:00
antirez
17b4ac3f8c Threaded IO: logging should be safe in I/O threads.
Potentially it is possible that we get interleaved writes, even if
serverLog() makes sure to write into a buffer and then use printf(), so
even this should be ok. However in general POSIX guarantees that writing
to the same file pointer object from multiple threads is safe. Anyway
currently we *reopen* the file at each call, but for the standard output
logging.

The logging functions actually also access global configuration while
performing the log (for instance in order to check the log level, the
log filename and so forth), however dunring the I/O threads execution
we cannot alter such shared state in any way.
2019-05-06 18:02:51 +02:00
antirez
4e10311fe5 Threaded IO: process read queue before stopping threads. 2019-05-06 18:02:51 +02:00
antirez
b9b29641cf Threaded IO: read side WIP 3. 2019-05-06 18:02:51 +02:00
antirez
1c3b585e78 Threaded IO: read side WIP 2. 2019-05-06 18:02:51 +02:00
antirez
7bae8afef8 Threaded IO: read side WIP. 2019-05-06 18:02:51 +02:00
antirez
793134d0e6 Threaded IO: hide more debugging printfs under conditional. 2019-05-06 18:02:51 +02:00
antirez
dc4d13e751 Threaded IO: make num of I/O threads configurable. 2019-05-06 18:02:51 +02:00
antirez
79626ca340 Threaded IO: use main thread if num of threads is 1. 2019-05-06 18:02:51 +02:00
Ubuntu
0353d13321 Threaded IO: stop threads when no longer needed + C11 in Makefile.
Now threads are stopped even when the connections drop immediately to
zero, not allowing the networking code to detect the condition and stop
the threads. serverCron() will handle that.
2019-05-06 18:02:51 +02:00
antirez
444b5f2af5 Threaded IO: 3rd version: use the mutex only to stop the thread. 2019-05-06 18:02:51 +02:00
antirez
df0a28f661 Threaded IO: second attempt without signaling conditions. 2019-05-06 18:02:51 +02:00
antirez
c1cb2ae695 Threaded IO: allow to disable debug printf. 2019-05-06 18:02:51 +02:00
antirez
62261aa905 Threaded IO: implement handleClientsWithPendingWritesUsingThreads().
This is just an experiment for now, there are a couple of race
conditions, mostly harmless for the performance gain experiment that
this commit represents so far.

The general idea here is to take Redis single threaded and instead
fan-out on expansive kernel calls: write(2) in this case, but the same
concept could be easily implemented for read(2) and protcol parsing.

However just threading writes like in this commit, is enough to evaluate
if the approach is sounding.
2019-05-06 18:02:51 +02:00
Salvatore Sanfilippo
179c89c6a4 Merge pull request #5971 from devnexen/unstable
build fix
2019-04-26 17:32:21 +02:00
David Carlier
0e00a99f32 build fix 2019-03-28 06:38:16 +00:00
Oran Agra
48d14e5aa7 slave corrupts replication stream when module blocked client uses large reply (or POSTPONED_ARRAY)
when redis appends the blocked client reply list to the real client, it didn't
bother to check if it is in fact the master client. so a slave executing that
module command will send replies to the master, causing the master to send the
slave error responses, which will mess up the replication offset
(slave will advance it's replication offset, and the master does not)
2019-03-24 14:17:37 +02:00