17 Commits

Author SHA1 Message Date
Oran Agra
5821e8cf82 fix processing of large bulks (above 2GB)
- protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
  readQueryFromClient potential overflow
- rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
- several places in sds.c that used int for string length or index.
- bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
- RM_SaveStringBuffer was limitted to 32bit length
2017-12-29 12:24:19 +02:00
antirez
9deb98167b Modules: support for modules native data types. 2016-06-03 18:14:04 +02:00
Oran Agra
7496636280 various cleanups and minor fixes 2016-04-25 16:49:57 +03:00
antirez
8339957bb2 Diskless replication: rio fdset target new supports buffering.
To perform a socket write() for each RDB rio API write call was
extremely unefficient, so now rio has minimal buffering capabilities.
Writes are accumulated into a buffer and only when a given limit is
reacehd are actually wrote to the N slaves FDs.

Trivia: rio lacked support for buffering since our targets were:

1) Memory buffers.
2) C standard I/O.

Both were buffered already.
2014-10-17 11:36:12 +02:00
antirez
32f90ad730 rio.c fdset target: tolerate (and report) a subset of FDs in error.
Fdset target is used when we want to write an RDB file directly to
slave's sockets. In this setup as long as there is a single slave that
is still receiving our payload, we want to continue sennding instead of
aborting. However rio calls should abort of no FD is ok.

Also we want the errors reported so that we can signal the parent who is
ok and who is broken, so there is a new set integers with the state of
each fd. Zero is ok, non-zero is the errno of the failure, if avaialble,
or a generic EIO.
2014-10-14 17:19:42 +02:00
antirez
4179f67af3 rio.c: draft implementation of fdset target implemented. 2014-10-10 17:44:06 +02:00
antirez
2973651071 Fixed typo in rio.h, simgle -> single. 2013-07-16 15:43:36 +02:00
yoav
dddfb15bc0 Chunked loading of RDB to prevent redis from stalling reading very large keys. 2013-07-16 15:41:24 +02:00
antirez
cac8706810 rio.c: added ability to fdatasync() from time to time while writing. 2013-04-24 10:26:30 +02:00
antirez
d8a59ffc18 Make rio.c comment 80-columns friendly. 2013-04-03 12:41:14 +02:00
antirez
a32d1ddff6 BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
antirez
78eeafc7cf Make inline functions rioRead/Write/Tell static. This fixes issue #447. 2012-04-11 11:58:32 +02:00
antirez
38b3cb4770 Fixed compilation of new rio.c changes (typos and so forth.) 2012-04-09 12:36:44 +02:00
antirez
ec83eacbde Add checksum computation to rio.c 2012-04-09 12:33:09 +02:00
antirez
3ad12623d4 rioInitWithFile nad rioInitWithBuffer functions now take a rio structure pointer to avoid copying a structure to return value to the caller. 2011-09-22 16:00:40 +02:00
antirez
06c356d912 comment on top of the _rio structure modified for correctness as actually fwrite/fread semantics is different in general, but was 0/1 in our old usage before rio.c as we always used 1 as number items, and the actual number of bytes to read as item length. 2011-09-22 15:47:48 +02:00
Pieter Noordhuis
ce9195ad16 Abstract file/buffer I/O to support in-memory serialization 2011-05-13 17:31:00 +02:00