21 Commits

Author SHA1 Message Date
John Sully
c2b5a8802e Support serializing all types
Former-commit-id: dfdd41f15c0567925f02fd5df779f597ab16894d
2019-10-16 14:23:28 -04:00
John Sully
725dd6539f Add back user space buffering of RDB save
Former-commit-id: d918ca6fa57a6149b86b4effc787dbdde7350133
2019-07-06 00:36:23 -04:00
John Sully
68bec6f239 Move remaning files dependent on server.h over to C++
Former-commit-id: 8c133b605c65212b023d35b3cb71e63b6a4c443a
2019-04-08 01:00:48 -04:00
John Sully
b660bfb6dd Make main headers C++ safe, and change rdb to use file descriptor instead of FILE pointer
Former-commit-id: 3c9dd6ffc254d089e4208ad39da7338b6fb0fba7
2019-02-05 23:36:40 -05:00
Oran Agra
5821e8cf82 fix processing of large bulks (above 2GB)
- protocol parsing (processMultibulkBuffer) was limitted to 32big positions in the buffer
  readQueryFromClient potential overflow
- rioWriteBulkCount used int, although rioWriteBulkString gave it size_t
- several places in sds.c that used int for string length or index.
- bugfix in RM_SaveAuxField (return was 1 or -1 and not length)
- RM_SaveStringBuffer was limitted to 32bit length
2017-12-29 12:24:19 +02:00
antirez
9deb98167b Modules: support for modules native data types. 2016-06-03 18:14:04 +02:00
Oran Agra
7496636280 various cleanups and minor fixes 2016-04-25 16:49:57 +03:00
antirez
8339957bb2 Diskless replication: rio fdset target new supports buffering.
To perform a socket write() for each RDB rio API write call was
extremely unefficient, so now rio has minimal buffering capabilities.
Writes are accumulated into a buffer and only when a given limit is
reacehd are actually wrote to the N slaves FDs.

Trivia: rio lacked support for buffering since our targets were:

1) Memory buffers.
2) C standard I/O.

Both were buffered already.
2014-10-17 11:36:12 +02:00
antirez
32f90ad730 rio.c fdset target: tolerate (and report) a subset of FDs in error.
Fdset target is used when we want to write an RDB file directly to
slave's sockets. In this setup as long as there is a single slave that
is still receiving our payload, we want to continue sennding instead of
aborting. However rio calls should abort of no FD is ok.

Also we want the errors reported so that we can signal the parent who is
ok and who is broken, so there is a new set integers with the state of
each fd. Zero is ok, non-zero is the errno of the failure, if avaialble,
or a generic EIO.
2014-10-14 17:19:42 +02:00
antirez
4179f67af3 rio.c: draft implementation of fdset target implemented. 2014-10-10 17:44:06 +02:00
antirez
2973651071 Fixed typo in rio.h, simgle -> single. 2013-07-16 15:43:36 +02:00
yoav
dddfb15bc0 Chunked loading of RDB to prevent redis from stalling reading very large keys. 2013-07-16 15:41:24 +02:00
antirez
cac8706810 rio.c: added ability to fdatasync() from time to time while writing. 2013-04-24 10:26:30 +02:00
antirez
d8a59ffc18 Make rio.c comment 80-columns friendly. 2013-04-03 12:41:14 +02:00
antirez
a32d1ddff6 BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
antirez
78eeafc7cf Make inline functions rioRead/Write/Tell static. This fixes issue #447. 2012-04-11 11:58:32 +02:00
antirez
38b3cb4770 Fixed compilation of new rio.c changes (typos and so forth.) 2012-04-09 12:36:44 +02:00
antirez
ec83eacbde Add checksum computation to rio.c 2012-04-09 12:33:09 +02:00
antirez
3ad12623d4 rioInitWithFile nad rioInitWithBuffer functions now take a rio structure pointer to avoid copying a structure to return value to the caller. 2011-09-22 16:00:40 +02:00
antirez
06c356d912 comment on top of the _rio structure modified for correctness as actually fwrite/fread semantics is different in general, but was 0/1 in our old usage before rio.c as we always used 1 as number items, and the actual number of bytes to read as item length. 2011-09-22 15:47:48 +02:00
Pieter Noordhuis
ce9195ad16 Abstract file/buffer I/O to support in-memory serialization 2011-05-13 17:31:00 +02:00