6574 Commits

Author SHA1 Message Date
antirez
8a08c7d570 Copyright notice added to module.c. 2016-10-06 08:48:21 +02:00
antirez
adcafbd283 Modules: API to save/load single precision floating point numbers.
When double precision is not needed, to take 2x space in the
serialization is not good.
2016-10-03 00:08:35 +02:00
antirez
8c561f8430 Modules: API to log from module I/O callbacks. 2016-10-02 16:51:37 +02:00
antirez
ac539449ea Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-10-02 16:50:37 +02:00
antirez
39b6ae46eb Add compiler optimizations to example module makefile. 2016-10-02 11:01:36 +02:00
antirez
48dbb54909 debug.c: include dlfcn.h regardless of BACKTRACE support. 2016-09-27 00:29:47 +02:00
antirez
43a1074bc5 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-09-26 09:10:52 +02:00
antirez
3b7916b7bb Security: CONFIG SET client-output-buffer-limit overflow fixed.
This commit fixes a vunlerability reported by Cory Duplantis
of Cisco Talos, see TALOS-2016-0206 for reference.

CONFIG SET client-output-buffer-limit accepts as client class "master"
which is actually only used to implement CLIENT KILL. The "master" class
has ID 3. What happens is that the global structure:

    server.client_obuf_limits[class]

Is accessed with class = 3. However it is a 3 elements array, so writing
the 4th element means to write up to 24 bytes of memory *after* the end
of the array, since the structure is defined as:

    typedef struct clientBufferLimitsConfig {
        unsigned long long hard_limit_bytes;
        unsigned long long soft_limit_bytes;
        time_t soft_limit_seconds;
    } clientBufferLimitsConfig;

EVALUATION OF IMPACT:

Checking what's past the boundaries of the array in the global
'server' structure, we find AOF state fields:

    clientBufferLimitsConfig client_obuf_limits[CLIENT_TYPE_OBUF_COUNT];
    /* AOF persistence */
    int aof_state;                  /* AOF_(ON|OFF|WAIT_REWRITE) */
    int aof_fsync;                  /* Kind of fsync() policy */
    char *aof_filename;             /* Name of the AOF file */
    int aof_no_fsync_on_rewrite;    /* Don't fsync if a rewrite is in prog. */
    int aof_rewrite_perc;           /* Rewrite AOF if % growth is > M and... */
    off_t aof_rewrite_min_size;     /* the AOF file is at least N bytes. */
    off_t aof_rewrite_base_size;    /* AOF size on latest startup or rewrite. */
    off_t aof_current_size;         /* AOF current size. */

Writing to most of these fields should be harmless and only cause problems in
Redis persistence that should not escalate to security problems.
However unfortunately writing to "aof_filename" could be potentially a
security issue depending on the access pattern.

Searching for "aof.filename" accesses in the source code returns many different
usages of the field, including using it as input for open(), logging to the
Redis log file or syslog, and calling the rename() syscall.

It looks possible that attacks could lead at least to informations
disclosure of the state and data inside Redis. However note that the
attacker must already have access to the server. But, worse than that,
it looks possible that being able to change the AOF filename can be used
to mount more powerful attacks: like overwriting random files with AOF
data (easily a potential security issue as demostrated here:
http://antirez.com/news/96), or even more subtle attacks where the
AOF filename is changed to a path were a malicious AOF file is loaded
in order to exploit other potential issues when the AOF parser is fed
with untrusted input (no known issue known currently).

The fix checks the places where the 'master' class is specifiedf in
order to access configuration data structures, and return an error in
this cases.

WHO IS AT RISK?

The "master" client class was introduced in Redis in Jul 28 2015.
Every Redis instance released past this date is not vulnerable
while all the releases after this date are. Notably:

    Redis 3.0.x is NOT vunlerable.
    Redis 3.2.x IS vulnerable.
    Redis unstable is vulnerable.

In order for the instance to be at risk, at least one of the following
conditions must be true:

    1. The attacker can access Redis remotely and is able to send
       the CONFIG SET command (often banned in managed Redis instances).

    2. The attacker is able to control the "redis.conf" file and
       can wait or trigger a server restart.

The problem was fixed 26th September 2016 in all the releases affected.
2016-09-26 08:47:52 +02:00
charsyam
da5dbbec00 Simple change just using slaves instead of server.slaves 2016-09-24 15:53:57 +09:00
Salvatore Sanfilippo
181f071efd Merge pull request #3511 from dvirsky/create_string_printf
added RM_CreateStringPrintf
2016-09-21 11:46:53 +02:00
Dvir Volk
b8a375fe59 added RM_CreateStringPrintf 2016-09-21 12:30:38 +03:00
antirez
e396deda94 dict.c: fix dictGenericDelete() return ASAP condition.
Recently we moved the "return ASAP" condition for the Delete() function
from checking .size to checking .used, which is smarter, however while
testing the first table alone always works to ensure the dict is totally
emtpy, when we test the .size field, testing .used requires testing both
T0 and T1, since a rehashing could be in progress.
2016-09-20 17:22:30 +02:00
antirez
3d37ed07c6 Clear child data when opening the pipes.
This is important both to reset the magic to 0, so that it will not
match if the structure is not explicitly set, and to initialize other
things we may add like counters and such.
2016-09-19 14:11:17 +02:00
antirez
62e7179de9 Child -> Parent pipe for COW info transferring. 2016-09-19 13:45:20 +02:00
antirez
70a27b5b58 zmalloc: Make fp var non local to fix build. 2016-09-19 10:34:39 +02:00
antirez
7e86bce6b0 zmalloc: zmalloc_get_smap_bytes_by_field() modified to work for any PID.
The goal is to get copy-on-write amount of the child from the parent.
2016-09-19 10:28:42 +02:00
antirez
e8ee32fcd6 redis-cli: "allocator-stats" -> "malloc-stats".
It was changed in Redis but not in redis-cli.
Thanks to @oranagra for signaling.
2016-09-19 09:47:35 +02:00
antirez
ed63dadc10 Typo fixed from MEMORY DOCTOR output. 2016-09-16 16:52:00 +02:00
antirez
44fedc3f7b Surround allocator name with quotes in MEMORY DOCTOR output. 2016-09-16 16:40:25 +02:00
antirez
6249fdc544 MEMORY DOCTOR initial implementation. 2016-09-16 16:36:53 +02:00
antirez
303d981cab Provide percentage of memory peak used info. 2016-09-16 10:43:19 +02:00
oranagra
c87f006951 add zmalloc used mem to DEBUG SDSLEN 2016-09-16 10:29:27 +02:00
antirez
9dce1c8dcd Memory related subcommands of DEBUG moved to MEMORY. 2016-09-16 10:26:23 +02:00
antirez
2decf8957e Group MEMORY command related APIs together in the source code. 2016-09-16 10:12:04 +02:00
antirez
28dc0a22a7 objectComputeSize(): skiplist nodes have different sizes.
The size of the node depends on the node level, however it is not stored
into the node itself, is an implicit information, so we use
zmalloc_size() in order to compute the sorted set size.
2016-09-15 17:43:13 +02:00
antirez
340c5546a1 MEMORY command: HELP + dataset percentage (like in INFO). 2016-09-15 17:33:16 +02:00
antirez
3819a712ac MEMORY USAGE: SAMPLES option added + fixes to size computation.
The new SAMPLES option is added, defaulting to 5, and with 0 being a
special value to scan the whole set of elements.

Fixes to the object size computation were made since the original PR
assumed data structures still contaning robj structures, while now after
the lazyfree changes, are all SDS strings.
2016-09-15 15:25:14 +02:00
antirez
0edf97bc7e INFO: new memory reporting fields added. 2016-09-15 10:33:23 +02:00
antirez
fa9588a275 C struct memoh renamed redisMemOverhead. API prototypes added. 2016-09-15 09:44:07 +02:00
antirez
a1045e9651 MEMORY OVERHEAD refactored into a generic API. 2016-09-15 09:37:55 +02:00
antirez
e2106f0281 dict.c: dictReplaceRaw() -> dictAddOrFind().
What they say about "naming things" in programming?
2016-09-14 16:43:38 +02:00
antirez
8c4add1a58 Trim comment to 80 cols. 2016-09-14 16:41:05 +02:00
antirez
7f608a61de Apply the new dictUnlink() where possible.
Optimizations suggested and originally implemented by @oranagra.
Re-applied by @antirez using the modified API.
2016-09-14 16:37:53 +02:00
oranagra
90af5e0825 dict.c: introduce dictUnlink().
Notes by @antirez:

This patch was picked from a larger commit by Oran and adapted to change
the API a bit. The basic idea is to avoid double lookups when there is
to use the value of the deleted entry.

BEFORE:

    entry = dictFind( ... ); /* 1st lookup. */
    /* Do somethjing with the entry. */
    dictDelete(...);         /* 2nd lookup. */

AFTER:

    entry = dictUnlink( ... ); /* 1st lookup. */
    /* Do somethjing with the entry. */
    dictFreeUnlinkedEntry(entry); /* No lookups!. */
2016-09-14 12:18:59 +02:00
antirez
f705ad52db MEMORY OVERHEAD implemented (using Oran Agra initial implementation).
This code was extracted from @oranagra PR #3223 and modified in order
to provide only certain amounts of information compared to the original
code. It was also moved from DEBUG to the newly introduced MEMORY
command. Thanks to Oran for the implementation and the PR.

It implements detailed memory usage stats that can be useful in both
provisioning and troubleshooting memory usage in Redis.
2016-09-13 17:39:25 +02:00
antirez
cf49be57a5 objectComputeSize(): estimate collections sampling N elements.
For most tasks, we need the memory estimation to be O(1) by default.
This commit also implements an initial MEMORY command.
Note that objectComputeSize() takes the number of samples to check as
argument, so MEMORY should be able to get the sample size as option
to make precision VS CPU tradeoff tunable.

Related to: PR #3223.
2016-09-13 10:28:23 +02:00
oranagra
41899d26ef Adding objectComputeSize() function. 2016-09-12 16:36:59 +02:00
oranagra
4448185b52 Optimize repeated keyname hashing.
(Change cherry-picked and modified by @antirez from a larger commit
provided by @oranagra in PR #3223).
2016-09-12 13:19:05 +02:00
Alexander Zhukov
a794bb296e Fix an article usage 2016-09-12 13:53:47 +03:00
Salvatore Sanfilippo
3a2c6c3eb9 Merge pull request #3492 from wyxustcsa09/fix-memory
fix memory error on module unload
2016-09-09 16:05:06 +02:00
antirez
f14ccc7d95 Merge branch 'unstable' of github.com:antirez/redis into unstable 2016-09-09 16:01:43 +02:00
antirez
eef33c77d3 Example modules: Add C99 standard to cflags. 2016-09-09 16:01:29 +02:00
antirez
19ba3dbb70 Merge branch 'aofrdb' into unstable 2016-09-09 15:03:21 +02:00
antirez
fdf93daf40 fix the fix for the TCP binding.
This commit attempts to fix a problem with PR #3467.
2016-09-09 14:59:48 +02:00
oranagra
639201a989 fix tcp binding when IPv6 is unsupported 2016-09-09 14:59:21 +02:00
antirez
b8f3c5dd36 debug.c: no need to define _GNU_SOURCE, is defined in fmacros.h. 2016-09-09 11:15:10 +02:00
antirez
309b7b3268 crash log - improve code dump with more info and called symbols. 2016-09-09 11:00:19 +02:00
wyx
5d070a76ba fix memory error on module unload 2016-09-09 10:22:57 +08:00
oranagra
ad0426e15b crash log - add hex dump of function code 2016-09-08 14:14:57 +02:00
antirez
c9e62f0056 dict.c benchmark minor improvements. 2016-09-07 15:28:40 +02:00