157 Commits

Author SHA1 Message Date
antirez
6249fdc544 MEMORY DOCTOR initial implementation. 2016-09-16 16:36:53 +02:00
antirez
303d981cab Provide percentage of memory peak used info. 2016-09-16 10:43:19 +02:00
antirez
9dce1c8dcd Memory related subcommands of DEBUG moved to MEMORY. 2016-09-16 10:26:23 +02:00
antirez
2decf8957e Group MEMORY command related APIs together in the source code. 2016-09-16 10:12:04 +02:00
antirez
28dc0a22a7 objectComputeSize(): skiplist nodes have different sizes.
The size of the node depends on the node level, however it is not stored
into the node itself, is an implicit information, so we use
zmalloc_size() in order to compute the sorted set size.
2016-09-15 17:43:13 +02:00
antirez
340c5546a1 MEMORY command: HELP + dataset percentage (like in INFO). 2016-09-15 17:33:16 +02:00
antirez
3819a712ac MEMORY USAGE: SAMPLES option added + fixes to size computation.
The new SAMPLES option is added, defaulting to 5, and with 0 being a
special value to scan the whole set of elements.

Fixes to the object size computation were made since the original PR
assumed data structures still contaning robj structures, while now after
the lazyfree changes, are all SDS strings.
2016-09-15 15:25:14 +02:00
antirez
fa9588a275 C struct memoh renamed redisMemOverhead. API prototypes added. 2016-09-15 09:44:07 +02:00
antirez
a1045e9651 MEMORY OVERHEAD refactored into a generic API. 2016-09-15 09:37:55 +02:00
antirez
f705ad52db MEMORY OVERHEAD implemented (using Oran Agra initial implementation).
This code was extracted from @oranagra PR #3223 and modified in order
to provide only certain amounts of information compared to the original
code. It was also moved from DEBUG to the newly introduced MEMORY
command. Thanks to Oran for the implementation and the PR.

It implements detailed memory usage stats that can be useful in both
provisioning and troubleshooting memory usage in Redis.
2016-09-13 17:39:25 +02:00
antirez
cf49be57a5 objectComputeSize(): estimate collections sampling N elements.
For most tasks, we need the memory estimation to be O(1) by default.
This commit also implements an initial MEMORY command.
Note that objectComputeSize() takes the number of samples to check as
argument, so MEMORY should be able to get the sample size as option
to make precision VS CPU tradeoff tunable.

Related to: PR #3223.
2016-09-13 10:28:23 +02:00
oranagra
41899d26ef Adding objectComputeSize() function. 2016-09-12 16:36:59 +02:00
antirez
ef404d604b Fix maxmemory shared integer check bug introduced with LFU. 2016-07-21 11:14:18 +02:00
antirez
368aa4da99 LFU: Initial naive eviction cycle.
It is possible to get better results by using the pool like in the LRU
case. Also from tests during the morning I believe the current
implementation has issues in the frequency decay function that should
decrease the counter at periodic intervals.
2016-07-18 13:50:19 +02:00
antirez
1e7f752571 LFU: Redis object level implementation.
Implementation of LFU maxmemory policy for anything related to Redis
objects. Still no actual eviction implemented.
2016-07-15 12:12:58 +02:00
antirez
c50290ee69 Expire and LRU related code moved into different files. 2016-07-06 15:24:06 +02:00
antirez
6ad67c330f getLongLongFromObject: use string2ll() instead of strict_strtoll().
strict_strtoll() has a bug that reports the empty string as ok and
parses it as zero.

Apparently nobody ever replaced this old call with the faster/saner
string2ll() which is used otherwise in the rest of the Redis core.

This commit close #3333.
2016-07-06 11:43:33 +02:00
Salvatore Sanfilippo
cea51549b8 Merge pull request #3336 from yossigo/create_string_from_string
Add RedisModule_CreateStringFromString().
2016-06-23 16:16:28 +02:00
tielei
46312a6cae A string with 21 chars is not representable as a 64-bit integer. 2016-06-23 19:53:56 +08:00
Yossi Gottlieb
c7ec9f280f Add RedisModule_CreateStringFromString(). 2016-06-22 21:02:40 +03:00
Yossi Gottlieb
83a4257f72 Use const in Redis Module API where possible. 2016-06-20 23:08:06 +03:00
Salvatore Sanfilippo
5ce6c56f98 Merge pull request #3283 from ideal/unstable
fix mistake in comment in object.c
2016-06-15 15:53:19 +02:00
antirez
9deb98167b Modules: support for modules native data types. 2016-06-03 18:14:04 +02:00
ideal
641f8c1c2c fix mistake comment in object.c 2016-05-30 16:57:36 +08:00
antirez
5eb85d8fe7 Lazyfree: Hash converted to use plain SDS WIP 4. 2015-10-01 13:02:25 +02:00
antirez
5ae12b50d1 Lazyfree: Hash converted to use plain SDS WIP 2. 2015-10-01 13:02:24 +02:00
antirez
5fd01e29d9 Lazyfree: Hash converted to use plain SDS WIP 1. 2015-10-01 13:02:24 +02:00
antirez
062bf5ce19 Lazyfree: Convert Sets to use plains SDS (several commits squashed). 2015-10-01 13:02:24 +02:00
antirez
ca8d13bd90 Lazyfree: a first implementation of non blocking DEL. 2015-10-01 13:00:19 +02:00
antirez
c15cac0d77 RDMF: More consistent define names. 2015-07-27 14:37:58 +02:00
antirez
8a893fa4cf RDMF: REDIS_OK REDIS_ERR -> C_OK C_ERR. 2015-07-26 23:17:55 +02:00
antirez
58844a7bfe RDMF: redisAssert -> serverAssert. 2015-07-26 15:29:53 +02:00
antirez
62b27ebc2a RDMF: OBJ_ macros for object related stuff. 2015-07-26 15:28:00 +02:00
antirez
fa26d3dd63 RDMF: use client instead of redisClient, like Disque. 2015-07-26 15:20:52 +02:00
antirez
6a424b5e36 RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
Oran Agra
c498bcbc7f sds size classes - memory optimization 2015-07-14 17:17:06 +02:00
antirez
804534462f CONFIG refactoring: configEnum abstraction.
Still many things to convert inside config.c in the next commits.
Some const safety in String objects creation and addReply() family
functions.
2015-03-11 17:00:13 +01:00
antirez
b8efe8b1c8 String: use new sdigits10() API in stringObjectLen().
Should be much faster, and regardless, the code is more obvious now
compared to generating a string just to get the return value of the
ll2stirng() function.
2015-02-27 16:09:17 +01:00
Matt Stancliff
e24ef16446 Add quicklist implementation
This replaces individual ziplist vs. linkedlist representations
for Redis list operations.

Big thanks for all the reviews and feedback from everybody in
https://github.com/antirez/redis/pull/2143
2015-01-02 11:16:08 -05:00
antirez
59d12da5d4 Handle infinite explicitly in createStringObjectFromLongLong(). 2014-12-03 10:37:01 +01:00
antirez
b81104a288 Use exp format and more precision output for ZSCAN.
Ref: issue #2175
2014-12-02 18:20:09 +01:00
antirez
f7b31f2cde tryObjectEncoding(): use shared objects with maxmemory and non-LRU policy.
In order to make sure every object has its own private LRU counter, when
maxmemory is enabled tryObjectEncoding() does not use the pool of shared
integers. However when the policy is not LRU-based, it does not make
sense to do so, and it is much better to save memory using shared
integers.
2014-07-18 10:09:51 +02:00
michael-grunder
0b57f71c74 Fix OBJECT arity
Previously, the command definition for the OBJECT command specified
a minimum of two args (and that it was variadic), which meant that
if you sent this:

OBJECT foo

When cluster was enabled, it would result in an assertion/SEGFAULT
when Redis was attempting to extract keys.

It appears that OBJECT is not variadic, and only ever takes 3 args.

https://gist.github.com/michael-grunder/25960ce1508396d0d36a
2014-07-11 13:03:26 -07:00
Matt Stancliff
2548b2d254 Fix lack of strtold under Cygwin
Renaming strtold to strtod then casting
the result is the standard way of dealing with
no strtold in Cygwin.
2014-05-12 11:11:09 -04:00
antirez
24cce99135 REDIS_ENCODING_EMBSTR_SIZE_LIMIT set to 39.
The new value is the limit for the robj + SDS header + string +
null-term to stay inside the 64 bytes Jemalloc arena in 64 bits
systems.
2014-05-07 17:05:09 +02:00
antirez
11cc33d48a tryObjectEncoding() refactoring.
We also avoid to re-create an object that is already in EMBSTR encoding.
2014-04-04 17:25:35 +02:00
antirez
6028bc5501 String value unsharing refactored into proper function.
All the Redis functions that need to modify the string value of a key in
a destructive way (APPEND, SETBIT, SETRANGE, ...) require to make the
object unshared (if refcount > 1) and encoded in raw format (if encoding
is not already REDIS_ENCODING_RAW).

This was cut & pasted many times in multiple places of the code. This
commit puts the small logic needed into a function called
dbUnshareStringValue().
2014-03-30 18:32:17 +02:00
Matt Stancliff
683edf1df2 Use LRU_CLOCK() instead of function getLRUClock()
lookupKey() uses LRU_CLOCK(), so it seems object
creation should use LRU_CLOCK() too.
2014-03-24 14:39:26 -04:00
antirez
7b50be26a9 Use getLRUClock() instead of server.lruclock to create objects.
Thanks to Matt Stancliff for noticing this error. It was in the original
code but somehow I managed to remove the change from the commit...
2014-03-21 09:08:20 +01:00
antirez
f73fcf5a18 Fix OBJECT IDLETIME return value converting to seconds.
estimateObjectIdleTime() returns a value in milliseconds now, so we need
to scale the output of OBJECT IDLETIME to seconds.
2014-03-20 11:55:18 +01:00