10 Commits

Author SHA1 Message Date
Oran Agra
d66739931b Adding real allocator fragmentation to INFO and MEMORY command + active defrag test
other fixes / improvements:
- LUA script memory isn't taken from zmalloc (taken from libc malloc)
  so it can cause high fragmentation ratio to be displayed (which is false)
- there was a problem with "fragmentation" info being calculated from
  RSS and used_memory sampled at different times (now sampling them together)

other details:
- adding a few more allocator info fields to INFO and MEMORY commands
- improve defrag test to measure defrag latency of big keys
- increasing the accuracy of the defrag test (by looking at real grag info)
  this way we can use an even lower threshold and still avoid false positives
- keep the old (total) "fragmentation" field unchanged, but add new ones for spcific things
- add these the MEMORY DOCTOR command
- deduct LUA memory from the rss in case of non jemalloc allocator (one for which we don't "allocator active/used")
- reduce sampling rate of the rss and allocator info
2018-03-12 15:08:52 +02:00
Oran Agra
eb83a56f8e active defrag v2
- big keys are not defragged in one go from within the dict scan
  instead they are scanned in parts after the main dict hash bucket is done.
- add latency monitor sample for defrag
- change default active-defrag-cycle-min to induce lower latency
- make active defrag start a new scan right away if needed, so it's easier
  (for the test suite) to detect when it's done
- make active defrag quick the current cycle after each db / big key
- defrag  some non key long term global allocations
- some refactoring for smaller functions and more reusable code
- during dict rehashing, one scan iteration of the dict, can end up scanning
  one bucket in the smaller dict and many many buckets in the larger dict.
  so waiting for 16 scan iterations before checking the time, may be much too long.
2018-03-12 15:07:43 +02:00
zhaozhao.zz
af08cd716d dict: fix the int problem for defrag 2017-12-05 15:38:03 +01:00
antirez
46f2263a45 Defrag: don't crash when a module value is encountered. 2017-01-12 09:50:40 +01:00
antirez
57c80d1fca Defrag: fix function name typo defarg -> defrag. 2017-01-11 15:38:12 +01:00
antirez
f11a12e915 Defrag: do not crash on empty quicklist. 2017-01-11 15:38:09 +01:00
antirez
da55ed0032 Defrag: fix comments & code to conform to the Redis code base.
Don't go over 80 cols. Start with captial letter, capital letter afer
point, end comment with a point and so forth. No actual code behavior
touched at all.
2017-01-10 11:33:50 +01:00
antirez
e4f25b015c Defrag: activate it only if running modified version of Jemalloc.
This commit also includes minor aesthetic changes like removal of
trailing spaces.
2017-01-10 11:25:39 +01:00
oranagra
763f49243d active defrag improvements 2017-01-02 09:42:32 +02:00
oranagra
53511a429c active memory defragmentation 2016-12-30 03:37:52 +02:00