24 Commits

Author SHA1 Message Date
YoongHM
0ab96fcc63 Fix compilation warnings in Lua and jemalloc dependencies (#7785)
- The argument `u` in for `ar` is ignored (and generates warnings since `D` became the default.
  All it does is avoid updating unchanged objects (shouldn't have any impact on our build)
- Enable `LUA_USE_MKSTEMP` to force the use of `mkstemp()` instead of `tmpname()` (which is dead
  code in redis anyway).
- Remove unused variable `c` in `f_parser()`
- Removed misleadingly indented space in `luaL_loadfile()` and ``addfield()`

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-29 17:10:54 +03:00
YoongHM
539712df55 Fix compilation warning in jemalloc's malloc_vsnprintf (#7789)
Change `val` to `unsigned char` before being tested.
The fix is identical to the one that's been made in upstream jemalloc.

warning is:
src/malloc_io.c: In function ‘malloc_vsnprintf’:
src/malloc_io.c:369:2: warning: case label value exceeds maximum value for type
  369 |  case '?' | 0x80:      \
      |  ^~~~
src/malloc_io.c:581:5: note: in expansion of macro ‘GET_ARG_NUMERIC’
  581 |     GET_ARG_NUMERIC(val, 'p');
      |     ^~~~~~~~~~~~~~~
2020-09-21 17:04:34 +03:00
Oran Agra
86b1b2e138 fix a rare active defrag edge case bug leading to stagnation
There's a rare case which leads to stagnation in the defragger, causing
it to keep scanning the keyspace and do nothing (not moving any
allocation), this happens when all the allocator slabs of a certain bin
have the same % utilization, but the slab from which new allocations are
made have a lower utilization.

this commit fixes it by removing the current slab from the overall
average utilization of the bin, and also eliminate any precision loss in
the utilization calculation and move the decision about the defrag to
reside inside jemalloc.

and also add a test that consistently reproduce this issue.
2020-05-20 16:04:42 +03:00
Oran Agra
0e07566591 Jemalloc: Avoid blocking on background thread lock for stats.
Background threads may run for a long time, especially when the # of dirty pages
is high.  Avoid blocking stats calls because of this (which may cause latency
spikes).

see https://github.com/jemalloc/jemalloc/issues/1502

cherry picked from commit 1a71533511027dbe3f9d989659efeec446915d6b
2019-06-02 15:27:38 +03:00
Oran Agra
36468e1c26 Active defrag fixes for 32bit builds (again)
* overflow in jemalloc fragmentation hint to the defragger
2018-07-11 16:09:00 +03:00
Oran Agra
ad7a05292e add defrag hint support into jemalloc 5 2018-06-27 10:52:39 +03:00
antirez
5b1a603cd3 Generate configure for Jemalloc. 2018-05-24 18:21:13 +02:00
antirez
e3d1823414 Jemalloc upgraded to version 5.0.1. 2018-05-24 17:17:37 +02:00
Oran Agra
0bd6e341f5 Active defrag fixes for 32bit builds
problems fixed:
* failing to read fragmentation information from jemalloc
* overflow in jemalloc fragmentation hint to the defragger
* test suite not triggering eviction after population
2018-05-17 09:52:00 +03:00
antirez
412b1733e2 Revert "Jemalloc updated to 4.4.0."
This reverts commit 36c1acc222d29e6e2dc9fc25362e4faa471111bd.
2017-04-22 13:17:07 +02:00
antirez
d69cc12bb3 Jemalloc updated to 4.4.0.
The original jemalloc source tree was modified to:

1. Remove the configure error that prevents nested builds.
2. Insert the Redis private Jemalloc API in order to allow the
Redis fragmentation function to work.
2017-01-30 09:58:34 +01:00
antirez
e901b0edbf Defrag: activate it only if running modified version of Jemalloc.
This commit also includes minor aesthetic changes like removal of
trailing spaces.
2017-01-10 11:25:39 +01:00
oranagra
c053025a0a active memory defragmentation 2016-12-30 03:37:52 +02:00
antirez
61cceef3a3 Jemalloc configure script fixed to work nested.
Now way to make unmodified Jemalloc configure to work when the jemalloc
source tree is inside a subdirectory of a different git repository.

Problem signaled here:
http://www.canonware.com/pipermail/jemalloc-discuss/2015-October/001174.html
2015-10-07 09:17:06 +02:00
antirez
e7e83e86f2 Jemalloc updated to 4.0.3. 2015-10-06 16:55:37 +02:00
antirez
0b469c7882 Jemalloc: use LG_QUANTUM of 3 for AMD64 and I386.
This gives us a 24 bytes size class which is dict.c dictEntry size, thus
improving the memory efficiency of Redis significantly.
Moreover other non 16 bytes aligned tiny classes are added that further
reduce the fragmentation of the allocator.

Technically speaking LG_QUANTUM should be 4 on i386 / AMD64 because of
SSE types and other 16 bytes types, however we don't use those, and our
jemalloc only targets Redis.

New versions of Jemalloc will have an explicit configure switch in order
to specify the quantum value for a platform without requiring any change
to the Jemalloc source code: we'll switch to this system when available.

This change was originally proposed by Oran Agra (@oranagra) as a change
to the Jemalloc script to generate the size classes define. We ended
doing it differently by changing LG_QUANTUM since it is apparently the
supported Jemalloc method to obtain a 24 bytes size class, moreover it
also provides us other potentially useful size classes.

Related to issue #2510.
2015-07-24 10:20:02 +02:00
antirez
72090a63a7 Jemalloc updated to 3.6.0.
Not a single bug in about 3 months, and our previous version was
too old (3.2.0).
2014-06-20 14:59:20 +02:00
antirez
deea56d5ce Redis/Jemalloc Gitignore were too aggressive.
Redis gitignore was too aggressive since simply broken.

Jemalloc gitignore was too agressive because it is conceived to just
keep the files that allow to generate all the rest in development
environments (so for instance the "configure" file is excluded).
2013-04-18 16:23:15 +02:00
antirez
24c94d3dc5 Jemalloc updated to version 3.2.0. 2012-11-28 18:39:35 +01:00
antirez
39f8289c41 Jemalloc updated to 3.0.0.
Full changelog here:

http://www.canonware.com/cgi-bin/gitweb.cgi?p=jemalloc.git;a=blob_plain;f=ChangeLog;hb=master

Notable improvements from the point of view of Redis:

1) Bugfixing.
2) Support for Valgrind.
3) Support for OSX Lion, FreeBSD.
2012-05-16 11:09:45 +02:00
antirez
ab9d925bb9 jemalloc gitignore updated to jemalloc 2.2.5 2011-11-25 16:42:10 +01:00
jbergstroem
705733c4bd Update to jemalloc 2.2.5 2011-11-23 21:36:25 +01:00
Pieter Noordhuis
dc2a073a65 Ignore jemalloc build artifacts 2011-06-20 11:38:25 +02:00
antirez
f675ff36b4 jemalloc source added 2011-06-20 11:30:06 +02:00