16 Commits

Author SHA1 Message Date
John Sully
d527bddb70 Improve multithreaded test reliability
Former-commit-id: effa53339cf272ced8f0661bc275628f87be5c73
2021-01-31 04:33:27 +00:00
christianEQ
358debebfa Merge tag 'tags/6.0.10' into redismerge_2021-01-20
Former-commit-id: dadce055f897cee83946c2d3e5cbb76341b94230
2021-01-26 21:43:09 +00:00
Oran Agra
5745b469d3 Improve stability of new CSC eviction test (#8160)
c4fdf09c0 added a test that now fails with valgrind
it fails for two resons:
1) the test samples the used memory and then limits the maxmemory to
   that value, but it turns out this is not atomic and on slow machines
   the background cron process that clean out old query buffers reduces
   the memory so that the setting doesn't cause eviction.
2) the dbsize was tested late, after reading some invalidation messages
   by that time more and more keys got evicted, partially draining the
   db. this is not the focus of this fix (still a known limitation)

(cherry picked from commit a102b21d178453247078b355a47762bfc121ac20)
2021-01-12 16:25:37 +02:00
Oran Agra
f2f57eb44f prevent client tracking from causing feedback loop in performEvictions (#8100)
When client tracking is enabled signalModifiedKey can increase memory usage,
this can cause the loop in performEvictions to keep running since it was measuring
the memory usage impact of signalModifiedKey.

The section that measures the memory impact of the eviction should be just on dbDelete,
excluding keyspace notification, client tracking, and propagation to AOF and replicas.

This resolves part of the problem described in #8069
p.s. fix took 1 minute, test took about 3 hours to write.

(cherry picked from commit c4fdf09c0584a3cee32b92f01b7958c72776aedc)
2021-01-12 16:25:37 +02:00
John Sully
b226b318ff Prevent crashes on shutdown due to lock being held
Former-commit-id: 6a74f524e558100dfb3e54779020cd3407706e08
2020-10-27 04:53:19 +00:00
John Sully
7b70d8c7d0 maxmem tests should be run solo for higher reliability
Former-commit-id: 95f46e60779fca715244745ced7b2ab04bbf9e3a
2020-06-01 16:50:30 -04:00
John Sully
a73e580ad4 Fix memory test failures with multithreading enabled
Former-commit-id: 58035404227a9ef1c3bd92623a333c915d50eab6
2020-05-30 00:06:15 -04:00
Oran Agra
ba809f26d4 make replication tests more stable on slow machines
solving few replication related tests race conditions which fail on slow machines

bugfix in slave buffers test: since the test is executed twice, each time with
a different commands count, the threshold for the delta can't be a constant.
2019-05-05 08:25:01 +03:00
antirez
52e7192e4a Slave removal: remove slave from top-level tests descriptions. 2018-09-11 15:32:28 +02:00
Oran Agra
c8452ab005 Fix unstable tests on slow machines.
Few tests had borderline thresholds that were adjusted.

The slave buffers test had two issues, preventing the slave buffer from growing:
1) the slave didn't necessarily go to sleep on time, or woke up too early,
   now using SIGSTOP to make sure it goes to sleep exactly when we want.
2) the master disconnected the slave on timeout
2018-08-21 11:46:07 +03:00
Oran Agra
d4ae76d1a6 fix slave buffer test suite false positives
it looks like on slow machines we're getting:
[err]: slave buffer are counted correctly in tests/unit/maxmemory.tcl
Expected condition '$slave_buf > 2*1024*1024' to be true (16914 > 2*1024*1024)

this is a result of the slave waking up too early and eating the
slave buffer before the traffic and the test ends.
2018-07-24 11:24:27 +03:00
Oran Agra
bf680b6f8c slave buffers were wasteful and incorrectly counted causing eviction
A) slave buffers didn't count internal fragmentation and sds unused space,
   this caused them to induce eviction although we didn't mean for it.

B) slave buffers were consuming about twice the memory of what they actually needed.
- this was mainly due to sdsMakeRoomFor growing to twice as much as needed each time
  but networking.c not storing more than 16k (partially fixed recently in 237a38737).
- besides it wasn't able to store half of the new string into one buffer and the
  other half into the next (so the above mentioned fix helped mainly for small items).
- lastly, the sds buffers had up to 30% internal fragmentation that was wasted,
  consumed but not used.

C) inefficient performance due to starting from a small string and reallocing many times.

what i changed:
- creating dedicated buffers for reply list, counting their size with zmalloc_size
- when creating a new reply node from, preallocate it to at least 16k.
- when appending a new reply to the buffer, first fill all the unused space of the
  previous node before starting a new one.

other changes:
- expose mem_not_counted_for_evict info field for the benefit of the test suite
- add a test to make sure slave buffers are counted correctly and that they don't cause eviction
2018-07-16 16:43:42 +03:00
Oran Agra
4acb4da1d1 add LFU policies to the test suite, just for coverage 2017-03-15 01:05:15 -07:00
Matt Stancliff
1cedebb799 Remove trailing spaces from tests 2014-09-29 06:49:08 -04:00
antirez
420584a46d Test: small integer sharing depends on maxmemory policy. 2014-07-18 10:55:08 +02:00
antirez
243b783fdc Maxmemory test 2011-07-28 12:32:52 +02:00