a18fcdb371
Fixes the unit test for hashtable random fairness intermittent failures when running with the `--accurate` flag. https://github.com/valkey-io/valkey/actions/runs/12969591890/job/36173815884#step:10:105 The test case picks a random element out of 400, repeated 1M times, and then checks that 60% of the elements are picked within 3 standard deviations from the number of times they're expected to be picked. In this test run (with `--accurate`), the expected number is 2500 and the standard deviation is 50, which is only 2% of the expected value. This makes the check too strict and makes the test flaky. As an alternative, we allow 80% of the elements to be picked within 10% of the expected number. With this alternative condition, we can also raise the check for the non-edge case from 60% to 80% of the elements to be within 3 standard deviations. (With fewer repetitions, 3 standard deviations is greater than 10% of the expected value, so this new condition only affects the `--accurate` test run.) Additional change: Set a random seed to the hash function in the test suite. Until now, we only seeded the random number generator. Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
Introduction
Valkey uses a very simple C testing framework, built up over time but now based loosely off of Unity.
All test files being test_ in the unit directory.
A single test file can have multiple individual tests, and they must be of the form int test_<test_name>(int argc, char *argv[], int flags) {, where test_name is the name of the test.
The test name must be globally unique.
A test will be marked as successful if returns 0, and will be marked failed in all other cases.
The test framework also parses several flags passed in, and sets them based on the arguments to the tests.
Tests flags:
- UNIT_TEST_ACCURATE: Corresponds to the --accurate flag. This flag indicates the test should use extra computation to more accurately validate the tests.
- UNIT_TEST_LARGE_MEMORY: Corresponds to the --large-memory flag. This flag indicates whether or not tests should use more than 100mb of memory.
- UNIT_TEST_SINGLE: Corresponds to the --single flag. This flag indicates that a single test is being executed.
- UNIT_TEST_VALGRIND: Corresponds to the --valgrind flag. This flag is just a hint passed to the test to indicate that we are running it under valgrind.
Tests are allowed to be passed in additional arbitrary argv/argc, which they can access from the argc and argv arguments of the test.
Assertions
There are a few built in assertions that can be used, that will automatically return from the current function and return the correct error code. Assertions are also useful as they will print out the line number that they failed on.
TEST_ASSERT(condition): Will evaluate the condition, and if it fails it will return 1 and print out the condition that failed.TEST_ASSERT_MESSAGE(message, condition): Will evaluate the condition, and if it fails it will return 1 and print out the provided message.
Other utilities
If you would like to print out additional data, use the TEST_PRINT_INFO(info, ...) option, which has arguments similar to printf.
This macro will also print out the function the code was executed from in addition to the line it was printed from.
Example test
int test_example(int argc, char *argv[], int flags) {
TEST_ASSERT(5 == 5);
TEST_ASSERT_MESSAGE("This should pass", 6 == 6);
return 0;
}
Running tests
Tests can be run by executing:
make valkey-unit-tests
./valkey-unit-tests
Running a single unit test file
./valkey-unit-tests --single test_crc64.c
Will just run the test_crc64.c file.