69 Commits

Author SHA1 Message Date
Oran Agra
9b2a42636f change SORT and SPOP to use lookupKeyWrite rather than lookupKeyRead
like in SUNIONSTORE etc, commands that perform writes are expected to open
all keys, even input keys, with lookupKeyWrite
2019-03-20 17:06:22 +02:00
zhaozhao.zz
10ffb2b722 try lazyfree temp set in SUNION & SDIFF 2019-03-07 22:08:04 +08:00
antirez
aae7e1bff0 Better distribution for set get-random-element operations. 2019-02-18 18:27:18 +01:00
antirez
c8304b099d RESP3: most null replies converted. 2019-01-09 17:00:29 +01:00
antirez
e95f9b664c RESP3: Use new aggregate reply API in t_set.c. 2019-01-09 17:00:29 +01:00
antirez
ec67ccc13c Tranfer -> transfer typo fixed. 2018-07-31 16:41:33 +02:00
zhaozhao.zz
b880a352d6 refactor dbOverwrite to make lazyfree work 2018-07-31 12:07:57 +08:00
zhaozhao.zz
3e4450e2a8 set: fix the int problem for qsort 2017-12-05 17:42:19 +08:00
zhaozhao.zz
0d56dd726c set: fix the int problem for SPOP & SRANDMEMBER 2017-12-05 17:19:19 +08:00
oranagra
4448185b52 Optimize repeated keyname hashing.
(Change cherry-picked and modified by @antirez from a larger commit
provided by @oranagra in PR #3223).
2016-09-12 13:19:05 +02:00
Yossi Gottlieb
83a4257f72 Use const in Redis Module API where possible. 2016-06-20 23:08:06 +03:00
oranagra
26e236b2a6 minor fixes - mainly signalModifiedKey, and GEORADIUS 2016-05-09 12:05:33 +03:00
antirez
062bf5ce19 Lazyfree: Convert Sets to use plains SDS (several commits squashed). 2015-10-01 13:02:24 +02:00
antirez
c15cac0d77 RDMF: More consistent define names. 2015-07-27 14:37:58 +02:00
antirez
8a893fa4cf RDMF: REDIS_OK REDIS_ERR -> C_OK C_ERR. 2015-07-26 23:17:55 +02:00
antirez
58844a7bfe RDMF: redisAssert -> serverAssert. 2015-07-26 15:29:53 +02:00
antirez
62b27ebc2a RDMF: OBJ_ macros for object related stuff. 2015-07-26 15:28:00 +02:00
antirez
fa26d3dd63 RDMF: use client instead of redisClient, like Disque. 2015-07-26 15:20:52 +02:00
antirez
6a424b5e36 RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
antirez
5f54f5b196 Rewrite smoveCommand test with ternary operator 2015-05-15 17:38:48 +02:00
Glenn Nethercutt
d8390522cb uphold the smove contract to return 0 when the element is not a member of the source set, even if source=dest 2015-04-17 09:27:54 -04:00
antirez
f9811b205d Fix setTypeNext call assuming NULL can be passed.
Segfault introduced during a refactoring / warning suppression a few
commits away. This particular call assumed that it is safe to pass NULL
to the object pointer argument when we are sure the set has a given
encoding. This can't be assumed and is now guaranteed to segfault
because of the new API of setTypeNext().
2015-03-31 15:26:35 +02:00
antirez
0a18237a52 Set: setType*() API more defensive initializing both values.
This change fixes several warnings compiling at -O3 level with GCC
4.8.2, and at the same time, in case of misuse of the API, we have the
pointer initialize to NULL or the integer initialized to the value
-123456789 which is easy to spot by naked eye.
2015-03-30 12:24:57 +02:00
antirez
38000e913a SPOP with count: fix replication for code path #3. 2015-02-11 10:52:28 +01:00
antirez
87a6696d89 SPOP: reimplemented for speed and better distribution.
The old version of SPOP with "count" argument used an API call of dict.c
which was actually designed for a different goal, and was not capable of
good distribution. We follow a different three-cases approach optimized
for different ratiion between sets and requested number of elements.

The implementation is simpler and allowed the removal of a large amount
of code.
2015-02-11 10:52:28 +01:00
antirez
9bffc3f0c6 Change alsoPropagate() behavior to make it more usable.
Now the API automatically creates its argv copy and increment ref count
of passed objects.
2015-02-11 10:52:27 +01:00
antirez
54def17f37 SPOP with count: initial fixes to the implementation.
Severan problems are addressed but still a few missing.
Since replication of this command was more complex than others since it
needs to replicate multiple SREM commands, an old API able to do this
was reused (it was taken inside the implementation since it was pretty
obvious soon or later that would be useful). The API was improved a bit
so that now a command may opt-out for the standard command replication
when the server.dirty counter is incremented, in order to "manually"
replicate what it wants.
2015-02-11 10:52:27 +01:00
Alon Diamant
3167c449e4 Following @mattsta's friendly review:
1. memory leak in t_set.c has been fixed
  2. end-of-line spaces has been removed (from all over the place)
  3. for loops have been ordered up to match existing Redis style (less weird)
  4. comments format has been fixed (added * in the beggining of every comment line)
2014-12-21 16:13:45 +02:00
Alon Diamant
4f5e9325cd Fix: case when SPOP with count>MAXINT, setTypeRandomElements() will get negative count argument due to signed/unsigned mismatch.
setTypeRandomElements() now returns unsigned long, and also uses unsigned long for anything related to count of members.
spopWithCountCommand() now uses unsigned long elements_returned instead of int, for values returned from setTypeRandomElements()
2014-12-18 14:38:20 +02:00
Alon Diamant
582c15db57 Added <count> parameter to SPOP:
spopCommand() now runs spopWithCountCommand() in case the <count> param is found.
Added intsetRandomMembers() to Intset: Copies N random members from the set into inputted 'values' array. Uses either the Knuth or Floyd sample algos depending on ratio count/size.
Added setTypeRandomElements() to SET type: Returns a number of random elements from a non empty set. This is a version of setTypeRandomElement() that is modified in order to return multiple entries, using dictGetRandomKeys() and intsetRandomMembers().
Added tests for SPOP with <count>: unit/type/set, unit/scripting, integration/aof
--
Cleaned up code a bit to match with required Redis coding style
2014-12-14 12:25:42 +02:00
antirez
1c94889182 No more trailing spaces in Redis source code. 2014-06-26 18:48:40 +02:00
antirez
69a3303c18 SDIFF iterator misuse fixed in diff algorithm #1.
The bug could be easily triggered by:

    SADD foo a b c 1 2 3 4 5 6
    SDIFF foo foo

When the key was the same in two sets, an unsafe iterator was used to
check existence of elements in the same set we were iterating.
Usually this would just result into a wrong output, however with the
dict.c API misuse protection we have in place, the result was actually
an assertion failed that was triggered by the CI test, while creating
random datasets for the "MASTER and SLAVE consistency" test.
2013-12-13 11:34:21 +01:00
antirez
1a1eb8bc8d SCAN code refactored to parse cursor first.
The previous implementation of SCAN parsed the cursor in the generic
function implementing SCAN, SSCAN, HSCAN and ZSCAN.

The actual higher-level command implementation only checked for empty
keys and return ASAP in that case. The result was that inverting the
arguments of, for instance, SSCAN for example and write:

    SSCAN 0 key

Instead of

    SSCAN key 0

Resulted into no error, since 0 is a non-existing key name very likely.
Just the iterator returned no elements at all.

In order to fix this issue the code was refactored to extract the
function to parse the cursor and return the error. Every higher level
command implementation now parses the cursor and later checks if the key
exist or not.
2013-11-05 15:47:50 +01:00
antirez
e96ffac563 SSCAN implemented. 2013-10-28 11:17:32 +01:00
antirez
aa32f92338 Introduction of a new string encoding: EMBSTR
Previously two string encodings were used for string objects:

1) REDIS_ENCODING_RAW: a string object with obj->ptr pointing to an sds
stirng.

2) REDIS_ENCODING_INT: a string object where the obj->ptr void pointer
is casted to a long.

This commit introduces a experimental new encoding called
REDIS_ENCODING_EMBSTR that implements an object represented by an sds
string that is not modifiable but allocated in the same memory chunk as
the robj structure itself.

The chunk looks like the following:

+--------------+-----------+------------+--------+----+
| robj data... | robj->ptr | sds header | string | \0 |
+--------------+-----+-----+------------+--------+----+
                     |                       ^
                     +-----------------------+

The robj->ptr points to the contiguous sds string data, so the object
can be manipulated with the same functions used to manipulate plan
string objects, however we need just on malloc and one free in order to
allocate or release this kind of objects. Moreover it has better cache
locality.

This new allocation strategy should benefit both the memory usage and
the performances. A performance gain between 60 and 70% was observed
during micro-benchmarks, however there is more work to do to evaluate
the performance impact and the memory usage behavior.
2013-07-22 10:31:38 +02:00
Rock Li
682eb6a6cd retval doesn't initalized
If each if conditions are all fail, variable retval will under uninitlized
2013-02-05 15:56:04 +08:00
Gengliang Wang
c0ed12e37f Fix a bug in srandmemberWithCountCommand()
In CASE 2, the call sunionDiffGenericCommand will involve the string "srandmember" 
> sadd foo one
(integer 1)
> sadd srandmember two
(integer 2)
> srandmember foo 3
1)"one"
2)"two"
2013-02-04 14:01:08 +08:00
antirez
3d30bb48c0 Generate del events when S*STORE commands delete the destination key. 2013-01-29 13:43:13 +01:00
antirez
f28d386cc5 Keyspace events: it is now possible to select subclasses of events.
When keyspace events are enabled, the overhead is not sever but
noticeable, so this commit introduces the ability to select subclasses
of events in order to avoid to generate events the user is not
interested in.

The events can be selected using redis.conf or CONFIG SET / GET.
2013-01-28 13:15:12 +01:00
antirez
f2d1105618 Keyspace events added for more commands. 2013-01-28 13:14:56 +01:00
guiquanz
df7a5b7157 Fixed many typos. 2013-01-19 10:59:44 +01:00
antirez
68e2d2ce07 SDIFF is now able to select between two algorithms for speed.
SDIFF used an algorithm that was O(N) where N is the total number
of elements of all the sets involved in the operation.

The algorithm worked like that:

ALGORITHM 1:

1) For the first set, add all the members to an auxiliary set.
2) For all the other sets, remove all the members of the set from the
auxiliary set.

So it is an O(N) algorithm where N is the total number of elements in
all the sets involved in the diff operation.

Cristobal Viedma suggested to modify the algorithm to the following:

ALGORITHM 2:

1) Iterate all the elements of the first set.
2) For every element, check if the element also exists in all the other
remaining sets.
3) Add the element to the auxiliary set only if it does not exist in any
of the other sets.

The complexity of this algorithm on the worst case is O(N*M) where N is
the size of the first set and M the total number of sets involved in the
operation.

However when there are elements in common, with this algorithm we stop
the computation for a given element as long as we find a duplicated
element into another set.

I (antirez) added an additional step to algorithm 2 to make it faster,
that is to sort the set to subtract from the biggest to the
smallest, so that it is more likely to find a duplicate in a larger sets
that are checked before the smaller ones.

WHAT IS BETTER?

None of course, for instance if the first set is much larger than the
other sets the second algorithm does a lot more work compared to the
first algorithm.

Similarly if the first set is much smaller than the other sets, the
original algorithm will less work.

So this commit makes Redis able to guess the number of operations
required by each algorithm, and select the best at runtime according
to the input received.

However, since the second algorithm has better constant times and can do
less work if there are duplicated elements, an advantage is given to the
second algorithm.
2012-11-30 16:36:42 +01:00
antirez
a32d1ddff6 BSD license added to every C source and header file. 2012-11-08 18:31:32 +01:00
antirez
43a30fce99 SRANDMEMBER <count> leak fixed.
For "CASE 4" (see code) we need to free the element if it's already in
the result dictionary and adding it failed.
2012-09-21 11:55:32 +02:00
antirez
8b697b838b Added the SRANDMEMBER key <count> variant.
SRANDMEMBER called with just the key argument can just return a single
random element from a Redis Set. However many users need to return
multiple unique elements from a Set, this is not a trivial problem to
handle in the client side, and for truly good performance a C
implementation was required.

After many requests for this feature it was finally implemented.

The problem implementing this command is the strategy to follow when
the number of elements the user asks for is near to the number of
elements that are already inside the set. In this case asking random
elements to the dictionary API, and trying to add it to a temporary set,
may result into an extremely poor performance, as most add operations
will be wasted on duplicated elements.

For this reason this implementation uses a different strategy in this
case: the Set is copied, and random elements are returned to reach the
specified count.

The code actually uses 4 different algorithms optimized for the
different cases.

If the count is negative, the command changes behavior and allows for
duplicated elements in the returned subset.
2012-09-21 11:55:28 +02:00
Erik Dubbelboer
1c82a561f1 Fixed some spelling errors in the comments 2012-04-07 14:40:29 +02:00
antirez
d6c3b3004e dict.c API names modified to be more coincise and consistent. 2011-11-08 17:07:55 +01:00
antirez
357f49db2f replaced redisAssert() with redisAssertWithInfo() in a shitload of places. 2011-10-04 18:43:03 +02:00
antirez
7192cd5cb7 Fix for bug 561 and other related problems 2011-06-20 17:19:36 +02:00
antirez
86ee8a8f25 Fix for the variadic version of SREM. Regression test added. 2011-05-31 20:14:29 +02:00