11882 Commits

Author SHA1 Message Date
John Sully
4568ecf9cd New expire datastructure and algorithm. Allows us to expire in sublinear time
Former-commit-id: ea3bd614b8b88b8de0b114f917fbd0de93557c72
2019-07-23 18:49:31 -04:00
John Sully
be123c44b9 Merge pull request #50 from JohnSully/expire
New Expire Datastructure which is faster and more memory efficient.  This allows us to process expiries in sublinear time.

Former-commit-id: d45edc493d111d4be81f2ce24e3022c8fffb3e2f
2019-07-23 18:40:48 -04:00
John Sully
d58b2575e5 Support TTL stats with the new expire datastructure
Former-commit-id: 271df3dad4f55f20177a8a9a065778f4943835f1
2019-07-23 18:30:10 -04:00
John Sully
d050d20a9f Implement hash table stats and fixup the hash function based on the results
Former-commit-id: 5a193872f8e002c97b7dc2c4bc3bab8e0478765f
2019-07-23 18:30:10 -04:00
John Sully
9420b9c1c2 Add back missing file lost in rebase
Former-commit-id: b5512d77a1299cf6ff960229cd47776b82eaba4b
2019-07-23 18:30:10 -04:00
John Sully
741029118e Add back file erroniously deleted in rebase
Former-commit-id: 42bda8eaba71c99c776100b225606c9aced1d2ba
2019-07-23 18:30:10 -04:00
John Sully
db206557e9 never make last minute changes before commiting
Former-commit-id: 7e5d3f4f160c1c6f91c42b19f95ad17fcb7f1590
2019-07-23 18:30:10 -04:00
John Sully
82bd5f0ecc Fix a few potential assert crashes
Former-commit-id: 5f3920e491a9632d3b84d9af7800c154f2be0809
2019-07-23 18:30:10 -04:00
John Sully
27cb5dba75 New expire datastructure and algorithm. Allows us to expire in sublinear time
Former-commit-id: 3880d2616c882e19169180dc10268564347b0279
2019-07-23 18:30:10 -04:00
Salvatore Sanfilippo
dbee7ca007
Merge pull request #6265 from madolson/dev-unstable-remove-create-object
Removed unnecessary creation of Redis objects
2019-07-23 17:05:10 +02:00
antirez
c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez
9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
Salvatore Sanfilippo
47ce1cebeb
Merge pull request #6267 from soloestoy/opt-claim-trackingtable
Client side caching: do not reclaim tracking table if it's empty
2019-07-23 09:43:41 +02:00
zhaozhao.zz
9012e587ca Client side caching: do not reclaim tracking table if it's empty 2019-07-23 15:25:00 +08:00
Madelyn Olson
7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Madelyn Olson
8fe173452a Removed unecessary creation of Redis objects 2019-07-22 22:52:16 -07:00
Oran Agra
3b6aeea44c Implement module api for aux data in rdb
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
2019-07-22 21:15:33 +03:00
antirez
c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
antirez
09c06698e9 Client side caching: redis-cli ability to enable tracking.
This is extremely useful in order to simulate an high load of requests
about different keys, and force Redis to track a lot of informations
about several clients, to simulate real world workloads.
2019-07-22 18:45:47 +02:00
antirez
436d02dd7b Move signalFlushedDb() into a better place.
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
2019-07-22 12:37:47 +02:00
antirez
7d3992a80a Client side caching: reclaim the tracking table on FLUSHALL. 2019-07-22 12:33:49 +02:00
antirez
842b44dc46 Client side caching: call the invalidation functions always.
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
2019-07-22 12:29:54 +02:00
antirez
f850201c64 Client side caching: don't hash the key if not needed. 2019-07-22 12:10:51 +02:00
antirez
e00442eb13 Clinet side caching: take count of used caching slots. 2019-07-22 11:47:44 +02:00
chendianqiang
004c26ec28 make memory usage consistent of robj with OBJ_ENCODING_INT 2019-07-22 17:45:30 +08:00
Oran Agra
d7d028a7a7 Allow modules to handle RDB loading errors.
This is especially needed in diskless loading, were a short read could have
caused redis to exit. now the module can handle the error and return to the
caller gracefully.

this fixes #5326
2019-07-21 18:19:32 +03:00
John Sully
ff682d7902 Modules must have execute permissions to load 2019-07-19 15:28:31 -04:00
John Sully
2e6f1098e4 Any +x bit is acceptable
Former-commit-id: 156e596f9c7a922bc3361652b74b78bbeab0f2dc
2019-07-19 15:26:17 -04:00
antirez
bc5cb168f5 RDB: fix MODULE_AUX loading by continuing to next opcode.
Thanks to @JohnSully for noticing this problem.
2019-07-19 11:12:46 +02:00
John Sully
b2aa7b0683 Any +x bit is acceptable
Former-commit-id: fc58516cca72fc9db97bc4c388f9fa692d115df4
2019-07-19 01:42:05 -04:00
John Sully
70534846d1 Modules must have execute permissions to load 2019-07-19 01:37:34 -04:00
John Sully
08d067fbcf Modules must have execute permissions to load
Former-commit-id: a4efcd35af52227a22daf7f882e8e14db3f8bf57
2019-07-19 01:31:10 -04:00
antirez
1aa5638b93 RDB: make sure to abort on LZF encoding error.
Former-commit-id: 27fe1658a2019bcd5d880e844bac21ccef8303f2
2019-07-19 01:03:16 -04:00
antirez
3e75e77dcd RDB: handle encoding errors with rdbExitReportCorruptRDB().
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.


Former-commit-id: 47feb2719ca7fd04e7e108ec1af0f777e536bf8a
2019-07-19 01:02:49 -04:00
John Sully
bb9b8ee164 Fix bad merge in SCAN KEYS command
Former-commit-id: c21af6b351328ffbdb1d1e2a7eed44f8f929f8b2
2019-07-19 00:43:23 -04:00
John Sully
15f5c7ccf7 Merge commit '722446510faf0debf0d309708b2ed4fe4d939319' into unstable
Former-commit-id: 821c12c482d20b15dfb5a6eeab52e167997627d8
2019-07-19 00:32:43 -04:00
John Sully
4088de0ed2 Merge commit '677e95e2f5ee903682ce44a7bfd9558a22577926' into unstable
Former-commit-id: 4602f8c391409e9dd59f1fbee6a5ef011b219ca1
2019-07-19 00:01:56 -04:00
John Sully
e642441354 Fix compile errors from merges
Former-commit-id: 27a927fe0011536c6539d7c2a79ccfdaf78cee22
2019-07-18 23:35:51 -04:00
Angus Pearson
7b615e7274 Add char* typeNameCanonicalize(robj*) to remove duplicate code between SCAN and TYPE commands,
and to keep OBJ_* enum to string canonicalization in one place.


Former-commit-id: 3cdc6e8d846e88cf4e250b2643662bde2a9317c5
2019-07-18 23:31:31 -04:00
John Sully
f84ee30d7d Merge commit '0cabe0cfa7290d9b14596ec38e0d0a22df65d1df' into unstable
Former-commit-id: 89b52da387079fb8693dcbced218b9c6244fb00c
2019-07-18 23:28:56 -04:00
Madelyn Olson
a8fc002adc Fixed some spelling issues in ACL codepath including user facing error
Former-commit-id: 50ad880ad55e7761fe2598e09be43947e88740fe
2019-07-18 23:28:36 -04:00
John Sully
e160fbdeb9 Merge commit 'de035c94816722dd923e4aedd852869de79a5185' into unstable
Former-commit-id: 15bed4bbcc962ec1e8ad5f57653534b721df9a70
2019-07-18 23:26:46 -04:00
John Sully
73df5494a8 Merge commit 'bf963253ecfd367b49081a26c1b5c410558aecfc' into unstable
Former-commit-id: 10986ae3a801c13013ec74880dd1f7969c284d01
2019-07-18 23:22:36 -04:00
John Sully
d129e3df0a Merge commit '7fababd44e8b8b2df53cad0906ec5ab7c678dd44' into unstable
Former-commit-id: 2baa76b2ee6e17c2cc44c559ba58b82875cf8f97
2019-07-18 23:21:13 -04:00
John Sully
f4c0c12e71 Merge commit '074d24df1e82b5c794a5da9fb6cda5b77b60b27b' into unstable
Former-commit-id: ee460301fd160cf3fc82e52cf47d4a5f7503d781
2019-07-18 23:20:46 -04:00
John Sully
c2eba1f742 Merge commit 'caf74e507e6535dab58ce6f3db19f8b594b5f7ca' into unstable
Former-commit-id: 8ebe2fce3da506639acb37fedf1a2d3e47669572
2019-07-18 23:18:42 -04:00
antirez
dec2345007 Test: fix slowlog test false positive.
In fast systems "SLOWLOG RESET" is fast enough to don't be logged even
when the time limit is "1" sometimes. Leading to false positives such
as:

[err]: SLOWLOG - can be disabled in tests/unit/slowlog.tcl
Expected '1' to be equal to '0'


Former-commit-id: 8198a697fd4455c88712099f20632e554fb564d4
2019-07-18 23:12:52 -04:00
antirez
a9dbdb3348 Make comment in getClientOutputBufferMemoryUsage() describing the present.
Former-commit-id: 35acae360a4c3c67370b03b4835c51b08194ca28
2019-07-18 23:12:39 -04:00
antirez
b70266f3d3 Fix test false positive introduced by threaded I/O.
Now clients that are ready to be terminated asynchronously are processed
more often in beforeSleep() instead of being processed in serverCron().
This means that the test will not be able to catch the moment the client
was terminated, also note that the 'omem' figure now changes in big
steps, because of the new client output buffers layout.

So we have to change the test range in order to accomodate for that.
Yet the test is useful enough to be worth taking, even if its precision
is reduced by this commit. Probably if we get more problems, a thing
that makes sense is just to check that the limit is < 200k. That's more
than enough actually.


Former-commit-id: 8aaa8b0b116dc86473b6a94bf2ff330dd4163ca1
2019-07-18 23:12:05 -04:00
John Sully
efc6bb9e0c Release version 5
Former-commit-id: 25e18a7de91d4af651f8f22674846cb0238dd5d3
2019-07-18 20:50:40 -04:00