14664 Commits

Author SHA1 Message Date
John Sully
bdff0121eb Make the replica weighting configurable
Former-commit-id: be6a8a7e68acb5cfbe950f13b903e6f7b98c5a39
2021-10-29 17:59:46 +00:00
John Sully
b4eda0ef34 Fix slower performance during replication by better balancing client load on threads servicing a replica
Former-commit-id: 496f91d3f169fcfe6d94c2ea69cee402f8eb60ca
2021-10-22 03:16:33 +00:00
John Sully
ee2677dac7 Merge branch 'COLLAB_RELEASE_6_4' into keydbpro_collab
Former-commit-id: 3d43e7a022250273ee5cfb5755ed1d6ca42b9889
2021-10-21 23:58:32 +00:00
malavan
bbad32ba8a null check for delete override
Former-commit-id: f5f2f5e200a5ff1b0306998624b758d5a4c10825
2021-10-21 23:56:45 +00:00
malavan
7f122b373e null check for delete override
Former-commit-id: a48b4cbdf24bfa5a1d13295c0202d1ed09115374
2021-10-21 23:50:50 +00:00
John Sully
57feba97d2 Do not dereference a nullptr if there are too many files open
Former-commit-id: 4674eb29a261e8b046953398c94354fc3e550c2a
2021-10-21 23:45:46 +00:00
John Sully
3b788bac1e Ensure async rehash completes before we start a new time. Degrad to sync hash if necessary to ensure this
Former-commit-id: 0f830facc7c6bc6668af9bb2e10b6e13a13227aa
2021-10-21 22:46:17 +00:00
John Sully
9955ecd94a Additional change to ensure FLASH storage goes through the multithread path
Former-commit-id: 422ea0723f0b8718f28ef9c1cc4d5f56d374af46
2021-10-20 03:13:36 +00:00
John Sully
2e65b905c1 Permit prefetch for FLASH scenarios in single thread mode
Former-commit-id: 6d0b90ed43cc9d1196903ddbc7d50cd40e439e42
2021-10-15 16:22:42 +00:00
jsully
5e4dec1a16 Merge branch 'multithread_load' into 'keydbpro'
Multithread load

See merge request external-collab/keydb-pro-6!5

Former-commit-id: 20e712244071028b0f75ccad477308efd139261f
2021-10-08 17:55:55 +00:00
John Sully
66af52120c disable key cache during load if necessary
Former-commit-id: 68dcf66909e2138da4902bdec98985f4fcd737cf
2021-10-04 07:35:36 +00:00
John Sully
db351b697a Handle the case where the key cache exceeds maxmemory
Former-commit-id: 01febf902267fec7fe87e6437b0b81fd08b50963
2021-10-04 07:34:05 +00:00
John Sully
e580edabac Don't expand the dictionary if a storage provider is set as we won't use the whole thing
Former-commit-id: 1f07b01144397cec59ec2d94f41c85eceb7248e2
2021-10-04 07:33:38 +00:00
John Sully
4650e051bd Use the concurrentqueue for multithread load instead of mutex and cvs
Former-commit-id: d5a59113dbfedaf7b62a650cff58a2e8ec01826f
2021-10-04 07:33:03 +00:00
John Sully
a4b8d420de Add in the concurrentqueue
Former-commit-id: 319cad462be502b0b7a8c45b634d578b2c1c4e9d
2021-10-02 18:27:21 +00:00
jsully
7cf8433745 Merge branch 'improve_overwrite_performance' into 'keydbpro'
Improve overwrite performance

See merge request external-collab/keydb-pro-6!4

Former-commit-id: 7e59e889f28e30fca409dfb951c31f8cb47effbd
2021-09-28 18:13:12 +00:00
John Sully
617519e021 Revert "Don't zero out potentially long buffers" - It has no benefit above the noise floor and potentially is the cause of a multithread slowdown
This reverts commit 711c08c62491d0670b329fa7760c6938689910f1 [formerly 91e76ab4a00546278100baf6dfe61c13e802b40e].


Former-commit-id: f651cf67d731a10c3a52e30261c8d8b9d291576c
2021-09-28 18:08:15 +00:00
John Sully
d5ee9cb1be Unify job types so everything is processed in order
Former-commit-id: 625aa97e4cf16337e8b052b7a27491a0ab09110f
2021-09-17 17:27:19 +00:00
Malavan Sotheeswaran
5ed9217c15 Merge fix to dict resize during rdb load
Former-commit-id: c398d5f8a027c67acac64bdbfbd01486dde555eb
2021-09-17 16:10:48 +00:00
malavan
deda1bdd44 if statement was actually correct
Former-commit-id: d44b99afdf1ec92f8a36f0c091c37328008d867b
2021-09-14 17:17:08 +00:00
malavan
51fe42b10e improve overwrite key performance
Former-commit-id: 56f9d5528385ea78074a308c6d3987b920d6cc35
2021-09-14 17:06:04 +00:00
John Sully
711c08c624 Don't zero out potentially long buffers
Former-commit-id: 91e76ab4a00546278100baf6dfe61c13e802b40e
2021-09-10 00:40:08 +00:00
John Sully
e62e367ed3 In single thread mode don't batch
Former-commit-id: 7daadae789cdca6f0eb0c3f553737d4f8efc0566
2021-09-10 00:39:47 +00:00
John Sully
df34078713 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/external-collab/keydb-pro-6 into keydbpro_collab
Former-commit-id: 067acbbfd5eb0c2b4515a5d2cc87970870a88acb
2021-09-03 17:20:54 +00:00
malavan
db84f52763 don't use flags that aren't supported by certain versions of OPENSSL
Former-commit-id: ca7342948fb12804905219ae106b4722368e9a2f
2021-09-02 19:06:35 +00:00
malavan
3e20f314ed Remove duplicate code
Former-commit-id: a4c3182afb71a0d7d6d25111c0e0cfe5dcfc6130
2021-09-02 19:06:12 +00:00
John Sully
29518a1399 We need to send keepalives to masters while waiting to prevent disconnects
Former-commit-id: 7cbd6758b1042198c14ca9e8da0f1f7bc05df93d
2021-09-01 04:15:59 +00:00
John Sully
40d5a2db44 Don't be in tracking mode during load as processChangesAsync works outside the normal system
Former-commit-id: 8d31ce6eafea1cea2f9f4ea25e44306efef28fa3
2021-09-01 04:15:28 +00:00
John Sully
ebb256d45b Merge branch 'keydbpro_collab' into multithread_load
Former-commit-id: f36e9efd8e0079650c78e1e8c19c5fa8553c8251
2021-08-11 02:54:02 +00:00
John Sully
5218d1b206 Fix crash loading RDB on start with a storage provider set
Former-commit-id: ace6a44399a15ab48a7ef798dc656dbc1207b58d
2021-08-11 02:53:44 +00:00
John Sully
1ed5c44c28 Fix crash in load with storage provider set
Former-commit-id: 6990818b7ca647819b50ae04224778e6f8f12a1a
2021-08-11 01:19:39 +00:00
John Sully
a560bcc4a2 Merge branch 'keydbpro_collab' into multithread_load
Former-commit-id: 8016c20f1f9a648e658c816e2f6777c5718d5e19
2021-08-09 20:20:34 +00:00
John Sully
6a5f39c48b Fix issue collab #32
Former-commit-id: 0d192cf00ebe9fc0d898404b86e1173476edaefb
2021-07-26 22:30:31 +00:00
John Sully
73560dc7b2 Initialize el so we can detect if it is null
Former-commit-id: ec0f833ea17c668971893aa8f198d22da2e1d289
2021-07-23 19:31:22 +00:00
John Sully
fa91bd9bfe We cannot create time events on threads that don't have an event loop
Former-commit-id: 3812586a41bb7f974b5d9820c8a68ff34ee8aa9a
2021-07-23 16:02:57 +00:00
John Sully
26750791b3 Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: e4e5c6696c6d831924f314a198b266b10d831e14
2021-07-19 20:51:27 +00:00
John Sully
795da380a2 Disable async rehash during load as it interferes with eviction
Former-commit-id: 54b4f39e9d634bf53b04cd94433b051b14323bc6
2021-07-19 18:17:54 +00:00
John Sully
2609b87a95 We need to free in order since the first big async free is likely the largest, so don't set the hipri bit
Former-commit-id: 76a9cefa94e0f446e12a690909cbda15d03ca211
2021-07-19 18:01:39 +00:00
John Sully
a3cf05574f We need to periodically flush the GC or we'll end up blocking with a huge backlog at the end of load
Former-commit-id: 29c0bf79ad1a810e808790de2f7db24f3cc603e8
2021-07-19 16:50:48 +00:00
John Sully
fc7221739c Info command should show how many keys are cached in RAM vs storage provider
Former-commit-id: 08597bee69bc16ca7c3d5ff31020472774c6eec9
2021-07-19 15:11:33 +00:00
John Sully
bd74913b8e StorageCache dtor leaks
Former-commit-id: 0262c4dc76a320141b8a4454df2f6baab4f74ab3
2021-07-19 15:10:48 +00:00
John Sully
48b30ec34a Do not resize replica buffer past the max client limit
Former-commit-id: ba116500ca4fd53e4e40f04fc33981e60bb21ab7
2021-07-18 20:48:08 +00:00
John Sully
29ef9708fa Return the ring buffer to its original size if we temporarily resized it
Former-commit-id: a12ce4a0d105bf7d6ccff95f7dc0044c4676b0a7
2021-07-18 20:45:32 +00:00
John Sully
653f7c96a0 Do not update batch variables when not in a batch
Former-commit-id: ad1e0286cf9b2d9de33c65e8e798a05ead3f7d5a
2021-07-18 20:28:42 +00:00
Malavan Sotheeswaran
4c2612a9fa Merge branch 'add_ci' into 'keydbpro'
test CI do not merge

See merge request keydb-dev/KeyDB-Pro!24

Former-commit-id: 047e32c63a329c9ebc6756db704eda8bbcc1b10a
2021-07-14 23:41:24 +00:00
Malavan Sotheeswaran
8a2b44c242 test CI do not merge
Former-commit-id: ccdf18b1bef07ba076e5f86d74fe1e1f6ae50a8c
2021-07-14 23:41:24 +00:00
John Sully
ca920fed38 Merge branch 'keydbpro' into keydbpro_collab
Former-commit-id: 8eec3e948ffd204bb2d6170ad3ca42fa8a2c6d8b
2021-07-09 05:25:04 +00:00
John Sully
6bb2075d3c Merge branch 'unstable' into keydbpro
Former-commit-id: 81ded8a35daa5100cac7299a7d0b5f43ee7ac74f
2021-07-09 04:41:47 +00:00
John Sully
da2aceabcf Merge tag '6.2.3' into unstable
Former-commit-id: 1895dbb7680fa9aadf6040912e89c733abc8c706
2021-07-09 04:40:31 +00:00
jsully
6637402bf6 Merge branch 'license_status_fix' into 'keydbpro'
license status OK if not checking license

Closes #14

See merge request keydb-dev/KeyDB-Pro!19

Former-commit-id: f66d0970b1eb9be50210a88bf773769a40513521
2021-07-07 18:07:30 +00:00