gvsafronov/futriix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,673 Commits 1 Branch 0 Tags
Commit Graph

14673 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
John Sully
c10f6d9b51 Fix compile error on some machines 
Former-commit-id: 5f491a38cb7eae63fd316c465f317005bbced385
 2021-11-10 17:39:00 +00:00
John Sully
5cd72115d1 Prevent track changes from clearing out the cache and misreporting the key count 
Former-commit-id: d7ceb59f11bc714e40f208cb8c774a5b762ad41e
 2021-11-10 06:40:43 +00:00
John Sully
ce2c49899d Fix snapshot count mismatch due to race 
Former-commit-id: b4175c9bf82cbec047b94403e2a3700f4544b4f1
 2021-11-10 05:29:13 +00:00
John Sully
1ebf62da26 Initial implementation of snapshot fast replication. There are still a few TODOs in progress 
Former-commit-id: 0febdcdab8693af443f350968ed3d8c80106675d
 2021-11-09 19:36:07 +00:00
John Sully
60f08d5f93 Smooth out performance fluctuations caused by variance in the rehash calibration 
Former-commit-id: 09580dedfef09deace7863bf68ba7e0f9edf3eb3
 2021-11-06 15:49:58 +00:00
John Sully
7962e0f5c7 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/external-collab/keydb-pro-6 into keydbpro_collab 
Former-commit-id: eca47e8d19defdc894b9dd09b2500d2e15e5fafa
 2021-11-02 19:39:53 +00:00
John Sully
9e5b7aaaa9 This change fixes out of file descriptor issues with FLASH. 
The change does 3 things:
1. It limits RocksDB to 256 file descriptors instead of unlimited
2. It includes the fd limit in its estimation for total file descriptors needed
3. It raises the system fd limit if possible before we open rocksdb but accounting for the 256 limit we added


Former-commit-id: 1447288209c5e7daf8a1203511fc262500ebe5e1
 2021-11-02 19:31:53 +00:00
malavan
4bb05e4685 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/external-collab/keydb-pro-6 into keydbpro 
Former-commit-id: 5b88af1067d344f121b2d65e34fb40e229722575
 2021-11-02 19:14:47 +00:00
malavan
519ba51343 don't delete db or snapshots on shutdown, still delete storage provider 
Former-commit-id: edb840ce10ea77ce654ba27c9eadbf98bbc13403
 2021-11-02 19:01:37 +00:00
John Sully
bdff0121eb Make the replica weighting configurable 
Former-commit-id: be6a8a7e68acb5cfbe950f13b903e6f7b98c5a39
 2021-10-29 17:59:46 +00:00
John Sully
b4eda0ef34 Fix slower performance during replication by better balancing client load on threads servicing a replica 
Former-commit-id: 496f91d3f169fcfe6d94c2ea69cee402f8eb60ca
 2021-10-22 03:16:33 +00:00
John Sully
ee2677dac7 Merge branch 'COLLAB_RELEASE_6_4' into keydbpro_collab 
Former-commit-id: 3d43e7a022250273ee5cfb5755ed1d6ca42b9889
 2021-10-21 23:58:32 +00:00
malavan
bbad32ba8a null check for delete override 
Former-commit-id: f5f2f5e200a5ff1b0306998624b758d5a4c10825
 2021-10-21 23:56:45 +00:00
malavan
7f122b373e null check for delete override 
Former-commit-id: a48b4cbdf24bfa5a1d13295c0202d1ed09115374
 2021-10-21 23:50:50 +00:00
John Sully
57feba97d2 Do not dereference a nullptr if there are too many files open 
Former-commit-id: 4674eb29a261e8b046953398c94354fc3e550c2a
 2021-10-21 23:45:46 +00:00
John Sully
3b788bac1e Ensure async rehash completes before we start a new time. Degrad to sync hash if necessary to ensure this 
Former-commit-id: 0f830facc7c6bc6668af9bb2e10b6e13a13227aa
 2021-10-21 22:46:17 +00:00
John Sully
9955ecd94a Additional change to ensure FLASH storage goes through the multithread path 
Former-commit-id: 422ea0723f0b8718f28ef9c1cc4d5f56d374af46
 2021-10-20 03:13:36 +00:00
John Sully
2e65b905c1 Permit prefetch for FLASH scenarios in single thread mode 
Former-commit-id: 6d0b90ed43cc9d1196903ddbc7d50cd40e439e42
 2021-10-15 16:22:42 +00:00
jsully
5e4dec1a16 Merge branch 'multithread_load' into 'keydbpro' 
Multithread load

See merge request external-collab/keydb-pro-6!5

Former-commit-id: 20e712244071028b0f75ccad477308efd139261f
 2021-10-08 17:55:55 +00:00
John Sully
66af52120c disable key cache during load if necessary 
Former-commit-id: 68dcf66909e2138da4902bdec98985f4fcd737cf
 2021-10-04 07:35:36 +00:00
John Sully
db351b697a Handle the case where the key cache exceeds maxmemory 
Former-commit-id: 01febf902267fec7fe87e6437b0b81fd08b50963
 2021-10-04 07:34:05 +00:00
John Sully
e580edabac Don't expand the dictionary if a storage provider is set as we won't use the whole thing 
Former-commit-id: 1f07b01144397cec59ec2d94f41c85eceb7248e2
 2021-10-04 07:33:38 +00:00
John Sully
4650e051bd Use the concurrentqueue for multithread load instead of mutex and cvs 
Former-commit-id: d5a59113dbfedaf7b62a650cff58a2e8ec01826f
 2021-10-04 07:33:03 +00:00
John Sully
a4b8d420de Add in the concurrentqueue 
Former-commit-id: 319cad462be502b0b7a8c45b634d578b2c1c4e9d
 2021-10-02 18:27:21 +00:00
jsully
7cf8433745 Merge branch 'improve_overwrite_performance' into 'keydbpro' 
Improve overwrite performance

See merge request external-collab/keydb-pro-6!4

Former-commit-id: 7e59e889f28e30fca409dfb951c31f8cb47effbd
 2021-09-28 18:13:12 +00:00
John Sully
617519e021 Revert "Don't zero out potentially long buffers" - It has no benefit above the noise floor and potentially is the cause of a multithread slowdown 
This reverts commit 711c08c62491d0670b329fa7760c6938689910f1 [formerly 91e76ab4a00546278100baf6dfe61c13e802b40e].


Former-commit-id: f651cf67d731a10c3a52e30261c8d8b9d291576c
 2021-09-28 18:08:15 +00:00
John Sully
d5ee9cb1be Unify job types so everything is processed in order 
Former-commit-id: 625aa97e4cf16337e8b052b7a27491a0ab09110f
 2021-09-17 17:27:19 +00:00
Malavan Sotheeswaran
5ed9217c15 Merge fix to dict resize during rdb load 
Former-commit-id: c398d5f8a027c67acac64bdbfbd01486dde555eb
 2021-09-17 16:10:48 +00:00
malavan
deda1bdd44 if statement was actually correct 
Former-commit-id: d44b99afdf1ec92f8a36f0c091c37328008d867b
 2021-09-14 17:17:08 +00:00
malavan
51fe42b10e improve overwrite key performance 
Former-commit-id: 56f9d5528385ea78074a308c6d3987b920d6cc35
 2021-09-14 17:06:04 +00:00
John Sully
711c08c624 Don't zero out potentially long buffers 
Former-commit-id: 91e76ab4a00546278100baf6dfe61c13e802b40e
 2021-09-10 00:40:08 +00:00
John Sully
e62e367ed3 In single thread mode don't batch 
Former-commit-id: 7daadae789cdca6f0eb0c3f553737d4f8efc0566
 2021-09-10 00:39:47 +00:00
John Sully
df34078713 Merge branch 'keydbpro' of https://gitlab.eqalpha.com/external-collab/keydb-pro-6 into keydbpro_collab 
Former-commit-id: 067acbbfd5eb0c2b4515a5d2cc87970870a88acb
 2021-09-03 17:20:54 +00:00
malavan
db84f52763 don't use flags that aren't supported by certain versions of OPENSSL 
Former-commit-id: ca7342948fb12804905219ae106b4722368e9a2f
 2021-09-02 19:06:35 +00:00
malavan
3e20f314ed Remove duplicate code 
Former-commit-id: a4c3182afb71a0d7d6d25111c0e0cfe5dcfc6130
 2021-09-02 19:06:12 +00:00
John Sully
29518a1399 We need to send keepalives to masters while waiting to prevent disconnects 
Former-commit-id: 7cbd6758b1042198c14ca9e8da0f1f7bc05df93d
 2021-09-01 04:15:59 +00:00
John Sully
40d5a2db44 Don't be in tracking mode during load as processChangesAsync works outside the normal system 
Former-commit-id: 8d31ce6eafea1cea2f9f4ea25e44306efef28fa3
 2021-09-01 04:15:28 +00:00
John Sully
ebb256d45b Merge branch 'keydbpro_collab' into multithread_load 
Former-commit-id: f36e9efd8e0079650c78e1e8c19c5fa8553c8251
 2021-08-11 02:54:02 +00:00
John Sully
5218d1b206 Fix crash loading RDB on start with a storage provider set 
Former-commit-id: ace6a44399a15ab48a7ef798dc656dbc1207b58d
 2021-08-11 02:53:44 +00:00
John Sully
1ed5c44c28 Fix crash in load with storage provider set 
Former-commit-id: 6990818b7ca647819b50ae04224778e6f8f12a1a
 2021-08-11 01:19:39 +00:00
John Sully
a560bcc4a2 Merge branch 'keydbpro_collab' into multithread_load 
Former-commit-id: 8016c20f1f9a648e658c816e2f6777c5718d5e19
 2021-08-09 20:20:34 +00:00
John Sully
6a5f39c48b Fix issue collab #32 
Former-commit-id: 0d192cf00ebe9fc0d898404b86e1173476edaefb
 2021-07-26 22:30:31 +00:00
John Sully
73560dc7b2 Initialize el so we can detect if it is null 
Former-commit-id: ec0f833ea17c668971893aa8f198d22da2e1d289
 2021-07-23 19:31:22 +00:00
John Sully
fa91bd9bfe We cannot create time events on threads that don't have an event loop 
Former-commit-id: 3812586a41bb7f974b5d9820c8a68ff34ee8aa9a
 2021-07-23 16:02:57 +00:00
John Sully
26750791b3 Merge branch 'keydbpro' into keydbpro_collab 
Former-commit-id: e4e5c6696c6d831924f314a198b266b10d831e14
 2021-07-19 20:51:27 +00:00
John Sully
795da380a2 Disable async rehash during load as it interferes with eviction 
Former-commit-id: 54b4f39e9d634bf53b04cd94433b051b14323bc6
 2021-07-19 18:17:54 +00:00
John Sully
2609b87a95 We need to free in order since the first big async free is likely the largest, so don't set the hipri bit 
Former-commit-id: 76a9cefa94e0f446e12a690909cbda15d03ca211
 2021-07-19 18:01:39 +00:00
John Sully
a3cf05574f We need to periodically flush the GC or we'll end up blocking with a huge backlog at the end of load 
Former-commit-id: 29c0bf79ad1a810e808790de2f7db24f3cc603e8
 2021-07-19 16:50:48 +00:00
John Sully
fc7221739c Info command should show how many keys are cached in RAM vs storage provider 
Former-commit-id: 08597bee69bc16ca7c3d5ff31020472774c6eec9
 2021-07-19 15:11:33 +00:00
John Sully
bd74913b8e StorageCache dtor leaks 
Former-commit-id: 0262c4dc76a320141b8a4454df2f6baab4f74ab3
 2021-07-19 15:10:48 +00:00