gvsafronov/futriix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
14,672 Commits 1 Branch 0 Tags
Commit Graph

35 Commits

Author SHA1 Message Date
John Sully
9e5b7aaaa9 This change fixes out of file descriptor issues with FLASH. 
The change does 3 things:
1. It limits RocksDB to 256 file descriptors instead of unlimited
2. It includes the fd limit in its estimation for total file descriptors needed
3. It raises the system fd limit if possible before we open rocksdb but accounting for the 256 limit we added


Former-commit-id: 1447288209c5e7daf8a1203511fc262500ebe5e1
 2021-11-02 19:31:53 +00:00
John Sully
fa9f1b4f95 Merge branch 'keydbpro_collab' into multithread_load 
Former-commit-id: 31ddf772cd89344017042a18547ae83a60ce02b9
 2021-06-01 19:18:19 +00:00
John Sully
44db4746cb Merge branch 'keydbpro' into keydbpro_collab 
Former-commit-id: 0f2132505470ca63375f53b54cb6751b604d92f9
 2021-06-01 19:17:54 +00:00
John Sully
e2000a6d3d Implement bulkInsert for rocksdb 
Former-commit-id: e9abae77d6ae18e21d7dac0cbdd563f572ac3d03
 2021-06-01 19:17:40 +00:00
John Sully
a8d177e758 Pause execution during rdbLoadProgressCallback as its too risky to let it run 
Former-commit-id: e70c01cb3e756d1e02ed190b76c73b7b7010c0d3
 2021-05-06 00:42:49 +00:00
John Sully
0adcb82652 Better RocksDB perf tuning 
Former-commit-id: cbe1e0a7c1b5dc49c1adff07b7c32042e673acf4
 2021-04-21 01:19:44 +00:00
John Sully
e94095913d Bump RocksDB version for better perf 
Former-commit-id: ab4ae61b9c54b3c28dc5fd775d0df3d377c4846a
 2021-03-24 20:12:43 +00:00
John Sully
e5a2edd641 Don't complain about unclean shutdowns with an empty database 
Former-commit-id: 99f5c02e87062552eaa2f26e960eb7c9dd977c84
 2021-02-26 06:06:58 +00:00
John Sully
649745924b RocksDB Read Performance Improvements 
Former-commit-id: 80cca4869888e048e10e11f1f20796c482c3e5b3
 2020-08-09 23:36:20 +00:00
John Sully
11a8ab1865 Disable compression it destroys read perf 
Former-commit-id: d3fffc12ae5339886f54c064127497f277393b00
 2020-08-06 22:30:56 +00:00
John Sully
655bc912e6 Add new storage-provider-options config 
Former-commit-id: 195a28beecc6094f959ddafef7fe33f5b55e4047
 2020-07-14 04:24:46 +00:00
John Sully
17661f2382 Implement storage key cache, and writeback memory model 
Former-commit-id: 732bd9c153459f1174475ad67de36c399ddbe359
 2020-07-11 21:23:48 +00:00
John Sully
3b483cb82d Report flash disk usage 
Former-commit-id: 8e4a71ccd31e3c643a6f7b5154d8f2a65d2de76f
 2020-06-07 16:40:03 -04:00
John Sully
d2a29bf343 Fix race when we free an item before we finish writing to disk making it temporarily unavailable 
Former-commit-id: 6139cfee295261d0210f2af6217c2a23eeaf52b8
 2020-06-04 22:18:03 -04:00
John Sully
6a9b345f56 RocksDB compression is too slow 
Former-commit-id: 665a8ff0a0a10ffecc36a8410030abd73879944c
 2020-06-03 22:21:03 -04:00
John Sully
b752ad4ceb Fix failure to count keys in cluster slots when reloading a FLASH database 
Former-commit-id: f6dd863e51f91620f184ff80f08cfe518d29c87f
 2020-04-28 20:48:46 -04:00
John Sully
bb3314c0d9 Fix incorrect prefix comparison 
Former-commit-id: 1ef167546be0678edd457d65a5368e8706fde0a3
 2020-03-23 22:51:46 -04:00
John Sully
0381e728ab Fix incorrect count when loading FLASH 
Former-commit-id: 38ee9c0df144621f7ca750527e9efb16e754ef40
 2020-03-23 19:47:48 -04:00
John Sully
82e1ed482a Prevent bookkeeping keys from leaking through 
Former-commit-id: 1a5af28e115fc123bf250b4a80ac69232bb3add0
 2020-03-23 18:06:47 -04:00
John Sully
ebe0b918bc Cache count of keys in database 
Former-commit-id: 7cd266030ad82b5fddd4668d666adc95e1bed475
 2020-03-17 17:11:41 -04:00
John Sully
003bfb6a04 Tune rocksdb 
Former-commit-id: 95d91b1ff8c8080dfcbf59e51d3eff73d27dcfb5
 2020-02-17 21:50:45 -05:00
John Sully
a086023407 Fix race condition with IStorage batches 
Former-commit-id: a2eabd044c9048662a63ff0f980ed947dc145912
 2020-02-17 19:00:31 -05:00
John Sully
5c657aefcb recommended rocksdb tuning 
Former-commit-id: 2a634365f901968f9758d2d362f211b8e0fddc2a
 2019-12-24 02:22:47 -05:00
John Sully
2f46f18c35 Perf remove extra lookup in rocksdb 
Former-commit-id: 8074472c7a25572a53f1166911920c2cb168c141
 2019-12-23 23:32:04 -05:00
John Sully
0ab5ea7b9b Add missing files 
Former-commit-id: 4eadf8f0923ee70ffa3af329a4d636d508bfad8d
 2019-12-23 19:07:53 -05:00
John Sully
033eda6016 Key count is innacurate after overwrite 
Former-commit-id: 180c5a8e401415cede36548ab8d01f3e4ff8fb7f
 2019-12-22 19:41:36 -05:00
John Sully
b43dda66e7 Fix FLASH eviction issue, the algo relies on correct key count 
Former-commit-id: dc260a7f44fe197b9b0022d044d559be88bf91be
 2019-12-22 18:38:10 -05:00
John Sully
232cc7c566 Fix issue where DELETE always returns 1 when using FLASH 
Former-commit-id: 0574db59fdc740255388ae77d2ece8622898bb49
 2019-12-22 17:30:15 -05:00
John Sully
6a77ceaa4a Evicted flash no longer keeps the key around 
Former-commit-id: 908b303d1a8dd032c092f0bc035361a09c0291a5
 2019-12-20 17:45:07 -05:00
John Sully
ac8450b332 FLASH storage no longer keeps evicted keys in memory 
Former-commit-id: 65c2b47c895d57231bbd3f451ecf1f6d9871f5f7
 2019-12-19 17:30:31 -05:00
John Sully
73a9564d27 Make the storage provider configurable, and show it in the info command 
Former-commit-id: e201ce622ff3c7832d03c6b5759386f0efc23cdf
 2019-12-09 21:07:37 -05:00
John Sully
c55904d9f4 Add support for multiple databases with rocksdb 
Former-commit-id: cfa3b760b00776876134692d42e25a60fcd8dea9
 2019-12-09 20:45:58 -05:00
John Sully
8f8f9b7a46 Flash running well 
Former-commit-id: 9cf393eb9fb69bcc7550cd8b83e1568b3f360310
 2019-12-08 20:06:22 -05:00
John Sully
5f481a206d Initial RocksDB integration 
Former-commit-id: 0de9e5b692c02e779e538ddd0a56d10215e501bb
 2019-12-06 20:39:32 -05:00
John Sully
0f5d2ccfe5 Initial standup of RocksDB dependency 
Former-commit-id: 19bb40ffdc794cf9cafa9f4132a0facf1e074101
 2019-12-06 17:33:18 -05:00