69 Commits

Author SHA1 Message Date
Guillem Jover
0731a0509a build: Add support for linking against a system libhiredis/libhiredis_ssl
Add a new USE_SYSTEM_HIREDIS make variable to select whether to link
against the system libhiredis (and libhiredis_ssl if BUILD_TLS is
enabled).

Move the sdscompat.h header from the vendored hiredis directory to src/,
as this file is not and has never been part of the upstream hiredis
project, it got added in commit bffbbeaa9a1a6b8e81384297272cb0631502e8fd
in redis itself.
2024-03-25 21:01:05 -04:00
Guillem Jover
ae130d1a7f build: Add support for linking against a system libjemalloc
Add a new USE_SYSTEM_JEMALLOC make variable to select whether to link
against the system libjemalloc.
2024-03-25 21:01:05 -04:00
Malavan Sotheeswaran
de3c218a02 add release notes up to 6.3.4 2023-10-29 17:39:55 -07:00
Malavan Sotheeswaran
b50f0cc21f
flash package update (#528)
* ship packages with flash option

* deb12 fix

Co-authored-by: benschermel <bschermel@snapchat.com>
2022-12-29 15:09:53 -05:00
Malavan Sotheeswaran
f5f1bd7605
Merge main with oss release sep29 2022 (#521)
* need to include stdint for uintptr_t

* need to include stdint for uintptr_t

* use atomic_load for g_pserver->mstime

* use atomic_load for g_pserver->mstime

* Integrate readwritelock with Pro Code

* Integrate readwritelock with Pro Code

* Defensive asserts for RWLock

* Defensive asserts for RWLock

* Save and restore master info in rdb to allow active replica partial sync (#371)

* save replid for all masters in rdb

* expanded rdbSaveInfo to hold multiple master structs

* parse repl-masters from rdb

* recover replid info from rdb in active replica mode, attempt partial sync

* save offset from rdb into correct variable

* don't change replid based on master in active rep

* save and load psync info from correct fields

* Save and restore master info in rdb to allow active replica partial sync (#371)

* save replid for all masters in rdb

* expanded rdbSaveInfo to hold multiple master structs

* parse repl-masters from rdb

* recover replid info from rdb in active replica mode, attempt partial sync

* save offset from rdb into correct variable

* don't change replid based on master in active rep

* save and load psync info from correct fields

* placement new instead of memcpy

* placement new instead of memcpy

* Remove asserts, RW lock can go below zero in cases of aeAcquireLock

* Remove asserts, RW lock can go below zero in cases of aeAcquireLock

* Inclusive language

* Inclusive language

* update packaging for OS merge

* update packaging for OS merge

* modify dockerfile to build within image

* modify dockerfile to build within image

* Make active client balancing a configurable option

* Make active client balancing a configurable option

* With TLS throttle accepts if server is under heavy load - do not change non TLS behavior

* With TLS throttle accepts if server is under heavy load - do not change non TLS behavior

* Only run the tls-name-validation test if --tls is passed into runtest

* Only run the tls-name-validation test if --tls is passed into runtest

* Fix KeyDB not building with TLS < 1.1.1

* Fix KeyDB not building with TLS < 1.1.1

* update changelog to use replica as terminology

* update changelog to use replica as terminology

* update copyright

* update copyright

* update deb copyright

* update deb copyright

* call aeThreadOnline() earlier

* call aeThreadOnline() earlier

* Removed mergeReplicationId

* Removed mergeReplicationId

* acceptTLS is threadsafe like the non TLS version

* acceptTLS is threadsafe like the non TLS version

* setup Machamp ci

* setup Machamp ci

* make build_test.sh executable

* make build_test.sh executable

* PSYNC production fixes

* PSYNC production fixes

* fix the Machamp build

* fix the Machamp build

* break into tests into steps

* break into tests into steps

* Added multimaster test

* Added multimaster test

* Update ci.yml

Change min tested version to 18.04

* Update ci.yml

Change min tested version to 18.04

* fork lock for all threads, use fastlock for readwritelock

* fork lock for all threads, use fastlock for readwritelock

* hide forklock object in ae

* hide forklock object in ae

* only need to include readwritelock in ae

* only need to include readwritelock in ae

* time thread lock uses fastlock instead of std::mutex

* time thread lock uses fastlock instead of std::mutex

* set thread as offline when waiting for time thread lock

* set thread as offline when waiting for time thread lock

* update README resource links

* update README resource links

* Fix MALLOC=memkind build issues

* Fix MALLOC=memkind build issues

* Fix module test break

* Fix module test break

* Eliminate firewall dialogs on mac for regular and cluster tests.  There are still issues with the sentinel tests but attempting to bind only to localhost causes failures

* Eliminate firewall dialogs on mac for regular and cluster tests.  There are still issues with the sentinel tests but attempting to bind only to localhost causes failures

* remove unused var in networking.cpp

* remove unused var in networking.cpp

* check ziplist len to avoid crash on empty ziplist convert

* check ziplist len to avoid crash on empty ziplist convert

* remove nullptr subtraction

* remove nullptr subtraction

* cannot mod a pointer

* cannot mod a pointer

* need to include stdint for uintptr_t

* need to include stdint for uintptr_t

* use atomic_load for g_pserver->mstime

* use atomic_load for g_pserver->mstime

* Integrate readwritelock with Pro Code

* Integrate readwritelock with Pro Code

* Defensive asserts for RWLock

* Defensive asserts for RWLock

* Save and restore master info in rdb to allow active replica partial sync (#371)

* save replid for all masters in rdb

* expanded rdbSaveInfo to hold multiple master structs

* parse repl-masters from rdb

* recover replid info from rdb in active replica mode, attempt partial sync

* save offset from rdb into correct variable

* don't change replid based on master in active rep

* save and load psync info from correct fields

* Save and restore master info in rdb to allow active replica partial sync (#371)

* save replid for all masters in rdb

* expanded rdbSaveInfo to hold multiple master structs

* parse repl-masters from rdb

* recover replid info from rdb in active replica mode, attempt partial sync

* save offset from rdb into correct variable

* don't change replid based on master in active rep

* save and load psync info from correct fields

* placement new instead of memcpy

* placement new instead of memcpy

* Remove asserts, RW lock can go below zero in cases of aeAcquireLock

* Remove asserts, RW lock can go below zero in cases of aeAcquireLock

* Inclusive language

* Inclusive language

* call aeThreadOnline() earlier

* call aeThreadOnline() earlier

* Removed mergeReplicationId

* Removed mergeReplicationId

* Make active client balancing a configurable option

* Make active client balancing a configurable option

* With TLS throttle accepts if server is under heavy load - do not change non TLS behavior

* With TLS throttle accepts if server is under heavy load - do not change non TLS behavior

* acceptTLS is threadsafe like the non TLS version

* acceptTLS is threadsafe like the non TLS version

* PSYNC production fixes

* PSYNC production fixes

* Ensure we are responsive during storagecache clears

* Ensure we are responsive during storagecache clears

* Ensure recreated tables use the same settings as ones made at boot

* Ensure recreated tables use the same settings as ones made at boot

* Converted some existing PSYNC tests for multimaster

* Converted some existing PSYNC tests for multimaster

* Inclusive language fix

* Inclusive language fix

* Cleanup test suite

* Cleanup test suite

* Updated test replica configs so tests make sense

* Updated test replica configs so tests make sense

* active-rep test reliability

* active-rep test reliability

* Quick fix to make psync tests work

* Quick fix to make psync tests work

* Fix PSYNC test crashes

* Fix PSYNC test crashes

* Ensure we force moves not copies when ingesting bulk insert files

* Ensure we force moves not copies when ingesting bulk insert files

* Disable async for hget commands as it is not ready

* Disable FLASH

* Fix crash in save of masterinfo

* Fix musl/Alpine build failures

* Remove unnecessary libs

* update readme

* update readme

* remove Enterprise references

* Limit max overage to 20% during RDB save

* Delete COPYING to replace with BSD license

* update deb master changelog

* Update license

* Fix Readme typo from github org transition

Replace mention of scratch-file-path with db-s3-object

* Fix reference counting failure in the dict.  This is caused by std::swap also swapping refcounts

* Fix assertion in async rehash

* Prevent crash on shutdown by avoiding dtors (they are unnecessary anyways)

* Initialize noshrink, it was dangling

* Prevent us from starting a rehash when one wasn't already in progress.  This can cause severe issues for snapshots

* Avoid unnecessary rehashing when a rehash is abandoned

* Dictionary use correct acquire/release semantics

* Add fence barriers for the repl backlog (important for AARCH64 and other weak memory models)

* Silence TSAN errors on ustime and mstime.  Every CPU we support is atomic on aligned ints, but correctness matters

* Disable async commands by default

* Fix TSAN warnings on the repl backlog

* Merge OSS back into pro

* Fix unmerged files

* Fix O(n^2) algorithm in the GC cleanup logic

* Fix crash in expire when a snapshot is in flight.  Caused by a perf optimization getting the expire map out of sync with the val

* On Alpine we must have a reasonable stack size

* Revert ci.yml to unstable branch version

* Implements the soft shutdown feature to allow clients to cooperatively disconnect preventing disruption during shutdown

* Ensure clean shutdown with multiple threads

* update dockerfiles

* update deb pkg references and changelog

* update gem reference

* lpGetInteger returns int64_t, avoid overflow (#10068)

Fix #9410

Crucial for the ms and sequence deltas, but I changed all
calls, just in case (e.g. "flags")

Before this commit:
`ms_delta` and `seq_delta` could have overflown, causing `currid` to be wrong,
which in turn would cause `streamTrim` to trim the entire rax node (see new test)

* Fix issue #454 (BSD build break)

* Do not allow commands to run in background when in eval, Issue #452

* Fix certificate leak during connection when tls-allowlists are used

* Fix issue #480

* Fix crash running INFO command while a disk based backlog is set

* check tracking per db

* fix warnings

* Fix a race when undoConnectWithMaster changes mi->repl_transfer_s but the connection is not yet closed and the event handler runs

* Fix a race in processChanges/trackChanges with rdbLoadRio by acquiring the lock when trackChanges is set

* Fix ASAN use after free

* Additional fixes

* Fix integer overflow of the track changes counter

* Fix P99 latency issue for TLS where we leave work for the next event loop

tlsProcessPendingData() needs to be called before we execute queued commands because it may enqueue more commands

* Fix race removing key cache

* Prevent crash on load in long running KeyDB instances

* Fixes a crash where the server assertion failed when the key exists in DB during RDB load

* Remove old assertion which is commented out.

* avoid from instatiating EpochHolder multiple times to improve performance and cpu utilization

* avoid from instatiating EpochHolder multiple times to improve performance and cpu utilization

* src\redis-cli.c: fix potential null pointer dereference found by cppcheck

src\redis-cli.c:5488:35: warning: Either the condition
'!table' is redundant or there is possible null pointer dereference:
table. [nullPointerRedundantCheck]

* Fix Issue #486

* Workaround bug in snapshot sync - abort don't crash

* Improve reliability of async parts of the soft shutdown tests

* Improve reliability of fragmentation tests

* Verify that partial syncs do indeed occur

* Fix O(n) algorithm in INFO command

* Remove incorrect assert that fires when the repl backlog is used fully

* Make building flash optional

* Remove unneeded gitlab CI file

* [BUG] Moves key to another DB, the source key was removed if the move failed due to the key exists in the destination db #497 (#498)

Co-authored-by: Paul Chen <mingchen@Mings-MacBook-Pro.local>

* trigger repl_curr_off!= master_repl_offset assert failure when having pending write case

* use debug for logging the message instead

* rocksdb log using up the diskspace on flash (#519)

* Fix OpenSSL 3.0.x related issues. (#10291)

* Drop obsolete initialization calls.
* Use decoder API for DH parameters.
* Enable auto DH parameters if not explicitly used, which should be the
  preferred configuration going forward.

* remove unnecessary forward declaration

* remove internal ci stuff

* remove more internal ci/publishing

* submodule update step

* use with syntax instead

* bump ci ubuntu old ver as latest is now 22.04

* include submodules on all ci jobs

* install all deps for all ci jobs

Co-authored-by: Vivek Saini <vsaini@snapchat.com>
Co-authored-by: Christian Legge <christian@eqalpha.com>
Co-authored-by: benschermel <bschermel@snapchat.com>
Co-authored-by: John Sully <john@csquare.ca>
Co-authored-by: zliang <zliang@snapchat.com>
Co-authored-by: malavan <malavan@eqalpha.com>
Co-authored-by: John Sully <jsully@snapchat.com>
Co-authored-by: jfinity <38383673+jfinity@users.noreply.github.com>
Co-authored-by: benschermel <43507366+benschermel@users.noreply.github.com>
Co-authored-by: guybe7 <guy.benoish@redislabs.com>
Co-authored-by: Karthick Ariyaratnam (A) <k00809413@china.huawei.com>
Co-authored-by: root <paul.chen1@huawei.com>
Co-authored-by: Ilya Shipitsin <chipitsine@gmail.com>
Co-authored-by: Paul Chen <32553156+paulmchen@users.noreply.github.com>
Co-authored-by: Paul Chen <mingchen@Mings-MacBook-Pro.local>
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2022-12-14 12:17:36 -05:00
John Sully
d2d76c2ed1 Fix unmerged files 2022-05-18 01:36:48 +00:00
John Sully
a265f815e2 Merge OSS back into pro 2022-05-18 01:29:15 +00:00
benschermel
bcead44d60 update deb copyright 2022-04-20 16:17:43 -04:00
benschermel
5c8926e2ca update changelog to use replica as terminology 2022-04-20 14:28:18 -04:00
benschermel
9031ac02f9 update packaging for OS merge 2022-04-15 23:09:44 -04:00
christianEQ
b785b03068 Merge remote-tracking branch 'opensource/RELEASE_6' into keydbpro
Former-commit-id: 439c655a543f4d3224d90bcdeb21ba43c2ab8ab7
2022-01-14 22:19:42 +00:00
benschermel
a1c9ef6800 update deb changelog
Former-commit-id: 5169f974b1ae05ce383242f3820033e9214b9a72
2021-11-17 18:22:36 +00:00
benschermel
35fcf97b8a update release packaging
Former-commit-id: 639179471ec155ce3ec4443095c44b66cfe66cfe
2021-11-13 17:50:42 +00:00
benschermel
a50f73fa2c update pkg conf file
Former-commit-id: 2a2ae61f07ecea8a178f4bc478bb7786d5be27d9
2021-08-22 22:28:18 +00:00
benschermel
d9536ec90f update pkg conf file
Former-commit-id: 03a52dade1fe8cf5a10bbab113e14f73fade0373
2021-08-22 22:18:22 +00:00
benschermel
1d871acbbe add keydb-diagnostic-tool to packaging
Former-commit-id: 971725cc8c9a4592d3e63f53e3dd10b6ca27e72f
2021-08-16 23:21:07 +00:00
Malavan Sotheeswaran
98fe0f9ed4 Merge branch 'pkg_update_tools' into 'keydbpro'
add diagnostic tool to packaging

See merge request keydb-dev/KeyDB-Pro!33

Former-commit-id: f87960ac26a710ed1a014a93e47740db1a81a07f
2021-08-16 22:44:53 +00:00
Malavan Sotheeswaran
3f6d551ecf Merge branch 'pkg_config' into 'keydbpro'
update packaging config files

See merge request keydb-dev/KeyDB-Pro!31

Former-commit-id: d6bce861e5bafa30d1cc31d1856c75f6e492d454
2021-08-16 22:42:29 +00:00
benschermel
48f6fc22ce add diagnostic tool to packaging
Former-commit-id: 37c3ca16162bd8a97ef35e2104ff5dc0ed6d7e48
2021-08-16 21:59:59 +00:00
benschermel
4ca625cd7c update deb master changelog
Former-commit-id: afd8ac5c386bb2265974407cf0a422015ad2fcd9
2021-08-13 00:49:41 +00:00
malavan
67ffa578d7 Merge remote-tracking branch 'opensource/unstable' into keydbpro
Former-commit-id: cf40fa5da29ba9574d20d706d0a41ca7001df95a
2021-08-11 20:41:54 +00:00
malavan
c36b790fcb Merge remote-tracking branch 'origin/unstable' into RELEASE_6
Former-commit-id: 16897d36b8f8c297185fc54f93cfb705e88da131
2021-08-11 19:59:28 +00:00
Paweł Sacawa
eb5a1a5e19 Add: zsh completions with descriptions
This commit add zsh completions for the keydb `client`. They have contextual host completion and full argument descriptions.
Vendor-distributed completions for zsh should end up in `/usr/share/zsh/vendor-completions`, but unfortunatly I'm not familiar with the packaging method for *.deb archives, so these completions will need to be moved to the appropriate directory.


Former-commit-id: f77980fce87f22b59677e374e0d5c113775cc08a
2021-08-11 15:05:11 -04:00
benschermel
a7b3ab45f2 update packaging config files
Former-commit-id: 0c5d12b91455db4acad57665db8aac79180b42ab
2021-08-10 20:08:39 -04:00
benschermel
dbd1a1e4af update package config files
Former-commit-id: fd367a91cc0feb22b67f034627d59feb679b81e8
2021-08-10 20:04:30 -04:00
malavan
e5c4cdbba7 Merge branch 'unstable' into RELEASE_6
Former-commit-id: 470523573932200c82c3ac02fb26f99892789a3d
2021-08-09 18:17:11 +00:00
christianEQ
c2655387fc updated pkg/deb/conf/keydb.conf to use keydb name throughout
Former-commit-id: 308cf77437374f01cc40101f2f35768db7a16302
2021-06-11 18:15:50 +00:00
John Sully
6b4f686f5f Merge branch 'unstable' into keydbpro
Former-commit-id: 205d8f18d2bb8df5253bab40578b006b7aa73fd5
2021-05-28 23:32:46 +00:00
John Sully
5267928381 Merge tag '6.2.2' into unstable
Former-commit-id: 93ebb31b17adec5d406d2e30a5b9ea71c07fce5c
2021-05-21 05:54:39 +00:00
John Sully
fe8efa916b Merge tag '6.2.1' into unstable
Former-commit-id: bfed57e3e0edaa724b9d060a6bb8edc5a6de65fa
2021-05-19 02:59:48 +00:00
benschermel
d42ad8f837 bump version
Former-commit-id: 4272a7de24eca935bfe1e13d44bdc0ece58efb55
2021-03-26 22:57:19 +00:00
benschermel
cd48001c21 addressing comments from merge review
Former-commit-id: 955aa69ca16ca7230d5099ad16f6f825402b5dd5
2021-03-23 18:41:59 +00:00
benschermel
88e6c73906 update enterprise wording for keydb.config lk
Former-commit-id: feb2d31e6544763c94f4abaea69bc98bd1cb8b17
2021-03-21 05:19:28 +00:00
benschermel
3f9a80917a minor fixes
Former-commit-id: ad7a27d41fb9e8e47966bbee72b28a4be66bbd8f
2021-03-18 01:16:57 +00:00
benschermel
05b2d1e798 update all packaging to support enterprise dist
Former-commit-id: 1fd4f27cd271090fc66dba3b09a3800bf1c33b0e
2021-03-17 02:30:34 +00:00
Ben Schermel
a472d76dd1 update deb changelog
Former-commit-id: 63c47a09e900e28f018cfb2c6d92901bd9e61e61
2020-09-28 23:42:54 +00:00
Ben Schermel
be7b877b8b update deb changelog
Former-commit-id: 087af94c229469da06dc9e26d3a65c1e9a5d141b
2020-09-28 20:51:31 +00:00
benschermel
46364b5b05 update deb changelog
Former-commit-id: a6782a3c0d35690d0a68da3e1decf1be723d01fd
2020-07-13 01:35:25 -04:00
John Sully
4e7027264f Bump version
Former-commit-id: ee5a281ed0012aca359b9448df5236bef719f4a7
2020-07-10 04:13:42 +00:00
benschermel
979d1389a7 update deb changelog
Former-commit-id: 85fe16238fdb692067f77f631b0589366823ba34
2020-06-24 00:32:10 -04:00
John Sully
65e9b5ccc7 Merge PKG files from unstable
No code change


Former-commit-id: 42c4d85d72af99322b21faab74ae23556b2054f0
2020-06-24 00:16:21 -04:00
Ben Schermel
ddde8abfc7 fix issues from rebase
Former-commit-id: 5d7cc28208bbe7b49eed0d1b1c2bb1c31e65e865
2020-06-09 00:10:58 -04:00
Ben Schermel
39a7500321 updating changelog
Former-commit-id: 5affe6dd268bb333214f70f5f809050e4b00b28b
2020-06-09 00:10:58 -04:00
Ben Schermel
a549aac9f8 dh9 for stretch deb pkg
Former-commit-id: 5cf02197ba418b5302759c14195ffa33a1db70fe
2020-06-09 00:10:58 -04:00
Ben Schermel
4c82fa9f4d fix missing sentinel
Former-commit-id: 01fe9274e8cb73a065d8550f220d07e5eacd0156
2020-06-09 00:10:58 -04:00
Ben Schermel
a7af0f2d5b update changelog
Former-commit-id: e2be81157e9cd1f57b9d734c5536dc0f4d5dbb0b
2020-06-09 00:10:58 -04:00
Ben Schermel
9399a8e6de update changelog
Former-commit-id: f0fe6c6b3f249a919ad5f4a233049c7a931bf5cc
2020-06-09 00:10:58 -04:00
Ben Schermel
5581f01efb update changelog
Former-commit-id: 5acb71ac79be4b0404fc56e82485a28f9664a017
2020-06-09 00:10:58 -04:00
Ben Schermel
12de8ba7a7 update changelog
Former-commit-id: adb5f42d945960cbe153e3614a1126972c8ce841
2020-06-09 00:10:58 -04:00
Ben Schermel
ee9c6923d1 include support for dists < dh11
Former-commit-id: 1930b759a9acce126342709acfb7884649023f3d
2020-06-09 00:10:58 -04:00