21057 Commits

Author SHA1 Message Date
d0de141b9a Delete src/modules/.gitignore
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:26:31 +00:00
31a7c701a6 Delete src/modules/keydb_modstatsd/Makefile
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:26:21 +00:00
c9e595e353 Delete src/modules/keydb_modstatsd/modmain.cpp
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:26:09 +00:00
016a8509a0 Delete src/modules/keydb_modstatsd/redismodule.h
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:25:57 +00:00
3917645451 Delete src/modules/Makefile
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:25:45 +00:00
21436a87e7 Delete src/modules/hellotype.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:25:35 +00:00
3affaadd6d Delete src/modules/gendoc.rb
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:25:21 +00:00
ae5e0a27e0 Delete src/modules/hellotimer.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:25:12 +00:00
b28ec47b7c Delete src/modules/helloacl.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:25:03 +00:00
29ee4b42b8 Delete src/modules/hellocluster.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:24:54 +00:00
00d60d5cfa Delete src/modules/hellohook.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:24:45 +00:00
97522720b8 Delete src/modules/helloblock.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:24:36 +00:00
b559a4f789 Delete src/modules/hellodict.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:24:26 +00:00
63c71b7593 Delete src/modules/helloworld.c
Some checks are pending
CI / test-ubuntu-latest (push) Waiting to run
CI / build-ubuntu-old (push) Waiting to run
CI / build-macos-latest (push) Waiting to run
CI / build-libc-malloc (push) Waiting to run
2025-04-15 17:24:18 +00:00
michael-grunder
603ebb27fb I think we want ARM's program counter here.
Fixes #797
2024-04-04 16:26:33 -04:00
Guillem Jover
0731a0509a build: Add support for linking against a system libhiredis/libhiredis_ssl
Add a new USE_SYSTEM_HIREDIS make variable to select whether to link
against the system libhiredis (and libhiredis_ssl if BUILD_TLS is
enabled).

Move the sdscompat.h header from the vendored hiredis directory to src/,
as this file is not and has never been part of the upstream hiredis
project, it got added in commit bffbbeaa9a1a6b8e81384297272cb0631502e8fd
in redis itself.
2024-03-25 21:01:05 -04:00
Guillem Jover
d67c65867a build: Add support for linking against a system librocksdb
Add a new USE_SYSTEM_ROCKSDB make variable to select whether to link
against the system librocksdb.
2024-03-25 21:01:05 -04:00
Guillem Jover
ae130d1a7f build: Add support for linking against a system libjemalloc
Add a new USE_SYSTEM_JEMALLOC make variable to select whether to link
against the system libjemalloc.
2024-03-25 21:01:05 -04:00
Guillem Jover
10c63d743c build: Add support for using the system concurrentqueue headers
Add a new USE_SYSTEM_CONCURRENTQUEUE make variable to select whether
to include the system concurrentqueue headers.
2024-03-25 21:01:05 -04:00
Tibault Damman
011fdb4772 fix build on 32bit systems
Code had an assert that tests if sizeof(long) == sizeof(long long),
which obviously fails on 32-bit architectures.
I believe the assert is incorrect on any architecture, considering
the following code is actually putting a long long in an _int_.

I replaced it with code that checks if the value fits in an int.

Signed-off-by: Tibault Damman <tibault.damman@basalte.be>
2024-03-07 19:50:35 -05:00
Roman Donchenko
d94fddae0a Add more detail to the panic messages in blocked.cpp 2024-03-07 19:50:00 -05:00
k00809413
2ad14dec88 Fix swapdb bugs when multiple clients involved. 2024-01-10 15:32:51 -05:00
Malavan Sotheeswaran
674d9fb7ee
stat64 is deprecated on mac (#737) 2023-11-16 03:05:43 -05:00
Malavan Sotheeswaran
05945b60a8 missed a refernce to isRocksdbSnapshotRepl 2023-10-26 11:40:50 -07:00
Malavan Sotheeswaran
486a6bb7b1 Merge remote-tracking branch 'internal/main' 2023-10-26 11:21:46 -07:00
John Sully
70503f6b58 Disable fastsync by default 2023-10-23 15:23:11 -04:00
John Sully
a7ac6c1e43 Remove unused variable 2023-10-23 15:23:11 -04:00
John Sully
d7977c468f Fastsync in-mem initial implementation 2023-10-23 15:23:11 -04:00
John Sully
7ae27f61f0 rename constants 2023-10-23 15:23:11 -04:00
Malavan Sotheeswaran
47f635ee16 Fix tls auditlog tests and enable tls for CI (#234)
* fix auditlog test titles, and enable tls tests for CI

* fix tls auditlog key string
2023-10-23 14:34:23 -04:00
Malavan Sotheeswaran
79b0c84710
Fixing some compile issues for alpine and centos, also fix endian issue for FLASH hashslot prefix (#722)
*fix for rocksdb centos compile fail

* fix alpine compile

* fix hashslot enumerate

* update alpine docker to latest version

* fix mac build, use big endian for expire prefix

* use endian.h for expires too
2023-10-13 15:00:35 -04:00
John Sully
cd42776c9f Make the connect timeout configurable when in cluster mode to prevent huge delays 2023-10-12 12:11:18 -04:00
Karthick Ariyaratnam
5b12251627
Fix integer overflow issue in the temp rdb file naming. (#702)
Co-authored-by: Karthick Ariyaratnam (A) <k00809413@china.huawei.com>
2023-09-28 21:01:47 -04:00
michieldwitte
a0f280e1a2
rename zset to iterzset (#706)
Co-authored-by: Michiel De Witte <michiel.dewitte@basalte.be>
2023-09-28 21:01:05 -04:00
Karthick Ariyaratnam
57f4a2e8b1
Fix a bug where flash CF options are being reset to default after flushall. (#718)
Co-authored-by: Karthick Ariyaratnam (A) <k00809413@china.huawei.com>
Co-authored-by: Malavan Sotheeswaran <105669860+msotheeswaran-sc@users.noreply.github.com>
2023-09-28 21:00:28 -04:00
Malavan Sotheeswaran
38169682f0
Merge latest internal to OSS (#720)
* add docker build

* fix the working dir in Dockerfile

* add release publish docker image

* address intentation and use default release

* migrate keydb_modstatsd to keydb-internal

* rm

* add submodule cpp-statsd-client

* include trigger keydb_modstatsd Makefile in modules Makefile

* update

* have clean also trigger keydb_modstatsd clean

* move cpp-statsd-client to deps

* checkout to a06a5b9359f31d946fe163b9038586982971ae49

* update relative path in compilation

* remove submodule instead use the source

* include building keydb statsd module

* fix check in Dockerfile docker-entrypoint.sh

* fix

* fix the comment caused stuck docker build

* use commit hash as tag template

* fix

* test tag

* Revert "test tag"

This reverts commit 9cbc57137d57aab4fdd5a9283bae07391b3c7f8b.

* make docker build independent

* add new build to ci

* emit system free metrics with '/proc/meminfo'

* have emit system free memory within metrics_time_taken_us and also add metric time taken for it

* Remove Expireset (#217)

Major refactor to place expiry information directly in the object struct.

* update MemFree to MemAvailable in keydb statsd

* add metric emit for non-empty primary with less than 2 connected replicas

* address comments

* Multiply CPU percent metric by 100

* Fix memory leaks

* Fix slow to free when low lock contention

* fix nodename metricsname

* fix unnecessary replace

* Make propagating before freeing module context optional (#225)

* don't propogate on module context free for rdb load

* default in wrong place

* Flash expiration (#197)

Design Doc: https://docs.google.com/document/d/1NmnYGnHLdZp-KOUCUatX5iXpF-L3YK4VUc9Lm3Tqxpo/edit?usp=sharing

* Emit more aggregate metrics in modstatsd (#223)

* Permit keys of differing slots as long as they are served by this cluster and we are not migrating

* Fix over pessimistic checks that prevent replicas from serving mget

* Fix logic bug

* async rehash is preventing rehashing during RDB load after a db flush.  Ensure it can't interefere after a flush

* make async rehash configurable

* only use internal locks when multithreaded (#205)

* Fix crossslot error migrating batches of keys

* Fix bug where we erroneously answer queries belonging to another shard

* fix mac compile

* enable level_compaction_dynamic_level_bytes after flush, and flush expires for FLASH (#229)

* enable level_compaction_dynamic_level_bytes after flush, and flush expires

* update debug reload for flash

* update debug reload for flash complete

* missing forward declare

* commit existing changes then track changes for debug reload

* missing args

* commitChanges is conditional

Co-authored-by: John Sully <jsully@snapchat.com>

---------

Co-authored-by: zliang <zliang@snapchat.com>
Co-authored-by: John Sully <jsully@snapchat.com>
Co-authored-by: Alex Cope <acope@snapchat.com>
Co-authored-by: John Sully <john@csquare.ca>
2023-09-28 18:13:27 -04:00
Malavan Sotheeswaran
dfcc848e1c enable level_compaction_dynamic_level_bytes after flush, and flush expires for FLASH (#229)
* enable level_compaction_dynamic_level_bytes after flush, and flush expires

* update debug reload for flash

* update debug reload for flash complete

* missing forward declare

* commit existing changes then track changes for debug reload

* missing args

* commitChanges is conditional

Co-authored-by: John Sully <jsully@snapchat.com>
2023-09-28 16:59:23 -04:00
John Sully
e7f90f0096 Fix bug where we erroneously answer queries belonging to another shard 2023-09-27 07:38:09 +00:00
John Sully
025b7077f7 Fix crossslot error migrating batches of keys 2023-09-26 00:11:21 -04:00
Malavan Sotheeswaran
596c513d3e only use internal locks when multithreaded (#205) 2023-09-25 15:53:47 -04:00
John Sully
570bcb55c7 make async rehash configurable 2023-09-23 16:32:28 -04:00
John Sully
68d751046e async rehash is preventing rehashing during RDB load after a db flush. Ensure it can't interefere after a flush 2023-09-23 16:32:28 -04:00
John Sully
cfe257a9c7 Fix logic bug 2023-09-20 12:57:10 -07:00
John Sully
c6e20ea396 Fix over pessimistic checks that prevent replicas from serving mget 2023-09-20 12:57:10 -07:00
a00817524
5f5753ae53 build failure 2023-09-17 19:06:00 -04:00
a00817524
e42d83d208 build failure 2023-09-17 19:06:00 -04:00
a00817524
9c32d4c1c8 removed proc call 2023-09-17 19:06:00 -04:00
a00817524
337eb297a1 review rework 2023-09-17 19:06:00 -04:00
a00817524
cf13e7b594 Async FLASH Design: AE event loop changes to understand StorageToken 2023-09-17 19:06:00 -04:00
John Sully
9251f1c244 Permit keys of differing slots as long as they are served by this cluster and we are not migrating 2023-09-14 21:55:04 -04:00