413 Commits

Author SHA1 Message Date
Salvatore Sanfilippo
36460f5680 Merge pull request #3383 from yossigo/datatype_load_save
Redis Module API calls to allow re-use of data type RDB save/load.
2019-11-19 10:55:42 +01:00
Oran Agra
10a834b48e Slightly more efficient RM_ReplyWithEmptyString
trimming talk about RESP protocol from API docs (should be independent to that anyway)
2019-11-14 09:46:46 +02:00
Oran Agra
c657b24c83 module docs, missing LOADING flag 2019-11-14 09:09:10 +02:00
Yossi Gottlieb
bf0931b57c Add RM_ModuleTypeReplaceValue.
This is a light-weight replace function, useful for use cases such as
realloc()ing an existing value, etc.  Using RM_ModuleTypeSetValue() in
such cases is wasteful and complex as it attempts to delete the old
value, call its destructor, etc.
2019-11-14 08:50:11 +02:00
Oran Agra
4c7b540ccd Add RM_ScanKey to scan hash, set, zset, changes to RM_Scan API
- Adding RM_ScanKey
- Adding tests for RM_ScanKey
- Refactoring RM_Scan API

Changes in RM_Scan
- cleanup in docs and coding convention
- Moving out of experimantal Api
- Adding ctx to scan callback
- Dont use cursor of -1 as an indication of done (can be a valid cursor)
- Set errno when returning 0 for various reasons
- Rename Cursor to ScanCursor
- Test filters key that are not strings, and opens a key if NULL
2019-11-11 16:05:55 +02:00
meir@redislabs.com
5642a1da20 Added scan implementation to module api.
The implementation expose the following new functions:
1. RedisModule_CursorCreate - allow to create a new cursor object for
keys scanning
2. RedisModule_CursorRestart - restart an existing cursor to restart the
scan
3. RedisModule_CursorDestroy - destroy an existing cursor
4. RedisModule_Scan - scan keys

The RedisModule_Scan function gets a cursor object, a callback and void*
(used as user private data).
The callback will be called for each key in the database proving the key
name and the value as RedisModuleKey.
2019-11-11 16:01:41 +02:00
Oran Agra
86e1f73bd6 rename RN_SetLRUOrLFU -> RM_SetLRU and RN_SetLFU
- the API name was odd, separated to two apis one for LRU and one for LFU
- the LRU idle time was in 1 second resolution, which might be ok for RDB
  and RESTORE, but i think modules may need higher resolution
- adding tests for LFU and for handling maxmemory policy mismatch
2019-11-10 09:27:01 +02:00
meir@redislabs.com
46ec9642ca fix documentation 2019-11-06 12:26:03 +02:00
Guy Benoish
ce913eea4c Support streams in general module API functions
Fixes GitHub issue #6492
Added stream support in RM_KeyType and RM_ValueLength.
Also moduleDelKeyIfEmpty was updated, even though it has
no effect now (It will be relevant when stream type direct
API will be coded - i.e. RM_StreamAdd)
2019-11-06 15:52:19 +05:30
meir@redislabs.com
0a6f42fc8e changed GetUsedMemoryPresentage -> GetUsedMemoryRatio and return value between 0 and 1. 2019-11-06 12:17:09 +02:00
meir@redislabs.com
667c9cd8c7 return value between 0 to 100 instead of 0 to 1. 2019-11-06 11:28:52 +02:00
meir@redislabs.com
2c5a3413dc expose used memory via redismodule api
The exposed functions:
1. RedisModule_GetUsedMemoryPercentage - return the used memory
2. RedisModue_MallocSize - return for a given pointer, the amount of memory allocated for this pointer
2019-11-06 11:28:52 +02:00
Yossi Gottlieb
a73881f818 Add ModuleDataType to/from string serialization.
Add two new functions that leverage the RedisModuleDataType mechanism
for RDB serialization/deserialization and make it possible to use it
to/from arbitrary strings:

* RM_SaveDataTypeToString()
* RM_LoadDataTypeFromString()
2019-11-04 21:53:59 +02:00
artix
29d1e22cd1 RM_CreateStringFromLongDouble: use new ld2string 'mode' type 2019-11-04 18:49:56 +01:00
artix
a24f1971f4 Start support for long double in modules
New API:

- RedisModule_StringToLongDouble
- RedisModule_CreateStringFromLongDouble
- RedisModule_ReplyWithLongDouble
2019-11-04 18:49:42 +01:00
Salvatore Sanfilippo
ab520ef80f Merge pull request #6531 from oranagra/rm_save_long_double
Module API for loading and saving long double
2019-11-04 17:33:35 +01:00
Salvatore Sanfilippo
cf2b0f6e86 Merge branch 'unstable' into module_lru_lfu 2019-11-04 11:04:49 +01:00
Salvatore Sanfilippo
574f611290 Merge pull request #6532 from oranagra/rm_misc_commands
Module API for PUBLISH, FLUSHALL, RANDOMKEY, DBSIZE
2019-11-04 11:02:37 +01:00
Salvatore Sanfilippo
9d9dbbf404 Merge branch 'unstable' into module_hooks 2019-11-04 10:37:06 +01:00
Oran Agra
ba1478c9f9 Add RM_ServerInfoGetFieldUnsigned
rename RM_ServerInfoGetFieldNumerical RM_ServerInfoGetFieldSigned
move string2ull to util.c
fix leak in RM_GetServerInfo when duplicate info fields exist
2019-11-04 08:50:29 +02:00
Oran Agra
8541c6a058 Adding RM_ServerInfoGetFieldC 2019-11-04 07:57:52 +02:00
Oran Agra
9cd3997851 Module API for PUBLISH, FLUSHALL, RANDOMKEY, DBSIZE 2019-11-04 07:40:52 +02:00
Oran Agra
04df0cc328 Module API for loading and saving long double
looks like each platform implements long double differently (different bit count)
so we can't save them as binary, and we also want to avoid creating a new RDB
format version, so we save these are hex strings using "%La".

This commit includes a change in the arguments of ld2string to support this.
as well as tests for coverage and short reads.

coded by @guybe7
2019-11-03 16:42:31 +02:00
Oran Agra
2bbd305a33 Add module api for looking into INFO fields
- Add RM_GetServerInfo and friends
- Add auto memory for new opaque struct
- Add tests for new APIs

other minor fixes:
- add const in various char pointers
- requested_section in modulesCollectInfo was actually not sds but char*
- extract new string2d out of getDoubleFromObject for code reuse

Add module API for
2019-11-03 15:02:25 +02:00
antirez
d44586c6aa Modules: fix thread safe context creation crash.
See #6525, this likely creates a NULL deference if the client was
terminated by Redis between the creation of the blocked client and the
creation of the thread safe context.
2019-10-31 18:07:33 +01:00
antirez
d576dde06b Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-10-31 17:55:18 +01:00
antirez
3b10c398f9 Modules: block on keys: finish implementing RM_UnblockClient(). 2019-10-31 17:39:05 +01:00
antirez
e6eaf0fce6 Modules: block on keys: fix bugs in processing order. 2019-10-31 12:23:55 +01:00
antirez
08df2470a9 Modules: block on keys: fix the top comments. 2019-10-31 11:43:45 +01:00
antirez
4af1df6b9f Modules: block on keys: use a better interface.
Using the is_key_ready() callback plus the reply callback later, creates
different issues AFAIK:

1. More complex API.
2. We need to call the reply callback() ASAP if the is_key_ready()
interface returned success, however the internals do not work in that
way, so when the reply callback is called the setup could be different.
To fix that, there is to break the current design that handles the
unblocked clients asyncrhonously, and run the list ASAP.
2019-10-31 11:35:07 +01:00
antirez
c9ad11d797 Modules: remove spurious call from moduleHandleBlockedClients().
Now we handle propagation when we free the context.
2019-10-31 10:32:59 +01:00
antirez
c4becaebaa Modules: block on keys: implement the internals. 2019-10-30 10:57:44 +01:00
antirez
fb56b8fd07 Modules: block on keys: export APIs. 2019-10-30 10:20:28 +01:00
antirez
5f2a2b974e Modules: block on keys functions layout and mechanism. 2019-10-30 10:11:58 +01:00
Oran Agra
68c6aacf3b Modules hooks: complete missing hooks for the initial set of hooks
* replication hooks: role change, master link status, replica online/offline
* persistence hooks: saving, loading, loading progress
* misc hooks: cron loop, shutdown, module loaded/unloaded
* change the way hooks test work, and add tests for all of the above

startLoading() now gets flag indicating what is loaded.
stopLoading() now gets an indication of success or failure.
adding startSaving() and stopSaving() with similar args and role.
2019-10-29 17:59:09 +02:00
Oran Agra
57649ce051 Module API for controlling LRU and LFU, and OpenKey without TOUCH
Some commands would want to open a key without touching it's LRU/LFU
similarly to the OBJECT or DEBUG command do.

Other commands may want to implement logic similar to what RESTORE
does (and in the future MIGRATE) and get/set the LRU or LFU.
2019-10-29 15:03:05 +02:00
Salvatore Sanfilippo
6f4ef70666 Merge pull request #6487 from oranagra/module_modified_key
Module API for explicit SignalModifiedKey instead of implicit one.
2019-10-29 10:40:22 +01:00
Salvatore Sanfilippo
9f31e02a79 Merge pull request #6495 from oranagra/rm_call_argv
Module api RM_CallArgv and test
2019-10-29 10:07:05 +01:00
Oran Agra
d725788873 Module api tests for RM_Call
Adding a test for coverage for RM_Call in a new "misc" unit
to be used for various short simple tests

also solves compilation warnings in redismodule.h and fork.c
2019-10-28 12:39:57 +02:00
Guy Benoish
5d01f01beb Modules: Allow notifying custom keyspace events
Also, add an API for getting server.notify_keyspace_events

Other (unrelated) changes:
Add RM_GetKeynameFromModuleKey
2019-10-28 15:53:35 +05:30
Oran Agra
6700a2dc66 Module API for explicit SignalModifiedKey instead of implicit one.
This commit also fixes an uninitialized module struct member (that luckily never got released)
2019-10-28 12:09:25 +02:00
Salvatore Sanfilippo
79e670d6ad Merge pull request #6498 from oranagra/rm_latency_add_sample
Module API for LatencyAddSample
2019-10-28 09:59:26 +01:00
Salvatore Sanfilippo
847dd92ef0 Merge pull request #6505 from swilly22/replaywith-module-API-additions
Introduce ReplyWithVerbatimString, ReplyWithEmptyArray, ReplyWithNull…
2019-10-28 09:56:56 +01:00
swilly22
29827d37a5 Introduce ReplyWithVerbatimString, ReplyWithEmptyArray, ReplyWithNullArray and ReplyWithEmptyString to redis module API 2019-10-28 08:50:25 +02:00
zhaozhao.zz
18c4ccf898 Modules: make unloading module more safe
As we know if a module exports module-side data types,
unload it is not allowed. This rule is the same with
blocked clients in module, because we use background
threads to implement module blocked clients, and it's
not safe to unload a module if there are background
threads running. So it's necessary to check if any
blocked clients running in this module when unload it.

Moreover, after that we can ensure that if no modules,
then no module blocked clients even module unloaded.
So, we can call moduleHandleBlockedClients only when
we have installed modules.
2019-10-28 11:31:58 +08:00
Oran Agra
5ba6fe50c7 Module API for LatencyAddSample 2019-10-24 14:24:55 +03:00
antirez
3bd7f715c8 Modules hooks: test flush event. 2019-10-24 10:51:03 +02:00
antirez
a398066625 Modules hooks: select DB zero by default. 2019-10-23 18:51:44 +02:00
antirez
938dafa9e3 Modules hooks: for nested calls, create new fake clients.
We can't use the same client at the same time when re-entering the hook.
2019-10-23 18:39:53 +02:00
antirez
d6739df4fc Modules hooks: do not re-enter in hooks.
Calling a module hook callback may result in callback operations in turn
triggering other events the module is subscribed too. We don't want to
trigger those, it's unsafe and quite confusing, and to do it correcly we
would need to maintain an event list: quite a more complex
implementation.
2019-10-23 18:39:53 +02:00