27350 Commits

Author SHA1 Message Date
Andreas Lind
8b497881f2
Support redis-cli -u rediss://... (#7900) 2020-10-11 18:14:02 +03:00
Yossi Gottlieb
a94ddb27fe Modules: fix RM_GetCommandKeys API. (#7901)
This cleans up and simplifies the API by passing the command name as the
first argument. Previously the command name was specified explicitly,
but was still included in the argv.
2020-10-11 18:10:55 +03:00
Yossi Gottlieb
056a43e1a6
Modules: fix RM_GetCommandKeys API. (#7901)
This cleans up and simplifies the API by passing the command name as the
first argument. Previously the command name was specified explicitly,
but was still included in the argv.
2020-10-11 18:10:55 +03:00
Meir Shpilraien (Spielrein)
e9c837ad31 Add Module API for version and compatibility checks (#7865)
* Introduce a new API's: RM_GetContextFlagsAll, and
RM_GetKeyspaceNotificationFlagsAll that will return the
full flags mask of each feature. The module writer can
check base on this value if the Flags he needs are
supported or not.

* For each flag, introduce a new value on redismodule.h,
this value represents the LAST value and should be there
as a reminder to update it when a new value is added,
also it will be used in the code to calculate the full
flags mask (assuming flags are incrementally increasing).
In addition, stated that the module writer should not use
the LAST flag directly and he should use the GetFlagAll API's.

* Introduce a new API: RM_IsSubEventSupported, that returns for a given
event and subevent, whether or not the subevent supported.

* Introduce a new macro RMAPI_FUNC_SUPPORTED(func) that returns whether
or not a function API is supported by comparing it to NULL.

* Introduce a new API: int RM_GetServerVersion();, that will return the
current Redis version in the format 0x00MMmmpp; e.g. 0x00060008;

* Changed unstable version from 999.999.999 to 255.255.255

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-10-11 17:21:58 +03:00
Meir Shpilraien (Spielrein)
adc3183cd2
Add Module API for version and compatibility checks (#7865)
* Introduce a new API's: RM_GetContextFlagsAll, and
RM_GetKeyspaceNotificationFlagsAll that will return the
full flags mask of each feature. The module writer can
check base on this value if the Flags he needs are
supported or not.

* For each flag, introduce a new value on redismodule.h,
this value represents the LAST value and should be there
as a reminder to update it when a new value is added,
also it will be used in the code to calculate the full
flags mask (assuming flags are incrementally increasing).
In addition, stated that the module writer should not use
the LAST flag directly and he should use the GetFlagAll API's.

* Introduce a new API: RM_IsSubEventSupported, that returns for a given
event and subevent, whether or not the subevent supported.

* Introduce a new macro RMAPI_FUNC_SUPPORTED(func) that returns whether
or not a function API is supported by comparing it to NULL.

* Introduce a new API: int RM_GetServerVersion();, that will return the
current Redis version in the format 0x00MMmmpp; e.g. 0x00060008;

* Changed unstable version from 999.999.999 to 255.255.255

Co-authored-by: Oran Agra <oran@redislabs.com>
Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
2020-10-11 17:21:58 +03:00
Yossi Gottlieb
929f1e2ec7 Module API: Add RM_GetClientCertificate(). (#7866)
This API function makes it possible to retrieve the X.509 certificate
used by clients to authenticate TLS connections.
2020-10-11 17:11:42 +03:00
Yossi Gottlieb
0aec98dce2
Module API: Add RM_GetClientCertificate(). (#7866)
This API function makes it possible to retrieve the X.509 certificate
used by clients to authenticate TLS connections.
2020-10-11 17:11:42 +03:00
Yossi Gottlieb
e270302cdf Modules: Add RM_GetDetachedThreadSafeContext(). (#7886)
The main motivation here is to provide a way for modules to create a
single, global context that can be used for logging.

Currently, it is possible to obtain a thread-safe context that is not
attached to any blocked client by using `RM_GetThreadSafeContext`.
However, the attached context is not linked to the module identity so
log messages produced are not tagged with the module name.

Ideally we'd fix this in `RM_GetThreadSafeContext` itself but as it
doesn't accept the current context as an argument there's no way to do
that in a backwards compatible manner.
2020-10-11 16:11:31 +03:00
Yossi Gottlieb
907da0580b
Modules: Add RM_GetDetachedThreadSafeContext(). (#7886)
The main motivation here is to provide a way for modules to create a
single, global context that can be used for logging.

Currently, it is possible to obtain a thread-safe context that is not
attached to any blocked client by using `RM_GetThreadSafeContext`.
However, the attached context is not linked to the module identity so
log messages produced are not tagged with the module name.

Ideally we'd fix this in `RM_GetThreadSafeContext` itself but as it
doesn't accept the current context as an argument there's no way to do
that in a backwards compatible manner.
2020-10-11 16:11:31 +03:00
Yossi Gottlieb
9e58b52a19 Modules: add RM_GetCommandKeys().
This is essentially the same as calling COMMAND GETKEYS but provides a
more efficient interface that can be used in every context (i.e. not a
Redis command).
2020-10-11 16:04:14 +03:00
Yossi Gottlieb
7d117d7591 Modules: add RM_GetCommandKeys().
This is essentially the same as calling COMMAND GETKEYS but provides a
more efficient interface that can be used in every context (i.e. not a
Redis command).
2020-10-11 16:04:14 +03:00
Yossi Gottlieb
bf5beab64a Introduce getKeysResult for getKeysFromCommand.
Avoid using a static buffer for short key index responses, and make it
caller's responsibility to stack-allocate a result type. Responses that
don't fit are still allocated on the heap.
2020-10-11 16:04:14 +03:00
Yossi Gottlieb
9b7f8ba84b Introduce getKeysResult for getKeysFromCommand.
Avoid using a static buffer for short key index responses, and make it
caller's responsibility to stack-allocate a result type. Responses that
don't fit are still allocated on the heap.
2020-10-11 16:04:14 +03:00
Uri Shachar
e66e79aa9f Support getting configuration from both stdin and file at the same time (#7893)
This allows supplying secret configuration (for example - masterauth) via a secure channel
instead of having it in a plaintext file / command line param, while still allowing for most
of the configuration to reside there.

Also, remove 'special' case handling for --check-rdb which hasn't been relevant
since 4.0.0.
2020-10-11 13:43:23 +03:00
Uri Shachar
dab5ec9b8d
Support getting configuration from both stdin and file at the same time (#7893)
This allows supplying secret configuration (for example - masterauth) via a secure channel
instead of having it in a plaintext file / command line param, while still allowing for most
of the configuration to reside there.

Also, remove 'special' case handling for --check-rdb which hasn't been relevant
since 4.0.0.
2020-10-11 13:43:23 +03:00
John Sully
27e17220c2 Remove dead code
Former-commit-id: c6f13892e04607700e27ec963fa1da695e784c11
2020-10-09 21:02:09 +00:00
John Sully
2c45f36dd4 Remove dead code
Former-commit-id: c6f13892e04607700e27ec963fa1da695e784c11
2020-10-09 21:02:09 +00:00
Felipe Machado
57dfccfbf9 Adds new pop-push commands (LMOVE, BLMOVE) (#6929)
Adding [B]LMOVE <src> <dst> RIGHT|LEFT RIGHT|LEFT. deprecating [B]RPOPLPUSH.

Note that when receiving a BRPOPLPUSH we'll still propagate an RPOPLPUSH,
but on BLMOVE RIGHT LEFT we'll propagate an LMOVE

improvement to existing tests
- Replace "after 1000" with "wait_for_condition" when wait for
  clients to block/unblock.
- Add a pre-existing element to target list on basic tests so
  that we can check if the new element was added to the correct
  side of the list.
- check command stats on the replica to make sure the right
  command was replicated

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-10-08 08:33:17 +03:00
Felipe Machado
c3f9e01794
Adds new pop-push commands (LMOVE, BLMOVE) (#6929)
Adding [B]LMOVE <src> <dst> RIGHT|LEFT RIGHT|LEFT. deprecating [B]RPOPLPUSH.

Note that when receiving a BRPOPLPUSH we'll still propagate an RPOPLPUSH,
but on BLMOVE RIGHT LEFT we'll propagate an LMOVE

improvement to existing tests
- Replace "after 1000" with "wait_for_condition" when wait for
  clients to block/unblock.
- Add a pre-existing element to target list on basic tests so
  that we can check if the new element was added to the correct
  side of the list.
- check command stats on the replica to make sure the right
  command was replicated

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-10-08 08:33:17 +03:00
Madelyn Olson
abe416c5f2 Fixed excessive categories being displayed from acls (#7889) 2020-10-07 22:09:09 -07:00
Madelyn Olson
2127f7c8eb
Fixed excessive categories being displayed from acls (#7889) 2020-10-07 22:09:09 -07:00
Oran Agra
38c7c62d22 Add some additional signal info to the crash log (#7891)
- si_code can be very useful info some day.
- a clear indication that redis was killed by an external user
2020-10-07 20:28:57 +03:00
Oran Agra
f659d23619
Add some additional signal info to the crash log (#7891)
- si_code can be very useful info some day.
- a clear indication that redis was killed by an external user
2020-10-07 20:28:57 +03:00
Oran Agra
ba61700db2 Allow blocked XREAD on a cluster replica (#7881)
I suppose that it was overlooked, since till recently none of the blocked commands were readonly.

other changes:
- add test for the above.
- add better support for additional (and deferring) clients for
  cluster tests
- improve a test which left the client in MULTI state.
2020-10-06 21:43:30 +03:00
Oran Agra
216c110609
Allow blocked XREAD on a cluster replica (#7881)
I suppose that it was overlooked, since till recently none of the blocked commands were readonly.

other changes:
- add test for the above.
- add better support for additional (and deferring) clients for
  cluster tests
- improve a test which left the client in MULTI state.
2020-10-06 21:43:30 +03:00
Wen Hui
bb40bd3d73 ACL : mentioning requirepass and aclfile not compatable in redis conf (#7883) 2020-10-05 18:49:47 +03:00
Wen Hui
6f3b91fe26
ACL : mentioning requirepass and aclfile not compatable in redis conf (#7883) 2020-10-05 18:49:47 +03:00
Hanif Bin Ariffin
b238eca1ce Fixed non-empty check in src/Makefile
Per [GNU Make Manual](https://www.gnu.org/software/make/manual/html_node/Conditional-Syntax.html).
To properly check for non-empty variable, one must strip whitespaces.

Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>

Former-commit-id: 05e0f323456b8e667d10dabfa804757a2fc81b04
2020-10-05 10:58:49 -04:00
Hanif Bin Ariffin
5c957fe228 Fixed non-empty check in src/Makefile
Per [GNU Make Manual](https://www.gnu.org/software/make/manual/html_node/Conditional-Syntax.html).
To properly check for non-empty variable, one must strip whitespaces.

Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>

Former-commit-id: 05e0f323456b8e667d10dabfa804757a2fc81b04
2020-10-05 10:58:49 -04:00
Wen Hui
bccea9ed92 support sentinel myid subcommand (#7858) 2020-10-05 13:37:20 +03:00
Wen Hui
57b8e00695
support sentinel myid subcommand (#7858) 2020-10-05 13:37:20 +03:00
Oran Agra
7481e513f0 memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.
2020-10-05 11:15:36 +03:00
Oran Agra
bea40e6a41
memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.
2020-10-05 11:15:36 +03:00
Oran Agra
14f13c34be fix new build warning about getGenericCommand (#7882) 2020-10-04 18:10:21 +03:00
Oran Agra
f87a47dc97
fix new build warning about getGenericCommand (#7882) 2020-10-04 18:10:21 +03:00
DvirDukhan
f0f8e9c824 redis-cli add control on raw format line delimiter (#7841)
Adding -D option for redis-cli to control newline between command
responses in raw mode.

Also removing cleanup code before calling exit, just in order
to avoid adding more adding more cleanup code (redis doesn't
bother to release allocations before exit anyway)

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-10-04 17:18:17 +03:00
DvirDukhan
6418d767e8
redis-cli add control on raw format line delimiter (#7841)
Adding -D option for redis-cli to control newline between command
responses in raw mode.

Also removing cleanup code before calling exit, just in order
to avoid adding more adding more cleanup code (redis doesn't
bother to release allocations before exit anyway)

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-10-04 17:18:17 +03:00
Hanif Bin Ariffin
4b63a5202b Removed dead code from a macro in zmalloc.cpp
I think the compiler would have removed this no-op anyways but it
definitely wasted me some 30 minutes on this :(

I was hoping I could remove the branch through some bit-hacking but
apparently its dead code :). Oh well, its 30 minutes of refreshing
bit hacking.

Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>

Former-commit-id: 8171e6de13311e3ad2e87c32d63060dcf3bd6055
2020-10-03 20:41:26 -04:00
Hanif Bin Ariffin
bb67320752 Removed dead code from a macro in zmalloc.cpp
I think the compiler would have removed this no-op anyways but it
definitely wasted me some 30 minutes on this :(

I was hoping I could remove the branch through some bit-hacking but
apparently its dead code :). Oh well, its 30 minutes of refreshing
bit hacking.

Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>

Former-commit-id: 8171e6de13311e3ad2e87c32d63060dcf3bd6055
2020-10-03 20:41:26 -04:00
Nykolas Laurentino de Lima
35bf5ad9c8 Add GET parameter to SET command (#7852)
Add optional GET parameter to SET command in order to set a new value to
a key and retrieve the old key value. With this change we can deprecate
`GETSET` command and use only the SET command with the GET parameter.
2020-10-02 15:07:19 +03:00
Nykolas Laurentino de Lima
66ee45b65c
Add GET parameter to SET command (#7852)
Add optional GET parameter to SET command in order to set a new value to
a key and retrieve the old key value. With this change we can deprecate
`GETSET` command and use only the SET command with the GET parameter.
2020-10-02 15:07:19 +03:00
Oran Agra
fc64fcce4d performance and memory reporting improvement - sds take control of it's internal frag (#7875)
This commit has two aspects:
1) improve memory reporting for all the places that use sdsAllocSize to compute
   memory used by a string, in this case it'll include the internal fragmentation.
2) reduce the need for realloc calls by making the sds implicitly take over
   the internal fragmentation of the block it allocated.
2020-10-02 08:19:44 +03:00
Oran Agra
3945a32177
performance and memory reporting improvement - sds take control of it's internal frag (#7875)
This commit has two aspects:
1) improve memory reporting for all the places that use sdsAllocSize to compute
   memory used by a string, in this case it'll include the internal fragmentation.
2) reduce the need for realloc calls by making the sds implicitly take over
   the internal fragmentation of the block it allocated.
2020-10-02 08:19:44 +03:00
John Sully
9d8897c50e Delete CONTRIBUTING
Former-commit-id: b042495f37f3d22f7759a4c59044af40273c3a1a
2020-10-01 11:17:56 -04:00
John Sully
71da0d44a5 Delete CONTRIBUTING
Former-commit-id: b042495f37f3d22f7759a4c59044af40273c3a1a
2020-10-01 11:17:56 -04:00
Oran Agra
86483e7952 Include internal sds fragmentation in MEMORY reporting (#7864)
The MEMORY command is used for debugging memory usage, so it should include internal
fragmentation, same as used_memory
2020-10-01 11:30:22 +03:00
Oran Agra
eb6241a3dd
Include internal sds fragmentation in MEMORY reporting (#7864)
The MEMORY command is used for debugging memory usage, so it should include internal
fragmentation, same as used_memory
2020-10-01 11:30:22 +03:00
Oran Agra
8cff3e0352 Fix crash in script timeout during AOF loading (#7870) 2020-10-01 11:27:45 +03:00
Oran Agra
dc803d25a6
Fix crash in script timeout during AOF loading (#7870) 2020-10-01 11:27:45 +03:00
Rafi Einstein
2636b760fb Makefile: enable program suffixes via PROG_SUFFIX (#7868) 2020-10-01 10:56:23 +03:00