14685 Commits

Author SHA1 Message Date
Uri Shachar
dab5ec9b8d
Support getting configuration from both stdin and file at the same time (#7893)
This allows supplying secret configuration (for example - masterauth) via a secure channel
instead of having it in a plaintext file / command line param, while still allowing for most
of the configuration to reside there.

Also, remove 'special' case handling for --check-rdb which hasn't been relevant
since 4.0.0.
2020-10-11 13:43:23 +03:00
John Sully
2c45f36dd4 Remove dead code
Former-commit-id: c6f13892e04607700e27ec963fa1da695e784c11
2020-10-09 21:02:09 +00:00
Felipe Machado
c3f9e01794
Adds new pop-push commands (LMOVE, BLMOVE) (#6929)
Adding [B]LMOVE <src> <dst> RIGHT|LEFT RIGHT|LEFT. deprecating [B]RPOPLPUSH.

Note that when receiving a BRPOPLPUSH we'll still propagate an RPOPLPUSH,
but on BLMOVE RIGHT LEFT we'll propagate an LMOVE

improvement to existing tests
- Replace "after 1000" with "wait_for_condition" when wait for
  clients to block/unblock.
- Add a pre-existing element to target list on basic tests so
  that we can check if the new element was added to the correct
  side of the list.
- check command stats on the replica to make sure the right
  command was replicated

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-10-08 08:33:17 +03:00
Madelyn Olson
2127f7c8eb
Fixed excessive categories being displayed from acls (#7889) 2020-10-07 22:09:09 -07:00
Oran Agra
f659d23619
Add some additional signal info to the crash log (#7891)
- si_code can be very useful info some day.
- a clear indication that redis was killed by an external user
2020-10-07 20:28:57 +03:00
Oran Agra
216c110609
Allow blocked XREAD on a cluster replica (#7881)
I suppose that it was overlooked, since till recently none of the blocked commands were readonly.

other changes:
- add test for the above.
- add better support for additional (and deferring) clients for
  cluster tests
- improve a test which left the client in MULTI state.
2020-10-06 21:43:30 +03:00
Wen Hui
6f3b91fe26
ACL : mentioning requirepass and aclfile not compatable in redis conf (#7883) 2020-10-05 18:49:47 +03:00
Hanif Bin Ariffin
5c957fe228 Fixed non-empty check in src/Makefile
Per [GNU Make Manual](https://www.gnu.org/software/make/manual/html_node/Conditional-Syntax.html).
To properly check for non-empty variable, one must strip whitespaces.

Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>

Former-commit-id: 05e0f323456b8e667d10dabfa804757a2fc81b04
2020-10-05 10:58:49 -04:00
Wen Hui
57b8e00695
support sentinel myid subcommand (#7858) 2020-10-05 13:37:20 +03:00
Oran Agra
bea40e6a41
memory reporting of clients argv (#7874)
track and report memory used by clients argv.
this is very usaful in case clients started sending a command and didn't
complete it. in which case the first args of the command are already
trimmed from the query buffer.

in an effort to avoid cache misses and overheads while keeping track of
these, i avoid calling sdsZmallocSize and instead use the sdslen /
bulk-len which can at least give some insight into the problem.

This memory is now added to the total clients memory usage, as well as
the client list.
2020-10-05 11:15:36 +03:00
Oran Agra
f87a47dc97
fix new build warning about getGenericCommand (#7882) 2020-10-04 18:10:21 +03:00
DvirDukhan
6418d767e8
redis-cli add control on raw format line delimiter (#7841)
Adding -D option for redis-cli to control newline between command
responses in raw mode.

Also removing cleanup code before calling exit, just in order
to avoid adding more adding more cleanup code (redis doesn't
bother to release allocations before exit anyway)

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-10-04 17:18:17 +03:00
Hanif Bin Ariffin
bb67320752 Removed dead code from a macro in zmalloc.cpp
I think the compiler would have removed this no-op anyways but it
definitely wasted me some 30 minutes on this :(

I was hoping I could remove the branch through some bit-hacking but
apparently its dead code :). Oh well, its 30 minutes of refreshing
bit hacking.

Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>

Former-commit-id: 8171e6de13311e3ad2e87c32d63060dcf3bd6055
2020-10-03 20:41:26 -04:00
Nykolas Laurentino de Lima
66ee45b65c
Add GET parameter to SET command (#7852)
Add optional GET parameter to SET command in order to set a new value to
a key and retrieve the old key value. With this change we can deprecate
`GETSET` command and use only the SET command with the GET parameter.
2020-10-02 15:07:19 +03:00
Oran Agra
3945a32177
performance and memory reporting improvement - sds take control of it's internal frag (#7875)
This commit has two aspects:
1) improve memory reporting for all the places that use sdsAllocSize to compute
   memory used by a string, in this case it'll include the internal fragmentation.
2) reduce the need for realloc calls by making the sds implicitly take over
   the internal fragmentation of the block it allocated.
2020-10-02 08:19:44 +03:00
John Sully
71da0d44a5 Delete CONTRIBUTING
Former-commit-id: b042495f37f3d22f7759a4c59044af40273c3a1a
2020-10-01 11:17:56 -04:00
Oran Agra
eb6241a3dd
Include internal sds fragmentation in MEMORY reporting (#7864)
The MEMORY command is used for debugging memory usage, so it should include internal
fragmentation, same as used_memory
2020-10-01 11:30:22 +03:00
Oran Agra
dc803d25a6
Fix crash in script timeout during AOF loading (#7870) 2020-10-01 11:27:45 +03:00
Rafi Einstein
b8187d39fb
Makefile: enable program suffixes via PROG_SUFFIX (#7868) 2020-10-01 10:56:23 +03:00
John Sully
2085e21dca Fix module test failures due to locking
Former-commit-id: 420ccdfbaebc452b9b374b54c6ebeec4a3ffea36
2020-10-01 00:08:54 +00:00
John Sully
51ae66539d Fix TLS test failure due to bad merge
Former-commit-id: 41466a4147bf675f69670016135f88589cc02a5c
2020-09-30 20:21:40 +00:00
John Sully
f7b7c5a168 Keep redis-cli.c C89 compatible
Former-commit-id: 9798f2d711939b03e972207638d18dcaa3b2e473
2020-09-30 20:17:51 +00:00
John Sully
a543a502b8 Mac build break fix
Former-commit-id: 5f3543921b4123c2216d9294c5eb7bfed007cbf5
2020-09-30 20:17:02 +00:00
John Sully
1fd53af71b Fix stream replication failure with active replication, issue #238
Former-commit-id: a41366cc3a6568c0249a5ee022e517add55e286d
2020-09-30 20:12:54 +00:00
John Sully
4f18a247e3 Merge tag '6.0.8' into unstable
Former-commit-id: 4c7e4b91a6bb2034636856b608b8c386d07f5541
2020-09-30 19:47:55 +00:00
nitaicaro
8fb89a5728
Fixed Tracking test “The other connection is able to get invalidations” (#7871)
PROBLEM:

[$rd1 read] reads invalidation messages one by one, so it's never going to see the second invalidation message produced after INCR b, whether or not it exists. Adding another read will block incase no invalidation message is produced.

FIX:

We switch the order of "INCR a" and "INCR b" - now "INCR b" comes first. We still only read the first invalidation message produces. If an invalidation message is wrongly produces for b - then it will be produced before that of a, since "INCR b" comes before "INCR a".

Co-authored-by: Nitai Caro <caronita@amazon.com>
2020-09-30 19:52:01 +03:00
Yossi Gottlieb
67b43f75e2 Modules: expose real client on conn events.
When REDISMODULE_EVENT_CLIENT_CHANGE events are delivered, modules may
want to mutate the client state (e.g. perform authentication).

This change links the module context with the real client rather than a
fake client for these events.
2020-09-30 17:06:57 +03:00
Yossi Gottlieb
cfccfbd6f4 Module API: Fail ineffective auth calls.
The client pointed to by the module context may in some cases be a fake
client. RM_Authenticate*() calls in this case would be ineffective but
appear to succeed, and this change fails them to make it easier to catch
such cases.
2020-09-30 17:06:57 +03:00
Yossi Gottlieb
1591e3479d
TLS: Do not require CA config if not used. (#7862)
The tls-ca-cert or tls-ca-cert-dir configuration parameters are only
used when Redis needs to authenticate peer certificates, in one of these
scenarios:

1. Incoming clients or replicas, with `tls-auth-clients` enabled.
2. A replica authenticating the master's peer certificate.
3. Cluster nodes authenticating other nodes when establishing the bus
   protocol connection.
2020-09-29 20:48:21 +03:00
YoongHM
448c435b1b
Fix compilation warnings in Lua and jemalloc dependencies (#7785)
- The argument `u` in for `ar` is ignored (and generates warnings since `D` became the default.
  All it does is avoid updating unchanged objects (shouldn't have any impact on our build)
- Enable `LUA_USE_MKSTEMP` to force the use of `mkstemp()` instead of `tmpname()` (which is dead
  code in redis anyway).
- Remove unused variable `c` in `f_parser()`
- Removed misleadingly indented space in `luaL_loadfile()` and ``addfield()`

Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-29 17:10:54 +03:00
Oran Agra
ef33252c43
warning: comparison between signed and unsigned integer in 32bit build (#7838) 2020-09-29 17:03:47 +03:00
David CARLIER
f971a5d8ed
Add support for Haiku OS (#7435) 2020-09-29 15:52:13 +03:00
Gavrie Philipson
49383a4af7
Fix typo in module API docs (#7861) 2020-09-29 13:10:08 +03:00
David CARLIER
ce8bfc56ad
getting rss size implementation for netbsd (#7293) 2020-09-29 08:49:35 +03:00
John Sully
3c0556093b Test RDB merge on load with active replication
Former-commit-id: 28183f4b66fc4c865048080b61e599eeb1d2293b
2020-09-29 03:26:06 +00:00
Ben Schermel
a472d76dd1 update deb changelog
Former-commit-id: 63c47a09e900e28f018cfb2c6d92901bd9e61e61
2020-09-28 23:42:54 +00:00
Ben Schermel
be7b877b8b update deb changelog
Former-commit-id: 087af94c229469da06dc9e26d3a65c1e9a5d141b
2020-09-28 20:51:31 +00:00
John Sully
0dd31eeed5 Drop min-clients-per-thread to a more reasonable number
Former-commit-id: bf3a3225fccfb63f7b5e03ef0b362b788b97026d
2020-09-28 19:19:47 +00:00
John Sully
a4aa190f20 Drop min-clients-per-thread to a more reasonable number
Former-commit-id: a0abc1eddd071f984950ad8918fad0259c495184
2020-09-28 19:17:36 +00:00
John Sully
5f01bf9c91 bump version 6.0.16
Former-commit-id: dab9296d7cede9d692e65403b5fa1fa2663fdc6c
2020-09-28 19:14:17 +00:00
John Sully
8b1dad9d92 Merge branch 'unstable' into RELEASE_6
Former-commit-id: a6f5a391a140aa9a28464a00ea4148f387bb8d98
2020-09-28 18:55:24 +00:00
WuYunlong
c2e5546071
Normalize sds test mechanism together with some compile warnings. (#7854) 2020-09-28 11:27:26 +03:00
Oran Agra
8aa083bd28
Fix new obuf-limits tests to work with TLS (#7848)
Also stabilize new shutdown tests on slow machines (valgrind)
2020-09-27 17:13:33 +03:00
caozb
a295770e32
ignore slaveof no one in redis.conf (#7842)
when slaveof config is "no one", reset any pre-existing config and resume.

also solve a memory leak if slaveof appears twice.
and fail loading if port number is out of range or not an integer.

Co-authored-by: caozhengbin <caozb@yidingyun.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-09-27 15:40:07 +03:00
Wang Yuan
c9f00bcce2
Don't support Gopher if enable io threads to read queries (#7851)
There's currently an issue with IO threads and gopher (issuing lookupKey from within the thread).
simply fix is to just not support it for now.
2020-09-27 12:35:16 +03:00
Wang Yuan
f1863a1fe7
Set 'loading' and 'shutdown_asap' to volatile sig_atomic_t type (#7845)
We may access and modify these two variables in signal handler function,
to guarantee them async-signal-safe, so we should set them to volatile
sig_atomic_t type.

It doesn't look like this could have caused any real issue, and it seems that
signals are handled in main thread on most platforms. But we want to follow C
and POSIX standard in signal handler function.
2020-09-25 16:25:47 +03:00
Uri Shachar
c30bd02c9d
Fix config rewrite file handling to make it really atomic (#7824)
Make sure we handle short writes correctly, sync to disk after writing  and use
rename to make sure the replacement is actually atomic.
In any case of failure old configuration will remain in place.

Also, add some additional logging to make it easier to diagnose rewrite problems.
2020-09-25 12:55:45 +03:00
WuYunlong
0d62caab21
Add fsync to readSyncBulkPayload(). (#7839)
We should sync temp DB file before renaming as rdb_fsync_range does not use
flag `SYNC_FILE_RANGE_WAIT_AFTER`.

Refer to `Linux Programmer's Manual`:
SYNC_FILE_RANGE_WAIT_AFTER
    Wait upon write-out of all pages in the range after performing any write.
2020-09-25 08:08:06 +03:00
John Sully
ad4c2390a8 Merge branch 'unstable' into keydbpro
Former-commit-id: d1986d9fcdfd56f2c30ee95edef6d5260e0aa777
2020-09-24 22:02:30 +00:00
Wen Hui
323029baa6
rdb.c: handle fclose error case differently to avoid double fclose (#7307)
When fclose would fail, the previous implementation would have attempted to do fclose again
this can in theory lead to segfault.

other changes:
check for non-zero return value as failure rather than a specific error code.
this doesn't fix a real bug, just a minor cleanup.
2020-09-24 18:17:53 +03:00