27628 Commits

Author SHA1 Message Date
Wen Hui
93d87d6d4c
[module] using predefined REDISMODULE_NO_EXPIRE in RM_GetExpire (#7669)
It was already defined in the API header and the documentation, but not used by the implementation.
2020-08-18 08:50:03 +03:00
Oran Agra
05a4af3464 Trim trailing spaces in error replies coming from rejectCommand (#7668)
fe8d6fe749 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.
2020-08-18 08:28:43 +03:00
Oran Agra
cdd925b289
Trim trailing spaces in error replies coming from rejectCommand (#7668)
65a3307bc9 added rejectCommand which takes an robj reply and passes it
through addReplyErrorSafe to addReplyErrorLength.
The robj contains newline at it's end, but addReplyErrorSafe converts it
to spaces, and passes it to addReplyErrorLength which adds the protocol
newlines.

The result was that most error replies (like OOM) had extra two trailing
spaces in them.
2020-08-18 08:28:43 +03:00
Yossi Gottlieb
2ec11f941a Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.
2020-08-17 17:46:54 +03:00
Yossi Gottlieb
64c360c515
Module API: fix missing RM_CLIENTINFO_FLAG_SSL. (#7666)
The `REDISMODULE_CLIENTINFO_FLAG_SSL` flag was already a part of the `RedisModuleClientInfo` structure but was not implemented.
2020-08-17 17:46:54 +03:00
Yossi Gottlieb
24efd22e89 TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.
2020-08-17 17:36:50 +03:00
Yossi Gottlieb
fb2a94af3f
TLS: relax verification on CONFIG SET. (#7665)
Avoid re-configuring (and validating) SSL/TLS configuration on `CONFIG
SET` when TLS is not actively enabled for incoming connections, cluster
bus or replication.

This fixes failures when tests run without `--tls` on binaries that were
built with TLS support.

An additional benefit is that it's now possible to perform a multi-step
configuration process while TLS is disabled. The new configuration will
be verified and applied only when TLS is effectively enabled.
2020-08-17 17:36:50 +03:00
John Sully
dcf607622c Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: ed98be0ba81ffdc501847ea0d2486f5f01391319
2020-08-17 02:26:51 +00:00
John Sully
7beb3401cf Merge branch 'keydbpro' into PRO_RELEASE_6
Former-commit-id: ed98be0ba81ffdc501847ea0d2486f5f01391319
2020-08-17 02:26:51 +00:00
John Sully
93a67abe52 Fix ARM build of RocksDB
Former-commit-id: 7301d6d5e76b65115e2610a5fc5afee443782d7c
2020-08-17 02:25:45 +00:00
John Sully
bfa8e2b6e2 Fix ARM build of RocksDB
Former-commit-id: 7301d6d5e76b65115e2610a5fc5afee443782d7c
2020-08-17 02:25:45 +00:00
John Sully
e8e7061a73 Fast cleanup of snapshots without leaving them forever
Former-commit-id: fdd83b2b49244ed2988b080892ee5cffe9fd2684
2020-08-17 00:33:37 +00:00
John Sully
035cde502a Fast cleanup of snapshots without leaving them forever
Former-commit-id: fdd83b2b49244ed2988b080892ee5cffe9fd2684
2020-08-17 00:33:37 +00:00
John Sully
c0586b3aed Allow garbage collection of generic data
Former-commit-id: feadb7fb1845027422bcfca43dbcb6097409b8dc
2020-08-17 00:32:48 +00:00
John Sully
b71f16471b Allow garbage collection of generic data
Former-commit-id: feadb7fb1845027422bcfca43dbcb6097409b8dc
2020-08-17 00:32:48 +00:00
Oran Agra
40ef9556c3 Merge pull request #7661 from michael-grunder/hiredis-unique-sds-symbols
This resolves an issue with sentinel that was created when hiredis was recently updated.
this was due to sds symbol names clashing, since hiredis now includes different implementation
of sdsrange than the one in redis.

The state of things is that redis-benchamrk and redis-cli include only hiredis sds implementation.
redis-cli even operates (calls sdscatlen) on sds that's allocated by hiredis.

Sentinel however has both implementations of the sds library in it (now each with it's own unique
symbols).
2020-08-16 08:54:36 +03:00
Oran Agra
1407a055ee
Merge pull request #7661 from michael-grunder/hiredis-unique-sds-symbols
This resolves an issue with sentinel that was created when hiredis was recently updated.
this was due to sds symbol names clashing, since hiredis now includes different implementation
of sdsrange than the one in redis.

The state of things is that redis-benchamrk and redis-cli include only hiredis sds implementation.
redis-cli even operates (calls sdscatlen) on sds that's allocated by hiredis.

Sentinel however has both implementations of the sds library in it (now each with it's own unique
symbols).
2020-08-16 08:54:36 +03:00
John Sully
00934cd341 Add snapshot depth to info string
Former-commit-id: 66aef678e9d6d7ab6c28622d54ada0ea8cdd2a99
2020-08-16 00:27:10 +00:00
John Sully
e97e4f1a35 Add snapshot depth to info string
Former-commit-id: 66aef678e9d6d7ab6c28622d54ada0ea8cdd2a99
2020-08-16 00:27:10 +00:00
John Sully
4a356ac13f Don't try and consolidate snapshots with a depth of 1
Former-commit-id: 26c298bd9bc4e2c6981de5c20284120ea54580c3
2020-08-16 00:26:05 +00:00
John Sully
6ba6652fcf Don't try and consolidate snapshots with a depth of 1
Former-commit-id: 26c298bd9bc4e2c6981de5c20284120ea54580c3
2020-08-16 00:26:05 +00:00
John Sully
3093d9e117 Only save MVCC tstamps if its an active replica
Former-commit-id: 5aded7dbad055308dc3932f7797fc71684b60966
2020-08-16 00:24:18 +00:00
John Sully
c3365964a0 Only save MVCC tstamps if its an active replica
Former-commit-id: 5aded7dbad055308dc3932f7797fc71684b60966
2020-08-16 00:24:18 +00:00
John Sully
cf4e74006f Don't free snapshot objects in a critical path (under the AE lock)
Former-commit-id: d0da3d3cb74334cc8a2d14f4bdaef7935181700a
2020-08-16 00:13:19 +00:00
John Sully
71506d7e0a Don't free snapshot objects in a critical path (under the AE lock)
Former-commit-id: d0da3d3cb74334cc8a2d14f4bdaef7935181700a
2020-08-16 00:13:19 +00:00
John Sully
311b286d41 BUG: Cannot have an EMBSTR bigger than 255
Former-commit-id: e2d6e2d9d585cb7a73f469a8580f9cb0ec71a429
2020-08-15 23:30:22 +00:00
John Sully
e946e5f907 BUG: Cannot have an EMBSTR bigger than 255
Former-commit-id: e2d6e2d9d585cb7a73f469a8580f9cb0ec71a429
2020-08-15 23:30:22 +00:00
John Sully
82989540c4 remove unnecessary allocation in rdb save
Former-commit-id: dfa76f04fd862e0cc7b4d3284b3922e3a5e5549d
2020-08-15 23:27:32 +00:00
John Sully
2268a2b761 remove unnecessary allocation in rdb save
Former-commit-id: dfa76f04fd862e0cc7b4d3284b3922e3a5e5549d
2020-08-15 23:27:32 +00:00
John Sully
ce69a765f8 Remove unnecessary key comparisons in perf critical snapshot paths
Former-commit-id: 40f8a8d102fdca9443399ef03a47df609b146d58
2020-08-15 23:25:58 +00:00
John Sully
a14b2097c3 Remove unnecessary key comparisons in perf critical snapshot paths
Former-commit-id: 40f8a8d102fdca9443399ef03a47df609b146d58
2020-08-15 23:25:58 +00:00
John Sully
1055543817 dictMerge perf improvements
Former-commit-id: 48401ec369c5693689ef658cca518dc94ab1402e
2020-08-15 23:14:29 +00:00
John Sully
0311011131 dictMerge perf improvements
Former-commit-id: 48401ec369c5693689ef658cca518dc94ab1402e
2020-08-15 23:14:29 +00:00
John Sully
eb572c57e2 Rehash efficiency
Former-commit-id: fab383156626ec683881101c22eb2f6c2cea4c5d
2020-08-15 23:05:56 +00:00
John Sully
34937b0ad5 Rehash efficiency
Former-commit-id: fab383156626ec683881101c22eb2f6c2cea4c5d
2020-08-15 23:05:56 +00:00
John Sully
6c6ca43eef Prevent unnecessary copy when overwriting a value from a snapshot
Former-commit-id: 654a7bc6ea82f4ac45a1c1a25c794e1c27c0d902
2020-08-15 22:59:01 +00:00
John Sully
07c019fd3d Prevent unnecessary copy when overwriting a value from a snapshot
Former-commit-id: 654a7bc6ea82f4ac45a1c1a25c794e1c27c0d902
2020-08-15 22:59:01 +00:00
John Sully
c804b28446 Merge branch 'unstable' into keydbpro
Former-commit-id: ae482585f0dc470efd73833f74111c2f87a172c5
2020-08-15 21:29:00 +00:00
John Sully
db193a1ef1 Merge branch 'unstable' into keydbpro
Former-commit-id: ae482585f0dc470efd73833f74111c2f87a172c5
2020-08-15 21:29:00 +00:00
michael-grunder
3db28bfe68 Use Hiredis' sdscompat.h to map sds* calls to hi_sds* 2020-08-15 13:13:23 -07:00
michael-grunder
275068c86f Use Hiredis' sdscompat.h to map sds* calls to hi_sds* 2020-08-15 13:13:23 -07:00
michael-grunder
82ae9ff451 Merge commit 'bffbbeaa9a1a6b8e81384297272cb0631502e8fd' into hiredis-unique-sds-symbols 2020-08-15 12:24:31 -07:00
michael-grunder
6866158f0d Merge commit 'bffbbeaa9a1a6b8e81384297272cb0631502e8fd' into hiredis-unique-sds-symbols 2020-08-15 12:24:31 -07:00
michael-grunder
bffbbeaa9a Squashed 'deps/hiredis/' changes from d5b4c69b7..00272d669
00272d669 Rename sds calls so they don't conflict in Redis.

git-subtree-dir: deps/hiredis
git-subtree-split: 00272d669b11e96b8311d9bfe167c117f8887dd6
2020-08-15 12:24:31 -07:00
John Sully
6b8e979434 Prehash the tombstone for cleanup
Former-commit-id: c9d97a7c7448fc769486175bea1648589487c87c
2020-08-14 16:05:39 +00:00
John Sully
6c83ecbb48 Prehash the tombstone for cleanup
Former-commit-id: c9d97a7c7448fc769486175bea1648589487c87c
2020-08-14 16:05:39 +00:00
Nathan Scott
9d4736b044 Annotate module API functions in redismodule.h for use with -fno-common (#6900)
In order to keep the redismodule.h self-contained but still usable with
gcc v10 and later, annotate each API function tentative definition with
the __common__ attribute.  This avoids the 'multiple definition' errors
modules will otherwise see for all API functions at link time.

Further details at gcc.gnu.org/gcc-10/porting_to.html

Turn the existing __attribute__ ((unused)), ((__common__)) and ((print))
annotations into conditional macros for any compilers not accepting this
syntax.  These macros only expand to API annotations under gcc.

Provide a pre- and post- macro for every API function, so that they can
be defined differently by the file that includes redismodule.h.

Removing REDISMODULE_API_FUNC in the interest of keeping the function
declarations readable.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-08-14 14:45:34 +03:00
Nathan Scott
11cd983d58
Annotate module API functions in redismodule.h for use with -fno-common (#6900)
In order to keep the redismodule.h self-contained but still usable with
gcc v10 and later, annotate each API function tentative definition with
the __common__ attribute.  This avoids the 'multiple definition' errors
modules will otherwise see for all API functions at link time.

Further details at gcc.gnu.org/gcc-10/porting_to.html

Turn the existing __attribute__ ((unused)), ((__common__)) and ((print))
annotations into conditional macros for any compilers not accepting this
syntax.  These macros only expand to API annotations under gcc.

Provide a pre- and post- macro for every API function, so that they can
be defined differently by the file that includes redismodule.h.

Removing REDISMODULE_API_FUNC in the interest of keeping the function
declarations readable.

Co-authored-by: Yossi Gottlieb <yossigo@gmail.com>
Co-authored-by: Oran Agra <oran@redislabs.com>
2020-08-14 14:45:34 +03:00
caozb
6ad1896ab3 wait command optimization (#7333)
Client that issued WAIT last will most likely have the highest replication offset, so imagine a probably common case where all clients are waiting for the same number of replicas. we prefer the loop to start from the last client (one waiting for the highest offset), so that the optimization in the function will call replicationCountAcksByOffset for each client until it found a good one, and stop calling it for the rest of the clients.
the way the loop was implemented would mean that in such case it is likely to call replicationCountAcksByOffset for all clients.

Note: the change from > to >= is not directly related to the above.

Co-authored-by: 曹正斌 <caozb@jiedaibao.com>
2020-08-13 14:30:51 +03:00
caozb
d10b2f3173
wait command optimization (#7333)
Client that issued WAIT last will most likely have the highest replication offset, so imagine a probably common case where all clients are waiting for the same number of replicas. we prefer the loop to start from the last client (one waiting for the highest offset), so that the optimization in the function will call replicationCountAcksByOffset for each client until it found a good one, and stop calling it for the rest of the clients.
the way the loop was implemented would mean that in such case it is likely to call replicationCountAcksByOffset for all clients.

Note: the change from > to >= is not directly related to the above.

Co-authored-by: 曹正斌 <caozb@jiedaibao.com>
2020-08-13 14:30:51 +03:00