2019-02-23 01:28:44 -05:00
What is KeyDB?
2015-02-13 15:34:22 +01:00
--------------
2019-02-23 01:28:44 -05:00
KeyDB is a high performance fork of Redis focussing on multithreading, memory efficiency, and high throughput. In addition to multithreading KeyDB also has features only available in Redis Enterprise such as FLASH storage support, and some not available at all such as direct backup to AWS S3.
2015-02-13 15:34:22 +01:00
2019-02-23 01:28:44 -05:00
On the same hardware KeyDB can perform twice as many queries per second as Redis, with 60% lower latency. Our work here is not complete and we expect substantially better results as we move more code away from the global lock.
2015-02-13 15:34:22 +01:00
2019-02-23 01:28:44 -05:00
Why fork Redis?
---------------
2015-02-13 15:34:22 +01:00
2019-02-23 01:28:44 -05:00
The Redis maintainers have continually reiterated that they do not plan to support multithreading. While we have great respect for the redis team, we feel the analysis justifying this decision is incorrect. In addition we are dismayed that features we believe belong in the core Redis engine are ending up in closed source modules. We feel a fork is the best way to accelerate development in the areas of most interest to us.
2015-02-13 15:34:22 +01:00
2019-02-23 01:28:44 -05:00
We plan to track the Redis repo closely and hope our projects can learn from each other.
New Configuration Options
-------------------------
2015-02-13 15:34:22 +01:00
2019-02-23 01:28:44 -05:00
With new features comes new options:
2012-04-21 12:08:26 +02:00
2019-02-23 01:28:44 -05:00
**server-threads** *N*
2019-02-23 01:32:50 -05:00
2019-02-23 01:33:27 -05:00
The number of threads used to serve requests. This should be related to the number of queues available in your network hardware, *not* the number of cores on your machine. Because KeyDB uses spinlocks to reduce latency; making this too high will reduce performance. We recommend using 4 here. By default this is set to one.
2012-04-21 12:08:26 +02:00
2019-02-23 01:28:44 -05:00
**scratch-file-path** *path*
2019-02-23 01:32:50 -05:00
If you would like to use the FLASH backed storage this option configures the directory for KeyDB's temporary files. This feature relies on snapshotting to work so must be used on a BTRFS filesystem. ZFS may also work but is untested. With this feature KeyDB will use RAM as a cache and page to disk as necessary.
2019-02-23 01:28:44 -05:00
**db-s3-object** *path*
2019-02-23 01:32:50 -05:00
If you would like KeyDB to dump directly to AWS S3 this option specifies the bucket. Using this option with the traditional RDB options will result in KeyDB backing up twice to both locations. This requires the AWS CLI tools to be installed and configured which are used under the hood to transfer the data.
All other configuration options behave as you'd expect. Your existing configuration files should continue to work unchanged.
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
Building KeyDB
--------------
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
KeyDB can be compiled and is tested for use on Linux. KeyDB currently relies on SO_REUSEADDR's load balancing behavior which is available only in Linux. When we support marshalling connections across threads we plan to support other operating systems such as FreeBSD.
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
Compiling is as simple as:
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
% make
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
You can enable flash support with (Note: autoconf and autotools must be installed):
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
% make MALLOC=memkind
2010-08-31 11:42:52 +02:00
2015-01-08 16:35:05 +01:00
Fixing build problems with dependencies or cached build options
2015-02-13 15:10:20 +01:00
---------
2019-02-23 01:28:44 -05:00
KeyDB has some dependencies which are included into the `deps` directory.
2016-01-21 16:57:36 +05:30
`make` does not automatically rebuild dependencies even if something in
the source code of dependencies changes.
2015-01-08 16:35:05 +01:00
When you update the source code with `git pull` or when code inside the
dependencies tree is modified in any other way, make sure to use the following
command in order to really clean everything and rebuild from scratch:
make distclean
This will clean: jemalloc, lua, hiredis, linenoise.
Also if you force certain build options like 32bit target, no C compiler
optimizations (for debugging purposes), and other similar build time options,
2015-02-13 15:10:20 +01:00
those options are cached indefinitely until you issue a `make distclean`
2015-01-08 16:35:05 +01:00
command.
2015-01-09 00:07:25 +09:00
2013-05-08 09:48:38 +02:00
Fixing problems building 32 bit binaries
---------
2019-02-23 01:28:44 -05:00
If after building KeyDB with a 32 bit target you need to rebuild it
2013-05-08 09:48:38 +02:00
with a 64 bit target, or the other way around, you need to perform a
2019-02-23 01:28:44 -05:00
`make distclean` in the root directory of the KeyDB distribution.
2013-05-08 09:48:38 +02:00
2019-02-23 01:28:44 -05:00
In case of build errors when trying to build a 32 bit binary of KeyDB, try
2013-05-08 09:48:38 +02:00
the following steps:
* Install the packages libc6-dev-i386 (also try g++-multilib).
2015-02-13 15:10:20 +01:00
* Try using the following command line instead of `make 32bit` :
`make CFLAGS="-m32 -march=native" LDFLAGS="-m32"`
2011-10-18 14:28:07 -04:00
Allocator
---------
2019-02-23 01:28:44 -05:00
Selecting a non-default memory allocator when building KeyDB is done by setting
the `MALLOC` environment variable. KeyDB is compiled and linked against libc
2011-11-15 13:28:57 -08:00
malloc by default, with the exception of jemalloc being the default on Linux
systems. This default was picked because jemalloc has proven to have fewer
fragmentation problems than libc malloc.
2011-10-18 14:28:07 -04:00
2011-11-15 13:28:57 -08:00
To force compiling against libc malloc, use:
2011-10-18 14:28:07 -04:00
2011-11-15 13:28:57 -08:00
% make MALLOC=libc
2011-10-18 14:28:07 -04:00
2011-11-15 13:28:57 -08:00
To compile against jemalloc on Mac OS X systems, use:
2011-10-18 14:28:07 -04:00
2011-11-15 13:28:57 -08:00
% make MALLOC=jemalloc
2011-10-18 14:28:07 -04:00
Verbose build
-------------
2019-02-23 01:28:44 -05:00
KeyDB will build with a user friendly colorized output by default.
2011-10-18 14:28:07 -04:00
If you want to see a more verbose output use the following:
2011-10-23 12:28:31 +02:00
% make V=1
2010-10-22 00:06:44 +02:00
2019-02-23 01:28:44 -05:00
Running KeyDB
2010-08-31 11:42:52 +02:00
-------------
2019-02-23 01:28:44 -05:00
To run KeyDB with the default configuration just type:
2010-08-31 11:42:52 +02:00
% cd src
2019-02-09 10:11:46 -05:00
% ./keydb-server
2015-11-15 20:03:18 +05:30
2019-02-23 01:28:44 -05:00
If you want to provide your keydb.conf, you have to run it using an additional
2010-08-31 11:42:52 +02:00
parameter (the path of the configuration file):
% cd src
2019-02-23 01:28:44 -05:00
% ./keydb-server /path/to/keydb.conf
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
It is possible to alter the KeyDB configuration by passing parameters directly
2011-12-01 15:40:49 +01:00
as options using the command line. Examples:
2019-02-09 10:11:46 -05:00
% ./keydb-server --port 9999 --replicaof 127.0.0.1 6379
2019-02-23 01:28:44 -05:00
% ./keydb-server /etc/keydb/6379.conf --loglevel debug
2011-12-01 15:40:49 +01:00
2019-02-23 01:28:44 -05:00
All the options in keydb.conf are also supported as options using the command
2011-12-01 15:40:49 +01:00
line, with exactly the same name.
2019-02-23 01:28:44 -05:00
Playing with KeyDB
2010-08-31 11:42:52 +02:00
------------------
2019-02-23 01:28:44 -05:00
You can use keydb-cli to play with KeyDB. Start a keydb-server instance,
2010-08-31 11:42:52 +02:00
then in another terminal try the following:
% cd src
2019-02-09 10:11:46 -05:00
% ./keydb-cli
2019-02-23 01:28:44 -05:00
keydb> ping
2010-08-31 11:42:52 +02:00
PONG
2019-02-23 01:28:44 -05:00
keydb> set foo bar
2010-08-31 11:42:52 +02:00
OK
2019-02-23 01:28:44 -05:00
keydb> get foo
2010-08-31 11:42:52 +02:00
"bar"
2019-02-23 01:28:44 -05:00
keydb> incr mycounter
2010-08-31 11:42:52 +02:00
(integer) 1
2019-02-23 01:28:44 -05:00
keydb> incr mycounter
2010-08-31 11:42:52 +02:00
(integer) 2
2019-02-23 01:28:44 -05:00
keydb>
2010-08-31 11:42:52 +02:00
2015-02-13 14:55:09 +01:00
You can find the list of all the available commands at http://redis.io/commands.
2010-08-31 11:42:52 +02:00
2019-02-23 01:28:44 -05:00
Installing KeyDB
2011-10-23 12:28:31 +02:00
-----------------
2019-02-23 01:28:44 -05:00
In order to install KeyDB binaries into /usr/local/bin just use:
2011-10-23 12:28:31 +02:00
% make install
2015-02-13 14:55:09 +01:00
You can use `make PREFIX=/some/other/directory install` if you wish to use a
2011-10-23 12:28:31 +02:00
different destination.
Make install will just install binaries in your system, but will not configure
init scripts and configuration files in the appropriate place. This is not
2019-02-23 01:28:44 -05:00
needed if you want just to play a bit with KeyDB, but if you are installing
2011-10-23 12:28:31 +02:00
it the proper way for a production system, we have a script doing this
for Ubuntu and Debian systems:
% cd utils
2014-03-15 14:43:50 +01:00
% ./install_server.sh
2011-10-23 12:28:31 +02:00
The script will ask you a few questions and will setup everything you need
2019-02-23 01:28:44 -05:00
to run KeyDB properly as a background daemon that will start again on
2011-10-23 12:28:31 +02:00
system reboots.
2019-02-23 01:28:44 -05:00
You'll be able to stop and start KeyDB using the script named
`/etc/init.d/keydb_<portnumber>` , for instance `/etc/init.d/keydb_6379` .
2011-10-23 12:28:31 +02:00
2012-11-08 18:43:20 +01:00
Code contributions
2015-11-15 20:03:18 +05:30
-----------------
2012-11-08 18:43:20 +01:00
2019-02-23 01:28:44 -05:00
Note: by contributing code to the KeyDB project in any form, including sending
2012-11-08 18:43:20 +01:00
a pull request via Github, a code fragment or patch via private email or
public discussion groups, you agree to release your code under the terms
2019-02-23 01:28:44 -05:00
of the BSD license that you can find in the COPYING file included in the KeyDB
2012-11-08 18:43:20 +01:00
source distribution.
2019-02-23 01:28:44 -05:00
Please see the CONTRIBUTING file in this source distribution for more
2012-11-08 18:43:20 +01:00
information.
2015-11-04 12:29:54 +01:00