Jordi Boggiano
@seldaek
http://nelm.io/


Redis - Your Advanced
In-Memory Key-Value Store

About Me

Belgian living in Zürich, Switzerland

Building the internet for 10 years
 http://seld.be

Symfony2, Composer and other OSS contributions
 http://github.com/Seldaek

Working at Nelmio
 http://nelm.io
 Symfony2 & frontend performance consulting

Agenda

Redis

Wut?

Store

You put data in it

Key-Value Store

Like NoSQL?

In-Memory Key-Value Store

Like Memcached?

In-Memory Key-Value Store

Memory is fast, but ephemeral

Persist to Disk

Fast and lasting

Created by @antirez

Sponsored by VMWare

So how does it work?

Simple Text Protocol

Human Readable!

Example - Client Library

SET key value
            

Example - Client Library

SET key value
> OK
            

Example - Client Library

SET key value
> OK
GET key
            

Example - Client Library

SET key value
> OK
GET key
> "value"
            

Example - Low Level

*3
$3
SET
$3
key
$5
value
            
+OK
            
*2
$3
GET
$3
key
            
$5
value
            

Example - Low Level

*3\r\n
$3\r\n
SET\r\n
$3\r\n
key\r\n
$5\r\n
value\r\n
            
+OK\r\n
            
*2\r\n
$3\r\n
GET\r\n
$3\r\n
key\r\n
            
$5\r\n
value\r\n
            

Data Types

Strings

SET name Bob
SET age 20
MGET name age
> Bob
> 20
                
GETSET name Alice
> Bob
                

Strings

SETEX age 3 20
GET age
> 20
// .. 3 seconds later ..
GET age
> null
                

Integers

INCR count
> 1
INCR count
> 2
INCRBY count 3
> 5
                

Hashes

HMSET user name Alice email alice@example.org
HGET user email
> alice@example.org
HKEYS user
> name
> email
HGETALL user
> name
> Alice
> email
> alice@example.org
                

Lists

RPUSH admins Alice
> 1
RPUSH admins Bob
> 2
LINDEX admins 0
> Alice
LLEN admins
> 2
RPOP admins
> Bob
                

Sets

SADD page:3:visitors 134
> 1
SADD page:3:visitors 253
> 1
SADD page:3:visitors 253
> 0
SCARD page:3:visitors
> 2
SMEMBERS page:3:visitors
> 134
> 253
SISMEMBER page:3:visitors 349
> 0
                

Sets

SADD page:6:visitors 253
SADD page:6:visitors 923
SADD page:6:visitors 13
SINTER page:3:visitors page:6:visitors
> 253
                

Sorted Sets

ZADD highscores 2930 Alice
ZADD highscores 1546 Bob
ZREVRANGE highscores 0 10 WITHSCORES
> Alice
> 2930
> Bob
> 1546
                

Sort of Lists? Listed Sets?

Lists

array('foo', 'bar')
                

Sets

shuffle(array('foo', 'bar'))
                

Sorted Sets

ksort(array(3 => 'foo', 1 => 'bar'))
                

The basic datatypes of every language exist in Redis

Process data in Redis
instead of PHP

A Few Features

Atomic Operations

SETNX name Alice
GET name
> Alice
SETNX name Bob
GET name
> Alice
                
INCR foo
GET foo
> 1
INCRBY foo 3
GET foo
> 4
                

EXPIRE / EXPIREAT / PERSIST

Pipelining

SORT - SQL in your NoSQL

SORT key
SORT key LIMIT 0 10 DESC
SORT key ALPHA
SORT page:6:visitors BY user_*->rank GET user_* DESC
                

Append-Only File & Snapshots

Pub/Sub

SUBSCRIBE foo1
PSUBSCRIBE foo*
PUBLISH foo0 message
                

Master/Slave Replication

Transactions: MULTI / EXEC / DISCARD

Optimistic Locking with WATCH

Upcoming Features

Millisecond expiration

PEXPIRE / PSETEX / PTTL

Lua Scripting: EVAL / EVALSHA

This is great.

EVAL

EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>]
            
GET A
EVAL "return redis.call('get', KEYS[1])" 1 A
            

EVAL

Example: Atomic Conditional Decrement, Client-Side

WATCH foo
$val = GET foo
$newVal = max(0, $val - 1); // decrement if foo > 0 client-side
MULTI
SET foo $newVal
EXEC
            

This may return -ERR.

EVAL

Example: Atomic Conditional Decrement, Server-Side

EVAL "local value = tonumber(redis.call('get', KEYS[1]))
    if value == nil
    then
        return {err="Value at key is not integer"}
    end
    if value > tonumber(ARGV[1])
    then
        value = value - 1
        redis.call('set', KEYS[1], value)
    end
    return value" 1 foo 0
            

This can not fail.

Redis Cluster

Almost as delayed as
Duke Nukem Forever

Speed

Redis is FAST.
CPU is unlikely to be the bottleneck.

What do you do when
you run out of RAM?

Wait for Redis Cluster?

If you can. Cluster looks great.

Use sharding, most client
libraries can do it.

Use Edis, a protocol compatible storage-oriented server.

http://inaka.github.com/edis/index.html

Use it without persistence,
store to disk/DB
and load hot data in memory.

A good example in these slides from Wooga

Ok, great. But..

What is it good for?

WEB SCALE

THE CLOUD

NOSQL

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

WEB SCALE

THE CLOUD

NOSQL

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

BULLSHIT

Get real

"If you do actually have to scale, then your database isn't going to magically do it for you."

Really.

What is it good for?

Storing Sessions

Caching Data

Other typical Memcached uses

Caching: Newspaper Site

Caching: Newspaper Site

Cache generic results:

SETEX <pageid>:content 600 <data>
            

Store user read-state in sets

SADD <pageid>:views <userid>
            

Combine and render for each user

GET <pageid>:content
SISMEMBER <pageid>:views <userid>
            

Logging, games and other write-heavy usages

LPUSH logs "Log message"
// keep the last 1000 entries
LTRIM logs 0 999
                

Highscore tables

ZADD scores 4290 <playerid>
ZADD scores 390 <playerid2>
// ...
ZREVRANK scores <playerid> // 0
ZREVRANGE scores 0 10 WITHSCORES
                

Note: players will only be listed once since it is a set.

Who's online now?

ZADD visits <unix> <userid>
ZADD visits 1327681399 52
ZADD visits 1327683245 18
ZRANGEBYSCORE visits <unix>-3600 <unix>
ZREMRANGEBYSCORE visits 0 <unix>-3600
                

Job Queues

Workers:

BRPOP queue
                

Application:

LPUSH queue "job description"
                

e.g. the Resque lib from GitHub

Behavior Tracking

SETBIT click:<item>:<date> <userid> 1
GETBIT click:3:2012-01-28 55239
                

Last but not least, it puts the fun back in Datafunbases

Usage with PHP

Extension: phpredis

Extension: phpredis

Comes with a session handler, sharding, up to date.

$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$redis->watch('x');
$val = $redis->get('x');
$newVal = max(0, $val - 1);
$result = $redis->multi()
    ->set('x', $newVal)
    ->exec();
echo $result !== false ? 'Success' : 'Race lost, try again';
            

http://github.com/nicolasff/phpredis

Extension: phpiredis

Extension: phpiredis

Basic hiredis bindings, protocol parsing, low level.

$redis = phpiredis_connect('127.0.0.1', 6379);
phpiredis_command($redis, 'WATCH x');
$val = phpiredis_command($redis, 'GET x');
$newVal = max(0, $val - 1);
phpiredis_command($redis, 'MULTI');
phpiredis_command($redis, 'SET x '.$newVal);
$result = phpiredis_command($redis, 'EXEC');
echo $result !== false ? 'Success' : 'Race lost, try again';
            

http://github.com/seppo0010/phpiredis

Plain PHP: Predis

Plain PHP: Predis

Very complete, sharding, master/slave auto-select,
can use phpiredis for parsing.

$redis = new Predis\Client('tcp://10.0.0.1:6379');

$options = array(
    'cas'   => true,   // enable Check-and-Set
    'watch' => 'x',
    'retry' => 10,     // automatic retries
);

$result = $redis->multiExec($options, function($transaction) {
    $val = $transaction->get('x');
    $newVal = max(0, $val - 1);
    $transaction->multi();
    $transaction->set('x', $newVal);
});

echo $result !== false ? 'Success' : 'Race lost 10 times, giving up';
            

http://github.com/nrk/predis

http://github.com/snc/SncRedisBundle (Symfony2 integration)

It's fast. It's fun. Try it.

try.redis-db.com

Thank you.

Find Out More

Questions?

jordi@nelm.io

@seldaek

slides.seld.be


Feedback:

http://joind.in/6079