Jordi Boggiano
@seldaek
http://nelm.io/

Redis - Your Advanced
In-Memory Key-Value Store

About Me

Belgian living in Zürich, Switzerland

Building the internet for 10 years
http://seld.be

Symfony2, Composer and other OSS contributions
http://github.com/Seldaek

Working at Nelmio
http://nelm.io
Symfony2 & frontend performance consulting

Agenda

Intro
Features
Use Cases
Using it with PHP

Redis

Wut?

Store

You put data in it

Key-Value Store

Like NoSQL?

In-Memory Key-Value Store

Like Memcached?

In-Memory Key-Value Store

Memory is fast, but ephemeral

Persist to Disk

Fast and lasting

Created by @antirez

So how does it work?

Simple Text Protocol

Human Readable!

Example - Client Library

SET key value

Example - Client Library

SET key value
> OK

Example - Client Library

SET key value
> OK
GET key

Example - Client Library

SET key value
> OK
GET key
> "value"

Example - Low Level

*3
$3
SET
$3
key
$5
value

+OK

*2
$3
GET
$3
key

$5
value

Example - Low Level

*3\r\n
$3\r\n
SET\r\n
$3\r\n
key\r\n
$5\r\n
value\r\n

+OK\r\n

*2\r\n
$3\r\n
GET\r\n
$3\r\n
key\r\n

$5\r\n
value\r\n

Data Types

Strings

SET name Bob
SET age 20
MGET name age
> Bob
> 20

GETSET name Alice
> Bob

Strings

SETEX age 3 20
GET age
> 20
// .. 3 seconds later ..
GET age
> null

Integers

INCR count
> 1
INCR count
> 2
INCRBY count 3
> 5

Hashes

HMSET user name Alice email alice@example.org
HGET user email
> alice@example.org
HKEYS user
> name
> email
HGETALL user
> name
> Alice
> email
> alice@example.org

Lists

RPUSH admins Alice
> 1
RPUSH admins Bob
> 2
LINDEX admins 0
> Alice
LLEN admins
> 2
RPOP admins
> Bob

Sets

SADD page:3:visitors 134
> 1
SADD page:3:visitors 253
> 1
SADD page:3:visitors 253
> 0
SCARD page:3:visitors
> 2
SMEMBERS page:3:visitors
> 134
> 253
SISMEMBER page:3:visitors 349
> 0

Sets

SADD page:6:visitors 253
SADD page:6:visitors 923
SADD page:6:visitors 13
SINTER page:3:visitors page:6:visitors
> 253

Sorted Sets

ZADD highscores 2930 Alice
ZADD highscores 1546 Bob
ZREVRANGE highscores 0 10 WITHSCORES
> Alice
> 2930
> Bob
> 1546

Sort of Lists? Listed Sets?

Lists

array('foo', 'bar')

Sets

shuffle(array('foo', 'bar'))

Sorted Sets

ksort(array(3 => 'foo', 1 => 'bar'))

The basic datatypes of every language exist in Redis

Process data in Redis
instead of PHP

A Few Features

Atomic Operations

SETNX name Alice
GET name
> Alice
SETNX name Bob
GET name
> Alice

INCR foo
GET foo
> 1
INCRBY foo 3
GET foo
> 4

EXPIRE / EXPIREAT / PERSIST

Pipelining

SORT - SQL in your NoSQL

SORT key
SORT key LIMIT 0 10 DESC
SORT key ALPHA
SORT page:6:visitors BY user_*->rank GET user_* DESC

Append-Only File & Snapshots

Pub/Sub

SUBSCRIBE foo1
PSUBSCRIBE foo*
PUBLISH foo0 message

Master/Slave Replication

Transactions: MULTI / EXEC / DISCARD

Optimistic Locking with WATCH

Upcoming Features

Millisecond expiration

PEXPIRE / PSETEX / PTTL

Lua Scripting: EVAL / EVALSHA

This is great.

EVAL

EVAL <body> <num_keys_in_args> [<arg1> <arg2> ... <arg_N>]

GET A
EVAL "return redis.call('get', KEYS[1])" 1 A

EVAL

Example: Atomic Conditional Decrement, Client-Side

WATCH foo
$val = GET foo
$newVal = max(0, $val - 1); // decrement if foo > 0 client-side
MULTI
SET foo $newVal
EXEC

This may return -ERR.

EVAL

Example: Atomic Conditional Decrement, Server-Side

EVAL "local value = tonumber(redis.call('get', KEYS[1]))
    if value == nil
    then
        return {err="Value at key is not integer"}
    end
    if value > tonumber(ARGV[1])
    then
        value = value - 1
        redis.call('set', KEYS[1], value)
    end
    return value" 1 foo 0

This can not fail.

Redis Cluster

Almost as delayed as
Duke Nukem Forever

Speed

Redis is FAST.
CPU is unlikely to be the bottleneck.

What do you do when
you run out of RAM?

Wait for Redis Cluster?

If you can. Cluster looks great.

Use sharding, most client
libraries can do it.

Use Edis, a protocol compatible storage-oriented server.

http://inaka.github.com/edis/index.html

Use it without persistence,
store to disk/DB
and load hot data in memory.

A good example in these slides from Wooga

Ok, great. But..

What is it good for?

WEB SCALE

THE CLOUD

NOSQL

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

WEB SCALE

THE CLOUD

NOSQL

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

BULLSHIT

Get real

"If you do actually have to scale, then your database isn't going to magically do it for you."

Really.

What is it good for?

Storing Sessions

Caching Data

Other typical Memcached uses

Caching: Newspaper Site

Expensive reads
Fast-changing pages
Dynamic tracking of what's read for users

Caching: Newspaper Site

Cache generic results:

SETEX <pageid>:content 600 <data>

Store user read-state in sets

SADD <pageid>:views <userid>

Combine and render for each user

GET <pageid>:content
SISMEMBER <pageid>:views <userid>

Logging, games and other write-heavy usages

LPUSH logs "Log message"
// keep the last 1000 entries
LTRIM logs 0 999

Highscore tables

ZADD scores 4290 <playerid>
ZADD scores 390 <playerid2>
// ...
ZREVRANK scores <playerid> // 0
ZREVRANGE scores 0 10 WITHSCORES

Note: players will only be listed once since it is a set.

Job Queues

Workers:

BRPOP queue

Application:

LPUSH queue "job description"

e.g. the Resque lib from GitHub

Behavior Tracking

SETBIT click:<item>:<date> <userid> 1
GETBIT click:3:2012-01-28 55239

Last but not least, it puts the fun back in Datafunbases

Usage with PHP

Extension: phpredis

Comes with a session handler, sharding, up to date.

$redis = new Redis();
$redis->connect('127.0.0.1', 6379);
$redis->watch('x');
$val = $redis->get('x');
$newVal = max(0, $val - 1);
$result = $redis->multi()
    ->set('x', $newVal)
    ->exec();
echo $result !== false ? 'Success' : 'Race lost, try again';

http://github.com/nicolasff/phpredis

Extension: phpiredis

Basic hiredis bindings, protocol parsing, low level.

$redis = phpiredis_connect('127.0.0.1', 6379);
phpiredis_command($redis, 'WATCH x');
$val = phpiredis_command($redis, 'GET x');
$newVal = max(0, $val - 1);
phpiredis_command($redis, 'MULTI');
phpiredis_command($redis, 'SET x '.$newVal);
$result = phpiredis_command($redis, 'EXEC');
echo $result !== false ? 'Success' : 'Race lost, try again';

http://github.com/seppo0010/phpiredis

Plain PHP: Predis

Very complete, sharding, master/slave auto-select,
can use phpiredis for parsing.

$redis = new Predis\Client('tcp://10.0.0.1:6379');

$options = array(
    'cas'   => true,   // enable Check-and-Set
    'watch' => 'x',
    'retry' => 10,     // automatic retries
);

$result = $redis->multiExec($options, function($transaction) {
    $val = $transaction->get('x');
    $newVal = max(0, $val - 1);
    $transaction->multi();
    $transaction->set('x', $newVal);
});

echo $result !== false ? 'Success' : 'Race lost 10 times, giving up';

http://github.com/nrk/predis

http://github.com/snc/SncRedisBundle (Symfony2 integration)

It's fast. It's fun. Try it.

Jordi Boggiano@seldaekhttp://nelm.io/

Redis - Your AdvancedIn-Memory Key-Value Store

About Me

Agenda

Redis

Wut?

Store

You put data in it

Key-Value Store

Like NoSQL?

In-Memory Key-Value Store

Like Memcached?

In-Memory Key-Value Store

Memory is fast, but ephemeral

Persist to Disk

Fast and lasting

Created by @antirez

Sponsored by VMWare

So how does it work?

Simple Text Protocol

Human Readable!

Example - Client Library

Example - Client Library

Example - Client Library

Example - Client Library

Example - Low Level

Example - Low Level

Data Types

Strings

Strings

Integers

Hashes

Lists

Sets

Sets

Sorted Sets

Sort of Lists? Listed Sets?

Lists

Sets

Sorted Sets

The basic datatypes of every language exist in Redis

Process data in Redisinstead of PHP

A Few Features

Atomic Operations

EXPIRE / EXPIREAT / PERSIST

Pipelining

SORT - SQL in your NoSQL

Append-Only File & Snapshots

Pub/Sub

Master/Slave Replication

Transactions: MULTI / EXEC / DISCARD

Optimistic Locking with WATCH

Upcoming Features

Millisecond expiration

PEXPIRE / PSETEX / PTTL

Lua Scripting: EVAL / EVALSHA

This is great.

EVAL

EVAL

EVAL

Redis Cluster

Almost as delayed asDuke Nukem Forever

Speed

Redis is FAST.CPU is unlikely to be the bottleneck.

What do you do whenyou run out of RAM?

Wait for Redis Cluster?

If you can. Cluster looks great.

Use sharding, most clientlibraries can do it.

Use Edis, a protocol compatible storage-oriented server.

Use it without persistence, store to disk/DBand load hot data in memory.

Ok, great. But..

What is it good for?

WEB SCALE

THE CLOUD

NOSQL

ELASTIC

HORIZONTAL SCALING

VERTICAL TOO

WEB SCALE

THE CLOUD

Jordi Boggiano
@seldaek
http://nelm.io/

Redis - Your Advanced
In-Memory Key-Value Store

Process data in Redis
instead of PHP

Almost as delayed as
Duke Nukem Forever

Redis is FAST.
CPU is unlikely to be the bottleneck.

What do you do when
you run out of RAM?

Use sharding, most client
libraries can do it.

Use it without persistence,
store to disk/DB
and load hot data in memory.