Jordi Boggiano
@seldaek
http://nelm.io/


High Performance Websites with Symfony2

About Me

Belgian living in Zürich, Switzerland

Weby stuff for 10 years
 http://seld.be

Symfony2 core dev and other OSS contributions
 http://github.com/Seldaek

Works at Nelmio
 http://nelm.io
 We do Symfony2 & Frontend Performance consulting

What about you?

Agenda

What is Performance?

What is Performance?

A word on Scalability

Scaling Vertically

Scaling Horizontally

Performance Analysis

Premature optimization

is the root of all evil

Don't rush things

Figure out the root cause first

Measure, change, measure

Performance matters are sometimes counter-intuitive

Let's get started

$ cd ~/www
$ git clone http://github.com/nelmio/slow.git
$ cd slow
$ app/console doctrine:schema:create
$ app/console slow:populate-db
                

Symfony2 Web Profiler

JMSDebuggingBundle

schmittjoh/JMSDebuggingBundle

Digging deeper with XHProf

XHProf
XHGui preinheimer/xhprof
jonaswouters/XhprofBundle

Note: Xdebug works too to profile on your machine

We have found a bottleneck

Let's see how to fix it

Backend Performance

Fixing Common
Performance Problems

Install APC

Seriously. Now.

Developer Mistakes

Usually easy to spot with a profiler, just fix them. A few examples:

Slow I/O: Filesystem

Bad:

echo file_get_contents('http://example.org/remote-page.html');
                

Better, with knplabs/Gaufrette

use Gaufrette\Filesystem;
use Gaufrette\Adapter\Ftp as FtpAdapter;
use Gaufrette\Adapter\Local as LocalAdapter;
use Gaufrette\Adapter\Cache as CacheAdapter;

$local = new LocalAdapter($cacheDirectory, true);
$ftp = new FtpAdapter($path, $host, $username, $password, $port);

// Cached Adapter with 3600 seconds time to live
$cachedFtp = new CacheAdapter($ftp, $local, 3600);

$filesystem = new Filesystem($cachedFtp);
                

It could use a pull request for an APC adapter (hint hint)

Slow I/O: DB

Bad:

mysqli_query('SELECT * FROM lala
    WHERE foo LIKE '%foo%'
    OR foo LIKE '%bar%'
    OR bar LIKE '%baz%'
    OR bar LIKE '%foo%'
');
                

Better:

Slow I/O: Emails

Build a queue if you need any kind of reliability, run it with Supervisord:

# /etc/supervisor/conf.d/email-dispatcher.conf

[program:email-dispatcher]
numprocs=1
autostart=true
autorestart=true
startretries=100
user=root
directory=/path/to/symfony/
stdout_logfile=/path/to/symfony/app/logs/email-dispatcher.log
command=/usr/local/bin/php /path/to/symfony/app/console email:send
                

More details in a talk from Jon Wage

Slow I/O: Emails

Best leave the actual sending to a third party (sendgrid, postmark, mailchimp, ..)

General Performance Advice

General Performance Advice

Service-Oriented Architecture

What is it?

General Performance Advice

Service-Oriented Architecture

Why is it good?

General Performance Advice

Cache, cache, cache

Symfony2 Tips & Tricks

Logging considerations

Default configuration

# PROD
monolog:
    handlers:
        main:
            type:         fingers_crossed
            action_level: error
            handler:      nested
        nested:
            type:  stream
            path:  %kernel.logs_dir%/%kernel.environment%.log
            level: debug

# DEV
monolog:
    handlers:
        main:
            type:  stream
            path:  %kernel.logs_dir%/%kernel.environment%.log
            level: debug
        firephp:
            type:  firephp # <-- Did you know?
            level: info
                

Fingers crossed is good for detailed reports in case of errors, but is still wasteful

Logging considerations

Solutions if this is a problem for you:

The Memory Monster: Symfony Profiler

It is disabled in production for a good reason

The Memory Monster: Symfony Profiler

You can also enable with custom rules by writing your own service:

framework:
    profiler:
        matcher:
            service: my_request_matcher # custom
                
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\RequestMatcherInterface;

class MyMatcher implements RequestMatcherInterface
{
    public function matches(Request $request)
    {
        // custom logic here
    }
}
                

The Memory Monster: Symfony Profiler

The Memory Monster: Symfony Profiler

Dump routes to Apache

$ app/console --env=prod router:dump

# skip "real" requests
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule .* - [QSA,L]

# nelmio_slow_default_index
RewriteCond %{REQUEST_URI} ^/noop$
RewriteRule .* $0/ [QSA,L,R=301]
RewriteCond %{REQUEST_URI} ^/noop/$
RewriteRule .* app.php [QSA, L,
    E=_ROUTING__route:nelmio_slow_default_index,
    E=_ROUTING__controller:Nelmio\\SlowBundle
        \\Controller\\DefaultController\:\:indexAction]
                

Bypass the Symfony Router entirely, only for Apache

Autoload faster with ApcUniversalClassLoader

// app/autoload.php

require __DIR__.'/../vendor/symfony/src/Symfony/'.
    'Component/ClassLoader/ApcUniversalClassLoader.php';

use Symfony\Component\ClassLoader\ApcUniversalClassLoader;

$loader = new ApcUniversalClassLoader('autoloader.');
                

~75% savings, possibly more on complex setups

Note: Make sure the prefix is unique if there are multiple apps on the server

Note: Using the same prefix could be better (less copies of the same files in APC), but risky

Autoload faster with MapClassLoader

// app/autoload.php

require __DIR__.'/../vendor/symfony/src/Symfony/'.
    'Component/ClassLoader/MapClassLoader.php';

use Symfony\Component\ClassLoader\MapClassLoader;

$loader = new MapClassLoader(array(
    'class' => '/path/to/class.php',
));
                

Fastest method, but requires more maintenance

Note: we'd need a pull request for a dumper script symfony/symfony#2407

Dependency Injection gotchas

Make sure instantiation is a cheap as possible

public function __construct($a, $b) {
    $this->a = $a;
    $this->b = $b;
    $this->init();
}

public function getStuff() {
    return $this->stuff;
}
                
public function __construct($a, $b) {
    $this->a = $a;
    $this->b = $b;
}

public function getStuff() {
    if (!$this->initialized) {
        $this->init();
    }

    return $this->stuff;
}
                

Be careful, if you forget to call init() in one of your method, you will likely get weird results

Dependency Injection gotchas

Until it's fixed, SwiftMailer is pretty expensive to inject. Another reason to offload email sending to a CLI worker.

Cache Warming

app/console cache:clear --env=prod --no-debug
                

The above builds a new cache, then swaps it off with the current one and finally removes the old cache.

The following clears the cache:

app/console cache:clear --env=prod --no-debug --no-warmup
                    

Event listener best practices

Listeners can be configured at runtime.

The DIC config is not the right place for conditional listeners.

Example:

<tag name="kernel.event_listener" event="security.interactive_login" method="onSecurityInteractiveLogin" />
<tag name="kernel.event_listener" event="kernel.response" method="onKernelResponse" />
                    
public function onSecurityInteractiveLogin(InteractiveLoginEvent $event)
{
    if ($event->getRequest()->checkSomething()) {
        $this->doSomething = true;
    }
}

public function onKernelResponse(FilterResponseEvent $event)
{
    if ($this->doSomething) {
        $event->setResponse(new Response('lala'));
    }
}
                    

Wasteful checks and calls at every request

Event listener best practices

Wire things at runtime instead:

<tag name="kernel.event_listener" event="security.interactive_login" method="onSecurityInteractiveLogin" />
                
public function onSecurityInteractiveLogin(InteractiveLoginEvent $event)
{
    if ($event->getRequest()->checkSomething()) {
        $this->eventDispatcher->addListener(
            KernelEvents::RESPONSE,
            array($this, 'onKernelResponse')
        );
    }
}

public function onKernelResponse(FilterResponseEvent $event)
{
    $event->setResponse(new Response('lala'));
}
                    

No more waste, and arguably simpler code

The downside is that it's not registered in the DIC anymore, but it expresses the intent and application flow better

Enable Doctrine Caches

doctrine:
    orm:
        metadata_cache_driver: apc
        query_cache_driver: apc
        result_cache_driver:
            type: memcache
            host: localhost
            port: 1234
                

Using the result cache

$query = $entityManager->createQuery('SELECT * FROM \Entity\Foo f');
$query->useResultCache(true, 86400, 'optional_query_id');
                

Last but not least

Symfony2 is a tool

If it gets in your way
replace bits and pieces

Dependency Injection means flexibility, use it

Web Performance Optimization

Analysis with
Webpagetest and YSlow

http://webpagetest.org/

http://developer.yahoo.com/yslow/

Merging and minifying assets with Assetic

Merging and minifying assets with Assetic

The configuration:

# app/config.yml
assetic:
    debug:          %kernel.debug%
    use_controller: false
    filters:
        cssrewrite: ~
        yui_css:
            jar: %kernel.root_dir%/Resources/bin/yuicompressor-2.4.6.jar
        yui_js:
            jar: %kernel.root_dir%/Resources/bin/yuicompressor-2.4.6.jar
                

Merging and minifying assets with Assetic

The template:

{% javascripts  '@NelmioSlowBundle/Resources/public/js/jquery.js'
                '@NelmioSlowBundle/Resources/public/js/main.js'
                filter="?yui_js"
                output="build/js/home.js"
-%}
    <script src="{{ asset_url }}"></script>
{%- endjavascripts %}

{% stylesheets  'bundles/nelmioslow/css/main.css'
                'bundles/nelmioslow/css/styles.css'
                filter="?yui_css,cssrewrite"
                output="build/css/home.css"
-%}
    <link type="text/css" href="{{ asset_url }}" rel="stylesheet" />
{%- endstylesheets %}
                

Merging and minifying assets with Assetic

Defining asset packages

The configuration:

# app/config.yml
assetic:
    # [...]

    assets:
        home_js:
            inputs:
                - @NelmioSlowBundle/Resources/public/js/jquery.js
                - @NelmioSlowBundle/Resources/public/js/main.js
            filters:
                - ?yui_js
        home_css:
            inputs:
                - bundles/nelmioslow/css/main.css
                - bundles/nelmioslow/css/styles.css
            filters:
                - ?yui_css
                - cssrewrite
                

Merging and minifying assets with Assetic

The template:

{% javascripts "@home_js" %}
    <script src="{{ asset_url }}"></script>
{% endjavascripts %}

{% stylesheets "@home_css" %}
    <link type="text/css" href="{{ asset_url }}" rel="stylesheet" />
{%- endstylesheets %}
                

Merging and minifying assets with Assetic

Dumping assets for production use:

$ app/console assetic:dump --env=prod --no-debug
                

Handling the organic DDoS

Load Testing with JMeter

http://jakarta.apache.org/site/downloads/downloads_jmeter.cgi

Adding Varnish to make it fly

http://varnish-cache.org

Adding Varnish to make it fly

# /etc/varnish/default.vcl

backend default {
    .host = "127.0.0.1";
    .port = "80";
}

sub vcl_recv {
    set req.backend = default;

    # Tell Symfony2 that varnish is there, supporting ESI
    set req.http.Surrogate-Capability = "abc=ESI/1.0";
}

sub vcl_fetch {
    # Enable ESI only if the backend responds with an ESI header
    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        unset beresp.http.Surrogate-Control;
        set beresp.do_esi = true;
    }
}
                

Adding AppCache to make it jump high

// web/app.php

require_once __DIR__.'/../app/bootstrap.php.cache';
require_once __DIR__.'/../app/AppKernel.php';
require_once __DIR__.'/../app/AppCache.php';    // <--

use Symfony\Component\HttpFoundation\Request;

$kernel = new AppKernel('prod', false);
$kernel->loadClassCache();
$kernel = new AppCache($kernel);                // <--
$kernel->handle(Request::createFromGlobals())->send();
                

Sending proper Cache-Control headers

Cache-Control: public, max-age=3600
Cache-Control: private, max-age=3600, must-revalidate
                

Sending proper Cache-Control headers

In practice:

public function indexAction()
{
    // response is private at this point
    $response = $this->render('FooBundle:Bar:index.html.twig');

    // response is implicitly set to public, with 5sec lifetime for proxies
    $response->setSharedMaxAge(5);
    return $response;
}
                

Going further with ESI

Configure Varnish:

# /etc/varnish/default.vcl

backend default {
    .host = "127.0.0.1";
    .port = "80";
}

sub vcl_recv {
    set req.backend = default;
    # Tell Symfony2 that varnish is there, supporting ESI
    set req.http.Surrogate-Capability = "abc=ESI/1.0";
}

sub vcl_fetch {
    # Enable ESI only if the backend responds with an ESI header
    if (beresp.http.Surrogate-Control ~ "ESI/1.0") {
        unset beresp.http.Surrogate-Control;
        set beresp.do_esi = true;
    }
}
                

Warning: Only applies to Varnish 3.0+

Going further with ESI

Enable ESI in Symfony2:

# app/config/config.yml
framework:
    # [...]
    esi: { enabled: true }
                

Going further with ESI

Leverage sub-requests in Symfony2:

{% render 'FooBundle:Bar:index' with {}, {'standalone': true} %}
                

Going further with ESI

Purging cached entries in Varnish:

acl purge {
    "localhost";
    "192.168.0.0/24";
}

sub vcl_recv {
    if (req.request == "PURGE") {
        if (!client.ip ~ purge) {
            error 405 "Not allowed.";
        }
        return (lookup);
    }
}

sub vcl_hit {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged.";
    }
}

sub vcl_miss {
    if (req.request == "PURGE") {
        purge;
        error 200 "Purged.";
    }
}
                

Going further with ESI

Purging cached entries in AppCache:

// app/AppCache.php
class AppCache extends Cache
{
    protected $whitelist = array('192.168.0.1', '127.0.0.1');

    protected function invalidate(Request $request) {
        if ('PURGE' !== $request->getMethod()) {
            return parent::invalidate($request);
        }

        // use a request matcher for smarter behavior
        if (!in_array($request->getClientIp(), $this->whitelist)) {
            return new Response('', 405);
        }

        $this->store->purge($request->getUri());

        $response = new Response();
        $response->setStatusCode(200, 'Purged');

        return $response;
    }
}
                

Let's Recap

Gather metrics.
If it ain't broken, don't fix it

Code legibility matters more
for non-critical paths

Heavy optimization often
comes with a cost

Do not optimize prematurely

Thank you.

Slides: http://slides.seld.be

Feedback & Questions:

jordi@nelm.io

@seldaek

joind.in/3695