<![CDATA[Oro Quickies]]>http://oro-quickies.alanstorm.com/Ghost 0.7Tue, 29 Mar 2016 20:00:13 GMT60<![CDATA[OroBAP's Twig Placeholder Tag]]>Poking around one of the base OroCRM templates, I came across an odd looking tag I'd never seen before.

#File: vendor/oro/platform/src/Oro/Bundle/UIBundle/Resources/views/Default/index.html.twig
...
{% placeholder scripts_after %}
...

Some quick Googling left me stumped — I couldn't find any reference to a placeholder

]]>
http://oro-quickies.alanstorm.com/orobaps-twig-placeholder-tag/04baad44-6a6f-44d7-b79a-56f511b9eea3Tue, 01 Apr 2014 01:19:59 GMTPoking around one of the base OroCRM templates, I came across an odd looking tag I'd never seen before.

#File: vendor/oro/platform/src/Oro/Bundle/UIBundle/Resources/views/Default/index.html.twig
...
{% placeholder scripts_after %}
...

Some quick Googling left me stumped — I couldn't find any reference to a placeholder twig tag. That's because it's a custom OroBAP tag, provided via a Twig extension.

Explaining how twig extensions work is more work than a quickies post entails. Fortunately, the README.md for the OroBAP UIBundle provided plenty of documentation on this custom tag.

#File: vendor/oro/platform/src/Oro/Bundle/UIBundle/README.md
## Introduction to placeholders

In order to improve layouts and make them more flexible a
new twig token `placeholder` is implemented. It allows us to
combine several blocks (templates or actions) and output
them in different places in twig templates. This way we can
customize layouts without modifying twig templates.    
...

The placeholder tag is way to add additional twig templates to certain areas without needing to modify the base application template files. For example, the OroBAP core team used the scripts_after placeholder in the the OroLocaleBundle bundle.

#File: vendor/oro/platform/src/Oro/Bundle/LocaleBundle/Resources/config/placeholders.yml
placeholders:
    scripts_after:
        items:
            locale_settings:
                order: 500

The inserts a locale_settings item into the HTML output wherever the {% placeholder scripts_after %} tag is used. The order field controls which order the placeholders are rendered in, since any bundle developer can add a placeholder via a placeholders.yml file.

What's a locale_settings item? That's defined later in the file.

#File: vendor/oro/platform/src/Oro/Bundle/NavigationBundle/Resources/config/placeholders.yml
items:
    locale_settings:
        template: OroLocaleBundle::locale_settings.html.twig

So, as you can see, the placeholder tag allowed the OroLocaleBundle author to insert the OroLocaleBundle::locale_settings.html.twig twig template into the document head without modifying the root document.

]]>
<![CDATA[WebSockets Overview]]>I've written up an overview of the various technologies involved in OroCRM's WebSockets implementation. This started as a quickies post, but ended up being complicated enough for a full length article.

WebSockets are surprisingly complicated, and require the involvment of numerous different fast moving, non-aligned open source projects. One of

]]>
http://oro-quickies.alanstorm.com/websockets-overview/01fa57cb-bdb0-410c-9215-040b76ffbba9Mon, 31 Mar 2014 17:57:27 GMTI've written up an overview of the various technologies involved in OroCRM's WebSockets implementation. This started as a quickies post, but ended up being complicated enough for a full length article.

WebSockets are surprisingly complicated, and require the involvment of numerous different fast moving, non-aligned open source projects. One of the neat things about full stack frameworks like OroCRM is they give developers a stable set of technologies to develop against when working with emerging technologies.

]]>
<![CDATA[No More Editing AppKernel to Enable Bundles]]>I'm not sure when this landed, but I just noticed the latest OroCRM RC2 builds have a new mechanism for adding bundles to an application without directly editing the AppKernel.php file.

Just add the following file to your bundle

Resources/config/oro/bundles.yml 

with the following contents.

bundles:
]]>
http://oro-quickies.alanstorm.com/no-more-editing-appkernel-to-enable-bundles/0053127d-70fd-4992-b63d-999fc01fa61dTue, 25 Mar 2014 22:38:40 GMTI'm not sure when this landed, but I just noticed the latest OroCRM RC2 builds have a new mechanism for adding bundles to an application without directly editing the AppKernel.php file.

Just add the following file to your bundle

Resources/config/oro/bundles.yml 

with the following contents.

bundles:
    - Namespace\Bundle\NameBundle\NamespaceNameBundle

Where Namespace\Bundle\NameBundle\NamespaceNameBundle is the full PHP class name of your bundle.

This feature will be vital to independant module distribution — with it in place end-users will be able to untar an archive into the src folder directly, clear the cache, and they'll be good to go. It'll also help with whatver official package managment system OroCRM Inc. comes up with.

One impotant gotcha in the RC2 builds: Right now this doesn't follow symlinks, so if you're developing bundles by symlinking your source controled code into a working OroCRM system, you'll want to manually apply the fix mentioned here. Hopefully a fix for this makes it into the final release.

(Between this and Magento's infamous problems with templates, I suspect Eastern European PHP developers are waging a secret war against the symlink)

]]>
<![CDATA[Magento Connector SOAP Calls]]>The new Magento Channel Connector support in the latest builds of OroCRM uses the Magento SOAP api to grab customer and order information from a Magento system.

As you'd expect in a "modern" Symfony application (if anyone can call mid-2000s java design patterns modern), there's literally dozens of different class

]]>
http://oro-quickies.alanstorm.com/magento-connector-soap-calls/4ddf9c11-0bda-4e17-b778-7a80ffbe9d45Sat, 22 Mar 2014 00:05:13 GMTThe new Magento Channel Connector support in the latest builds of OroCRM uses the Magento SOAP api to grab customer and order information from a Magento system.

As you'd expect in a "modern" Symfony application (if anyone can call mid-2000s java design patterns modern), there's literally dozens of different class files involved in scheduling and running a synchronization job. Ultimately, the following Symfony app console command is what runs a scheduled job

$ php app/console oro:cron:channels:sync --channel-id=4 -v --env=prod

I was running into some weird behavior with the customer import, and ended up debugging the call all the way down to the SOAP transport layer.

#File: vendor/oro/platform/src/Oro/Bundle/IntegrationBundle/Provider/SOAPTransport.php
public function call($action, $params = [])
{
    if (!$this->client) {
        throw new InvalidConfigurationException("SOAP Transport does not configured properly.");
    }
    $result = $this->client->__soapCall($action, $params);
    return $result;
}    

This is the ultimate method that PHP calls for any SOAP related API method invoked during a Magento channel sync. The object in $this->client is a native PHP Soap object. If you're having trouble with the syncing, dropping some debugging code in here is a good idea.

    if($action == 'customerCustomerList')
    {
        var_dump($params['filters']['complex_filter']);
    }
    $result = $this->client->__soapCall($action, $params);
]]>
<![CDATA[PHP Namesapces and call_user_func]]>I was doing a little experimental meta-programming in OroCRM and stumbled across another unknown-to-me quirk w/r/t PHP namespaces and the call_user_func function. It turns out PHP's callback pseudo-types don't inherit the namespace they're called from.

That is, consider something like this

namespace Pulsestorm\Example;
class Main
]]>
http://oro-quickies.alanstorm.com/php-namesapces-and-call_user_func/24c3bd74-61c5-4daa-afe1-de32d4eb2456Sun, 23 Feb 2014 20:25:23 GMTI was doing a little experimental meta-programming in OroCRM and stumbled across another unknown-to-me quirk w/r/t PHP namespaces and the call_user_func function. It turns out PHP's callback pseudo-types don't inherit the namespace they're called from.

That is, consider something like this

namespace Pulsestorm\Example;
class Main
{
    static public function test()
    {

    }

    static public function main()
    {
        call_user_func(array('Main',test));    
    }
}

My previous assumptions here were that call_user_func would look for a Main class in the currently defined namespace. However, that's not the case. Instead, it looks for a Main class in the global namespace.

In order to do what I wanted, I needed the full namespace path in the call_user_func function

call_user_func(array('Pulsestorm\Example\Main',test));    

You can also use the PHP magic constant __NAMESPACE__ if you like

call_user_func(array(__NAMESPACE__.'\Main',test));    

I'm sure there's good reasons for this tradeoff on the implementation level, but it's another of example of PHP's attempt to solve one problem (global everything) creating more mental work for full-stack, multi-framework developers.

]]>
<![CDATA[OroCRM Job Queue]]>Once you have OroCRM installed, if you navigate to

System -> Job Queue

you'll see OroCRM features a job queue. You won't, however, find a lot of explanation about what this is.

If you're not familiar with the concept, a job queue is a way for a programmer to

]]>
http://oro-quickies.alanstorm.com/orocrm-job-queue/ca0fd761-b379-4a2d-93b1-31da78b5be44Wed, 05 Feb 2014 06:04:44 GMTOnce you have OroCRM installed, if you navigate to

System -> Job Queue

you'll see OroCRM features a job queue. You won't, however, find a lot of explanation about what this is.

If you're not familiar with the concept, a job queue is a way for a programmer to say

Hey, I want this to happen, but it can happen later, so don't stop doing what you're doing

Job queues became popular in web programming when everyone realized it was probably a bad idea to have users wait for some long running thing to finish before showing the next page, and also realized that threads were a two edged sword — with both edges on the handle.

In keeping with lessons learned from Magento, the OroCRM team didn't reinvent the wheel when it came to job queues. They use the popular JMSJobQueueBundle for their implementation. Johannes site contains almost all the documentation for running the job queue you'll need.

I say almost, because there's a few things about the Oro UI implementation you'll want to be aware of.

First — you can start processing jobs in JMSJobQueueBundle by running the following app/console command

$ app/console jms-job-queue:run --max-concurrent-jobs=5 --env='prod'

This process will stick around for 15 minutes and run though any jobs in the queue. This is how I'd recommend you run these jobs as a developer working with OroCRM. (The instalation instructions recommend using something like supervisord for production systems).

What I wouldn't recommend is the "Run daemon" button. This launches an ajax request that attempts to start the job queue using a Symfony Process object. Running system processes from PHP is always a dodgey proposition, especially when the generated command line string looks like this

$ nohup php app/console jms-job-queue:run --max-runtime=999999999 --max-concurrent-jobs=5 --env='prod' > /dev/null 2>&1 &

Any output (standard or error) is redirect to /dev/null. Also, the use of the nohup utility from a web context is a problematic. The nohup command behaves different under different unixes and different shells. Add in the non-privlaged PHP/Apache user problem and you're in for some fun times. My attempts to get this feature to work from my local development Mac have met with failure, and the opaque nature of shell processes from PHP mean it's pretty hard to debug why.

So, skip the Run daemon button, and just run the job queue yourself from the command line.

]]>
<![CDATA[Symfony's Third Generated PHP File]]>This is a correction to a previous post where I asserted the Symfony framework (which OroCRM is built in), has two generated PHP files to contend with. There are actually (at least) three.

In addition to the app container file and the bootstrap cache file, there's also an application class

]]>
http://oro-quickies.alanstorm.com/symfonys-third-generated-php-file/bfe0dd79-6608-4b10-bdcb-71f9f3cde7f8Wed, 29 Jan 2014 08:55:57 GMTThis is a correction to a previous post where I asserted the Symfony framework (which OroCRM is built in), has two generated PHP files to contend with. There are actually (at least) three.

In addition to the app container file and the bootstrap cache file, there's also an application class cache file. Like the container class, this file is generated and stored in the cache

app/cache/dev/classes.php
app/cache/prod/classes.php    

This file is similar to that bootstrap cache, in that it combines a number of Symfony class files into a single PHP include. The difference is, unlike app/bootstrap.php.cache, classes in classes.php come from bundles used in the specific application, and are not limited to files from Symfony itself.

Generating classes.php

The generation of classes.php is a bit complicated. First, this file is not generated as part of Symfony's cache warmup. You can call

php app/console cache:clear

all you like, but that won't regenerate classes.php. Instead, you need to load an actual application page, or otherwise boot your application kernel.

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
public function boot()
{
    //...
    if ($this->loadClassCache) {
        $this->doLoadClassCache($this->loadClassCache[0], $this->loadClassCache[1]);
    }
    //...
}    

It's the call to doLoadClassCache which triggers a call to ClassCollectionLoader::load

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
protected function doLoadClassCache($name, $extension)
{
    if (!$this->booted && is_file($this->getCacheDir().'/classes.map')) {
        ClassCollectionLoader::load(include($this->getCacheDir().'/classes.map'), $this->getCacheDir(), $name, $this->debug, false, $extension);
    }
}

The work of generating classes.php is done in ClassCollectionLoader::load. Simple enough — except you need to pass in a list of classes you want to combine. These classes come from the classes.map file.

app/cache/dev/classes.map
app/cache/prod/classes.map

Of course, now our question is "where does classes.map come from?". The classes.map file is generated at the same time as the app container file, right after the call to $container->compile() in initializeContainer.

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
protected function initializeContainer()
{
    //...
    if (!$cache->isFresh()) {
        //...
        $container->compile();
        //...
    }
    //...
}

Container compilation is a bit much to get into for a quickies post, but from a high level view the compiler looks through every installed bundle in the system, and checks if it has a dependency injection extension. A dependency injection extension class is a file located in a bundle's DependencyInjection folder, ending in the word Extension, and extending the base class Sensio\Bundle\FrameworkExtraBundle\DependencyInjectionExtension.

One example is the following file

vendor/sensio/framework-extra-bundle/Sensio/Bundle/FrameworkExtraBundle/DependencyInjection/SensioFrameworkExtraExtension.php

The dependency injection extension is a standard part of bundles generated with the app/console generate:bundle command.

If the bundle has this sort of extension, the container compiler asks it if it wants to put any files in the classes.php cache. A bundle developer specifies if they want any class files dropped into this class cache in the extension's load method.

#File: vendor/sensio/framework-extra-bundle/Sensio/Bundle/FrameworkExtraBundle/DependencyInjection/SensioFrameworkExtraExtension.php
public function load(array $configs, ContainerBuilder $container)
{
    //...
    $this->addClassesToCompile(array(
        // cannot be added because it has some annotations
        //'Sensio\\Bundle\\FrameworkExtraBundle\\Configuration\\ParamConverter',
        'Sensio\\Bundle\\FrameworkExtraBundle\\EventListener\\ParamConverterListener',
        'Sensio\\Bundle\\FrameworkExtraBundle\\Request\\ParamConverter\\DateTimeParamConverter',
        'Sensio\\Bundle\\FrameworkExtraBundle\\Request\\ParamConverter\\DoctrineParamConverter',
        'Sensio\\Bundle\\FrameworkExtraBundle\\Request\\ParamConverter\\ParamConverterInterface',
        'Sensio\\Bundle\\FrameworkExtraBundle\\Request\\ParamConverter\\ParamConverterManager',
    ));        
    //...
}

This tells the extension about the classes, the extension tells the container compiler, the container compiler generates classes.map, and the app kernel uses classes.map to generate classes.php.

Not the most straight forward process for a newcomer — but not something a standard system user is going to encounter on a regular basis. The most important takeaway is if you're the sort of developer who likes adding debug code to core framework files, you may need to do it in classes.php instead of the files in vendor/.

]]>
<![CDATA[OroCRM Virtual Machine]]>The OroCRM has has released a virtual machine containing the latest source as a virtual machine. This is welcome news for developers who hate setting things up themselves, and also welcome news for developers looking for a reference implementation.

]]>
http://oro-quickies.alanstorm.com/orocrm-virtual-machine/a1a452c5-acd2-444d-9795-ce507da57e00Tue, 28 Jan 2014 19:26:08 GMTThe OroCRM has has released a virtual machine containing the latest source as a virtual machine. This is welcome news for developers who hate setting things up themselves, and also welcome news for developers looking for a reference implementation.

]]>
<![CDATA[Twig Templates, Initial Cache Hit]]>OroCRM uses Symfony's delegate template engine, which means developers are free to use any template service added to the Symfony configuration. This usually means twig.

I'm tracing the execution of a full twig render, and I'm starting to see why twig's initial cache hit is so performance heavy.

Here's one

]]>
http://oro-quickies.alanstorm.com/twig-templates-initial-cache-hit/1a7a9b76-b9de-47fe-9579-3a325ce8e4f6Thu, 23 Jan 2014 19:37:32 GMTOroCRM uses Symfony's delegate template engine, which means developers are free to use any template service added to the Symfony configuration. This usually means twig.

I'm tracing the execution of a full twig render, and I'm starting to see why twig's initial cache hit is so performance heavy.

Here's one example — the isTemplateFresh method is where some of the "do I need to recompile the twig templates" logic lives. In plain english, this method checks if any of the configured twig extension classes have changed, and then checks if the specific twig template has changed. If so, it signals to the system that a recompile is needed.

This is a valuable developer convenience, and means we're not recompiling templates after making small changes. (What are we — C programmers?)

In php code, it looks like this

#File: vendor/twig/twig/lib/Twig/Environment.php
public function isTemplateFresh($name, $time)
{
    foreach ($this->extensions as $extension) {
        $r = new ReflectionObject($extension);
        if (filemtime($r->getFileName()) > $time) {
            return false;
        }
    }

    return $this->getLoader()->isFresh($name, $time);
}

This method loops over each configured twig extension and

  • Creates a reflection object to grab the file name
  • Then stats the extension file to get its last modified date
  • Assuming the extensions haven't changed, it stats the specific twig template with isFresh

That's a lot of work — and this will happen for each twig template in the layout. If you have 5 twig extensions and a simple layout with 3 templates, that's 15 times through the loop. Add a new template, that's 20 times through the loop. This is the sort of thing that would crush a production server under heavy load.

Of course, the mantra of premature optimization is the root of all evil is on your lips. Hop up to where isTemplateFresh gets called.

if (!is_file($cache) || ($this->isAutoReload() && !$this->isTemplateFresh($name, filemtime($cache)))) {
    $this->writeCacheFile($cache, $this->compileSource($this->getLoader()->getSource($name), $name));
}

and you'll see it's easily short circuited if isAutoReload returns false, which is how a production system using twig should be configured.

Ever after all these years of programming, my instinctual mind wants nothing to do with this sort of code. However, like anyone who has a long career in programming, learning to accept this is how the bowels of systems look is step one to putting yourself in a position to make things better.

]]>
<![CDATA[Symfony, OroCRM, and Views]]>The concept of a "view" is a little tricky in Symfony, which makes it a little tricky in OroCRM/OroBAP.

As of Symfony 2.4 (the latest version) and 2.3 (the latest version used with OroCRM), there are two primary ways for a controller action method to hand off

]]>
http://oro-quickies.alanstorm.com/symfony-orocrm-and-views/b47a64ec-ea0e-4679-b342-bac2ac83f2fbMon, 20 Jan 2014 02:57:18 GMTThe concept of a "view" is a little tricky in Symfony, which makes it a little tricky in OroCRM/OroBAP.

As of Symfony 2.4 (the latest version) and 2.3 (the latest version used with OroCRM), there are two primary ways for a controller action method to hand off to a view. You can tell your controller to render a specific view identifier, or you can tell your controller to render the view it thinks it should render, (you can also simply return a raw Response object, but that's off topic for this post)

To tell your controller to render a specific view, just have your controller action return the results of the render function.

#File: vendor/oro/platform/src/Oro/Bundle/DashboardBundle/Controller/DashboardController.php
public function indexAction($name = null)
{    
    return $this->render(
        'OroDashboardBundle:Index:default.html.twig',
        [
            'pageTitle'     => $dashboard['label'],
            'dashboardName' => $name,
            'dashboards'    => $manager->getDashboards(),
            'widgets'       => $manager->getDashboardWidgets($name),
        ]
    );
}

Behind the scenes the render method will use the view identifier OroDashboardBundle:Index:default.html.twig and the templating service's renderResponse method to create a Response object

#File: vendor/oro/platform/src/Oro/Bundle/DashboardBundle/Controller/DashboardController.php
public function render($view, array $parameters = array(), Response $response = null)
{
    return $this->container->get('templating')
        ->renderResponse($view, $parameters, $response);
}

Pretty straight forward so far. However, you'll also see controller action methods returning plain old PHP arrays.

#File: vendor/oro/crm/src/OroCRM/Bundle/AccountBundle/Controller/AccountController.php
/**
 * @Route("/view/{id}", name="orocrm_account_view", requirements={"id"="\d+"})
 * @Template
 * @Acl(
 *      id="orocrm_account_view",
 *      type="entity",
 *      permission="VIEW",
 *      class="OroCRMAccountBundle:Account"
 * )
 */    
public function viewAction(Account $account)
{
    return ['entity'  => $account];
}

This can be confusing the first time you encounter it — the trick here is the annotation configuration format. Specifically the @Template annotation. The @Template annotation lets you skip the explicit call to render, and tells Symfony which template to use. The returned PHP array becomes the variables for the view.

When, as above, there's no template listed after @Template, Symfony will construct a template name using the bundle name, controller class name, and action method name.

In the above example, this generated view name is

OroCRMAccountBundle:Account:view.html.twig

The OroCRMAccountBundle part comes from the controller's bundle, the Account comes from the controller's class name, and the view comes from the action method name.

For people curious about the implementation details, this template name gets created in the following Symfony core file, called as a kernel.controller listener.

#File: vendor/sensio/framework-extra-bundle/Sensio/Bundle/FrameworkExtraBundle/EventListener/TemplateListener.php
public function onKernelController(FilterControllerEvent $event)
{
    //...
    if (!$configuration->getTemplate()) {
        $guesser = $this->container->get('sensio_framework_extra.view.guesser');
        $configuration->setTemplate($guesser->guessTemplateName($controller, $request, $configuration->getEngine()));
    }        
    //...
}

More specifically, it's the call to the sensio_framework_extra service's guessTemplateName that does it.

Symfony's internal use of the annotation system seems to be their compromise in the "opinionated framework" wars. Symfony, by itself, is a configuration based framework. However, the annontation features are often used to implement a simplier, "opinioned" version of a feature.

Also, I thought about going on a strongly types rant about how PHP's lack of return types combined with their embrace of Java/C# style classes make this feel extra weird — but decided it fit better as a snarky post script.

]]>
<![CDATA[Symfony's Two Generated PHP Files]]>Update: The information in the post below is correct, except for the part where it talks about Symfony have two generated PHP files. It actually has three.

More as a reminder to myself than anything else, it's important to remember a Symfony application has two generated PHP files.

The first

]]>
http://oro-quickies.alanstorm.com/symfonys-two-generated-php-files/3e67f6ef-6aee-4916-afa8-171ec1b45980Sat, 18 Jan 2014 20:36:47 GMTUpdate: The information in the post below is correct, except for the part where it talks about Symfony have two generated PHP files. It actually has three.

More as a reminder to myself than anything else, it's important to remember a Symfony application has two generated PHP files.

The first is the bootstrap cache file. Despite being a cache file, this does not live in the cache folder. Instead, it lives directly in app/

app/bootstrap.php.cache

This file contains the Symfony Component PHP classes needed to bootstrap a basic Symfony/HttpKernel environment. It lives outside the cache folder because it's environment independent. Put another way, these are the classes that make the environment system possible.

The second generated file is the container class file. Its name/location is dependent on the type of app kernel and the environment you're running in. Most of the time it lives in the following locations for the dev and prod environments.

app/cache/dev/appDevDebugProjectContainer.php
app/cache/prod/appProdProjectContainer.php

This file contains the generated class that implements the container for a specific project. That is, when the framework generates a container file, it looks at the configuration for each bundle and generates a class with methods and configuration that allow a a Symfony programmer to use the container.

For example, when you say

$this->get('logger');

from a controller in Symfony, the framework eventually calls the method

#File: app/cache/dev/appDevDebugProjectContainer.php
protected function getLoggerService()
{
    $this->services['logger'] = $instance = new \Symfony\Bridge\Monolog\Logger('app');

    $instance->pushHandler($this->get('monolog.handler.console'));
    $instance->pushHandler($this->get('monolog.handler.main'));
    $instance->pushHandler($this->get('monolog.handler.debug'));

    return $instance;
}

By generating a container class with these methods pre-defined, the framework code avoids the performance overhead of looking up the configured class names for every service call, as well as some of the PHP overhead involved in using magic methods like __call.

Regenerating

If you're creating new services in Symfony, you'll often need to re-generate you container file. Doing so is as simple as clearing your Symfony cache — the Symfony kernel will regenerate these files automatically. If you're curious, the code that does this starts here

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php
protected function initializeContainer()
{
    $class = $this->getContainerClass();
    $cache = new ConfigCache($this->getCacheDir().'/'.$class.'.php', $this->debug);
    $fresh = true;
    if (!$cache->isFresh()) {
        $container = $this->buildContainer();
        $container->compile();
        $this->dumpContainer($cache, $container, $class, $this->getContainerBaseClass());

        $fresh = false;
    }

    require_once $cache;

    $this->container = new $class();
    $this->container->set('kernel', $this);

    if (!$fresh && $this->container->has('cache_warmer')) {
        $this->container->get('cache_warmer')->warmUp($this->container->getParameter('kernel.cache_dir'));
    }
}

Regenerating the bootstrap cache file is a little trickier. In theory, only a Symfony internals developer ever needs to regenerate this file. However, if you're like me, you'll often end up dropping some var_dump's in core system files to figure out what the heck's going on with a bug. This, in turn, means you can sometimes accidentally delete something important and need to regenerate the file.

Fortunately, while it takes a little digging, it is possible to generate the bootstrap cache file. Symfony ships with a file named build_bootstrap.php. Just run this from the console, and your file should be regenerated.

./vendor/sensio/distribution-bundle/Sensio/Bundle/DistributionBundle/Resources/bin/build_bootstrap.php

This file has a history of moving around, but a quick

$ find . -name 'build_bootstrap.php`

should point you towards the correct file. Finally, if you're using composer to manage your Symfony project, running

$ composer.phar update

should automatically regenerate the bootstrap cache file.

]]>
<![CDATA[Symfony Aliases and the Event Dispatcher]]>In what I hope is the last word on the previously discussed event_dispatcher service, I downloaded the latest version of Symfony (2.4.1, the OroCRM Beta 6 uses Symfony 2.3.6), and discovered a change in the generated dev container file. Specifically,

#File: app/cache/dev/appDevDebugProjectContainer.
]]>
http://oro-quickies.alanstorm.com/symfony-aliases-and-the-event-dispatcher/7966f7ff-099d-4b89-b6c0-10defb9c376dSat, 18 Jan 2014 19:30:36 GMTIn what I hope is the last word on the previously discussed event_dispatcher service, I downloaded the latest version of Symfony (2.4.1, the OroCRM Beta 6 uses Symfony 2.3.6), and discovered a change in the generated dev container file. Specifically,

#File: app/cache/dev/appDevDebugProjectContainer.php
$this->aliases = array(
    //...
    'event_dispatcher' => 'debug.event_dispatcher',
    //...
);    

That is, for the dev environment, the event_dispatcher service is aliased as debug.event_dispatcher. This means all users of the event_dispatch service are actually using the debug.event_dispatcher service, (when running in the dev environment). This solves the web profiler's incorrect tracking of third party bundle events, and is an implicit endorsement that third party bundles can and should use the bundled event dispatcher. One assumes we'll see this change come to OroCRM/OroBAP in an upcoming release.

]]>
<![CDATA[More Notes on Symfony Event Dispatch]]>As hinted at by the last post, I've spent more of my "Oro Time" this week mired in Symfony's event dispatcher. There's a few other things I've found that are worth sharing.

First, the web profiler appears to have a bug where it will incorrectly list a called observer as

]]>
http://oro-quickies.alanstorm.com/more-notes-on-symfony-event-dispatch/7c53acc8-c5d4-457e-aa58-544c41f4bd7dFri, 17 Jan 2014 00:56:21 GMTAs hinted at by the last post, I've spent more of my "Oro Time" this week mired in Symfony's event dispatcher. There's a few other things I've found that are worth sharing.

First, the web profiler appears to have a bug where it will incorrectly list a called observer as uncalled. Or it may be that the event dispatcher that ships with Symfony is only intended for kernel events and bundle authors are misusing it for their own event systems.

Speaking of kernel events — my initial event debugging indicated the OroCRM beta has over 450 kernel.request listeners, over 200 kernel.response listeners, and near another 200 kernel.request listeners. I haven't done a lot of Symfony 2 development, but that seemed excessive. Fortunately, it was a misdiagnosis on my part. The special TraceableEventDispatcher dispatcher sets and resets it's listeners in the preDispatch method

#File: vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Debug/TraceableEventDispatcher.php
private function preDispatch($eventName, Event $event)
{
    //...
    foreach ($listeners as $listener) {
        $this->dispatcher->removeListener($eventName, $listener);
        $wrapped = $this->wrapListener($eventName, $listener);
        $this->wrappedListeners[$this->id][$wrapped] = $listener;
        $this->dispatcher->addListener($eventName, $wrapped);
    }
    //...    
}

This, combined with my simple debugging in addListener led me to think there were a crazy number of listeners setup.

Finally, one reason it was plausible Symfony had that many listeners setup is the system includes the ability for an event to stop propagation. If you call stopPropagation on your event object

$event->stopPropagation();

you're telling Symfony to skip running future listeners for this event dispatch. I'm not sure I like this feature in a modular full stack framework's event system as it gives a single bundle developer the ability to radically interfere with other bundle developer's listeners — but the feature is there, so make sure you include it in your debugging regime.

]]>
<![CDATA[Extending Symfony's Event Service]]>One important concept in Symfony 2 (the framework OroBAP/OroCRM is built-in/on-top-of) is the service container. Programmers create classes and class hierarchies for myriad reasons — one way of looking at a service container is it allows a programmer to say

Hey, this class? It's a service. It provides an

]]>
http://oro-quickies.alanstorm.com/extending-symfonys-event-service/923f92af-2a0d-4775-b052-2d2d4409b0e2Wed, 15 Jan 2014 21:49:21 GMTOne important concept in Symfony 2 (the framework OroBAP/OroCRM is built-in/on-top-of) is the service container. Programmers create classes and class hierarchies for myriad reasons — one way of looking at a service container is it allows a programmer to say

Hey, this class? It's a service. It provides an important single unit of functionality to programmers using this system.

Service containers are an abstraction on top of the programming language itself.

Service containers make a lot of things easy and consistant. One of those things is dependency injection. One way of looking at dependency injection is

Hey, that class? I want to change how it works. Use this other class instead.

Symfony's configuration system allows a developer to swap out service classes without changing the underlying PHP code.

With that in mind, I wanted to inject a different event dispatcher into my system for debugging purposes. However, when I went digging into the system, I ran into a few problems.

The first step of injecting a different class for a service container is finding the name/identifier of the service. I knew, (through some debug_backtrace debugging in an event listener), that there were two event dispatchers in Symfony — one for dev, one for prod

//dev dispather
Symfony\Component\HttpKernel\Debug\TraceableEventDispatcher

//prod dispatcher
Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher

This makes sense, as the web profiler available in dev mode reports on dispatched events, and you wouldn't want a production system wasting resources doing that.

So, understanding how Symfony works, I assumed there was some extra dev configuration that injected a different dispatcher into the service for dev mode. The next step was to search the default Symfony configuration for the ContainerAwareEventDispatcher class. I found it in

#File: vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Resources/config/services.xml
<!-- ... -->
<parameters>
    <parameter key="event_dispatcher.class">Symfony\Component\EventDispatcher\ContainerAwareEventDispatcher</parameter>
</parameters>

Being in an older bundle (that's part of the core system), it's not surprising the service is configured in XML (newer bundles seem to use yaml files). This configuration file has a paramater named event_dispatcher.class. This isn't the service definition, it's just a configuration paramater. However, if we jump down further in the file

#File: vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Resources/config/services.xml

<services>
    <service id="event_dispatcher" class="%event_dispatcher.class%">
        <argument type="service" id="service_container" />
    </service>
</services>

We see the configuration for a service named event_dispatcher that uses the event_dispatcher.class paramater to define the service class

class="%event_dispatcher.class%"

This is what I was looking for.

Or so I thought.

Normally, my assumptions above would be correct. However, the event dispatcher is special. Looking through my ack results, I also saw the following configuration.

#File: vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Resources/config/debug.xml
<parameters>
    <parameter key="debug.event_dispatcher.class">Symfony\Component\HttpKernel\Debug\TraceableEventDispatcher</parameter>
    <!-- ... -->
</paramaters>

<!-- ... -->

<service id="debug.event_dispatcher" class="%debug.event_dispatcher.class%">
    <tag name="monolog.logger" channel="event" />
    <argument type="service" id="event_dispatcher" />
    <argument type="service" id="debug.stopwatch" />
    <argument type="service" id="logger" on-invalid="null" />
    <call method="setProfiler"><argument type="service" id="profiler" on-invalid="null" /></call>
</service>    

Earlier I assumed Symfony injected the TraceableEventDispatcher class into the system for the dev environment. However, the above configuration makes it look like the event service is named debug.event_dispatcher. This doesn't match event_dispatcher.

At this point my brain shuts down because I'm trying to hold two impossible-to-resolve ideas in my head. There's a name for this condition — it's called being a programmer.

Pre-Bootstrapped System Behavior

The answer, of course, was my assumption about the Symfony event dispatcher was incorrect. The event dispatcher is part of the HTTP kernel object, and the system needs to (or the cire developers have chosen to) instantiate this object before the system is fully bootstrapped.

The event dispatchers are hard coded in the generated container classes. For the dev container that's

#File: app/cache/dev/appDevDebugProjectContainer.php
protected function getHttpKernelService()
{
    return $this->services['http_kernel'] = 
        new \Symfony\Component\HttpKernel\DependencyInjection\ContainerAwareHttpKernel(
            $this->get('debug.event_dispatcher'), 
            $this, 
            $this->get('debug.controller_resolver')
        );
}    

and for the prod container that's

protected function getHttpKernelService()
{
    return $this->services['http_kernel'] = 
        new \Symfony\Component\HttpKernel\DependencyInjection\ContainerAwareHttpKernel(
            $this->get('event_dispatcher'), 
            $this, 
            new \Symfony\Bundle\FrameworkBundle\Controller\ControllerResolver($this, $this->get('controller_name_converter'), $this->get('monolog.logger.request', ContainerInterface::NULL_ON_INVALID_REFERENCE)));
}

The first argument to the HTTP Kernel's (ContainerAwareHttpKernel) constructor is the event dispatcher object, and you can see the different hard coded service values.

$this->get('debug.event_dispatcher')
$this->get('event_dispatcher')

So, unlike the rest of the system, the event dispatcher (and controller resolver) don't fully conform to what a Symfony developer would consider best practices.

This means I'd need to inject two service classes. One for the event_dispatcher service, and another for the debug.event_dispatcher service. It also means they'd probably need to be different classes in order to ensure each obeyed the implicit interfaces for the originals.

I don't bring this up to condem the Symfony or Oro core teams. I bring it up to identify a pattern that exists in almost every programatic computer system out there. These systems are built, in large part, by programmers to solve programmer's problems. However, the code that bootstraps these systems needs to solve problems the old, pre-system way.

In a sufficiently complex system the bootstrap code ends up resembling a weird hybrid that's not quite raw programming language, but not fully conformant with the rules of the system. This is as true for Symfony as it is for the *nix bash shell.

This both because a partially bootstraped system doesn't have accesses to the same resource, and also because the developers are used to working with the patterns they've developed for the system itself and start incorporating different versions of them in the bootstrap code. It's systems all the way down.

If you're planning on sticking to the programmers path, learning to accept this will help you quickly identify the parts of a system that behave like this, and quickly track down bugs related to these differenence.

]]>
<![CDATA[Flexible Entity Manager]]>While OroCRM uses the Doctrine 2 Object Relational Mapper (ORM), the OroCRM core development team has extended standard Doctrine with a number of enhancements (working mostly within the systems provided by Doctrine) to implement a Flexible Entity System.

The Flexible Entity system offers an end-user with little to no development

]]>
http://oro-quickies.alanstorm.com/flexible-entity-manager/9c53bc32-3796-40ad-a6e8-ef025e9a19e3Sun, 12 Jan 2014 03:07:24 GMTWhile OroCRM uses the Doctrine 2 Object Relational Mapper (ORM), the OroCRM core development team has extended standard Doctrine with a number of enhancements (working mostly within the systems provided by Doctrine) to implement a Flexible Entity System.

The Flexible Entity system offers an end-user with little to no development skill the ability to add new fields to objects, and have user interface elements for those fields in an entity's main editing form.

In other words, it's UI configurable CRUD. Not exactly rocket science in 2014, but a none-the-less important part of any CRM system. While a business could install a stock OroCRM system and get some value from it, the real value of a CRM is the customizations made for a specific business, or business type. The bet OroCRM the company is making is that companies will prefer working with OroCRM, or OroCRM based development teams to build a system they own vs. relying on a cloud provider's vision for what their CRM should be.

You can find the user interface for this section at

System -> Entities

While day-to-day users of an OroCRM application will spend little time here, I suspect OroCRM developers will become very familiar with this particular screen.

]]>