/usr/portage

Polite Exceptions – Fixing the stepchild of API design 2

Every API has an visible, an invisibile and a hidden part. The visible part is obvious: public methods and properties but also constants and parameter values. That’s the most visible part to any client (read: user) of your API. The invisible part is everything private, you can’t really see it and – more important – you can’t use it (except if you resort to reflection). The hidden part consists of all the protected symbols, as you can’t really see them until you extend a class. The other hidden part are Exceptions. You can’t really see them and there is no common expectation what methods throw what kind of exception. Yes, throws@-docblocks help, but that’s mostly all we have.

Exceptions handling: the problem

The usability of hidden parts of an API is all about expectations: people love languages like Ruby because once you learned a certain set of API (e.g. the string API), you can instinctively infer a large part of other APIs. This is good and keeps learning costs down. PHP, on the other hand, with its historically grown standard extension is on the opposite site of the fence: various parameter order the naming scheme is unreliable at best.
The future is multi-lingual, you need to know more than one programming language and speed of learning matters. Like, a lot. Because “X programmers”, for any value of X, are weak players. What type of exception a class might throw should be defined by clear expectations for the general case. If you use a preconceived HTTP client httpFoo, call method request() and want to handle exception cases, what exactly do you catch?

Talent borrows, genius steals

Zend Framework 2 has a lot of problems but there are two things they did particularly well: naming of abstract classes and interfaces and how they treat exceptions. Every component (see, component is a lie here, as they aren’t really stand alone components but I digress) has its own exception subpackage which has extension specific exceptions. Those exception all implement a single marker interface called ExceptionInterface. If you use Zend\Something and want to handle all exceptions, just catch Zend\Something\Exception\ExceptionInterface.

Programming transaction costs

Time to relevant data is the new time to market. We no longer optimize for feature-complete products shipping on a certain date but relevant changes generating relevant data as soon as possible. Therefore programmer round-trips matter. I consider everything that is not core domain or core UI a round trip:

These steps aren’t worthless, they are worth less from a business perspective as they don’t generate revenue very soon. However they are needed to keep revenue over time. So let’s make those things cheaper.

When dealing with Exceptions in Symfony 2 projects, two steps are particularly expensive:

Especially the latter can be simplified quite dramatically.

Simplifying

To simplify Exception handling, we just open sourced a bundle we developed at InterNations. Let’s create a few custom exceptions:

php app/console exception:generate app/src/MyVendor/MyBundle "MyVendor\MyBundle" \
 ExceptionInterface RuntimeException DomainException RuntimeException:SpecificRuntimeException
Create directory app/src/MyVendor/MyBundle/Exception
Writing app/src/MyVendor/MyBundle/Exception/ExceptionInterface.php
Writing app/src/MyVendor/MyBundle/Exception/RuntimeException.php
Writing app/src/MyVendor/MyBundle/Exception/SpecificRuntimeException.php
Writing app/src/MyVendor/MyBundle/Exception/DomainException.php

Let’s rewrite an existing bundle to use custom exceptions:

php app/console exception:rewrite app/src/MyVendor/MyBundle "MyVendor\MyBundle"
Found bundle specific exception class BadFunctionCallException
Found bundle specific exception class BadMethodCallException
Found bundle specific exception class DomainException
Found bundle specific exception class InvalidArgumentException
Found bundle specific exception class LengthException
Found bundle specific exception class LogicException
Found bundle specific exception class OutOfBoundsException
Found bundle specific exception class OutOfRangeException
Found bundle specific exception class OverflowException
Found bundle specific exception class RangeException
Found bundle specific exception class RuntimeException
Found bundle specific exception class UnderflowException
Found bundle specific exception class UnexpectedValueException
...............
------------------------------------------------------------
------------------------------------------------------------
SUMMARY
------------------------------------------------------------
------------------------------------------------------------
Files analyzed:               15
Files changed:                1
------------------------------------------------------------
"throw" statements found:     2
"throw" statements rewritten: 1
------------------------------------------------------------
"use" statements found:       1
"use" statements rewritten:   1
"use" statements added:       1
------------------------------------------------------------
"catch" statements found:     0

You’ll find the ExceptionBundle over at github. It uses PHP Parser to rewrite code which proves again to be a wonderful project.

Filed on 20-12-2012, 10:10 under , , , , & two comments & no trackbacks

The state of meta programming in PHP 11

Quoting Wikipedia

Metaprogramming is the writing of computer programs that write or manipulate other programs (or themselves) as their data, or that do part of the work at compile time that would otherwise be done at runtime

Metaprogramming is quite an interesting sub-discipline and knowing about certain techniques and tools allows you to cut corners quite dramatically for certain tasks. As always, don’t overdo but to find out when you are overdoing, first start doing, get excited, overdo, find out the right dose. Let’s have a look at what kind of tools you have available in PHP to solve typical meta programming problems.

What kind of questions can meta programmatic APIs answer?

I would group metaprogramming into three sub areas: type introspection, lower level syntax inspection and metadata management. Typical type introspection questions are:

On a lower level you typically interact with a certain kind of syntax tree to answer questions like:

A third category is adding metadata to the declared types: Java, C# and a few others have first-class Annotation support for this kind of things but PHP only has user space solutions so far. A few things you need metadata for:

The toolkit

Reflection APIs

PHP core delivers 2.5 key APIs for meta programming. The first one is ext/reflection. You can create reflection classes form a lot of things, functions, classes, extensions and use them to make programming assumptions about the APIs you are introspecting.

A simple example to find out the number of required parameters for each method in the class DirectoryIterator:

<?php
$class = new ReflectionClass('DirectoryIterator');
foreach ($class->getMethods() as $method) {
    $numberOfRequiredParameters = $method->getNumberOfRequiredParameters();
}

Refection is all nice and shiny, except when you don’t want to include everything you want to inspect. This is of interest if you inspect various source trees at once that declare duplicate symbols. To do so, there is PHP-Token-Reflection by Ondřej Nešpor. It’s a pretty nifty replacement for ext/reflection completely built in user land and on top of ext/tokenizer that even copes with invalid declarations. Additionally it fixes some oddities of the internal reflection API but tries to keep it as close as possible. I’ve played around with it a bit and I quite like it.

<?php
$broker = new TokenReflection\Broker(new TokenReflection\Backend\Memory());
$broker->processDirectory("path/to/src");
$class = $broker->getClass('MyClass');
foreach ($class->getMethods() as $method) {
   ...
}

Tokenizer

Another core API, this time much more low level, is ext/tokenizer. If enabled at compile time it allows you to parse PHP source code into a list of tokens. Because the API is so low level it is quite hard to use without a proper abstraction layer on top of it. Most of the successful projects built upon ext/tokenizer have built one. One of them is phpcs by Greg Sherwood that built an Token Stream abstraction on top of ext/tokenizer that allows much more convenient navigation in the token stream. Another one shipping its own token stream abstraction is pdepend by Manuel Pichler. Another noteworthy, standalone abstraction is php-manipulator.
For an example on how the raw API can be used, I once wrote this little script to apply a few transformations to source trees to ease converting source trees to PHP 5.4.

PHP Parser: a fully fledged AST parser for PHP

Between a high level API like Reflection and a low level API like ext/tokenizer there surely is a gap: what if I want to work on an AST data structure. There is this beautiful project PHP-Parser by Nikita Popov. This is quite interesting for more complex transformations like user space AOP, all kinds of static code analysis and so on. If ext/tokenizer feels way underpowered, have a look at this project.

Aspect oriented programming

While we are talking about AOP: a relative newcomer is PECL AOP that provides a quite simple API for aspect orientated programming in PHP. For Zend Framework 2 there is also an AOP module available. Let’s stick to AOP for a moment: for Symfony 2 there is JMSAopBundle by Johannes Schmitt. It provides basic AOP functionality for Symfony 2. JMSSecurityExtraBundle and JMSDiExtraBundle use it to provide annotation support for Symfony security bundle and the Symfony dependency injection component.

Metadata management

Traditionally, every docblock documentation parser rolled it’s own annotation system. This changed a little with the rise of Symfony and Doctrine 2. Doctrine 2 allows you to use annotations for persistence definition and Symfony allows you to use annotations for a lot of things (routes, security, etc.). While Doctrine still ships it’s own metadata handling component in doctrine-common, there is another library by Johannes Schmitt, Metadata that aims to consolidate metadata handling for PHP. The API of the Metadata library as well as the one of doctrine-common is quite simple: you have some sort of annotation reader that maps metadata information to classes. Think about this annotation:

<?php
use My\Annotation\Some;
/**
  * @Some(foo="bar")
  */
class MyClass
{}

This kind of annotation will map to an instance of My\Annotation\Some with the property $foo set to “bar”.

Radioactive, specialized or obscure

Ever dreamed of renaming functions, redeclaring classes and so on? Let us not discuss whether this is a good idea or not, but if you would like, look no further: there is runkit for that (I think this is the most current fork).

If you want to access the opcodes of a your code, Stefan Esser wrote bytekit for you (bytekit.org is no longer available, I only found Tyrael/bytekit and Mayflower/Bytekit). To make working with bytekit data a little more convenient, Sebastian Bergmann wrote bytekit-cli.

To register callbacks at every function call, there is funcall by Chen Ze and intercept by Gabriel Ricard.

One should not forget about xdebug by Derick Rethans that provides a quite specialized sub-sub-sub-discipline: code coverage analyis.

The future

PHP core itself could really use native support for annotations. This would fix little differences in how annotations are used nowadays by major projects. Another very interesting development is quite definitely PHP AOP. I would consider that a candidate for core inclusion at some point.

The userland libraries could see some consolidation and now that we have composer dependency management isn’t so much of a problem. Especially in the Symfony 2 world, reusing the same metadata framework would make totally sense. A first step is that Zend Framework 2 uses doctrine-common for annotations support.

Filed on 02-12-2012, 17:05 under , , , , , , , & eleven comments & no trackbacks

Dependency Injection Container Refactorings, Part Two 0

This is part of a mini-series about typical refactorings when using DI containers. Read part one.


(c) Jil A. Brown

Introduce Parameter

When configuring objects you will stumble upon occurrences of duplicated configuration. As configuration duplication is as bad as code duplication, making refactorings and maintenance time-intense and error-prone, we try to avoid them. Occurrences I had, started from defining the same hosts over and over for different services and quasi hard-coded upload prefixes for files sprinkled all over my configuration. I will illustrate this refactoring with the image upload example. We configure Zend_File_Transfer and add a few validators to allow image uploads but only specific ones:

<?xml version="1.0"?>
<container>
   <services>
      <service id="fileTransferService" class="Zend_File_Transfer">
          …
         <call method="addValidator">
            <argument>Count</argument>
            …
            <argument>photo</argument>
         </call>
         <call method="addValidator">
            <argument>Size</argument>
            …
            <argument>photo</argument>
         </call>
         <call method="addValidator">
            <argument>MimeType</argument>
            …
            <argument>photo</argument>
         </call>
         <call method="addValidator">
            <argument>ImageSize</argument>
            …
            <argument>photo</argument>
         </call>
      </service>
   <services>
</container>

When adding validators to Zend_File_Transfer the fourth argument (in this case photo) is the name of the array key of the file. In our case the markup would look like this:

<input type="file" name="photo"/>

The specific key is important if you allow the upload of various file types in one request. Now we change the requirements and allow not only photos but photos and PDFs (in the same input as photos, so that the user does not need to use different inputs based on file formats). To not mislead the next programmer working on this piece of code, we should change the markup to something like this (give me a better name please):

<input type="file" name="photoOrPdf"/>

Now we open our container configuration and change every occurrence of “photo” to “photoOrPdf” and hope not to forget one. Except the one you’ll find out two month later. To avoid this duplication of configuration, we introduce a parameter and our container configuration changes.

<?xml version="1.0"?>
<container>
   <parameters>
       <parameter key="filePrefix">photoOrPdf</parameter>
   </parameters>
   <services>
      <service id="fileTransferService" class="Zend_File_Transfer">
          …
          <call method="addValidator">
             <argument>Count</argument>
             …
             <argument>%filePrefix%</argument>
          </call>
          <call method="addValidator">
             <argument>Size</argument>
             …
             <argument>%filePrefix%</argument>
          </call>
          <call method="addValidator">
             <argument>MimeType</argument>
             …
             <argument>%filePrefix%</argument>
          </call>
          <call method="addValidator">
             <argument>ImageSize</argument>
             …
             <argument>%filePrefix%</argument>
          </call>
       </service>
   </services>
</container>

To make things even more smooth we could inject that parameter into the view and into the controller to make sure, configuration value duplication is no longer an issue with this specific module.

Parametererize Service

Excluded, as I no longer think this is actually a good idea.

Allow Environment Specific configuration

When you have a development process where you pass several acceptance stages before an artefact goes into production, these stages are typically slightly different from each other. Starting from different service IP addresses over single machine vs. multi machine, there will definitely be some variance among them. Typical variances are:

One way to do so is to sprinkle conditions all over your application and check on which host you are but that will lead to an application well beyond manageability. That’s why I was never happy (at least for large applications >100 person-days) with typical PHP application configurations like the preposterous config.inc.php. Having a touring complete programming language at hand for configuration will eventually introduce ugly conditionals making configurations unreadable. But I digress.

There are various models for stage configuration, including inheritance from each former stage, inheritance from a main configuration, standalone configuration and all mixes of these models. All of them are well implementable with the Symfony 2 dependency injection container. Let’s start with the most simplistic one, standalone configuration for each stage:

<?php
$container = new ContainerBuilder();
$loader = new XmlFileLoader($container, new FileLocator(…));
$loader->import($currentStage . '.xml');

A more complicated one is main configuration + override per stage:

<?php
$container = new ContainerBuilder();
$baseLoader = new XmlFileLoader($container, new FileLocator(…));
$baseLoader->import('main.xml');
$stageLoader = new XmlFileLoader($container, new FileLocator(…));
$stageLoader->import($currentStage . '.xml');

The most “complicated” would be linear inheritance, where testing extends development, staging extends testing and so on:

<?php
$container = new ContainerBuilder();
$loader = new XmlFileLoader($container, new FileLocator(…));
foreach (array('development', 'testing', 'staging', 'production') as $stage) {
    $loader->import($stage . '.xml');
    if ($stage == $currentStage) {
        break;
    }
}

With this kind of setup you can override configuration.

Example main.xml:

<?xml version="1.0"?>
<container>
   <parameters>
        <parameter key="database.name">application</parameter>
        …
   </parameter>
   <services>
       <service id="component" class="MyComponent"/>
       <service id="component2" class="MyComponent2"/>
   </services>
</container>

testing.xml with different database name and an alternative for component2:

<?xml version="1.0"?>
<container>
   <parameters>
        <parameter key="database.name">another_database</parameter>
   </parameter>
   <services>
      <service id="component2" class="MyAlternativeComponent2"/>
   </services>
</container>

Filed on 06-05-2011, 17:05 under , , , & no comments & no trackbacks

Dependency Injection Container Refactorings, Part One 15

This is part of a mini-series about typical refactorings when using DI containers. Read part two.


(c) Jil A. Brown

Working heavily with the Symfony2 Dependency Injection Container, I feel that we found some typical refactorings towards a DI container that emerge during the introduction of such a component. I want to write down the preliminary results of trying to systematize more or less as a draft. I will use the Symfony2 DI container configuration as an example but most of the refactorings should be applicable to other containers as well, some of them even to dependency injection without a container.

Make Dependency Explicit

This is typically the first step towards Dependency Injection: make a dependency explicit. There are three typical ways to do so, first is constructor injection, second is setter injection and third and less preferred is property injection. I roughly prefer constructor injection for invariant dependency in my domain and setter injection for infrastructure (setNotifier e.g.). Consider this example:

<?php
namespace Example;
class Client
{
    public function execute()
    {
        $dependency = new Dependency();
        $dependency->execute();
    }
}

Client creates a new instance of Dependency and call execute(). Bad for testing and for configuration, Dependency will always be hard coded there. To make it easier manageable we refactor towards setter injection:

<?php
namespace Example;
class Client
{
    public function setDependency(Dependency $dependency)
    {
        $this->_dependency = $dependency;
    }
 
    public function execute()
    {
        $this->_dependency->execute();
    }
}

Now we can manage Client in the DI container like this:

<?xml version="1.0"?>
<container xmlns="http://www.symfony-project.org/schema/dic/services">
    <services>
        <service name="example.client" class="Example\Client">
            <call method="setDependency">
                <argument type="service">
                    <service class="Example\Dependency"/>
                </argument>
            </call>
        </service>
    </services>
</container>

We see that the dependency is explicit: we specifically configure Example\Client and pass a specific Example\Dependency object.

Introduce Interface Injection

After a number of Explicit Dependency refactorings our configuration file for the service container will become huge. We will notice that we have common dependencies that are used at various places, an event manager for example. To fix that rapid growth we choose to utilize Interface Injection to ease configuration.

This is the configuration starting point:

<?xml version="1.0"?>
<container xmlns="http://www.symfony-project.org/schema/dic/services">
    <services>
        <service name="example.client" class="Example\Client">
            <call method="setDependency">
                <argument type="service">
                    <service class="Example\Dependency"/>
                </argument>
            </call>
        </service>
        <service name="example.anotherClient" class="Example\AnotherClient">
            <call method="setDependency">
                <argument type="service">
                    <service class="Example\Dependency"/>
                </argument>
            </call>
            <call method="setOtherDependency">
                <argument type="service">
                    <service class="Example\OtherDependency"/>
                </argument>
            </call>
        </service>
    </services>
</container>

We notice that both Example\Client and Example\AnotherClient depend on Example\Dependency. First of all we need an interface contracting setDependency. This is basically the Extract Interface refactoring. We call the newly extracted interface Example\DependencyAware.

The interface:

<?php
namespace Example;
interface DependencyAware
{
    public function setDependency(Dependency $dependency);
}

And we refactor both Example\Client and Example\AnotherClient to implement Example\DependencyAware.

Now we change our configuration to call setDependency no longer explicitly for Example\Client and Example\AnotherClient but for every object implementing Example\DependencyAware.

<?xml version="1.0"?>
<container xmlns="http://www.symfony-project.org/schema/dic/services">
    <services>
        <service name="example.client" class="Example\Client"/>
        <service name="example.anotherClient" class="Example\AnotherClient">
            <call method="setOtherDependency">
                <argument type="service">
                    <service class="Example\OtherDependency"/>
                </argument>
            </call>
        </service>
    </services>
    <interfaces>
        <interface class="Example\DependencyAware">
            <call method="setDependency">
                <argument type="service">
                    <service class="Example\Dependency"/>
                </argument>
            </call>
        </interface>
    </interfaces>
</container>

Expose Service

Really simple, but …
Expose Service is applied when a service has been only a dependency but should be used as a top level service. We start with the well known example:

<?xml version="1.0"?>
<container xmlns="http://www.symfony-project.org/schema/dic/services">
    <services>
        <service name="example.client" class="Example\Client">
            <call method="setDependency">
                <argument type="service">
                    <service class="Example\Dependency"/>
                </argument>
            </call>
        </service>
    </services>
</container>

Consider we want to expose Example\Dependency as a service directly, we need to change from the configuration above to
reference the service by ID.

<?xml version="1.0"?>
<container xmlns="http://www.symfony-project.org/schema/dic/services">
    <services>
        <service name="example.dependency" class="Example\Dependency"/>
        <service name="example.client" class="Example\Client">
            <call method="setDependency">
                <argument type="service" id="example.dependency"/>
            </call>
        </service>
    </services>
</container>

Simple.

Next topics would be: Introduce Parameter, Parametererize Service and Allow Environment Specific configuration

Filed on 19-04-2011, 21:09 under , , , & 15 comments & one trackback