iDatamining.org

I am looking for projects to work on
Please contact with me at yiyu.jia@iDataMining.org!

Friday, July 1, 2011

ExtJS 4 REST proxy and Zend framework Zend_Rest_Route are incompatible.

I noticed a very strange thing when I worked on Ext JS 4 v4.0.2 and Zend Framework. I find that every time when Ext JS grid sends new created Record to the server through its REST proxy, the POST method was teated as PUT method in Zend_Rest_Route. This cause troublesome because my code have to know if the incoming record a new one for insert or exsiting one for update. Otherwise, I have to use some UPSERT SQL statement to UPDATE or INSERT record into Database table. Unfortunately, I use DB2 for i5/OS V5R4, which does not have built-in UPSERT command and it even does nor support MERGE statement.

I do not want to walkaround this by doing SELECT first and judging if the code need to run INSERT or UPDATE statement. So, I traced the Ext JS 4 code and Zend Framework 1.11 code. I found that these two are not compatiable. I am not sure if we can say it is BUG as this wont happen if either side change their code. Or, both side can change the code to avoid the confilicaton if we say bugs are from both sides.

The key for this issue is that Ext JS 4 append record's id at the end of base URL, which points to RESTful service. Below is a psuedo POST URL create by Ext JS 4 REST proxy. I highlight the unexpected friend in red color.

http://hostname/yourRestServiceURI/yourRecordId/?dc=randomNum


We understand that adding record id as part of URI can allow server side script easily find it. But, why this design is implemented in a general SDK like Ext JS? Especially, I do not see good reason for a POST request to have it as record in POST are suppoe not to have been existing on server side. Therefore, the ID of the new record might not have been generated if it will be generated on the server side only.

In Ext JS 4 source code \src\data\proxyRest.js, we can see what happens,
/**
     * Specialized version of buildUrl that incorporates the {@link #appendId} and {@link #format} options into the
     * generated url. Override this to provide further customizations, but remember to call the superclass buildUrl
     * so that additional parameters like the cache buster string are appended
     */
    buildUrl: function(request) {
        var me        = this,
            operation = request.operation,
            records   = operation.records || [],
            record    = records[0],
            format    = me.format,
            url       = me.getUrl(request),
            id        = record ? record.getId() : operation.id;
        
        if (me.appendId && id) {
            if (!url.match(/\/$/)) {
                url += '/';
            }
            
            url += id; //Why add ID here even for POST? Yiyu Jia
        }
        
        if (format) {
            if (!url.match(/\.$/)) {
                url += '.';
            }
            
            url += format;
        }
        
        request.url = url;
        
        return me.callParent(arguments);
    }
}

Now, people will ask why adding id in URL cause the problem in Zend Framework? Well, let's trace Zend Framework code now.

On Zend Framework API document about Zend_Rest_Route, we can see Zend clearly described their design how RESTful URI should looks like,
Zend_Rest_Route Behavior
MethodURIModule_Controller::action
GET/product/ratings/Product_RatingsController::indexAction()
GET/product/ratings/:idProduct_RatingsController::getAction()
POST/product/ratingsProduct_RatingsController::postAction()
PUT/product/ratings/:idProduct_RatingsController::putAction()
DELETE/product/ratings/:idProduct_RatingsController::deleteAction()
POST/product/ratings/:id?_method=PUTProduct_RatingsController::putAction()
POST/product/ratings/:id?_method=DELETEProduct_RatingsController::deleteAction()

As we can see, Zend Framework does not expect any thing attached to method URI when PHP programmer intend to use POST method. In Zend_Rest_Route source code, we can see a case switch statement as below. You will see when the POST method will be "magically" routed to PUT method if pathElementCount is larger than zero.

switch( $values[$this->_actionKey] ){
                    case 'post':
                        if ($pathElementCount > 0) {
                            $values[$this->_actionKey] = 'put';
                        } else {
                            $values[$this->_actionKey] = 'post';
                        }
                        break;
                    case 'put':
                        $values[$this->_actionKey] = 'put';
                        break;
                }


But, where is the $pathElementCount set? In the same source code file, we can see below statement,

//Store path count for method mapping
$pathElementCount = count($path);

So, where is value of $path populated? Still, in the same source code, there is a function call match. Part of it is cited as below.

/**
     * Matches a user submitted request. Assigns and returns an array of variables
     * on a successful match.
     *
     * If a request object is registered, it uses its setModuleName(),
     * setControllerName(), and setActionName() accessors to set those values.
     * Always returns the values as an array.
     *
     * @param Zend_Controller_Request_Http $request Request used to match against this routing ruleset
     * @return array An array of assigned values or a false on a mismatch
     */
    public function match($request, $partial = false)
    {
        if (!$request instanceof Zend_Controller_Request_Http) {
            $request = $this->_front->getRequest();
        }
        $this->_request = $request;
        $this->_setRequestKeys();

        $path   = $request->getPathInfo();
        $params = $request->getParams();
        $values = array();
        $path   = trim($path, self::URI_DELIMITER);

Ok, now, it is not the end of code tracing yet. Where is the getPathInfo() function? It is in Zend_Controller_Request_Http. The following code citation will show us that getPathInfo() eeturns everything between the BaseUrl and QueryString and deos something special.

/**
     * Returns everything between the BaseUrl and QueryString.
     * This value is calculated instead of reading PATH_INFO
     * directly from $_SERVER due to cross-platform differences.
     *
     * @return string
     */
    public function getPathInfo()
    {
        if (empty($this->_pathInfo)) {
            $this->setPathInfo();
        }

        return $this->_pathInfo;
    }

/**
     * Set the PATH_INFO string
     *
     * @param string|null $pathInfo
     * @return Zend_Controller_Request_Http
     */
    public function setPathInfo($pathInfo = null)
    {
        if ($pathInfo === null) {
            $baseUrl = $this->getBaseUrl(); // this actually calls setBaseUrl() & setRequestUri()
            $baseUrlRaw = $this->getBaseUrl(false);
            $baseUrlEncoded = urlencode($baseUrlRaw);
        
            if (null === ($requestUri = $this->getRequestUri())) {
                return $this;
            }
        
            // Remove the query string from REQUEST_URI
            if ($pos = strpos($requestUri, '?')) {
                $requestUri = substr($requestUri, 0, $pos);
            }
            
            if (!empty($baseUrl) || !empty($baseUrlRaw)) {
                if (strpos($requestUri, $baseUrl) === 0) {
                    $pathInfo = substr($requestUri, strlen($baseUrl));
                } elseif (strpos($requestUri, $baseUrlRaw) === 0) {
                    $pathInfo = substr($requestUri, strlen($baseUrlRaw));
                } elseif (strpos($requestUri, $baseUrlEncoded) === 0) {
                    $pathInfo = substr($requestUri, strlen($baseUrlEncoded));
                } else {
                    $pathInfo = $requestUri;
                }
            } else {
                $pathInfo = $requestUri;
            }
        
        }

        $this->_pathInfo = (string) $pathInfo;
        return $this;
    }

So, what is the conclusion? The conclusion is that both Ext JS and Zend framework has their own design of RESTful URI pattern. Unfortunately, they are not compatible with each other. They are too nice to do more than developer expected. As libraries vendor, they maybe should do less sometimes in order to make their library as general as possible. Ext JS, as a pure client side JavaScript library, should not guide server side script designer how to implement RESTful Web service. Zend Framework, as a pure server side script library, should not make design for client side application either. In fact, RESTful Web service is a set of guide lines for developer to make scalable application over HTTP methods. But, it does not define how the URI pattern must be.

5 comments:

  1. Hi, I have recently run into this exact problem.
    To resolve it I created a new Extjs Zend specific REST Proxy. See my implementation here:
    http://techfrere.blogspot.com/2011/08/linking-extjs4-to-zend-using-rest.html

    The class is specific to my requirements, but can be easily customised.
    I am sure by now you have found a solution, but thought it might be of interest...

    ReplyDelete
  2. Hi freremark,

    Thank you for comment! And your post is nice one.

    Yes. This can be solved by either overwrite JavaScript function or ZF class function. Or, we can even over write both side.

    I have nothing to say if Ext JS or ZF say we can solve it by over write methods. However, as I said in my post, this could be their design problem as I do not think both sides need to do such specific URI pattern design.

    At least, our posts can be an additional document for other developers who might run into same problem.

    ReplyDelete
  3. Hy there, I am working at the moment also with Zend and Ext4, so I stumbled upon your post.

    Imho the error is neither in Ext nor in Zend.
    The "if (me.appendId && id) {"
    arround the "url += id;" should not lead to the question "Why add ID here even for POST?"
    Instead the question should be:
    Why is there an id set, although its an POST Request?

    ReplyDelete
  4. Hi Thomas, I agree with you. I was asking exactly same question as yours. I should put my comments there in a proper way of English writing. Thanks you.

    ReplyDelete
  5. Yiyu,

    I enjoyed your port.
    I implemented the RESTful web service for ZF server in Android, but I am new to ExtJS. Can you post a complete example of ExtJS 4?

    Thanks
    David

    ReplyDelete