Category Archives: Developer

Unleash the true power of YQL

What is YQL

It is a SQL like language created by Yahoo that harvests the information from the Net like how you interface the relational database. Underneath the hood, someone has done the job to bind the data sources and expose it as open table. As many has shared their works, more open tables are created for your usage. Apart from solving integration headaches, its async processing plus parallelism provides you a highly performed data hub. With the nice abstraction provided by YQL, we now can easily and quickly tap into the data from various sources for our app.

Taste the power of YQL

There are tons of information in the Net that tells people how to use the YQL. So, I don’t plan to repeat it here. If you are interested in learning it, you can read the YQL guide here. If you are impatient like me, you may want to see it in action first before diving in. Below is a great presentation from Nagesh Susarla.

Design Robust Restful API

URL Path Design

Check out this video first:

  • Use resource oriented model – Every resource type should be a Noun that is normally represented as a collection. On the other hand, HTTP verbs (GET, PUT, POST, DELETE) will be used to manipulate the resources.
    • GET: read (cacheable)
    • PUT: modify (caller provides id)
    • POST: create the resource and use to call method as RPC.
    • DELETE: remove
  • Each resource should have 2 apis only: one for the collection in plural form, one for a particular entity.
  • The collection one can have ?search=… for locate the set of entities you want.
  • No verb should be in the url path.
  • Complex variation should use ?xxxx to take care of. Don’t complicate your url.
  • Pagination prefers to use ?offset=50&limit=200
  • Field extraction: field extraction uses ?fields=xxx,yyy
  • Formatting: take advantage of file extension like dogs.json
  • For input parameters, put everything into the URL and not using the HTTP headers, which is used for OAuth headers.
  • Error handling: Use HTTP error code to indicate error in the server side. (ie. 200 = succeed, 400=application error and 500 = wrong request, is used if the API). A human readable error message, together with the hint to fix that, should be sent back in the HTTP body response.
  • Operations can be sync vs async.
  • In GET operation, by default the container only return the URL reference of its immediate children. An optional parameter “expand” can be used to request the actual representation of all children and descendant.
  • API should only expose the function semantics but nothing about its implementation details, which allows the implementation to continuously evolve without breaking the client interface. And a good API should focus to do one thing well, rather than multiple things of different purposes. Each API must be self-contained and not relying on any specific call sequence to work correctly.


  1. List all persons. GET /persons
  2. Find a person with a particular id. GET /persons/123
  3. Get partial fields. GET /persons/123?fields=(name,age)
  4. Find a person’s particular friend. GET/persons/123/friends/456
  5. Find all persons named John. GET /persons/search?q=(name,eq,John)
  6. Find all dogs whose master is John. GET /persons/search?q=(name,eq,John)/dogs
  7. Create a person with a server assigned id. POST /persons?name=Dave&age=10
  8. Create a person with a client assigned id. PUT /persons/123?name=Dave&age=10
  9. Ask the person to perform an action. POST /persons/123/action/travel?location=Euro
  10. Remove a person. DELETE /persons/123
  11. Return a page of result. GET /persons/search?q=(name,eq,John)&offset=1&limit=25

Async call handling

In case when the operation takes a long time to complete, an asynchronous mode should be used. In a polling approach, a transient transaction object is return immediately to the caller. The caller can then use GET request to poll for the result of the operation. We can also use a notification approach. In this case, the caller pass along a callback URI when making the request. The server will invoke the callback URI to POST the result when it is done.


  • If you follow the minimal API design approach, the newer version is usually about adding parameters to your original API rather than removing parameters.
  • Backward compatibility via using the same URL. (e.g.…). On the implementation side, you only have the implementation that takes the latest version API parameters as input. In other words, you are prepared to receive request of the older version as well as the latest version. But you substitute the default value of the parameters of the newer version that is missing in the older version. And then send this request (with all the parameters filled) to the latest implementation.
  • For incompatible change, use a different URL endpoint for the new version (e.g.…). You also keep the corresponding implementation (v1 and v2) behind those endpoints. Depends on your decision whether to keep supporting the older version, you may want to introduce a deprecation process. Unfortunately there is no standard way to indicate an API will be deprecated in the response. One possible way is to put a flag in the HTTP header of the response to indicate when the API will be deprecated.
  • Most people put versioning info as part of url like (e.g.…). But it is debatable as others see versioning shouldn’t be part of url instead you can play trick in content type to specify what version of content you want.

Authentication and Authorization

Authentication call which must the first call to make and precede any other application API calls. As far as API security, App level key, with OAuth2.0 protocol should be used for authentication and authorization purpose.