Wednesday, July 6, 2011

Content negotiation with Figment and FsConneg

A couple of posts ago I introduced FsConneg, a stand-alone HTTP content negotiation library written in F#. One of my goals in making it stand-alone is that it could be reused across projects, maybe eventually getting integrated into Frank or MonoRail3. Here I will show some potential integrations with Figment.

Let's start with a trivial function:

let connegAction _ = "hello world"

We want to bind this to an URL (i.e. GETting the URL would return "hello world" obviously), and negotiate the response media type with the user agent. The server will support XML and JSON.

We can build a dispatch table with the supported media types and corresponding actions:

let writers = [ 
                ["text/xml"; "application/xml"], Result.xml 
                ["application/json"], Result.json 

Just as a reminder, Result.xml and Result.json are of type 'a -> FAction, that is, they take some value and return an action where the value is serialized as XML or JSON respectively.

Wrapping actions

Now with FsConneg and this table, we write a generic function that wraps the action with content negotiation (this is all framework-level code):

let internal accepted (ctx: ControllerContext) = 

let negotiateActionMediaType writers action = 
    let servedMedia = List.collect fst writers 
    let bestOf = accepted >> FsConneg.bestMediaType servedMedia >> fst 
    let findWriterFor mediaType = List.find (fst >> List.exists ((=)mediaType)) >> snd 
    fun ctx -> 
        let a = 
            match bestOf ctx with 
            | Some mediaType -> 
                let writer = writers |> findWriterFor mediaType 
                action >>= writer >>. vary "Accept" 
            | _ -> Result.notAcceptable 
        a ctx

Briefly, this function takes a table of acceptable media types and associated writers (just like the table we created above) and a "partial" action, and returns an action where the media type is negotiated with the user agent.

Armed with this function, let's bind the negotiated action to an URL:

get "conneg1" (negotiateActionMediaType writers connegAction)

For a second URL we'd also like to offer text/html. Here's a simple parameterized Wing Beats page:

let wbpage title =
    [e.Html [ 
        e.Head [ 
            e.Title [ &title ] 
        e.Body [ 
            e.H1 [ &title ] 

We want to make this an action:

let html = wbpage >> wbview

I defined wbview in a previous article, it's not instrumental to this post. What's important is that html is a function string -> FAction so we can now add it to our dispatch table:

let conneg2writers = (["text/html"], html)::writers

and bind it to an URL:

get "conneg2" (negotiateActionMediaType conneg2writers connegAction)

Using routing

An entirely different approach is to use routing to select the appropriate 'writer' (or serializer, or formatter, whatever you want to call it)

let ifConneg3Get = ifMethodIsGet &&. ifPathIs "conneg3" 
action (ifConneg3Get &&. ifAcceptsAny ["application/xml"; "text/xml"]) (connegAction >>= Result.xml)

I think this snippet is pretty intuitive, even if you're not familiar with Figment or functional programming. I'll explain anyway:

ifMethodIsGet and ifPathIs are routing functions built into Figment. The &&. operator composes these routing functions as expected, i.e. the resulting routing function must satisfy both conditions. This is explained in more detail in my introduction to Figment.

The >>= operator is a monadic bind. The function

connegAction >>= Result.xml

is equivalent to:

result { 
    let! result = connegAction 
    return! Result.xml result }


fun ctx -> 
    let result = connegAction ctx 
    Result.xml result ctx

Except the first one is evidently more concise. I explained this in more detail in my last post.

ifAcceptsAny uses FsConneg to determine if any of the media types in the list is acceptable to the client. Its definition is quite simple:

let ifAcceptsAny media = 
    fun (ctx: HttpContextBase, _) -> 
        let acceptable = FsConneg.negotiateMediaType media ctx.Request.Headers.["Accept"] 
        acceptable.Length > 0

Similarly, let's add JSON and HTML support:

action (ifConneg3Get &&. ifAcceptsAny ["application/json"]) (connegAction >>= Result.json) 
action (ifConneg3Get &&. ifAcceptsAny ["text/html"]) (connegAction >>= html)

We close by stating that all other media types are not acceptable:

action ifConneg3Get Result.notAcceptable

The HTTP RFC says it's ok to respond with some non-acceptable media type, so you could also use this to define a default media type instead of a "not acceptable".

The important thing to notice about this last example is that using routing like this doesn't yield proper content negotiation. If a user-agent requests with "Accept: application/json, application/xml;q=0.8" (i.e. prefers application/json), the code above will respond with application/xml, disregarding the client's preferences, simply because the action for application/xml was defined before application/json.

Many frameworks don't handle this properly. If you're planning to build a RESTful application I recommend testing the framework you'll use for this. For example, OpenRasta does the right thing, but Lift had issues until some months ago, and WCF Web API doesn't handle it correctly at the moment.

Using extensions

A common way to select a response media type is using extensions in the URL. Twitter used to do this, until they scrapped XML support altogether. MySpace still does it. Similarly, others use a query string parameter to select the media type, like

This isn't really content negotiation as defined by HTTP, but some people do it in the name of simplicity, or to work around client issues. It could be considered as part of a client-driven negotiation, though.

At any rate, implementing extension-driven media types is quite easy. Similarly to the first example, we can build a dispatch table of extensions and then act on it:

let extensions = [
                    "xml", Result.xml
                    "json", Result.json
                    "html", html
for ext,writer in extensions do
    let ifConneg4 = ifPathIsf "conneg4.%s" ext
    action (ifMethodIsGet &&. ifConneg4) (connegAction >>= writer)

Using extensions + conneg

Some prefer a compromise between conneg and extensions, by implementing an extensionless URL that supports content negotiation and then the same URL with extensions as a means to override conneg and work around possible client issues, or just selecting a media type without messing with headers.

For our example we'd want an URL /conneg5 that supports content negotiation, plus /conneg5.xml, /conneg5.json and /conneg5.html to force a particular media type.

As with the previous approaches, let's build a table:

let writers = [ 
                "xml", ["application/xml"; "text/xml"], Result.xml 
                "json", ["application/json"], Result.json 
                "html", ["text/html"], html 

Now let's map the extensions, just as in the last example:

let basePath = "conneg5"

for ext,_,writer in writers do 
    let ifBasePath = ifPathIsf "%s.%s" basePath ext 
    action (ifMethodIsGet &&. ifBasePath) (connegAction >>= writer)

Finally, the conneg'd URL:

let mediaTypes = (fun (_,a,b) -> a,b) writers 
let ifBasePath = ifPathIs basePath 
action (ifMethodIsGet &&. ifBasePath) (negotiateActionMediaType mediaTypes connegAction)

If you do this for all your actions you could easily extract this to a reusable function.

Final words

As I said before, these are all potential integrations. I specifically want these libraries to be as non-opinionated as possible. But at the same time, I want to provide all the tools to let the developer easily create her own opinions/conventions to fit her project, using library calls and standard language constructs instead of having to learn framework extension points. For example, notice that the dispatch tables I explained are all regular lists of strings and functions, which are then mapped and iterated just like any other list. More on this in a future post.

All code posted here is part of the FigmentPlayground repository.

No comments: