Sunday, December 4, 2011

SolrNet 0.4.0 beta 1 released

SolrNet is a Solr client for .NET, and I just released version 0.4.0 beta 1.

  • Core functionality is stable.
  • New features are unit-tested, but may not be battle-tested.
  • Feature-frozen, no new features will be allowed between this beta and the final release. You can of course still send in pull requests for new features, they will be included in the next release, just not in 0.4.0; and in fact there is a pending pull request implementing sharding that I have to review, work on and eventually merge.
  • API-frozen, unless there's some critical flaw or an obvious win with a small change.
  • Little to no documentation about new features or changes. Tests are the primary or maybe only source of usage.

New features


  • Fixed an intermitent bug in NHibernate integration.
  • Fixed bug with LocalParams in date facets.
  • Fixed pre/post delimiters for fastVectorHighlighter.
  • Fixed exception when using SolrNet in Application_Start under IIS 7+

Breaking changes

  • Removed query result interfaces (ISolrQueryResults). Just replace it with SolrQueryResults in your code.
  • Deprecated Add(IEnumerable<T>). Use AddRange() instead.

Other stuff

  • Upgraded to NHibernate 3.2
  • Upgraded to Autofac 2.5.2

Contributors since 0.4.0 alpha 1

Huge kudos to them all! Also thanks to Stephen Pope and Paige Cook for picking up many questions about SolrNet in the mailing list and Stackoverflow.


Binaries available on Google Code as usual.

If you're upgrading from 0.3.1, see also the 0.4.0 alpha 1 release notes.

Monday, November 28, 2011

A few FSharpx examples

FSharpx doesn't currently have any proper documentation, and even comments are lacking despite what ohloh says. We will definitely improve the comments, but a reference documentation isn't going to help at all if you don't know the underlying concepts and how things fit together. A good way to introduce these concepts is by using examples that deal with familiar problems. Stackoverflow has proven to be a great source of real-world use cases for many FSharpx features, which also serve to grow it with new features. Here are the F# questions I answered last week using FSharpx I'd like to elaborate. I recommend following the links to the original questions and reading the other answers for comparison.

List multiplication

The question is about doing a cartesian product. Of course, in F# this is pretty trivial to achieve thanks to list comprehensions, but FSharpx has a couple of functions to help express this more directly:

First there's List.lift2. Its signature is ('a -> 'b -> 'c) -> 'a list -> 'b list -> 'c list .

If you're familiar with F# you'll notice this is similar to the standard function List.map2, which has the exact same signature. So what's the difference?

List.map2 works pairwise. Its result has the exact same number of elements as the parameters. Let's see an example:

> List.map2 (+) [0;1] [0;2];;


That is, it adds 1st element in the first list to 1st element in the second list, and 2nd to 2nd.

List.lift2 does precisely a cartesian product. Let's try it with the same parameters as above:

> List.lift2 (+) [0;1] [0;2];;


It adds 1st to 1st, 1st to 2nd, 2nd to 1st, 2nd to 2nd.

Same signature, very different semantics.

The name lift2 comes from lifting a function (a binary function, in this case, hence '2') to a monad (the F# list as a monad in this case). In Haskell this function is generalized to all monads and called liftM2. In F# it's also the same as liftA2, which lifts a function to an applicative functor, because there isn't really a first-class concept of monads or applicatives in F#. As an exercise, try writing the monadic 'bind' and 'return' for an F# list, then lift2 in terms of them. You can also implement lift2 using applicative functors, try it. Both exercises are quite enlightening once they 'click'.

But the question asked for a result in tuples. So we could say:

List.lift2 (fun a b -> a,b) [1..30] [1..30]

Tupling elements is a pretty common operation, so FSharpx offers a function tuple2 that does just that (and tuple3 up to tuple6. If you need more than that you probably shouldn't be using a tuple). To summarize:

List.lift2 tuple2 [1..30] [1..30]

With list comprehensions:

[ for x in [1..30] do
    for y in [1..30] do
        yield x,y ]

I think pointfree style wins here if you're familiar with monads.

Updating nested immutable data structures

In which the developer has a few nested immutable records and wants to write a mapping function that reaches deep inside this nesting. The record types model a role-playing game:

type Monster = { 
    Awake: bool 

type Room = { 
    Locked: bool 
    Monsters: Monster list 

type Level = { 
    Illumination: int 
    Rooms: Room list 

type Dungeon = { 
    Levels: Level list 

How to write a mapping function that transforms all Monsters in all Rooms within a particular Level of a Dungeon?

In other words, a function mapMonstersOnLevel : int -> (Monster -> Monster) -> Dungeon -> Dungeon .

Let's roll up our sleeves!

let mapMonstersOnLevel nLevel f dungeon =
  let levelMap (level: Level) =
    { level with Rooms = 
        (fun room -> 
                            { room with Monsters = f room.Monsters }) level.Rooms }
  { dungeon with Levels = 
                 List.mapi (fun i level -> 
                             if i = nLevel then levelMap level else level) dungeon.Levels }

Using lenses we can express this much more concisely and elegantly. After the dreaded but necessary boilerplate for lenses (each field in each record must have an associated lens explicitly spelled out), we are left with:

let mapMonstersOnLevel nLevel f =
    Dungeon.levels >>| Lens.forList nLevel >>| Level.rooms >>| Lens.listMap Room.monsters
    |> Lens.update (f |> |>

Lenses are quite intuitive, I think it's pretty easy to grasp what's hapenning there, even if you've never seen lenses before. The only difference is that because a lens Update is a Get, followed by the transforming function, followed by a Set, this isn't as effective as the first version. It can be optimized, but I'm not sure it's worth it, updating an element in an immutable list is still O(n), it doesn't matter much if it needs an extra pass.

Exception handling in pipeline sequence

An initial state is pipelined along a series of functions that modify this state. The state in question is a point on a plane, but that's not important. Problem is, each function can throw, and when that happens the whole computation must be aborted and an error must be shown.

Initially, the problem is posed like this:

let startingPosition = 0. ,0.

let moveByLengthAndAngle l a (x,y) = x,y // too lazy to do the math
let moveByXandY dx dy (x,y) = x+dx, y+dy
let moveByXandAngle dx a (x,y) = x+dx, y

let finalPosition =
    |> moveByLengthAndAngle x1 a1 
    |> moveByXandY x2 y2
    |> moveByXandAngle x3 a3
    |> moveByLengthAndAngle x4 a4
    // etc...

Instead of doing this, let's represent the computation as a list of partially applied functions with the associated error messages:

let actions = 
        moveByLengthAndAngle x1 a1, "failed first moveByLengthAndAngle"
        moveByXandY x2 y2, "failed moveByXandY"
        moveByXandY x3 y3, "failed moveByXandY"
        moveByXandAngle x3 a3, "failed moveByXandAngle"
        moveByLengthAndAngle x4 a4, "failed second moveByLengthAndAngle"

And then fold over this list of actions, starting with the initial point and accumulating the result of each function. In the folding function we must also catch any potential exception and put the error message instead, and abort the rest of the computation. But the state is a pair of floats and the error message obviously a string, so we model the actual state as a Choice of either the pair of floats or a string (i.e. Choice<float * float, string> ):

let finalPosition = 
    let evalToChoice (f,message) a =
            f a |> Choice1Of2
        with _ -> Choice2Of2 message
    let folder a f = 
        match a with 
        | Choice2Of2 a -> Choice2Of2 a 
        | Choice1Of2 a -> f a
    |> evalToChoice
    |> List.fold folder (Choice1Of2 startingPosition)

All that's left to do is pattern match on finalPosition to see if the computation ended up as error or completed successfully:

match finalPosition with
| Choice1Of2 (x,y) -> 
    printfn "final position: %f,%f" x y
| Choice2Of2 error -> 
    printfn "error: %s" error

FSharpx has a couple of abstractions (mostly borrowed from Haskell, as usual) that help reduce the clutter here:

Firstly, FSharpx defines a monad around Choice1Of2 / Choice2Of2. This is like the Either monad defined in Haskell or Scalaz, except we call it Choice because... well, the F# type is already called Choice. So we have Choice.bind and returnM (actually 'return' is just the constructor Choice1Of2). The convention here is that Choice1Of2 carries the successful branch of the computation and Choice2Of2 carries the error.

FSharpx also defines bifunctor for Choice. In Scalaz and Haskell this is a typeclass (of which Either is an instance), but in FSharpx this is simply two functions bimap and mapSecond. Bimap takes two mapping functions, and uses the first one to map the Choice1Of2 part of a Choice, and the other one to map the Choice2Of2 part. mapSecond only maps the Choice2Of2 part of a choice. What about mapFirst? It's simply map.

Since Choice/Either is frequently used in pure functional programming to represent errors, and we're working in an impure language, it makes sense to have a library function to convert a function that throws as a means to report an error into a function that returns a Choice, where the Choice2Of2 part is the potential exception. This seems to be pretty common in Scala, and it's what Choice.protect in FSharpx does. The signature is pretty explicit: ('a -> 'b) -> 'a -> Choice<'b,exn>

Putting all the pieces together:

let finalPosition = 
    let folder position (f,message) =
        Choice.bind (Choice.protect f >> Choice.mapSecond (konst message)) position
    List.fold folder (Choice1Of2 startingPosition) actions

Haskellers have long ago recognized this folding with a monadic result and abstracted it into a generic function foldM. As usual, in F# we can't express this generically for any monad, but we can write it for every monad. With foldM the code looks like this:

let finalPosition = 
    let folder position (f,message) = 
        Choice.protect f position |> Choice.mapSecond (konst message)
    Choice.foldM folder startingPosition actions

A different approach to the problem is to rethrow the message as an exception, instead of using Choices. This is quite concise as well:

let finalPosition() = 
    let evalRethrow (f,message) a =
        try f a with _ -> failwith message
    |> evalRethrow
    |> List.fold (|>) startingPosition

Instead of pattern matching over the result, you'd invoke this function in a try..with block:

    let x,y = finalPosition()
    printfn "final position: %f,%f" x y
with e -> 
    printfn "error: %s" e.Message

The problem with using exceptions is that they're not really part of the type system. The function finalPosition is of type unit -> float*float . This doesn't say it might throw an exception, so you can't statically enforce handling the error case. Also, exceptions are actually misused here: having an error in the computation is an expected result, not a programming error. I recommend reading Errors vs Exceptions for more information on this. The terms 'exception' and 'error' may sometimes be used in one way or the other, but it's always useful to make the distinction.

Also worth noting is how representing the computations as a list of actions with associated data (in this case, the messages) allows us to freely choose how to evaluate it.

Using F# datatypes in C#

FSharpx is not just for F# consumers. I blogged before about using the F# runtime as a library in C# apps. FSharpx adds some sugar so you can use F# persistent collections in C# quite comfortably. Here's an example:

var a = FSharpList.Create(1, 2, 3);
var b = a.Cons(0);
b.TryFind(x => x > 4)
 .Match(v => Console.WriteLine("I found a value {0}", v),
        () => Console.WriteLine("I didn't find anything"));

Tuesday, November 15, 2011

Lenses in F#

Consider the following record types in F# :

type Car = { 
    Make: string 
    Model: string 
    Mileage: int 

type Editor = { 
    Name: string 
    Salary: int 
    Car: Car 

type Book = { 
    Name: string 
    Author: string 
    Editor: Editor 

Given a Book we can trivially retrieve its editor's car mileage:

let mileage = aBook.Editor.Car.Mileage

Setting the mileage is a bit more problematic though. If this were mutable, we could just do:

aBook.Editor.Car.Mileage <- 1000

But it's not, so we use F# copy-and-update syntax:

let book2 = { aBook with Editor = 
                { aBook.Editor with Car = 
                    { aBook.Editor.Car with Mileage = 1000 } } }

That's a lot of fun! Not. Can we make this prettier?

Or if we want to modify a property, for example add 1000 to the mileage, even with mutable properties we have to do:

aBook.Editor.Car.Mileage <- aBook.Editor.Car.Mileage + 1000

If this were C# we could just say:

aBook.Editor.Car.Mileage += 1000;

That's pretty convenient, so how can we implement something similar in F# with immutable records?

In summary, can we gain back some of the convenience of mutable properties?

The key is to make properties first-class values.

First we ask ourselves: what's a property getter? It's a function 'a -> 'b , where 'a is the record type and 'b is the property type.

A setter for a mutable property is a function 'b -> 'a -> unit . But we're interested in setters for immutable records. Such a setter is a function 'b -> 'a -> 'a , i.e. it takes the new value, the record, and returns the modified record.

So we have a pair of functions:

Get : 'a -> 'b

Set : 'b -> 'a -> 'a

Let's put them together in a record, which we'll call "Lens":

type Lens<'a,'b> = {
    Get: 'a -> 'b
    Set: 'b -> 'a -> 'a

Modeling a 'modify' operation on top of this is easy: you get the original value, modify it with some function, then set the modified value back:

with member l.Update f a = 
    let value = l.Get a 
    let newValue = f value 
    l.Set newValue a

Note that this Update is still purely functional.

Let's create some lenses for the record types we declared above:

type Car with
    static member mileage = 
        { Get = fun (c: Car) -> c.Mileage
          Set = fun v (x: Car) -> { x with Mileage = v } }

type Editor with
    static member car = 
        { Get = fun (x: Editor) -> x.Car 
          Set = fun v (x: Editor) -> { x with Car = v } }

type Book with
    static member editor = 
        { Get = fun (x: Book) -> x.Editor 
          Set = fun v (x: Book) -> { x with Editor = v } }

What we need now is a way to compose these lenses, so we can go from a book to a mileage and update it:

let inline (>>|) (l1: Lens<_,_>) (l2: Lens<_,_>) = 
    { Get = l1.Get >> l2.Get 
      Set = l2.Set >> l1.Update }

Lenses are closed under composition. That is, the sequential composition of any two lenses is a lens.

We can now create a lens that goes from a book instance to the mileage, by composing primitive lenses:

let bookEditorCarMileage = Book.editor >>| >>| Car.mileage

let mileage = bookEditorCarMileage.Get aBook

let book2 = aBook |> bookEditorCarMileage.Set 1000

We can also implement (+=) generically for any lens focused on a type supporting (+) (e.g. int, float, decimal):

let inline (+=) (l: Lens<_,_>) v = l.Update ((+) v)

let book2 = aBook |> bookEditorCarMileage += 1000

Thanks to lenses we now have much of the convenience of the mutable properties, while at the same time making properties first-class composable objects and retaining purity.

Lenses and the State monad

It's also possible to lift lens operations to the State monad: this gives an even more imperative feel, while still retaining referential transparency. Here's an example (using FSharpx's State monad):

let promote =
    state {
        let! oldSalary = Lens.getState Editor.salary
        do! Editor.salary += 1000
        return oldSalary
let oldSalary, promotedTom = promote tom
printfn "Tom used to make %d, after promotion he now makes %d" oldSalary promotedTom.Salary


I've barely scratched the surface of lenses here. The general concept of lenses is that a lens allows to focus on a particular element in a data structure, both to view it and to update it. More formally, lenses can be described as well-behaved bidirectional transformations. Lenses must follow some laws (which I haven't shown here) in order to be well-behaved.

Lenses are being actively researched, and because of their genericity they have been applied in widely different areas, for example functional reactive AJAX applications (lenses for Flapjax) and configuration management.

There's even a whole research programming language around lenses called Boomerang.

Virtual Combat Cards, a tool to assist Dungeons & Dragons Dungeon masters, written in Scala, makes use of lenses as first-class properties (code is here if you want to check it out).

Scalaz implements the basic plumbing and definition of lenses for Scala, and this F# implementation draws heavily from that code. Hat tip to the Scalaz devs. Haskell has a similar package Data.Lenses.

When using lenses for properties as I've shown here, there's the problem of having to define the primitive lenses that wrap the actual .NET properties. Haskell solves that using Template Haskell, and Scala has a compiler plugin currently under development which automatically generates this boilerplate. It may be possible to write a type provider to do this in F# 3.0, but I haven't looked into this yet.

F# code shown here coming soon to FSharpx.

Saturday, October 29, 2011

Poor man's option type in C#

I've blogged before about the virtues of the Option type. However, it's currently not part of the standard .NET library, so either you have to use F#, code it yourself, or use a library.

But there is something built-in and almost equivalent: lists! You can use any list-like container (List<T>, T[]) as an option type, and all operations are already built-in.

Option has None (no value) and Some (a value). This corresponds exactly to an empty list and a singleton list respectively. Let's see how we map operations on Options (using FSharpx) to operations on arrays using standard .NET 3.5:


Option Array / IEnumerable
Constructor: Some
new[] { 5 }
Constructor: None
new int[0]
Check if the option has a value
FSharpOption<int> o = ...
bool hasValue = o.HasValue();
IEnumerable<int> o = ...
bool hasValue = o.Any();
Get the value associated with the option
FSharpOption<int> o = ...
int value = o.Value;
IEnumerable<int> o = ...
int value = o.First();
Pattern matching
FSharpOption<int> o = ...
int b = o.Match(x => x + 2, () => 99);
IEnumerable<int> o = ...
int b = o.Aggregate(99, (_, x) => x + 2);
or lazier:
int b = singleVersion.Any() ? singleVersion.First() + 2 : 99;

And thanks to LINQ, you also get mapping and monadic syntax for free. Remember that code I refactored to monads a while ago? Here's the last part of it side-by-side with a translation using Array/IEnumerable:

var maxVersion = L.F((string[] parts) => {
    var p = parts.Length == 2 ? parts[1] : parts[0];
    if (string.IsNullOrWhiteSpace(p))
        return FSharpOption<FSharpOption<Version>>.Some(FSharpOption<Version>.None);
    return ParseVersion(p).Select(v => v.ToOption());

var singleVersion =
    from v in ParseVersion(value)
    select (IVersionSpec) new VersionSpec {IsMinInclusive = true, MinVersion = v};

var versionRange = L.F(() => from x in checkLength(value)
                             from isMin in minInclusive(value)
                             from isMax in maxInclusive(value)
                             let val = value.Substring(1, value.Length - 2)
                             let parts = val.Split(',')
                             from y in checkParts(parts)
                             from min in minVersion(parts)
                             from max in maxVersion(parts)
                             select (IVersionSpec) new VersionSpec {
                                 IsMinInclusive = isMin,
                                 MinVersion = min.HasValue() ? min.Value : null,
                                 IsMaxInclusive = isMax,
                                 MaxVersion = max.HasValue() ? max.Value : null,

return singleVersion.OrElse(versionRange)();
var maxVersion = L.F((string[] parts) => {
    var p = parts.Length == 2 ? parts[1] : parts[0];
    if (string.IsNullOrWhiteSpace(p))
        return new[] {new Version[0]};
    return ParseVersion(p).Select(v => new[] {v});

var singleVersion =
    from v in ParseVersion(value)
    select (IVersionSpec) new VersionSpec {IsMinInclusive = true, MinVersion = v};

var versionRange = L.F(() => from x in checkLength(value)
                             from isMin in minInclusive(value)
                             from isMax in maxInclusive(value)
                             let val = value.Substring(1, value.Length - 2)
                             let parts = val.Split(',')
                             from y in checkParts(parts)
                             from min in minVersion(parts)
                             from max in maxVersion(parts)
                             select (IVersionSpec) new VersionSpec {
                                 IsMinInclusive = isMin,
                                 MinVersion = min.Any() ? min.First() : null,
                                 IsMaxInclusive = isMax,
                                 MaxVersion = max.Any() ? max.First() : null,

return singleVersion.Any() ? singleVersion : versionRange();

If you want to compare the whole code: here's the original (using option) and here's the one using arrays.

You've even probably used this already, perhaps without realizing this relation. However, while an option type can either have exactly one or zero value, an array can have any number of values. And if you see a method returning an IEnumerable<T>, you wouldn't think you're supposed to treat it as an option.

So IEnumerable<T> as a monad (the List monad, that is) is sort of an extension of the option type (i.e. the Maybe monad): instead of just supporting one successful computation, it supports many. I think using the List monad as an Option is acceptable locally, and only if you can't use a proper option type or for some reason don't want to take the dependency on a library. It's a useful hack, but still a hack. They're different things really.

Tuesday, October 18, 2011

10 reasons to use the F# runtime in your C# app

Most people have at least noticed that F# shipped with Visual Studio 2010. I mean, you click File –> New Project and there's the F# project templates, you can't miss them.

What most people probably didn't realize is that even if you don't use the F# language or aren't even interested in it, you can still profit from using the F# runtime in C# / VB.NET projects. The F# runtime is just a regular DLL named FSharp.Core.dll you can reference just like any other assembly. It's available for .NET 2.0 and 4.0 (separate DLLs). This availability for .NET 2.0 is particularly valuable for projects that for some reason can't be upgraded to newer versions of .NET.

Granted, the library is designed to be used from F#, so sometimes it looks weird in C#, but we'll see how to work around some of the oddities with FSharpx.

Let's see some of the things FSharp.Core gives you, in no particular order:


So you want to use tuples but you can't upgrade to .NET 4 because of company policies or some obscure dependency that breaks. No problem, FSharp.Core.dll implements them, so you can use tuples in .NET 2.0 with exactly the same API and namespace as .NET 4 tuples. If you then upgrade you don't have to change anything.

Tuples are simple but not trivial to implement, for example some forget to implement equality / hashing so you'd end up with "WTF moments" at some point. It's worth using a library that implements them properly.

As usual, keep in mind that tuples are essentially anonymous. Item1, Item2, etc don't convey any information about what they're holding, only their types.

Persistent collections

Persistent lists are one of the most frequently used data structures in functional programming. They're so prevalent that F# has special syntax for them. For example, to define an empty list in F# :

let empty = []

F# infers the list element type. In C# things are more verbose:

var empty = FSharpList<int>.Empty;

To add an element to a list you actually create a new list that has the new element as head and the other list as tail. Again, F# has special syntax:

let a = 1::empty

While in C#:

var a = new FSharpList<int>(1, empty);


var a = FSharpList<int>.Cons(1, empty);

FSharpx helps with a little sugar here:

var a = empty.Cons(1);

You can also create an immutable list from any IEnumerable<T>:

var b = SeqModule.ToList(new[] { 1, 2, 3 });

Again, FSharpx adds some sugar:

var b = new[] { 1, 2, 3 }.ToFSharpList();


var b = FSharpList.Create(1, 2, 3);

How do you use a FSharpList? You can access a particular element just as with a regular mutable list:

Console.WriteLine(b[2]); // prints "3"

Be aware that random access in an immutable linked list is O(n).

FSharpList implement IEnumerable<T>, so you can traverse it with foreach and use all LINQ functions (Aggregate, Select, Where, etc).

Functional languages often use pattern matching and recursion to process a list. The F# wikibook has a great chapter explaining it. FSharpx implements basic pattern matching on lists for C#, so you can write this to reverse a list:

void Reverse() {
    var a = Enumerable.Range(0, 1000).ToFSharpList();
    var r = Loop(FSharpList<int>.Empty, a);

static FSharpList<T> Loop<T>(FSharpList<T> acc, FSharpList<T> l) {
    return l.Match(() => acc,
                   (head, tail) => Loop(acc.Cons(head), tail));

But be careful! F# compiles the equivalent code using tail call optimization, while C# doesn't have that feature, so the above code blows with a StackOverflowException when given a sufficiently big list (unless you've compiled with optimizations and running in a 64-bit CLR !)

When recursively processing lists, it's best to use Aggregate() instead if possible (usually called fold in functional languages), which encapsulates recursion without blowing the stack. It's also simpler:

var a = Enumerable.Range(0, 1000000).ToFSharpList();
var r = a.Aggregate(FSharpList<int>.Empty, (acc, i) => acc.Cons(i));

Of course, this is just demo code. If you really want to reverse a list just call ListModule.Reverse(a);

FSharp.Core also implements a persistent set and dictionary.

Imperative programmers might wonder why they should use an immutable collection when the BCL already has several perfectly good mutable collections.

One of the most cited reasons for using persistent collections (and functional programming in general) is multithreading. Indeed you can freely and safely pass persistent collections around threads, which makes multithreaded development easier. However, the same can be said about passing collections around regular functions: you can be sure that no function can ever modify a list, therefore you have one less thing to keep track of in your head and you statically eliminate a whole class of bugs. Immutability makes all kinds of programming simpler, multithreaded or not. Of course, for immutable collections to really work as immutable, the underlying element type must be also immutable.

Reactive extensions also includes an ImmutableList class, although it's internal.

The Option type

I have blogged before about using the F# Option type in C# projects here and here. Options are pervasively used in F#, for example several functions on collections use options. The problem is, these functions take the equivalent of a Func but in F#, which is an FSharpFunc, which makes it very inconvenient to use them from C#.

FSharpx wraps these F# functions so they can be used with System.Func and System.Action. For example:

var a = FSharpList.Create(1, 2, 3);
a.TryFind(x => x > 4) // returns FSharpOption<int>
    .Match(v => Assert.Fail("shouldn't have found value {0}", v),
           () => { /* nothing found */ });

The Unit type

Many functional languages like F# have a type called "Unit", which is just like "void" in C-like languages, except it's actually usable as a proper type.

By "usable" I mean you can actually define something like a Func<Unit> (you can't have a Func<void>, it's not even syntactically correct even though there is a type System.Void). A Func<Unit> is just like an Action, except it's obviously a Func so it can be used for example in a LINQ expression (i.e. a monad).

FSharpx includes a ToFunc() extension method on Action, Action<T>, Action<T1,T2>, etc. to respectively convert them to Func<Unit>, Func<T,Unit>, Func<T1,T2,Unit> and so on.

You can also use it for types like FSharpOption<Unit> as I blogged about before.

Reactive Extensions also includes a Unit type.

Discriminated unions

I have blogged before about using F# discriminated unions in C# here and here, in the context of validation. They're very useful to express things like "either this or that" without having to introduce a whole class hierarchy implementing equality / hash / comparison.

Just as with other things, using them in C# is more verbose than in F#.

Let's see an example:

var a = FSharpChoice<int, string>.NewChoice1Of2(1);
if (a.IsChoice1Of2) {
    var x = ((FSharpChoice<int, string>.Choice1Of2)a).Item;
    Console.WriteLine(x + 2);
} else if (a.IsChoice2Of2) {
    var x = ((FSharpChoice<int, string>.Choice2Of2)a).Item;
    Console.WriteLine(x + ";");

Now that looks really ugly. And what's with the downcasting?!

FSharpx makes this more usable by implementing pattern matching (basically a visitor) so you can write instead:

var a = FSharpChoice.New1Of2<int, string>(1);
a.Match(x => Console.WriteLine(x + 2),
        x => Console.WriteLine(x + ";"));

FSharpx also implements LINQ operators around 2-choice and integrates with Option. Here's an example:

object a = 40;
const string b = "60";
var r = from i in FSharpOption.ParseInt(b).ToFSharpChoice("Invalid value b")
        from j in FSharpChoice.Cast<int>(a).SelectSecond(_ => "Invalid value a")
        select i + j;
r.Match(i => Assert.AreEqual(100, i),

Just as with tuples, discriminated unions are essentially anonymous. Tuples are the generic, anonymous product types. Discriminated unions are the generic, anonymous sum types.

Reactive extensions uses an internal Either<TLeft, TRight> type.


Once again, you're stuck with .NET 3.5 drooling over the Task Parallel Library in .NET 4.

Reactive extensions used to include a backport of System.Threading.dll, but it was unsupported and it's not included in recent releases any more.

F# has asynchronous workflows, which is similar yet somewhat different from C# 5 await/async (see differences in this series of posts by Tomas Petricek)

FSharpx has LINQ bindings for this so you can write:

static FSharpAsync<string> Get(string u) {
    var web = new WebClient();
    return web.AsyncDownloadString(new Uri(u));
var qq = // qq is of type FSharpAsync<string>
    from google in Get("")
    from bing in Get("")
    select google + bing;

string result = qq.Run();

Or you can run multiple requests in parallel:

var urls = FSharpList.Create(
    , ""
    , ""
    , ""
var result = FSharpAsync.Parallel(urls.Select(Get)).Select(s => string.Join("", s)).Run();

It may not be as powerful as F# async workflows, but still useful.


Another one for .NET 2.0 / 3.5 users. FSharp.Core includes System.Numerics.BigInteger for arbitrary-precision arithmetic. It doesn't have all of .NET 4 BigInteger's methods, but it implements the basic operations. Want to calculate 23^25 + 4? No problem:

var a = new BigInteger(23);
var b = BigInteger.Pow(a, 25);
b += new BigInteger(4);

Result: 11045767571919545466173812409689947


The Lazy<T> type is yet another feature that .NET 4 copied from F#, or so it seems. Are you still writing singletons the "old" way? With Lazy you can just do this in .NET 3.5 (using FSharpx-added sugar):

class MySingleton {
    private MySingleton() {}

    private static readonly Lazy<MySingleton> instance = 
        FSharpLazy.Create(() => new MySingleton());

    public static MySingleton Instance {
        get { return instance.Value; }

Although to be honest, I don't think I've ever used this.

Enumerable cache

Sometimes you have a forward-only iterator wrapped in an IEnumerable, like database results or some data parsed lazily from a web request, and you want to traverse it more than once, but you also want to keep it lazy, so ToList() doesn't cut it. With FSharp.Core you can cache it on demand using Seq.cache, named SeqModule.Cache in C# / VB.NET.

System.Interactive also has a function like this, it's called MemoizeAll, although I like the F# name better as it seems to be more an application of caching than memoization.

Enumerable zip

Another nifty operator that is only available in .NET 4+. The one in FSharp.Core is slightly different: Enumerable.Zip includes a mapper, its signature is:

IEnumerable<TResult> Zip<TFirst, TSecond, TResult>(
    this IEnumerable<TFirst> first,
    IEnumerable<TSecond> second,
    Func<TFirst, TSecond, TResult> resultSelector)

while the one in F# (also in the static SeqModule class) zips directly to a tuple:

IEnumerable<Tuple<T1, T2>> Zip<T1, T2>(IEnumerable<T1> first, IEnumerable<T2> second)


If you're working with Visual Studio 2010, the F# runtime is a great library you can take advantage of, even in .NET 2.0 projects. And you already have it, so use it!

If you run .NET 3.5 or better, FSharpx makes it more C# friendly. It also makes it easier to interop with F# projects if you ever need it, since they use the same underlying types.

Even in .NET 4, persistent collections, discriminated unions and Option alone are easily worth the dependency.

Also worth mentioning is the F# PowerPack, a separate library implementing additional collections like HashMultiMap and LazyList and math-specific facilities such as rational and complex numbers, matrix, vectors.

And it's all open source, Apache-licensed.

PS: did you know the VB.NET runtime has a CSV parser?

Friday, October 7, 2011

Introducing FSharpx

A couple of months ago I started writing FSharp.Core.CS, a project to bridge F# core constructs to C#, such as the Option type. I think I never mentioned it explicitly but I did blog about it, for example back when I wrote about validating with applicative functors with LINQ.

I realized I was going to implement several monads in F# and there was an excellent project already doing that: Ryan Riley's FSharp.Monad. It only missed C# compatibility, exactly what I was doing in FSharp.Core.CS among other things, and FSharp.Monad already was doing some things other than monads, so it seemed like the perfect moment to merge both projects, and so we created FSharpx.

FSharpx aims to create a wider and richer foundation for programming in .NET, building on top of the F# core library and the PowerPack. It targets mainly F# but strives to be usable from all .NET languages wherever possible.

It's similar in spirit to Scalaz, even though Scala can do things like typeclasses which F#/C# cannot.

Here's a brief summary of what FSharpx.Core currently implements:

Tomas Petricek's async extensions were also merged, under the name FSharpx.AsyncExtensions, which implements:

Finally, Steffen Forkmann has started a branch for F# 3.0 type providers, which includes AppSettings, File system, Regex, CSV. These are implemented using a DSL on top of the base classes provided by Microsoft.

It's still early days for FSharpx, we're frequently breaking things, and there's almost no documentation. Still, I'm already using it in some of my projects (Figment, FsFormlets, CsFormlets) and in real-world production C#/VB.NET code.

I'm very excited about FSharpx as it is truly community-driven with many people involved. Contributors so far include Ryan Riley, Steffen Forkmann, Tomas Petricek, Daniel Mohl, Gustavo Guerra, Hodza Nassredin and yours truly.

So come check it out! If you have any questions, drop by our mailing list.

Monday, August 29, 2011

Validating with applicative functors in F#

In my last post I showed how to validate using applicative functors in C#. LINQ helps a lot with making the syntax bearable.

In F# we have other tools to encode applicative functors: custom operators and type inference. And thanks to pattern matching it's much easier to define the validation applicative functor operations in F# (but I won't show that here).

So let's see what the C# code from the last post looks like in F#. Hopefully this article will serve both C# developers interested in learning F# and F# developers interested in learning more about applicative functors. F# programmers will probably find the first part boring and might want to go directly to the second part.

Defining the primitives

Starting from the bottom up, here's the function that builds a validator from a predicate:

static Func<T, FSharpChoice<T, Errors>> Validator<T>(Predicate<T> pred, string error) {
    return x => {
        if (pred(x))
            return FSharpChoice.Ok(x);
        return FSharpChoice.Error<T>(error);

Same thing in F#:

let validator pred error x =
    if pred x
        then Choice1Of2 x
        else Choice2Of2 [error]

We don't need the Ok and Error constructors because F# correctly infers the appropriate types. In fact, we did not use a single type annotation!

We don't need to explicitly state that we're returning a function, as we'll see now:

static FSharpChoice<T, Errors> NonNull<T>(T value, string err) where T: class {
    return Validator<T>(x => x != null, err)(value);

Same in F# :

let (==) = LanguagePrimitives.PhysicalEquality
let inline (!=) a b = not (a == b)

let nonNull e = validator ((!=) null) e

Readers unfamiliar with F# might wonder about the definition of == and !=. The answer is: F# doesn't have a built-in reference equality operator, and with good reason: usually you want to compare things by their structure rather than by reference. We could have used F# structural equality here (the <> operator in this case) but this would have placed an additional equality type restriction. It doesn't really matter in this case but we'll try to keep things as close as possible to the equivalent C# code. If you want to learn more about structural equality I refer you to this very thorough article by Brian McNamara.

C# programmers that are still paying any attention to this code (gotcha! ;-) should have noticed that I defined the validator function with three parameters (pred, error, x), yet I only defined the first two arguments when defining nonNull. Where did the other parameter go? This is an example of partial application of a curried function. Currying and partial application are often confused with each other, but they're not the same thing. Ivan Towlson has a great series of articles about partial application in F# and comparing it to C#. I also recomend this article (and related) by Dustin Campbell.

Moving on:

static FSharpChoice<T, Errors> NotEqual<T>(T value, T other, string err) {
    return Validator<T>(v => !Equals(v, other), err)(value);

Same in F# :

let notEqual a = validator ((<>) a)

Here we did use structural equality, we certainly don't want to compare things by reference. Object.Equals is not the same as F#'s structural equality. Something probably a little closer would be requiring T to implement IEquatable<T> in the C# function, but still not the same.

static Func<Address, FSharpChoice<Address, Errors>> ValidateAddressLines = 
    Validator<Address>(x => x.Line1 != null || x.Line2 == null, 
                       "Line1 is empty but Line2 is not");

Same in F# :

let validateAddressLines =
        (fun (a: Address) -> a.Line1 != null || a.Line2 == null) 
        "Line1 is empty but Line2 is not"

Nothing interesting here. Next:

static FSharpChoice<T?, Errors> GreaterThan<T>(T? value, T? other, string err) where T: struct, IComparable<T> {
    if (!value.HasValue && !other.HasValue || value.HasValue != other.HasValue || value.Value.CompareTo(other.Value) > 0)
        return FSharpChoice.Ok(value);
    return FSharpChoice.Error<T?>(err);

The bad news here is that F# doesn't have built-in support for nullable types. Fortunately, it's not too hard to write a few functions and get decent syntax. I already did that about a year ago so I'll just use it here:

let greaterThan o = validator ((<?) o)

No, that's not a typo: due to the way operators are defined, they have to be read backwards when partially applied. You can be a bit more explicit/verbose and instead say:

let greaterThan o = validator (fun a -> a >? o)

Combining functions through monads and applicative functors

Now that we have all primitives defined, let's get to the interesting part: combine them with an applicative functor and monad.

I've blogged about applicative functors in F# many times... very briefly, there are two operations: pure (puree in F#) and apply (<*> in F#). And we define this convenience function (this is standard for applicative functors):

let inline (<!>) f x = puree f <*> x

Here's ValidateAddress() in C# using LINQ to encode the applicative functor:

static FSharpChoice<Address, Errors> ValidateAddress(Address a) {
    return from x in NonNull(a.Postcode, "Post code can't be null")
           join y in ValidateAddressLines(a) on 1 equals 1
           select a;

Here's the equivalent in F# :

let validateAddress (a: Address) = 
    fun x y -> a
    <!> nonNull "Post code can't be null" a.Postcode
    <*> validateAddressLines a

If you squint a little, they're not that different really. We can go a bit further to avoid having those dummy values x and y that we end up ignoring anyway. We only need to define (these are also standard functions):

let inline lift2 f a b = f <!> a <*> b
let inline ( <*) a b = lift2 (fun z _ -> z) a b

Now we can say:

let validateAddress (a: Address) = 
    puree a
    <* nonNull "Post code can't be null" a.Postcode
    <* validateAddressLines a

Validating a single Order in C#:

static FSharpChoice<Order, Errors> ValidateOrder(Order o) {
        from name in NonNull(o.ProductName, "Product name can't be null")
        from cost in GreaterThan(o.Cost, 0, string.Format("Cost for product '{0}' must be positive", name))
        select o;

Unlike the previous validators, this one is monadic, not applicative. In F# we have some alternatives. We can use a computation expression:

let validateOrder (o: Order) =
    validation {
        let! name = nonNull "Product name can't be null" o.ProductName
        let! _ = greaterThan (0m).n (sprintf "Cost for product '%s' must be positive" name) o.Cost
        return o

Or we can use monadic operators directly:

let validateOrder (o: Order) =
    let nameNotNull = nonNull "Product name can't be null" o.ProductName
    let positiveCost n = greaterThan (0m).n (sprintf "Cost for product '%s' can't be negative" n) o.Cost
    nameNotNull >>= positiveCost |> map (fun _ -> o)

The higher-order function to make this operate on sequences of orders:

static FSharpChoice<FSharpList<Order>, Errors> ValidateOrders(IEnumerable<Order> orders) {
    var zero = ListModule.Empty<Order>().PureValidate();
    return orders
        .Aggregate(zero, (e, c) => from a in e
                                   join b in c on 1 equals 1
                                   select a.Cons(b));

Same in F# :

let inline flip f a b = f b a
let inline cons a b = a::b
let seqValidator f = 
    let zero = puree [] f >> Seq.fold (lift2 (flip cons)) zero
let validateOrders c = seqValidator validateOrder c

And the final validations with an sample application:

let customer = 
        Surname = "foo",
        Address = Address(Postcode = "1424"),
        Orders = ResizeArray([
                                Order(ProductName = "Foo", Cost = (5m).n)
                                Order(ProductName = "Bar", Cost = (-1m).n)
                                Order(ProductName = null , Cost = (-1m).n)
let result = 
    puree customer
    <* nonNull "Surname can't be null" customer.Surname
    <* notEqual "foo" "Surname can't be foo" customer.Surname
    <* validateAddress customer.Address
    <* validateOrders customer.Orders
match result with
| Choice1Of2 c -> printfn "Valid customer: %A" c
| Choice2Of2 errors -> printfn "Invalid customer. Errors:\n%A" errors

Wrap it up already!

I know, I know, this post is too long already. Hopefully this step-by-step, side-by-side comparison will help C# programmers see some of the power of F# through currying, partial application, custom operators and type inference, in a non-trivial example. And F# developers can see another use for applicative functors: validation.

Code is here.

Monday, August 22, 2011

Validating with applicative functors and LINQ

In my last post I introduced the basics of validation with applicative functors. I used a very simple example that didn't make it justice, so let's fix that now. I'll borrow a more complex example from the FluentValidation wiki:

class Address {
    public string Line1 { get; set; }
    public string Line2 { get; set; }
    public string Town { get; set; }
    public string County { get; set; }
    public string Postcode { get; set; }        

class Order {
    public string ProductName { get; set; }
    public decimal? Cost { get; set; }

class Customer {
    public int Id { get; set; }
    public string Surname { get; set; }
    public string Forename { get; set; }
    public decimal Discount { get; set; }
    public Address Address { get; set; }
    public IList<Order> Orders { get; set; }

In this domain, we'll do a few somewhat arbitrary validations:

  • A customer's surname can't be null.
  • A customer's surname can't be 'Foo'.
  • An address postcode can't be null
  • Address lines are optional, but content in the second line is not allowed if there is no content in the first line.
  • An order's product name can't be null
  • If the order's product name is valid, check that its cost is positive.

Let's do this top-down; starting with the customer:

var customer = new Customer { ... }
var result =
    from surname in NonNull(customer.Surname, "Surname can't be null")
    join surname2 in NotEqual(customer.Surname, "foo", "Surname can't be foo") on 1 equals 1
    join address in ValidateAddress(customer.Address) on 1 equals 1
    join orders in ValidateOrders(customer.Orders) on 1 equals 1
    select customer;

surname, surname2, etc, are all dummies here, we never really use them. Now we'll describe each of these functions, starting with NonNull() which is almost trivial:

static FSharpChoice<T, Errors> NonNull<T>(T value, string err) where T: class {
    if (value == null)
        return FSharpChoice.Error<T>(err);
    return FSharpChoice.Ok(value);

where Errors is just a type alias for FSharpList<string>, and FSharpChoice.Error() and FSharpChoice.Ok() are just less-verbose FSharpChoice constructors for this purpose.

NotEqual() is similar and, just as one would expect, validates that a value is not equal to some other value and also returns a FSharpChoice<T, Errors>

Now let's see how ValidateAddress() looks like:

static FSharpChoice<Address, Errors> ValidateAddress(Address a) {
    return from x in NonNull(a.Postcode, "Post code can't be null")
           join y in ValidateAddressLines(a) on 1 equals 1
           select a;

Ok, but what's in ValidateAddressLines() ?

static FSharpChoice<Address, Errors> ValidateAddressLines(Address a) {
    if (a.Line1 != null || a.Line2 == null)
        return FSharpChoice.Ok(a);
    return FSharpChoice.Error<Address>("Line1 is empty but Line2 is not");

Note how both NonNull() and ValidateAddressLines() use the form "if condition then ok else error". It's a pretty common pattern so let's abstract that to a higher-order function:

static Func<T, FSharpChoice<T, Errors>> Validator<T>(Predicate<T> pred, string err) {
    return x => {
        if (pred(x))
            return FSharpChoice.Ok(x);
        return FSharpChoice.Error<T>(err);

and now we can write:

static readonly Func<Address, FSharpChoice<Address, Errors>> ValidateAddressLines =
    Validator<Address>(x => x.Line1 != null || x.Line2 == null, 
                       "Line1 is empty but Line2 is not");

Only ValidateOrders() is left to explain. Let's see first how to validate a single order, and then we'll figure out how to make that operate on a list of orders:

static FSharpChoice<Order, Errors> ValidateOrder(Order o) {
        from name in NonNull(o.ProductName, "Product name can't be null")
        from cost in GreaterThan(o.Cost, 0, string.Format("Cost for product '{0}' can't be negative", name))
        select o;

Here we used monadic validation (from...from...) which, as explained before, causes the second validation to run only if the first was successful.

Now the tricky part: making this work on IEnumerable<Order>. I'll use explicit types here so you can better see what's going on in each step, and inline comments:

static FSharpChoice<IEnumerable<Order>, Errors> ValidateOrders(IEnumerable<Order> orders) {
    // first we apply the validator to all orders
    IEnumerable<FSharpChoice<Order,Errors>> validatedOrders = orders.Select(ValidateOrder);

    // now we fold (Aggregate) over the list of validated orders...
    // collect all orders (or their concatenated errors) in a single FSharpChoice

    // we need first an empty list of orders in the domain of validation:
    FSharpChoice<FSharpList<Order>, Errors> zero = ListModule.Empty<Order>().PureValidate();

    // now the actual fold
    return validatedOrders
            (FSharpChoice<FSharpList<Order>, Errors> e, FSharpChoice<Order, Errors> c) => 
                from a in e
                join b in c on 1 equals 1
                select a.Cons(b))

        // finally we need to upcast the list to match the return type
        .Select(x => (IEnumerable<Order>)x);

Just as before with the conditional validator, we can (and should) abstract this to a higher-order function, and then we can express ValidateOrders() as:

static Func<IEnumerable<Order>, FSharpChoice<IEnumerable<Order>, Errors>> ValidateOrders =


Validating with applicative functors is nothing new. Haskell and Scala developers have been doing it for quite some time now.

This approach to validation is attractive because it's very simple. It all revolves around the FSharpChoice type which is one of the basic building blocks in functional programming, and a very simple type: it's either this or that, either the validated value or the list of errors.

There are no ad-hoc concepts about validation here: validators are functions returning FSharpChoice<T, Errors> . We use higher-order functions to abstract validators. Folding and mapping, to manipulate validations. Validators are composed through an applicative functor or monad. These are all very general concepts. Accidental difficulty (sometimes also called "accidental complexity") is low. The core that enables applicative functor validation can be expressed in around 30 lines of F# code. In contrast, commonly used validation libraries in .NET have lots of types representing non-generic, ad-hoc concepts, abstractions, with ad-hoc interactions. Therefore, concept count and accidental complexity are high. 

Of course, applicative functors are best expressed and understood in functional languages, but C# 3 seems to be a good enough vehicle for them. They can be expressed even in Java (the Functional Java library implements the validation applicative functor) although it's quite verbose. If you want to learn more about applicative functors, see:

If you found this post interesting, stay tuned: Steffen, Ryan and I are planning to create a general FP library for C# and F#, where this code would be part of said library, among many other things. In the meantime, you can find the code here.

Thursday, August 18, 2011

Refactoring to monadic C# - applicative functors

In the last part of this series, we "upgraded" the Maybe monad to an Either monad in order to add error messages when parsing two integers. To recap, the code using the Either monad via LINQ looked like this:

var somethingOrError =
    from userID in FSharpOption.TryParseInt(req_userID).ToFSharpChoice("Invalid User ID")
    from id in FSharpOption.TryParseInt(req_otherID).ToFSharpChoice("Invalid ID")
    select doSomething(userID, id);

somethingOrError.Match(Console.WriteLine, setError);

That code, however, only dealt with setting one error message for the whole computation. When it got the first error, it aborted the rest of the computation.

Sometimes we do want this shortcut evaluation, but more generally, when validating data, we want to validate the whole thing, and then display a list of errors, not abort at the first error we find.

To do this, we need to step down from monads into a weaker abstraction: applicative functors. I've already blogged about applicative functors in F# and C#, mostly in the context of formlets. In a nutshell:

  • Applicative functors are defined by two functions: pure and apply.
  • These functions must satisfy some laws, which I won't mention here.
  • Pure is basically a Func<T, M<T>> for some M, i.e. it puts some value into M
  • Apply is basically a Func<M<Func<A,B>>,M<A>,M<B>>, i.e. it applies a function within M
  • All monads are applicative functors (but not necessarily the other way around).
  • All applicative functors are functors (but not necessarily the other way around), which means you can Select() over them with LINQ.

But I don't want to get into much detail here. What's interesting here is that we won't use the "default" implementation of the Either applicative functor (i.e. the one derived from the monad) because it would only keep the first error message, so it would be pretty useless in this case! The Either applicative functor we'll use will of course collect all validation messages in a list.

But first, for comparison, here's some simple imperative code to parse two numbers and then do something with them or display all errors:

var errors = new List<string>();

int userID;
var userID_ok = int.TryParse(req_userID, out userID);
if (!userID_ok)
    errors.Add("Invalid user ID");

int id;
var id_ok = int.TryParse(req_otherID, out id);
if (!id_ok)
    errors.Add("Invalid ID");

if (errors.Count > 0)
    Console.WriteLine(doSomething(userID, id));

Just like with CsFormlets, this Either applicative functor can be expressed either "directly", or using LINQ as explained by Tomas Petricek here. In this post I'll skip the direct syntax and use LINQ instead. And using LINQ, the above imperative code is translated into this:

var userID = FSharpOption.TryParseInt(req_userID)
    .ToFSharpChoice(FSharpList.New("Invalid User ID"));
var id = FSharpOption.TryParseInt(req_otherID)
    .ToFSharpChoice(FSharpList.New("Invalid ID"));

var result =
    from a in userID
    join b in id on 1 equals 1
    select doSomething(a, b);

result.Match(Console.WriteLine, setErrors);

Note that it's very similar to the monadic code we were first using, only instead of from...from... (which compiles to SelectMany(), i.e. monadic bind) we do from...join... .
"on 1 equals 1" is the syntactic tax we have to pay for bending LINQ like this.

result is of type FSharpChoice<int, FSharpList<string>> , that is it contains either the result of doSomething() or a list of errors.

This was a trivial example of using applicative functors for validation; just an introduction. It doesn't show one of its biggest strengths: composability. In the next post we'll look into validating a more complex domain.

Code is here.