Email validation with FParsec

Email address validation is one of those topics that keep coming up again and again. It seems that we developers never get it quite right, but with good reason: the spec is downright insane. There are six RFCs involved, which obsolete some other RFCs, and in turn have some erratas. I hope the guys at the IETF had some very good reasons to make this so damn complex!

Anyway, you can validate all the RFCs you want and you can still get an invalid address. joe@example.com is syntactically correct yet there isn't any Joe that works at example.com :-)

What's interesting here is the different approaches taken to cope with such a monster:

• Validate lightly, trying not to incur into false negatives, then check it by actually sending a confirmation email to that address. This is IMO the only sensible way to do it.
• Perl's email validation includes (of course) a 6700 character regex and optionally does DNS validation of the domain.
• Phil Haack's validation (C#) uses just a regex, which can't technically handle all possible cases because the RFCs allow arbitrarily nested comments.
• Other single-regex implementations
• Dominic Sayers' PHP implementation includes what is probably the most thorough analysis and testcase for the various RFCs. The testcase has 228 tests, and (obviously) his implementation passes all the tests. Also includes optional DNS check.
• In comparison, System.Net.Mail.MailAddress does some quite poor validation, it passes 40% of Dominic's tests.
• George Pollard's Haskell validation uses Parsec which results in an interesting pseudotranslation of the ENBF to executable code.

The last one really caught my interest so I ported it (mostly as an exercise) to F# + FParsec. Then I grabbed Dominic's testcase and ran it with FsUnit. The result? It passes 84% of Dominic's tests (with no false negatives). Here's the code (updated to F# 2.0 / FParsec trunk 5/17/2010):

module EmailValidation.EmailValidator

open System
open FParsec
open FParsec.Primitives
open FParsec.CharParsers

let isValidEmail email =
    let wsp = anyOf " \t" >>% ()
    let crlf = pchar '\n' >>% ()
    let nullChar = pchar (char 0) >>% ()
    let ranges = Seq.map (Seq.map char) >> Seq.concat >> Seq.toArray >> (fun x -> String x) >> (fun x -> anyOf x >>% ())
    let vchar = ranges [{0x21..0x7e}]
    let obsNoWsCtl = ranges [{1..8};{11..12};{14..31};{127..127}]
    let atomText = digit <|> letter <|> anyOf "!#$%&'*+-/=?^_`{|}~"
    let atom = many1 atomText >>% ()
    let fws = (many1 wsp >>. optional (crlf >>. many1 wsp)) <|> (many1 (crlf >>. many1 wsp) >>% ())
    let commentText = ranges [{33..39};{42..91};{93..126}] <|> obsNoWsCtl
    let quotedPair = pchar '\\' >>. (vchar <|> wsp <|> crlf <|> obsNoWsCtl <|> nullChar)
    let rec commentContent x = (commentText <|> quotedPair <|> comment) x
    and comment = between (pchar '(') (pchar ')') (many (commentContent <|> fws)) >>% ()
    let cfws = many (comment <|> fws)
    let quotedText = ranges [{33..33};{35..91};{93..126}] <|> obsNoWsCtl
    let quotedContent = quotedText <|> quotedPair
    let quotedString = between (pchar '"') (pchar '"') (many (optional fws >>. quotedContent) >>. optional fws)
    let dottedAtoms = sepBy1 (optional cfws >>. (atom <|> quotedString) >>. optional cfws) (pchar '.') >>% ()
    let localPart = dottedAtoms
    let domainText = ranges [{33..90};{94..126}] <|> obsNoWsCtl
    let domainLiteral =  between (optional cfws >>. pchar '[') (pchar ']' >>. optional cfws) (many (optional fws >>. domainText) >>. optional fws)
    let domain = dottedAtoms <|> domainLiteral 
    let addrSpec = localPart >>. pchar '@' >>. domain >>. eof
    match run addrSpec email with
    | Failure (msg, _, _) -> false
    | Success _ -> true

Just for reference, here's the actual test output:

192 passed.
36 failed.
0 erred.
----
Failed: ID "21": "123456789012345678901234567890123456789012345678901234567890@1
2345678901234567890123456789012345678901234567890123456789.123456789012345678901
23456789012345678901234567890123456789.12345678901234567890123456789012345678901
234567890123456789.1234.example.com"
Expected: false
Actual: true
----
Failed: ID "23": "12345678901234567890123456789012345678901234567890123456789012
345@example.com"
Expected: false
Actual: true
----
Failed: ID "31": """@example.com"
Expected: false
Actual: true
----
Failed: ID "34": "x@x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.
x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.
x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.x23456789.
x23456789.x23456789.x23456789.x23456"
Expected: false
Actual: true
----
Failed: ID "35": "first.last@[.12.34.56.78]"
Expected: false
Actual: true
----
Failed: ID "36": "first.last@[12.34.56.789]"
Expected: false
Actual: true
----
Failed: ID "37": "first.last@[::12.34.56.78]"
Expected: false
Actual: true
----
Failed: ID "38": "first.last@[IPv5:::12.34.56.78]"
Expected: false
Actual: true
----
Failed: ID "39": "first.last@[IPv6:1111:2222:3333::4444:5555:12.34.56.78]"
Expected: false
Actual: true
----
Failed: ID "40": "first.last@[IPv6:1111:2222:3333:4444:5555:12.34.56.78]"
Expected: false
Actual: true
----
Failed: ID "41": "first.last@[IPv6:1111:2222:3333:4444:5555:6666:7777:12.34.56.7
8]"
Expected: false
Actual: true
----
Failed: ID "42": "first.last@[IPv6:1111:2222:3333:4444:5555:6666:7777]"
Expected: false
Actual: true
----
Failed: ID "43": "first.last@[IPv6:1111:2222:3333:4444:5555:6666:7777:8888:9999]
"
Expected: false
Actual: true
----
Failed: ID "44": "first.last@[IPv6:1111:2222::3333::4444:5555:6666]"
Expected: false
Actual: true
----
Failed: ID "45": "first.last@[IPv6:1111:2222:3333::4444:5555:6666:7777]"
Expected: false
Actual: true
----
Failed: ID "46": "first.last@[IPv6:1111:2222:333x::4444:5555]"
Expected: false
Actual: true
----
Failed: ID "47": "first.last@[IPv6:1111:2222:33333::4444:5555]"
Expected: false
Actual: true
----
Failed: ID "48": "first.last@example.123"
Expected: false
Actual: true
----
Failed: ID "49": "first.last@com"
Expected: false
Actual: true
----
Failed: ID "50": "first.last@-xample.com"
Expected: false
Actual: true
----
Failed: ID "51": "first.last@exampl-.com"
Expected: false
Actual: true
----
Failed: ID "52": "first.last@x23456789012345678901234567890123456789012345678901
2345678901234.example.com"
Expected: false
Actual: true
----
Failed: ID "97": "test@123.123.123.123"
Expected: false
Actual: true
----
Failed: ID "115": "test@12345678901234567890123456789012345678901234567890123456
78901234567890123456789012345678901234567890123456789012345678901234567890123456
78901234567890123456789012345678901234567890123456789012345678901234567890123456
789012345678901234567890123456789012.com"
Expected: false
Actual: true
----
Failed: ID "116": "test@example"
Expected: false
Actual: true
----
Failed: ID "153": "first."".last@example.com"
Expected: false
Actual: true
----
Failed: ID "158": "first.last@[IPv6:1111:2222:3333:4444:5555:6666:12.34.567.89]"

Expected: false
Actual: true
----
Failed: ID "159": ""test\
 blah"@example.com"
Expected: false
Actual: true
----
Failed: ID "190": "a@b"
Expected: false
Actual: true
----
Failed: ID "199": "aaa@[123.123.123.333]"
Expected: false
Actual: true
----
Failed: ID "201": "a@bar"
Expected: false
Actual: true
----
Failed: ID "205": "a@-b.com"
Expected: false
Actual: true
----
Failed: ID "206": "a@b-.com"
Expected: false
Actual: true
----
Failed: ID "213": "invalid@special.museum-"
Expected: false
Actual: true
----
Failed: ID "216": "foobar@192.168.0.1"
Expected: false
Actual: true
----
Failed: ID "227": ""null \0"@char.com"
Expected: false
Actual: true

Using Windsor in F#

I've been doing some experiments with F# which involves using Windsor (my default choice for a IoC container) and found that Windsor's fluent interface is... not so fluent when used in F#.

UPDATE 10/21/2010: The F# team has loosened the syntax a lot, F# 2.0 can now consume the Windsor fluent API without any changes.

Fluent interface as-is in F#

Example: given this code in C#

container.Register(Component.For<IMyServiceContract>().ImplementedBy<MyServiceImpl>());

Let's try to translate this to F#. (I'll do it step by step so it's more didactic)

Code: container.Register(Component.For<IMyServiceContract>().ImplementedBy<MyServiceImpl>())
Compiler says: Error: Successive arguments should be separated by spaces or tupled, and arguments involving function or method applications should be parenthesized.
Explanation: Component.For<IMyServiceContract>() is a method application, so it has to be parenthesized:

Code: container.Register((Component.For<IMyServiceContract>()).ImplementedBy<MyServiceImpl>())
Compiler says: Error: Type constraint mismatch. The type ComponentRegistration<IMyServiceContract> is not compatible with type IRegistration array.
The type 'ComponentRegistration<IMyServiceContract>' is not compatible with the type 'IRegistration array'.
Explanation: The reason for these errors is that F# expects an array as the parameter for Register, since it's signature is

IWindsorContainer Register(params IRegistration[] registrations)

and F# doesn't support params arrays, so we have to explicitly construct an array:

Code: container.Register([|(Component.For<IMyServiceContract>()).ImplementedBy<MyServiceImpl>()|])
Compiler says: Error: This expression has type ComponentRegistration<IMyServiceContract> but is here used with type IRegistration.
Explanation: What? But ComponentRegistration<IMyServiceContract> implements the IRegistration interface! Yeah, but Register() takes an array of IRegistration, not an array of ComponentRegistration<IMyServiceContract>, and F# doesn't implement array covariance. You can write this in C#:

IMyServiceContract[] arr = new MyServiceImpl[] { new MyServiceImpl() };

but you can't write this in F#:

let arr: IMyServiceContract[] = [| MyServiceImpl() |]

This is actually a Good Thing since this kind of covariance has some nasty consequences. So we have no option but to cast:

Code: container.Register [| (((Component.For<IMyServiceContract>()).ImplementedBy<MyServiceImpl>()) :> IRegistration) |]
Compiler says: Warning: This expression should have type 'unit', but has type 'IWindsorContainer'.
Explanation: We have to do something about the return value. We're not using it in this example, so we'll just discard it:

Code: let _ = container.Register [| (((Component.For<IMyServiceContract>()).ImplementedBy<MyServiceImpl>()) :> IRegistration) |]

This finally compiles, but it's way too noisy. Not fluent at all!

We could have also written:

let _ = container.Register(Seq.to_array (Seq.cast [(Component.For<IMyServiceContract>()).ImplementedBy<MyServiceImpl>()]))

but it's just as ugly.

Extension method solution

We could hide the casting and array stuff in an extension method:

module WindsorContainerExtensions = 
    type IWindsorContainer with
        member x.RegisterComponents (r: seq<#IRegistration>) = 
            let _ = x.Register (Seq.to_array (Seq.cast r))
            () 

which allows us to write:

container.RegisterComponents [(Component.For<IMyServiceContract>()).ImplementedBy<MyServiceImpl>()]

Much better, but it doesn't quite fit the F# spirit.

Function pipelines solution

A better-yet solution is to use function pipelining to build a DSL, like FsUnit and FsTest do, so we can write:

typeof<IMyServiceContract>
    |> implementedBy (typeof<MyServiceImpl>)
    |> registerIn container

Here are the functions that support this:

let implementedBy impl (service: Type) =
    (Component.For service).ImplementedBy impl       
    
let registerIn (container: IWindsorContainer) registration = 
    container.RegisterComponents [ registration ]

It's easy to follow this pattern and implement similar functions to cover more functionality. For example, this sets the lifestyle for a registration:

let withLifestyle lifestyle (registration: ComponentRegistration<_>) = 
    (registration.LifeStyle).Is lifestyle

Usage:

typeof<IMyServiceContract>
    |> implementedBy (typeof<MyServiceImpl>)
    |> withLifestyle LifestyleType.Transient
    |> registerIn container

Conclusion

Fluent interfaces are cool but quite language-dependent, so if you're designing an API targeting the CLR and thinking of building a fluent interface, make sure you also provide an alternative, simpler, non-fluent API so other languages can build their own flavor of fluent interface (Windsor does provide this, of course)

Block components in ASP.NET MVC (part 2)

In my last post I explored the possibility of block components in ASP.NET MVC. Actually, these posts should be titled "Block components in ASP.NET MVC / WebFormViewEngine". But WebFormViewEngine is the default view engine (a strong default) so whenever I say ASP.NET MVC it's WebFormViewEngine. Of course we could replace it and implement block components any way we want, but today we'll stick to defaults as much as we can, trying to keep it simple.

So far, I have only considered a single block. Let's now see what happens when trying to introduce several named blocks. We want to map the name of the block (a string) to the block itself (an Action). So... IDictionary<string, Action>:

box.ascx

<%@ Control Language="C#" Inherits="System.Web.Mvc.ViewUserControl<IDictionary<string, Action>>" %>
<div class="box">
    <div class="top"></div>
    <div class="title"><% Model["title"](); %></div>
    <div class="content">
        <% Model["content"](); %>
    </div>
    <div class="bottom"></div>
</div>

OK, that wasn't so bad... now let's see how we define the sections.

First attempt

It's a dictionary, so we could just give it a dictionary:

<% Html.RenderPartial("box", new Dictionary<string, Action> {
   {"title", () => {%>
<h1>the title</h1>
<%}}, {"content", () => { %>
<div>some content</div>
<%}}}); %>

This doesn't require any extra code but it's very ugly... we could try defining a small fluent interface for this:

Second attempt

<% Html.Partial("box")
.WithSection("title", () => {%>
    <h1>the title</h1>
<% })
.WithSection("content", () => {%>
    <div>some content</div>
<% })
.Render(); %>

Implementation:

public static class HtmlHelperExtensions {
    public static PartialWithSectionsRendering Partial(this HtmlHelper html, string viewName) {
        return new PartialWithSectionsRendering(html, viewName);
    }

    public class PartialWithSectionsRendering {
        private readonly HtmlHelper html;
        private readonly string viewName;
        private readonly IDictionary<string, Action> sections = new Dictionary<string, Action>();

        public PartialWithSectionsRendering(HtmlHelper html, string viewName) {
            this.html = html;
            this.viewName = viewName;
        }

        public PartialWithSectionsRendering WithSection(string section, Action render) {
            sections[section] = render;
            return this;
        }

        public void Render() {
            html.RenderPartial(viewName, sections);
        }
    }
}

Still too awkward... plus we could easily forget to call the final Render().

Third attempt

<% Html.RenderPartialWithSections("box", section => {
    section("title", () => { %>
        <h1>the title</h1>
    <% });
    section("content", () => { %>
        <div>some content</div>
    <% });
}); %>

This certainly looks better... the intention is expressed quite clearly, and the implementation is very simple:

public static class HtmlHelperExtensions {
    public static void RenderPartialWithSections(this HtmlHelper html, string viewName, Action<Action<string, Action>> sections) {
        var dict = new Dictionary<string, Action>();
        sections.Invoke((section, render) => dict[section] = render);
        html.RenderPartial(viewName, dict);
    }
}

Let's try this one more time:

Fourth attempt

<% Html.RenderPartialWithSections2("box", 
title => {%>
    <h1>the title</h1>
<% },
content => {%>
    <div>some content</div>
<% }); %>

This is shorter that the last attempt, but what's really that title variable? What would happen if we use it in the Action block? Let's see the implementation first:

public static class HtmlHelperExtensions {
    public static void RenderPartialWithSections2(this HtmlHelper html, string viewName, params Action<object>[] sections) {
        var dict = new Dictionary<string, Action>();
        foreach (var a in sections) {
            var name = a.Method.GetParameters()[0].Name;
            var render = a; // copied to avoid access to modified closure
            dict[name] = () => render.Invoke(null);
        }
        html.RenderPartial(viewName, dict);
    }
}

So if you use the title variable in your title Action block, Bad Things Will Happen. Namely, a NullReferenceException since it's just a dummy null object!

Conclusion

I'll stop there for now, but I have to confess that I don't really like any of these solutions :-) They work, but this problem can be expressed more elegantly in Brail. DSLs win here... unless I'm missing something!

Block components in ASP.NET MVC

A couple of days ago I was browsing stackoverflow for some questions to answer when I hit this one:

I have repetitious markup for elements, like:

<div class="box">
    <div class="top"></div>

    <div class="content">
      content goes here
    </div>
    <div class="bottom"></div>
</div>

Top and bottom are styled with CSS to include images for borders.

I could use JQuery to inject tags, but is there a better way to template controls in ASP.NET MVC?

MonoRail users might recognize this as a block component, that is, a ViewComponent (ViewUserControl in ASP.NET MVC / WebFormViewEngine terms) that accepts a block (or several blocks, also called sections) of HTML to be inserted somewhere within the ViewComponent.

The author of the question goes on to answer himself with two HtmlHelper extensions methods:

  public static void BeginBox(this HtmlHelper htmlHelper)
  {
      StringBuilder box = new StringBuilder();

      box.AppendLine("<div class=\"box\">");
      box.AppendLine("    <div class=\"top\"></div>");
      box.AppendLine("        <div class=\"content\">");

      htmlHelper.ViewContext.HttpContext.Response.Write(box.ToString());
  }

  public static void EndBox(this HtmlHelper htmlHelper)
  {
      StringBuilder box = new StringBuilder();

      box.AppendLine("        </div>");
      box.AppendLine("    <div class=\"bottom\"></div>");
      box.AppendLine("</div>");

      htmlHelper.ViewContext.HttpContext.Response.Write(box.ToString());
  }

This works just fine, but I strongly dislike having markup in code. Plus, we could forget to call Html.EndBox() which would break everything. Can we somehow move this to a ViewUserControl? By using an Action as the Model of the ViewUserControl, we can:

box.ascx

<%@ Control Language="C#" Inherits="System.Web.Mvc.ViewUserControl<Action>" %>
<div class="box">
    <div class="top"></div>
    <div class="content">
        <% Model(); %>
    </div>
    <div class="bottom"></div>
</div>

Then in the view:

<% Html.RenderPartial("box", Lambda.Action(() => { %>
content goes here
<% })); %>

Note the Lambda.Action helper method, which is defined as:

public static class Lambda {
    public static Action Action(Action a) {
        return a;
    }
}

This keeps the Action as an Action, otherwise the runtime tries to convert it to a ViewDataDictionary with the obvious conversion error that causes.

Also note that you could write any markup, call any helper, or reference any ViewData or Model property of the view, instead of the simple text "content goes here".

The thing gets more interesting as we add support for multiple sections, which I'll blog about next. (see second part)

Windsor facility for Quartz.NET

A few months ago I wrote a couple of adapters to integrate Quartz.Net into Windsor. These were loose components that you had to register yourself and the configuration wasn't very friendly. So I decided to wrap them in a facility to make things cleaner and easier.

This is what the facility provides:

• Some Windsor-managed Quartz entities:
  • IJob / IStatefulJob
  • ISchedulerListener
  • IJobListener (both global and per-job)
  • ITriggerListener (both global and per-trigger)
  • IScheduler
  • IJobFactory (although you probably won't use this...)
• Automatic FileScanJob registering
• IJobScheduler: a simpler IScheduler
• Automatic scheduler starting if the Startable facility is present (otherwise you'll have to call IScheduler.Start() manually)

Here's what it doesn't support:

• These entities are NOT Windsor-managed (they are instantiated normally by Quartz instead)
  • IThreadPool
  • IJobStore
  • ISchedulerPlugin
  • All other interfaces and/or quartz entities
  • Listeners declared in quartz_jobs.xml
• Trigger and job group names

Note that it's up to you to register jobs with the appropriate lifestyle. Listeners can only have singleton lifestyle since they're injected in IScheduler which is itself a singleton.

Here are the bits:

• Binaries
• Sample app (source code)