Customizing SolrNet

One of the most voted enhancement requests for SolrNet (an Apache Solr client for .NET) right now is to add support for POSTing when querying.

Let me explain: queries are serialized by SolrNet and sent to Solr via HTTP. Normally, queries are issued with a GET request and the query itself goes in the query string part of the URL. A simple query URL might look like this: http://localhost:8983/solr/select?q=id:123 .

The problem arises when the query is too long to fit in the query string. Even though the HTTP protocol does not place any a priori limit on the length of a URI, most (all?) servers do, for performance and security reasons.

Here's a little program that reproduces this issue:

internal class Program {
    private const string serverURL = "http://localhost:8983/solr";

    private static void Main(string[] args) {
        Startup.Init<Dictionary<string, object>>(serverURL);
        var solr = Startup.Container.GetInstance<ISolrOperations<Dictionary<string, object>>>();
        solr.Query(Query.Field("id").In(Enumerable.Range(0, 1000).Select(x => x.ToString()).ToArray()));
    }
}

This creates the query "id:0 OR id:1 OR ... OR id:999", it's about 10KB after encoding, more than enough for our tests. Running this against Solr on Jetty 6 makes Jetty throw:

2010-12-13 17:52:33.362::WARN:  handle failed 
java.io.IOException: FULL 
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:274) 
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202) 
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378) 
        at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226) 
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)

Not very graceful... it should probably respond with 414 Request-URI Too Long instead of throwing like this, but clients shouldn't send such long URIs anyway.

Steven Livingston has a good blog post describing a patch modifying some classes in SolrNet to deal with this issue. However, even though I never foresaw this problem when writing SolrNet, solving it does not really require any changes to the existing codebase.

In this particular case, what we need to do concretely is override the Get() method of the ISolrConnection service and make it issue POST requests instead of GET. We can write a decorator to achieve this:

public class PostSolrConnection : ISolrConnection {
    private readonly ISolrConnection conn;
    private readonly string serverUrl;

    public PostSolrConnection(ISolrConnection conn, string serverUrl) {
        this.conn = conn;
        this.serverUrl = serverUrl;
    }

    public string Post(string relativeUrl, string s) {
        return conn.Post(relativeUrl, s);
    }

    public string Get(string relativeUrl, IEnumerable<KeyValuePair<string, string>> parameters) {
        var u = new UriBuilder(serverUrl);
        u.Path += relativeUrl;
        var request = (HttpWebRequest) WebRequest.Create(u.Uri);
        request.Method = "POST";
        request.ContentType = "application/x-www-form-urlencoded";
        var qs = string.Join("&", parameters
            .Select(kv => string.Format("{0}={1}", HttpUtility.UrlEncode(kv.Key), HttpUtility.UrlEncode(kv.Value)))
            .ToArray());
        request.ContentLength = Encoding.UTF8.GetByteCount(qs);
        request.ProtocolVersion = HttpVersion.Version11;
        request.KeepAlive = true;
        try {
            using (var postParams = request.GetRequestStream())
            using (var sw = new StreamWriter(postParams))
                sw.Write(qs);
            using (var response = request.GetResponse())
            using (var responseStream = response.GetResponseStream())
            using (var sr = new StreamReader(responseStream, Encoding.UTF8, true))
                return sr.ReadToEnd();
        } catch (WebException e) {
            throw new SolrConnectionException(e);
        }
    }
}

Now we have to apply this decorator:

private static void Main(string[] args) {
    Startup.Init<Dictionary<string, object>>(new PostSolrConnection(new SolrConnection(serverURL), serverURL));
    var solr = Startup.Container.GetInstance<ISolrOperations<Dictionary<string, object>>>();
    solr.Query(Query.Field("id").In(Enumerable.Range(0, 1000).Select(x => x.ToString()).ToArray()));
}

That's it! If you're using Windsor, applying the decorator looks like this:

private static void Main(string[] args) {
    var container = new WindsorContainer();
    container.Register(Component.For<ISolrConnection>()
        .ImplementedBy<PostSolrConnection>()
        .Parameters(Parameter.ForKey("serverUrl").Eq(serverURL)));
    container.AddFacility("solr", new SolrNetFacility(serverURL));
    var solr = container.Resolve<ISolrOperations<Dictionary<string, object>>>();
    solr.Query(Query.Field("id").In(Enumerable.Range(0, 1000).Select(x => x.ToString()).ToArray()));
}

This is the real benefit of writing decoupled code. Not testability, but flexibility. Testability is nice of course, but not the primary purpose.
When your code is decoupled, you can even implement entire features mostly by rearranging the object graph. This is pretty much how I implemented multicore support in SolrNet.

The PostSolrConnection implementation above works with SolrNet 0.3.0 and probably also 0.2.3. PostSolrConnection is not the default because: a) it needs to be tested thoroughly, and b) Solr doesn't emit cache headers when POSTing so it precludes caching.

Windsor-managed MembershipProviders

I see many questions on Stackoverflow that are basically variants of this: "How can I integrate my custom MembershipProvider into my IoC container?"

Integrating your custom MembershipProvider to a IoC container has many advantages, since it would let you treat it just like any other service: you could manage its lifetime, dependencies, configuration, even proxy it if you wanted.

Problem is, MembershipProviders are one of those things that are managed by the ASP.NET runtime. You just configure it in your web.config and the runtime instantiates it when it's needed. You don't really get much control over its creation.

A cheap solution is to use the container as a service locator directly in your membership provider (using something like CommonServiceLocator), e.g.:

public class MyMembershipProvider : MembershipProvider {
    private IUserRepository repo {
        get { return ServiceLocator.Current.GetInstance<IUserRepository>(); }
    }

    public override string GetUserNameByEmail(string email) {
        return repo.First(u => u.Email == email);
    }

    ...
}

Using a service locator like this should be avoided as much as possible. Mark Seeman explains it thoroughly in this article. In a nutshell, you want to limit the usage of the service locator pattern to glue code (i.e. very low-level infrastracture), even there use it as little as possible, and never use it in application-level code.

As usual with this kind of problems, the solution is to write a wrapper/adapter/bridge/whatever-you-want-to-call-it that isolates the issue so that client code doesn't have to suffer it. It's similar in concept to the implementation of Windsor-managed HttpModules. It's actually simpler than that, we don't need a custom lifestyle manager here.

In fact, Spring.NET has had such an adapter for quite some time. The only problem with that implementation is that you can't change the lifetime of the custom provider, it's always a singleton. My implementation doesn't have this limitation: your provider can be transient, singleton, per web request, whatever. The price for this is that you can't use Initialize() (more precisely, it won't do anything), but since it's managed by the container, you can use the container to provide any configuration, which is much more flexible. The implementation is about 200 lines of boring, simple code so I'm not going to post it here. It does use Windsor as a service locator, but this is low-level infrastracture/glue code. The goal here is to keep your code clean.

The code is here, and here's how to use it:

1. Write your custom MembershipProvider as a regular component, using constructor or property injection as you see fit.
   
2. Implement IContainerAccessor in your global HttpApplication class. Use this article as reference.
   
3. Register your custom provider in Windsor and assign a name to the component. E.g.:
   
   

   container.Register(Component.For<MyMembershipProvider>()
       .LifeStyle.Transient
       .Named("myProvider"));

4. Register your custom provider in your web.config using the adapter and referencing the name of the corresponding Windsor component in a "providerId" attribute. E.g.:
   
   

       <membership defaultProvider="customProvider">
         <providers>
           <clear/>
           <add name="customProvider" type="ProviderInjection.WebWindsorMembershipProvider, ProviderInjection" providerId="myProvider"/>
         </providers>
       </membership>

That's it. Here's a sample app that you can use as reference. This can be easily ported to any IoC container, and for any provider like RoleProvider, ProfileProvider, etc. I haven't used this in anger so let me know if you have any problems with it.

Hybrid lifestyles in Windsor

All Inversion of Control (IoC) containers offer some way to control the lifecycle or scope of the component instances. Singleton (one instance per container) and Transient (a new instance each time your request it from the container) are the two most common scopes.

Castle Windsor calls these scopes "lifestyles" (the term "lifecycle" is used for something else) and it comes with these built-in lifestyles (I'll just quote the documentation):

• Singleton (default): one instance per container
• Transient: a new instance each time your request it from the container
• PerThread: one instance per thread
• PerWebRequest: one instance per HttpContext (HTTP request)
• Pooled: instances will be pooled to avoid unnecessary constructions

More importantly, a custom lifestyle manager can be plugged-in, just by implementing the ILifestyleManager interface. Some examples of this are the WCF facility PerWcfSession and PerWcfOperation lifestyles and the PerHttpApplication lifestyle I implemented to inject dependencies to HttpModules.

There are other lifestyles that come in handy sometimes, especially in web applications. I created the Castle.Windsor.Lifestyles contrib project to host them. I have implemented these lifestyles so far:

• Abstract hybrid lifestyle
• Abstract hybrid PerWebRequest + X
• Hybrid PerWebRequest + Transient
• PerWebSession: one instance per HTTP session
• PerHttpApplication (mentioned above)

An hybrid lifestyle is one that actually blends two underlying lifestyles: a main lifestyle and a secondary lifestyle. The hybrid lifestyle first tries to use the main lifestyle; if it's unavailable for some reason, it uses the secondary lifestyle. This is commonly used with PerWebRequest as the main lifestyle: if the HTTP context is available, it's used as the scope for the component instance; otherwise the secondary lifestyle is used.

The PerWebSession has a couple of caveats:

• Components using this lifestyle have to be serializable if you're using a session-state mode other than InProc.
• Components using this lifestyle will not be properly released, since the session end event only fires when using the InProc session-state mode.

Usage:

1. Add a reference to Castle.Windsor.Lifestyles.dll
2. Use the appropriate lifestyle descriptor in your registration, e.g.:

   container.Register(Component.For<SomeService>()
                   .LifeStyle.HybridPerWebRequestTransient());

Low-level SolrNet

I recently got a question about how to handle multi-faceting in SolrNet, a nice feature of Solr that can be very useful to the end-user. eBay uses a kind of multi-faceting interface.
If you know nothing about Solr or SolrNet, read on, this article isn't so much about Solr as API design.

The Solr wiki has an example query with multi-faceting:

q=mainquery&fq=status:public&fq={!tag=dt}doctype:pdf&facet=on&facet.field={!ex=dt}doctype

For those of you that are not into Solr, this is just a regular URL query string that is passed to the Solr endpoint. The final URL looks like this (modulo encoding):

http://localhost:9983/solr/select/?q=mainquery&fq=status:public&fq={!tag=dt}doctype:pdf&facet=on&facet.field={!ex=dt}doctype 

And this is how you represent this query in the SolrNet object model:

var solr = ServiceLocator.Current.GetInstance<ISolrOperations<Document>>(); 
ISolrQueryResults<Document> results = solr.Query("mainquery", new QueryOptions { 
    FilterQueries = new[] { 
        Query.Field("status").Is("public"), 
        new LocalParams {{"tag", "dt"}} + Query.Field("doctype").Is("pdf") 
    }, 
    Facet = new FacetParameters { 
        Queries = new[] { 
            new SolrFacetFieldQuery(new LocalParams {{"ex", "dt"}} + "doctype") 
        } 
    } 
});

We build object models like this one because they're programmable, objects and methods can be programmatically combined to build (or compose) our intention. Like Hibernate's Criteria API.

Opposed to this is the string, and most of the time we hate it because it's opaque, it doesn't have any syntactical meaning within our object-oriented code. It has no programmability, no composability. We use very generic classes to build strings, like StringBuilders or StringWriters, which don't convey any syntactical information about what we're actually doing. If we need to extract information from a string, we have to write a parser, which is not a trivial task. But the string also has its advantages: it's naturally serializable (or should I say already serialized), it can be more readable and more concise. And those are some of the reasons why Hibernate also provides the HQL API. You might be thinking that this dichotomy of objects and strings is really a matter of serialization and deserialization, but I'm talking about human-readable strings here, whereas a serialized format is frequently for machine consumption only.

So if we already know what the query string is, how can we simplify the chunk of code above? Thanks to IoC, we can easily tap into some of SolrNet's "internal" components without worrying about what dependencies they need:

Func<string, string, KeyValuePair<string, string>> kv = (k, v) => new KeyValuePair<string, string>(k, v); 
var connection = ServiceLocator.Current.GetInstance<ISolrConnection>(); 
var xml = connection.Get("/select", new[] { 
    kv("q", "mainquery"), 
    kv("fq", "status:public"), 
    kv("fq", "{!tag=dt}doctype:pdf"), 
    kv("facet", "on"), 
    kv("facet.field", "{!ex=dt}doctype"), 
}); 
var parser = ServiceLocator.Current.GetInstance<ISolrQueryResultParser<Document>>(); 
ISolrQueryResults<Document> results = parser.Parse(xml); 

ISolrConnection is just a wrapper over the HTTP request, we give it the querystring parameters and get Solr's XML response, then we feed the response to the parser component and voilà, we have our results.

And since it's just a regular HTTP request, we can go even lower:

using (var web = new WebClient()) { 
    var xml = web.DownloadString("http://localhost:9983/solr/select/?q=mainquery&fq=status%3Apublic&fq=%7B!tag%3Ddt%7Ddoctype%3Apdf&facet=on&facet.field=%7B!ex%3Ddt%7Ddoctype");
    var parser = ServiceLocator.Current.GetInstance<ISolrQueryResultParser<Document>>(); 
    ISolrQueryResults<Document> results = parser.Parse(xml); 
} 

I'll leave it to you to decide which one to use. Like the choice between HQL and Criteria, sometimes you might prefer one over the other depending on the context. Just keep in mind that these components' interfaces are not as stable as the "really public" documented interfaces, they might have breaking changes more often.

Windsor-managed HttpModules

Another interesting question from stackoverflow:

I have a custom HTTP Module. I would like to inject the logger using my IoC framework, so I can log errors in the module. However, of course I don't get a constructor, so can't inject it into that. What's the best way to go about this?

First thing we need to recognize is that there are actually two problems here:

1. Taking control of IHttpModule instantiation in order to inject it services, and
2. Managing the lifecycle of the IHttpModule

Let's start with the lifecycle. Ayende has the best summary of the lifecycle of HttpApplication and its related IHttpModules I've found. If you're not familiar with this, go read it now, I'll wait.

Back so soon? Ok, we can implement this in Windsor with a pluggable lifestyle. Lifestyle managers dictate when it's necessary to create a new instance of the component, but not how. They don't get to actually create the instance, that's the responsibility of the component activator. Here's the full component creation flow reference.

So we need to write a new lifestyle manager that allows at most one component instance per HttpApplication instance. I won't bother you with the implementation details since it's very similar to the per-request lifestyle: it's composed of the LifestyleManager itself and a HttpModule as a helper to store component instances.

Now we need a way to manage the IHttpModule instantiation. Unsurprisingly, we can do that with another IHttpModule, which I'll call WindsorHttpModule.
This WindsorHttpModule will be responsible for the initialization of the "user-level", Windsor-managed IHttpModules.

So, to summarize:

1. Register PerHttpApplicationLifestyleModule and WindsorHttpModule:
   

   <httpModules>
       <add name="PerHttpApplicationLifestyleModule" type="HttpModuleInjection.PerHttpApplicationLifestyleModule, HttpModuleInjection"/>
       <add name="WindsorModule" type="HttpModuleInjection.WindsorHttpModule, HttpModuleInjection"/>
   </httpModules>

2. Write your http modules using normal dependency injection style, e.g.:
   

   public class Service {
       public DateTime Now {
           get { return DateTime.Now; }
       }
   }
   
   public class UserHttpModule : IHttpModule {
       private readonly Service s;
   
       public UserHttpModule(Service s) {
           this.s = s;
       }
   
       public void Init(HttpApplication context) {
           context.BeginRequest += context_BeginRequest;
       }
   
       private void context_BeginRequest(object sender, EventArgs e) {
           var app = (HttpApplication) sender;
           app.Response.Write(s.Now);
       }
   
       public void Dispose() {}
   }

3. Make your HttpApplication implement IContainerAccessor
4. Register your http modules in Windsor, using the custom lifestyle:
   

   container.AddComponent<Service>();
   container.Register(Component.For<IHttpModule>()
                          .ImplementedBy<UserHttpModule>()
                          .LifeStyle.Custom<PerHttpApplicationLifestyleManager>());

   
   Note that UserHttpModule is not registered in the <httpModules> section of web.config, since it's managed by Windsor now.

You can also combine this with Rashid's BaseHttpModule to make your modules more testable.

Full source code is here.

Visualizing Windsor components dependencies

When working with Windsor on complex applications, it's pretty common to have hundreds or even thousands of components managed by the container. The dependencies can get quite intricate and sometimes you need to see the big picture. Static analyzers like NDepend don't cut it since components are wired by Windsor at runtime.

So here's a little guide to output a PNG of a container's components and dependencies, using QuickGraph and GLEE.

First we need to model the components and dependencies as a regular Dictionary. The key of this dictionary will be a description of the component (a regular string) and the value of this dictionary will be a list of dependencies.

var container = new WindsorContainer();

// ...add components...

var dependencyDict = container.Kernel.GraphNodes
    .Distinct(new EqComparer<GraphNode>((a, b) => a.Describe() == b.Describe(), n => n.Describe().GetHashCode()))
    .ToDictionary(n => n.Describe(), n => n.Dependents.Select(a => a.Describe()));

EqComparer is just a functional IEqualityComparer. Distinct() is needed because when using forwarded types you'd get duplicate dictionary keys.

Now we dump this into a QuickGraph digraph:

var graph = dependencyDict.ToVertexAndEdgeListGraph(kv => kv.Value.Select(n => new SEquatableEdge<string>(kv.Key, n)));

and finally we render the digraph to a PNG using GLEE:

graph.ToGleeGraph().ToBitmap().Save("windsor.png");

Here's a sample output from SolrNet's inner components (click to zoom):

 

 

Only problem with this is that it doesn't take custom subdependency resolvers (like an ArrayResolver) into account, since they can't be modeled in the kernel's GraphNodes.

Here's what it looks like after some ad-hoc fixes:

Here's the full code.

Related articles:

• Evgeniy Kucheruk outputs dependencies using Graphviz.
• Mike Hadlow outputs dependencies as text.
• UPDATE: just found some old articles by Hammett doing the same thing (couldn't find the source code, though)

 

Poor man's profiler with Windsor

I generally advise to keep the IoC container out of the tests, but sometimes it is a convenient tool to have. Take for example the auto-mocking container concept. It's just too useful so I make an exception to the previous rule sometimes.

Here's another use for a container in a test: a profiler. All IoC containers have some proxying abilities (or rather, they delegate to one of the proxying libraries), which can be leveraged to build a basic profiler. A profiler is especially useful in integration tests where there's something that's taking too much time but you can't put your finger on the culprit simply because there are too many components involved.

A real-world example: an application which uses SolrNet was taking too long when getting lots of documents from Solr. There was nothing weird in Solr's log so it was a client issue. Let's profile it!

Setting up the profiler

We create a ProfilingContainer and add all the components we want to profile:

IWindsorContainer container = new ProfilingContainer(); 
container.AddComponent<ISolrDocumentResponseParser<Document>, SolrDocumentResponseParser<Document>>(); 
container.AddComponent<ISolrResponseParser<Document>, ResultsResponseParser<Document>>("resultsParser"); 
container.AddComponent<IReadOnlyMappingManager, AttributesMappingManager>(); 
container.Register(Component.For<ISolrQueryResultParser<Document>>().ImplementedBy<SolrQueryResultParser<Document>>() 
    .ServiceOverrides(ServiceOverride.ForKey("parsers").Eq(new[] { "resultsParser" }))); 
container.AddComponent<ISolrFieldParser, DefaultFieldParser>(); 
container.AddComponent<ISolrDocumentPropertyVisitor, DefaultDocumentVisitor>();

Adding code to profile

We add the code that exercises the components:

var parser = container.Resolve<ISolrQueryResultParser<Document>>(); 

for (int i = 0; i < 1000; i++) { 
    parser.Parse(responseXMLWithArrays); 
} 

Getting the actual profile data

Finally, we get the profile data:

Node<KeyValuePair<MethodInfo, TimeSpan>> rawProfile = container.GetProfile();

This returns a tree of method calls, each with its logged duration. We just want a list of aggregated durations, so first we flatten the tree:

IEnumerable<KeyValuePair<MethodInfo, TimeSpan>> profile = Flatten(rawProfile);

And now we can query and write the profile:

var q = from n in profile 
        group n.Value by n.Key into x 
        let kv = new { method = x.Key, count = x.Count(), total = x.Sum(t => t.TotalMilliseconds)} 
        orderby kv.total descending 
        select kv; 

foreach (var i in q) 
    Console.WriteLine("{0} {1}: {2} executions, {3}ms", i.method.DeclaringType, i.method, i.count, i.total); 

Which prints something like this (prettified):

Method	Executions	Total own time
AttributesMappingManager.GetFields(Type)	21000	2803.23ms
DefaultDocumentVisitor.Visit(Object, String, XmlNode)	10000	432.19ms
DefaultFieldParser.Parse(XmlNode, Type)	10000	122.77ms
SolrQueryResultParser<Document>.Parse(String)	1000	93.52ms
DefaultFieldParser.CanHandleType(Type)	10000	44.71ms
SolrDocumentResponseParser<Document>.ParseResults(XmlNode)	1000	43.11ms
ResultsResponseParser<Document>.Parse(XmlDocument, SolrQueryResults<Document>)	1000	30.28ms
DefaultFieldParser.CanHandleSolrType(String)	10000	23.78ms

 

Analyzing the profile

Well, it's obvious that AttributesMappingManager.GetFields() is to be blamed here, it's taking a lot of time... which is only logical since it uses a lot of reflection...
But wait a minute, I already thought about that when I wrote that MemoizingMappingManager (which is a memoizer decorator for the AttributesMappingManager)... only I forgot to add it to the container!

Fixing and re-running the profile

After adding the MemoizingMappingManager, the profile comes out like this:

Method	Executions	Total own time
DefaultDocumentVisitor.Visit(Object, String, XmlNode)	10000	358.64ms
SolrQueryResultParser<Document>.Parse(String)	1000	166.64ms
DefaultFieldParser.Parse(XmlNode, Type)	10000	87.13ms
MemoizingMappingManager.GetFields(Type)	21000	50.5ms
DefaultFieldParser.CanHandleType(Type)	10000	42.03ms
SolrDocumentResponseParser<Document>.ParseResults(XmlNode)	1000	34.93ms
ResultsResponseParser<Document>.Parse(XmlDocument, SolrQueryResults<Document>)	1000	30.83ms
DefaultFieldParser.CanHandleSolrType(String)	10000	22.22ms
AttributesMappingManager.GetFields(Type)	1	19.82ms

 

Conclusion

This cheap profiler has a number of caveats:

• Since it works by adding a DynamicProxy interceptor to everything, it can only profile interceptable classes and methods managed by Windsor.
• Times should be taken with a grain of salt. Based on a couple of unscientific tests, it can introduce an error of 10ms (on my machine), rendering a 10ms measured time useless. I'd only use these times for a qualitative comparison. There's a reason why there's a specific .Net profiling API.
• Multithreading: each thread will get its own interceptor, so each thread gets its own profile.
• What I just described is not really a test. There is nothing asserted, it can't pass or fail. The "test" here is just a means to run the profile. It should be [Ignore]d.

All in all, I found this to be quite useful. ProfilingContainer is just a thin wrapper around WindsorContainer that adds a ProfilerFacility, which in turn sets up a ProfilingInterceptor. So you could, for example, add the ProfilingInterceptor to your application container with a per-web-request lifestyle and code a HttpModule to profile an entire http request.

Code is here. Sample profile shown in this post is here.

Redundancy + Dependency injection

= Reliability.

Today, one of the biggest payment gateways, Authorize.Net, went down because of a big power outage caused by a fire. Everyone is twittering like crazy, ranting and asking customers to order by phone.

Authorize.Net is also our payment gateway at work, but this wasn't a major issue for us. We just had to swap the payment processing implementation from Authorize.Net with the one that uses a Verisign Payflow Pro account, and that's it. We even didn't have to shut down the site because we have this set up via dynamic.web.config.

Really, you gotta love dependency injection in times like these!

Duck typing extensions for Windsor

Whenever you have to use some piece of legacy code that's badly designed (i.e. static, no coding to interfaces, god classes), your own code gets tainted with some unwanted dependencies or it gets harder to test because of that unmockable legacy code.

The usual cure for this is to isolate the legacy code with adapters, factories, etc. But what if the legacy code isn't so bad? What if we only need to extract an interface? The resulting adapter would be just trivially forwarding calls to the real object... which is boring code. Can't we do better?

After my last post I've been playing a bit more with David Meyer's Duck Typing Project (aka DeftTech) and thought it would be nice to have the ability to expose a registered Windsor component under one or more duck-typed interfaces. Kind of like forwarded types, but without having the service actually implement those interfaces.

So I came up with a couple of simple extension methods that allow this:

public class Duck {
    public bool HasQuacked { get; private set; }
    public bool HasSwum { get; private set;}

    public void Quack() {
        HasQuacked = true;
    }

    public void Swim() {
        HasSwum = true;
    }
}

public interface IQuack {
    void Quack();
}

public interface ISwimmer {
    void Swim();
}

[Test]
public void WindsorDuckTyping() {
    var container = new WindsorContainer();
    DuckComponentExtensions.Kernel = container.Kernel;
    container.AddFacility<FactorySupportFacility>();
    container.Register(Component.For(typeof(Duck))
        .Duck<IQuack>()
        .Duck<ISwimmer>());
    ISwimmer swimmer = container.Resolve<ISwimmer>();
    swimmer.Swim();
    IQuack quack = container.Resolve<IQuack>();
    quack.Quack();
    Duck duck = container.Resolve<Duck>();
    Assert.IsTrue(duck.HasQuacked);
    Assert.IsTrue(duck.HasSwum);
}

So the Duck can be resolved as IQuack even though it doesn't implement IQuack. Here's the code that enables this:

public static class DuckComponentExtensions {
    public static IKernel Kernel { get; set; }

    public static ComponentRegistration<object> Duck<TDuck>(this ComponentRegistration<object> reg) {
        var targetType = reg.Implementation ?? reg.ServiceType;
        if (!DuckTyping.CanCast<TDuck>(targetType))
            throw new ApplicationException(string.Format("Can't duck type '{0}' to type '{1}'", targetType, typeof(TDuck)));
        Kernel.Register(Component.For<TDuck>()
                            .UsingFactoryMethod(k => DuckTyping.Cast<TDuck>(k.Resolve(reg.Name, reg.ServiceType))));
        return reg;
    }
}

Full source code with tests available here.
Apparently the proxies generated by DeftTech can't be proxied by DynamicProxy... so don't try to define any interceptors for these components! Again, the optimal solution would be to implement proper duck-typing with DynamicProxy...

Abusing using to prioritize threads

Welcome to another chapter of "Abusing Using"! IDisposable has been used or abused (depending on who you ask) to create scopes/contexts of all sorts, from database testing to HTML generation to clipboard overriding. Here's another little (ab)use: thread priority setting, for all those background tasks that eat our CPUs:

public static class ThreadSettings {
    private class DisposableThreadPriority: IDisposable {
        private readonly ThreadPriority originalPriority;

        public DisposableThreadPriority(ThreadPriority p) {
            originalPriority = Thread.CurrentThread.Priority;
            Thread.CurrentThread.Priority = p;
        }

        public void Dispose() {
            Thread.CurrentThread.Priority = originalPriority;
        }
    }

    public static IDisposable LowestPriority {
        get {
            return new DisposableThreadPriority(ThreadPriority.Lowest);
        }
    }
}

I like to keep this in an interceptor so I can easily apply it to different components just by adding a couple of lines in my web.config:

public class LowPriorityInterceptor : IInterceptor {
    public void Intercept(IInvocation invocation) {
        using (ThreadSettings.LowestPriority) {
            invocation.Proceed();
        }
    }
}

Windsor - configurable component initialization

This question was raised on the Castle forums a couple of days ago:

I have a third party dependency. It uses a property class for configuration with only a default constructor.

public class Properties { 
      public Properties() {...} 

      public void Add(string name, string value) {...} 
} 

I would love to be able to invoke the .Add to setup this object. Something like this:

<components> 
  <component id="Properties" type="Example.Properties, thirdParty"> 
    <Add> 
      <name>key1</name> 
      <value>value1</value> 
    </Add> 
    <Add> 
      <name>key2</name> 
      <value>value2</value> 
    </Add> 
  </component> 
</components> 

I do realize that I could write an adapter class to interact with the Properties class and then di the adapter. But I'm wondering if I'm missing something or if there is a reason that method invocation is not supported.

Windsor has so many extensibility points that sometimes I have a hard time picking the right one. Windsor lets you change, override or customize almost every aspect of its behaviour thanks to its extensible design. To solve this one, I chose to override the default component activator. The component activator is the internal service responsible for instantiating the component object. To quote Windsor's reference manual:

The ComponentActivator takes a few steps to create the instance

• Selects the constructor it can satisfy more parameters
• Creates the instance using the constructor selected
• Tries to supply dependencies to properties
• Runs the commission phase lifecycle steps (if any was registered)

Our custom activator will call the default activator, then "deserialize" the method calls from the configuration to the appropriate MethodInfo objects, and finally call those methods on the component instance. We can use the componentActivatorType attribute to select the custom activator.

Here's a demo:

[TestFixture]
public class Tests {
    public class MyComponent {
        public int C { get; private set; }

        public void Add(int i) {
            C += i;
        }

        public void NoParameters() {
            C += 2;
        }
    }

    [Test]
    public void Init() {
        var c = new WindsorContainer(new XmlInterpreter(new StaticContentResource(@"<castle>
<components>
<component id=""mycomponent"" type=""WindsorInitConfig.Tests+MyComponent, WindsorInitConfig"" componentActivatorType=""WindsorInitConfig.InitComponentActivator, WindsorInitConfig"">
<init>
    <Add>
        <i>5</i>
    </Add>
    <Add>
        <i>3</i>
    </Add>
    <NoParameters/>
</init>
</component>
</components>
</castle>")));
        Assert.AreEqual(10, c.Resolve<MyComponent>().C);
    }
}

 

You can checkout the whole code here. Note that this is not really production-quality code: it's not properly tested and it probably won't work on generic methods and overloaded methods with the same parameter names, but it's enough for most cases. Please feel free to enhance it and send me a patch! :-)

Windsor facility for Quartz.NET

A few months ago I wrote a couple of adapters to integrate Quartz.Net into Windsor. These were loose components that you had to register yourself and the configuration wasn't very friendly. So I decided to wrap them in a facility to make things cleaner and easier.

This is what the facility provides:

• Some Windsor-managed Quartz entities:
  • IJob / IStatefulJob
  • ISchedulerListener
  • IJobListener (both global and per-job)
  • ITriggerListener (both global and per-trigger)
  • IScheduler
  • IJobFactory (although you probably won't use this...)
• Automatic FileScanJob registering
• IJobScheduler: a simpler IScheduler
• Automatic scheduler starting if the Startable facility is present (otherwise you'll have to call IScheduler.Start() manually)

Here's what it doesn't support:

• These entities are NOT Windsor-managed (they are instantiated normally by Quartz instead)
  • IThreadPool
  • IJobStore
  • ISchedulerPlugin
  • All other interfaces and/or quartz entities
  • Listeners declared in quartz_jobs.xml
• Trigger and job group names

Note that it's up to you to register jobs with the appropriate lifestyle. Listeners can only have singleton lifestyle since they're injected in IScheduler which is itself a singleton.

Here are the bits:

• Binaries
• Sample app (source code)

Simple dependency injection container with CommonServiceLocator

When I started developing SolrNet, one of my goals was to have a DI-friendly library but without taking a direct dependency on any specific container. I wanted to enable both IoC and non-IoC users to use the library with equal ease. So, for example, non-IoC consumers would code like this:

var solr = new SolrServer<Document>(ConfigurationManager.AppSettings["solrURL"]);

while IoC users would write (Windsor sample):

container.Register(Component.For<ISolrServer<Document>>()
	.ImplementedBy<SolrServer<Document>>()
	.Parameters(Parameter.ForKey("serverURL").Eq(ConfigurationManager.AppSettings["solrURL"]))); 

and then inject ISolrServer<Document> where they needed it.

But SolrServer itself had a number of dependent components. At first, this wasn't a problem, this number of components was low and component composition was shallow and simple. But as I added features to the library, there were more and more components, interacting in more complex ways. Since I wanted to keep the possibility of writing "new SolrServer...", every component was responsible for instantiating its own dependencies. Soon, I had component instances duplicated everywhere. Dependencies were not clear because I had "shortcut" constructors for each component. In a word, a mess.

It was time to refactor, make dependencies clear and explicit, and introduce a container to manage dependencies. But I didn't want to depend on any specific container! So I took a look at CommonServiceLocator, which is an abstraction over IoC containers. Just what I needed! Except for one little thing: it's a read-only abstraction, that is, it provides an interface to get things out of the container, but no methods to put things in the container. So it's up to the consumer of the library to appropriately register the library's components. This is stated in the docs:

Libraries should not configure the container
As a library or framework author, understand that you should not be putting anything into the container - that's the job of your caller. Allow the application writers to choose whatever container they want. You need to document what services you need registered, and if you're using the ServiceLocation.Current ambient container.

Well... I just don't agree with that. Consumers shouldn't have to know about the library's internal components lifestyles and dependencies. That's why complex stuff with many components are usually packaged into facilities (Windsor) or modules (Ninject, Autofac). But if I used a facility or module I'd be again depending on a single container!

Actually, there are very valid reasons why the CommonServiceLocator doesn't include Register(). Like the name says, it's a locator (read-only) abstraction, not a factory (creation) abstraction. Each container/factory has unique semantics for registering and creating components, different ways to handle lifestyles, disposable components, registering dependencies, and so on. More on this in this discussion between Hammett and Ayende.

But it still wasn't good enough for me, so the only way out was to write my own simple container on top of IServiceLocator. Based on this article from Ayende, I started with the following interface:

public interface IContainer : IServiceLocator {
    /// <summary>
    /// Adds a component implementing <typeparamref name="T"/>
    /// Component key is <typeparamref name="T"/>'s <see cref="Type.FullName"/>
    /// </summary>
    /// <typeparam name="T">Service type</typeparam>
    /// <param name="factory">Component factory method</param>
    void Register<T>(Converter<IContainer,T> factory);

    /// <summary>
    /// Adds a component implementing <typeparamref name="T"/> with the specified key
    /// </summary>
    /// <typeparam name="T">Service type</typeparam>
    /// <param name="factory">Component factory method</param>
    /// <param name="key">Component key</param>
    void Register<T>(string key, Converter<IContainer,T> factory);
}

Implementing this interface, I could register transient components:

[Test]
public void Transient() {
    var container = new Container();
    container.Register<IService>(c => new ServiceImpl());
    var inst1 = container.GetInstance<IService>();
    var inst2 = container.GetInstance<IService>();
    Assert.AreNotSame(inst1, inst2);
}

Also singletons:

[Test]
public void Singleton() {
    var container = new Container();
    var inst = new ServiceImpl();
    container.Register<IService>(c => inst);
    var inst1 = container.GetInstance<IService>();
    var inst2 = container.GetInstance<IService>();
    Assert.AreSame(inst, inst1);
    Assert.AreSame(inst, inst2);
}

Being such a simple container, injection is less declarative than in real containers:

[Test]
public void Injection() {
    var container = new Container();
    container.Register(c => new AnotherService(c.GetInstance<IService>()));
    var inst = new ServiceImpl();
    container.Register<IService>(c => inst);
    var svc = container.GetInstance<AnotherService>();
    Assert.AreSame(inst, svc.Svc);
}

And that's it for the basic container. Later I added component removal for easier testing and I also experimented with per-thread and per-HttpContext lifestyles just for fun, but I didn't really test them. Check out the full tests if you're interested.

Armed with this little container, I was able to clean up my dependencies and hide the internal components to non-IoC consumers while also allowing for other containers to be used, thanks to CommonServiceLocator. More on this when I release the next version of SolrNet.

Source code:

• IContainer
• Container (implementation)

Castle Windsor factory method support

After using ninject for a short project (some modifications to a Subtext blog) I came back to Windsor and found myself missing ToMethod(). ToMethod() in ninject gives you factory method support, that is, you pass it a Func and the container will call it whenever it needs to instantiate the component.

Now Windsor has had factory support for years, but it requires the factory to be a class, registered as a component in the container. This is ok as it's the most flexible approach, but sometimes all you need is a factory method, and creating a class and then registering it seems a bit overkill. It would be nicer to write something like:

Container.Register(Component.For<HttpServerUtilityBase>()
   .FactoryMethod(() => new HttpServerUtilityWrapper(HttpContext.Current.Server))
   .LifeStyle.Is(LifestyleType.Transient));

With this little extension method, you can do just that:

    public static class ComponentRegistrationExtensions {
        public static IKernel Kernel { private get; set; }

        public static ComponentRegistration<T> FactoryMethod<T, S>(this ComponentRegistration<T> reg, Func<S> factory) where S: T {
            var factoryName = typeof(GenericFactory<S>).FullName;
            Kernel.Register(Component.For<GenericFactory<S>>().Named(factoryName).Instance(new GenericFactory<S>(factory)));
            reg.Configuration(Attrib.ForName("factoryId").Eq(factoryName), Attrib.ForName("factoryCreate").Eq("Create"));
            return reg;
        }

        private class GenericFactory<T> {
            private readonly Func<T> factoryMethod;

            public GenericFactory(Func<T> factoryMethod) {
                this.factoryMethod = factoryMethod;
            }

            public T Create() {
                return factoryMethod();
            }
        }
    }

 

Of course, this needs the FactorySupportFacility to be installed:

Container.AddFacility("factory.support", new FactorySupportFacility());

And since this isn't really a part of the Kernel, it needs a reference to the container's kernel:

ComponentRegistrationExtensions.Kernel = Container.Kernel;

UPDATE 5/7/2009: I submitted a patch which Ayende applied in revision 5650 so this is now part of the Windsor trunk, only instead of FactoryMethod() it's called UsingFactoryMethod()

UPDATE 7/30/2010: as of Windsor 2.5, UsingFactoryMethod() no longer requires the FactorySupportFacility.

Basic Quartz.Net-Windsor integration

A couple of months or so ago, I had to code some periodic maintenance jobs to be run in-process. So I was faced with the same options that Ayende had a year ago (except the out-of-process options). I finally decided on using Quartz.NET, since I needed the dynamic update capabilities (that is, it watches the config file for changes and re-schedules accordingly) which Castle.Components.Scheduler doesn't have yet as far as I know... (BTW Castle.Components.Scheduler is now part of the trunk).

So I set to code the wrappers needed to make it play with Windsor, and the resulting code is here. There's a sample app that shows how to set up job and trigger listeners (both global and job-specific). The actual scheduling configuration is managed by Quartz.NET of course. In the sample I used the external quartz_jobs.xml config with the default RAMJobStore, but you could easily change that to a ADOJobStore or anything, just by setting the props dictionary on the QuartzNetScheduler component, that dictionary is passed as-is to Quartz.NET.

Why did I write "basic" integration on the title? Well, I just needed the basic features of Quartz.NET, so I didn't even try to integrate stuff like clustering, remote servers, etc. I have no idea if those will work with my code. If anyone gives it a try, I'd love to hear about it :-)

UPDATE 4/3/2009: I wrapped the components in a facility for easier usage and configuration.

Changing Windsor components configuration at runtime

Most of the time you want your services/websites/systems running non-stop. I mean, why would you ever want to be restarting them every 4 minutes? You want to minimize downtime. One cause of app restarting in .NET is having to modify web.config / app.config. To partially address that, I wrote a DynamicConfigurationSettings class some time ago, which basically mimics a web.config but freely modifiable at runtime. Now, let's say you have an app with hundreds of components registered in a Windsor container, all configured in your web.config or windsor.boo file. How could you change a component's properties without restarting your app? Even if you put your container config in another file, if you change it nothing happens until the file is re-parsed and re-processed, which basically means recycling the app.

Example

A sample case could be a SMTP mail sender component, which could look like this:

public class SmtpMailSender : IEmailSender
{
    public SmtpMailSender(int port, string host)
    {
        ...
    }  
    ...
}

and configured like this:

<configuration>
    <components>
        <component id="smtp.sender" 
            service="Namespace.IEmailSender, AssemblyName"
            type="Namespace.SmtpMailSender, AssemblyName">
            <parameters>
                <port>10</port>
                <host>smtp.mycompany.com</host>
            </parameters>
        </component>
    </components>
</configuration>

Now suddenly, the mail server goes down! The ops team, a.k.a. you :-), quickly sets up a second mail server at smtp6.myothercompany.com:25, but you have to point the smtp component to the new server while you investigate what happened to the other. If you touched the config, the site would be recycled, and bad things could happen:

• If you use InProc sessions (which is the default), all your users lose their sessions.
• If you use inproc cache (which is the default), you lose all your cache.
• If any user was in the middle of a payment transaction (using a payment gateway like VeriSign PayPal), you could lose the transaction

Solution

But Windsor is very flexible, so you could tie your components configuration to a dynamic.web.config using this simple SubDependencyResolver:

public class DynamicAppSettingsResolver : ISubDependencyResolver {
   public string Key(ComponentModel model, DependencyModel dependency) {
       return string.Format("{0}.{1}", model.Implementation, dependency.DependencyKey);
   }

   public object Resolve(CreationContext context, ISubDependencyResolver parentResolver, ComponentModel model, DependencyModel dependency) {
       var key = Key(model, dependency);
       return Convert.ChangeType(DynamicConfigurationSettings.AppSettings[key], dependency.TargetType);
   }

   public bool CanResolve(CreationContext context, ISubDependencyResolver parentResolver, ComponentModel model, DependencyModel dependency) {
       var key = Key(model, dependency);
       return dependency.DependencyType == DependencyType.Parameter &&
              !string.IsNullOrEmpty(DynamicConfigurationSettings.AppSettings[key]);
   }
}

Adding this resolver using AddSubResolver, gives you the ability to override components parameters in the appSettings section of your dynamic.web.config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>    
    <appSettings>
        <add key="Namespace.SmtpMailSender.host" value="smtp6.myothercompany.com"/>
        <add key="Namespace.SmtpMailSender.port" value="25"/>
    </appSettings>
</configuration>

Caveats

This solution has some caveats:

• The component modifiable with this method (let's call it component A) has to be transient (default is singleton)! Otherwise the subresolver will never get a chance to set the new values...
  Also, as a consequence, other components (let's call them group B) depending on component A have to be transient as well, or they will be stuck with a SmtpMailSender with the old values! And yet other components (group C), depending on any component of group B, have to be transient for the same reason, and so on.
• This subresolver only works with basic types. Ints and strings are all I have needed so far. Of course, you could swap that Convert.ChangeType() with type converters or the other TypeConverters, or store the settings in a configSection of its own instead of appSettings for more flexibility.
• It does not support (although it wouldn't be very difficult to do) service overriding, it only does parameter overriding.

Castle Facility for SolrSharp

Some months ago I wrote SolrNet, an interface to Solr, mainly because I thought SolrSharp (the standard Solr interface for .net) was too verbose and IoC-unfriendly. I still think that way, but I always wondered what it would take to integrate it to Windsor (or any other IoC container, for that matter).

And the answer is... 241 lines of code (as per "wc -l *.cs"). That's it. That's all it took to write some interfaces and adapters and then wrap them in a Castle facility. That should teach me to try a bit harder next time! :-)

SolrNet was designed with IoC in mind, so integration is just a matter of registering a component:

<component id="solr" service="SolrNet.ISolrOperations`1, SolrNet" type="SolrNet.SolrServer`1, SolrNet">
    <parameters>
        <serverURL>http://localhost:8983/solr</serverURL>
    </parameters>
</component> 

Code is here. Note that it's just a proof of concept, I barely tested it!

Injectable file adapters

Someone must have done this, but I really couldn't find it. I'm talking about an IoC-friendly System.IO.File replacement. You know, unit tests shouldn't touch the file system, etc. So if your code writes a file and want to unit-test it, you pretty much have to mock the writing of the file. Except there's a known problem, System.IO.File is a static class, and those are not mockable. Even TypeMock can't mock System.IO.File since it's part of mscorlib. So the only solution left is to build an interface and a wrapper around File. So, here's the source, probably the most boring code I've ever written. I've also included a static locator that gets the IFile implementation using Ayende's IoC static accessor to the Windsor Container, so you can write code like this:

[Test]
public void Copy() {
    var mocks = new MockRepository();
    var container = mocks.CreateMock<IWindsorContainer>();
    var fileImpl = mocks.CreateMock<IFile>();
    IoC.Initialize(container);
    With.Mocks(mocks).Expecting(delegate {
        SetupResult.For(container.Resolve<IFile>()).Return(fileImpl);
        Expect.Call(() => fileImpl.Copy(null, null))
            .IgnoreArguments()
            .Repeat.Once();
    }).Verify(delegate {
        FileEx.Copy("source", "dest");                
    });
}

So you only have to replace your calls to System.IO.File to FileEx and that's it. You get the benefits of testability and extensibility (yes, sometimes you need to provide a different behavior for File.WriteAllText()) and you don't have to deal with interfaces, implementations, etc once you have set up the container. Personally, I prefer to make explicit the dependency for IFile in my components.

Like I said, I was very surprised that I couldn't find a working implementation of this... specially because there was a big debate a year ago about maintenability, YAGNI, dependency injection, coupling and more, and Anders Norås commented this solution on one of his own posts.

Well, I hope someone finds this useful.