Here's another embeddable web console: a Boo interpreter. You enter some code, hit the Execute button and it compiles and runs the code. You get access to all of the assemblies in your app.
I use this to run short ad-hoc "queries" against a running web app, like:
print (context.Cache["pepe"] == null)
import System.Web for s in context.ApplicationInstance.Container.ResolveAll(typeof(IHttpModule)): print s
import System.Diagnostics
pc = PerformanceCounterCategory("ASP.NET")
for counter in pc.GetCounters():
print counter.CounterName, counter.NextValue()import NHibernate
sf = context.ApplicationInstance.Container.Resolve(typeof(ISessionFactory))
using s = sf.OpenSession():
users = s.CreateQuery("from User order by newid()").SetMaxResults(5).List()
for u in users:
print u.Id Why Boo? Because it's pretty much the perfect .net scripting language to embed: small (this console with boo merged in is about 1.9MB), type inference, duck typing, low-ceremony syntax.
Setup:
<add verb="*" path="boo/*" validate="false" type="BooWebConsole.ControllerFactory, BooWebConsole"/>
Visit /boo/index.ashx
Notes:
UPDATE: it's now also available on NuGet
When developing web apps, I sometimes need to poke into the app's database, or try out some query. When it's a SQL Server database, I have Management Studio. But what if it's SQLite? Or Firebird? Even when it's SQL Server, most of my apps use NHibernate and I'm really spoiled by HQL. And if it's a local embedded database, you have to browse and open the db file. Boring. I already have the app URL, that should be enough.
So I built a NHibernate web console. It's kind of like NHibernate Query Analyzer, but more web-oriented. Here's a screencast showing its features:
A couple of features that are not evident from the screencast:
Yeah, I know it's butt-ugly and a little rough around the edges, but still it's pretty usable (at least for me).
DISCLAIMER / WARNING: it's up to you to secure this if you put it in a production website! Also, there is no query cancellation, so if you issue a 1M-results query you will effectively kill your app!
UPDATE: just added RSS support for query results.
Another interesting question from stackoverflow:
I have a custom HTTP Module. I would like to inject the logger using my IoC framework, so I can log errors in the module. However, of course I don't get a constructor, so can't inject it into that. What's the best way to go about this?
First thing we need to recognize is that there are actually two problems here:
Let's start with the lifecycle. Ayende has the best summary of the lifecycle of HttpApplication and its related IHttpModules I've found. If you're not familiar with this, go read it now, I'll wait.
Back so soon? Ok, we can implement this in Windsor with a pluggable lifestyle. Lifestyle managers dictate when it's necessary to create a new instance of the component, but not how. They don't get to actually create the instance, that's the responsibility of the component activator. Here's the full component creation flow reference.
So we need to write a new lifestyle manager that allows at most one component instance per HttpApplication instance. I won't bother you with the implementation details since it's very similar to the per-request lifestyle: it's composed of the LifestyleManager itself and a HttpModule as a helper to store component instances.
Now we need a way to manage the IHttpModule instantiation. Unsurprisingly, we can do that with another IHttpModule, which I'll call WindsorHttpModule.
This WindsorHttpModule will be responsible for the initialization of the "user-level", Windsor-managed IHttpModules.
So, to summarize:
<httpModules> <add name="PerHttpApplicationLifestyleModule" type="HttpModuleInjection.PerHttpApplicationLifestyleModule, HttpModuleInjection"/> <add name="WindsorModule" type="HttpModuleInjection.WindsorHttpModule, HttpModuleInjection"/> </httpModules>
public class Service { public DateTime Now { get { return DateTime.Now; } } } public class UserHttpModule : IHttpModule { private readonly Service s; public UserHttpModule(Service s) { this.s = s; } public void Init(HttpApplication context) { context.BeginRequest += context_BeginRequest; } private void context_BeginRequest(object sender, EventArgs e) { var app = (HttpApplication) sender; app.Response.Write(s.Now); } public void Dispose() {} }
container.AddComponent<Service>();
container.Register(Component.For<IHttpModule>()
.ImplementedBy<UserHttpModule>()
.LifeStyle.Custom<PerHttpApplicationLifestyleManager>());
You can also combine this with Rashid's BaseHttpModule to make your modules more testable.
Some websites and products that use SolrNet:
I'll keep an updated list on the project's wiki. If you have a public website using SolrNet, let me know!
When working with Windsor on complex applications, it's pretty common to have hundreds or even thousands of components managed by the container. The dependencies can get quite intricate and sometimes you need to see the big picture. Static analyzers like NDepend don't cut it since components are wired by Windsor at runtime.
So here's a little guide to output a PNG of a container's components and dependencies, using QuickGraph and GLEE.
First we need to model the components and dependencies as a regular Dictionary. The key of this dictionary will be a description of the component (a regular string) and the value of this dictionary will be a list of dependencies.
var container = new WindsorContainer(); // ...add components... var dependencyDict = container.Kernel.GraphNodes .Distinct(new EqComparer<GraphNode>((a, b) => a.Describe() == b.Describe(), n => n.Describe().GetHashCode())) .ToDictionary(n => n.Describe(), n => n.Dependents.Select(a => a.Describe()));
EqComparer is just a functional IEqualityComparer. Distinct() is needed because when using forwarded types you'd get duplicate dictionary keys.
Now we dump this into a QuickGraph digraph:
var graph = dependencyDict.ToVertexAndEdgeListGraph(kv => kv.Value.Select(n => new SEquatableEdge<string>(kv.Key, n)));
and finally we render the digraph to a PNG using GLEE:
graph.ToGleeGraph().ToBitmap().Save("windsor.png");
Here's a sample output from SolrNet's inner components (click to zoom):
Only problem with this is that it doesn't take custom subdependency resolvers (like an ArrayResolver) into account, since they can't be modeled in the kernel's GraphNodes.
Here's what it looks like after some ad-hoc fixes:
Related articles:
I'm currently working on migrating the Castle project Subversion repository to git. There are already a couple of svn mirrors on github, but this migration is intended to eventually replace svn as the official repository. With over 6000 commits in 5 years of history, it's not a trivial migration.
One of the issues is that all subprojects are currently being split from the main trunk to make them more independent. I'll leave that for another post.
Another issue is the committer mapping. Each svn username needs to be mapped to a github account (name + email). Roelof Blom kindly provided this map, so I was set to import with git-svn. Two days later git-svn finished and I pushed the repository to github.
Much to my dismay, I found that some committers on my repository weren't matching their github accounts. The Ken Egozi on my repo wasn't the same Ken Egozi on github!
I had two options: either fix the user mappings and re-run git-svn or change the committers with git filter-branch. I went with filter-branch as described in the Pro Git book. It was processing about 1 commit/s so I left it working and went to sleep.
The next morning I went to see and it had segfaulted halfway through. Now that did it. I updated my GitSharp fork and wrote this kind of specific filter-branch:
internal class Program { private static void Main(string[] args) { var committerMap = new Dictionary<string, string> { {"Ken Egozi", "a@b.com"}, {"Krzysztof Koźmic", "c@d.com"}, }; var repo = Repository.Open(args[0]); var refs = repo.getAllRefs() .Where(x => x.Key.StartsWith("refs/heads") || x.Key.StartsWith("refs/tags")) .ToDictionary(x => x.Key, x => x.Value); var commitMap = new Dictionary<string, string>(); foreach (var r in refs) { Console.WriteLine("Processing ref {0}", r.Key); var startCommit = repo.MapCommit(r.Value.ObjectId); var newHead = commitMap.ContainsKey(r.Value.ObjectId.Name) ? repo.MapCommit(commitMap[r.Value.ObjectId.Name]) : Rewrite(repo, startCommit, commitMap, committerMap); var newRef = repo.UpdateRef(r.Value.Name); newRef.NewObjectId = newHead.CommitId; newRef.IsForceUpdate = true; newRef.Update(); } } public static Commit Rewrite(Repository repo, Commit startCommit, Dictionary<string, string> commitMap, Dictionary<string, string> committerMap) { Commit lastCommit = null; var walker = new RevWalk(repo); walker.sort(RevSort.Strategy.REVERSE); walker.markStart(walker.parseCommit(startCommit.CommitId)); foreach (var rcommit in walker.iterator().AsEnumerable()) { var commit = rcommit.AsCommit(walker); if (commitMap.ContainsKey(commit.CommitId.Name)) { lastCommit = repo.MapCommit(commitMap[commit.CommitId.Name]); Console.WriteLine("{0} already visited, skipping", commit.CommitId.Name); continue; } if (committerMap.ContainsKey(commit.Author.Name)) commit.Author = new PersonIdent(commit.Author.Name, committerMap[commit.Author.Name], commit.Author.When, commit.Author.TimeZoneOffset); var newCommit = new Commit(repo) { TreeId = commit.TreeId, Author = commit.Author, Committer = commit.Author, Message = commit.Message, ParentIds = commit.ParentIds.Select(x => repo.MapCommit(commitMap[x.Name]).CommitId).ToArray(), }; newCommit.Save(); commitMap[commit.CommitId.Name] = newCommit.CommitId.Name; lastCommit = newCommit; } return lastCommit; } }
It took this little code 3 minutes to rewrite the whole repository. That's 34 commits/s !
When time allows, I'll try and clean this up, then merge it into the new GitSharp.CLI project. Instead of using a bash script to define transformations like the original filter-branch, this could use an embedded Boo or IronPython script!
DISCLAIMER: The code shown here is completely throwaway quality. It does not intend to be a reference GitSharp app or anything like that. It does not intend to be a general filter-branch replacement. It works on my machine, etc. Do not run this code on your repositories unless you know what you're doing. It will rewrite your whole repository! You have been warned.
I've recently received several questions about the relationship between Solr, SolrNet, NHibernate, Lucene, Lucene.Net, etc, how they fit together, how they should be used, what features does each provide. Here's an attempt at elucidating the topic:
Let's start from the bottom up:
In case this wasn't totally clear, here's a diagram depicting a possible NHibernate-SolrNet architecture:
There is a killer characteristic of open source that seems to be often overlooked: the ability to browse the source code history.
This will sound trivial to many, but it's something that I've seen a lot in questions asked on stackoverflow.com and other forums. People ask: "why has feature X changed?" "Why am I getting an exception P since I upgraded project Q to version R?"
The first thing you need to do is assume there is a good reason for that change and set out to find it, instead of considering it a bug in the OSS. Keep yourself humble, remember that the people working on OSS are generally very smart and really know what they're doing.
And the really generic answer to such questions would be (without any intention of being harsh): you can usually find out yourself.
An example: this stackoverflow question (here summarized):
In the previous version of NHibernate (2.0.1) the following property will validate:
internal virtual BusinessObject Parent
{
get { /*code*/ }
}However, in 2.1 it errors saying that the types should be 'public/protected virtual' or 'protected internal virtual'. Why is this requirement now there?
Now I've been using NHibernate since 0.83 but I'm no expert in NHibernate internals. Despite that, I was able to answer the question by doing this:
NHibernate.Proxy.DynProxyTypeValidator. That's it. No special knowledge needed. This process could be further simplified by using Fisheye's search instead of locally grepping (but I never seem to get good results from Fisheye) or getting NHibernate from github instead of svn and then grepping with git-grep or gitk. Browsing history with svn is so painfully slow that I prefer to do it with Fisheye.
By the way, this is the same approach I use on closed code at work when my boss asks me "Hey, this feature used to behave differently! When and why did we change it?" (and this happens a lot)
This is where best practices like specific commit messages, atomic commits and good issue tracking really pay off.
Documentation? I save that for stuff that this approach can't possibly handle.
If you're running IIS 6 like me, you know there aren't many options if you want extensionless URLs. I already had a custom URL rewriting engine in place with Windsor integration and stuff, but it couldn't handle extensionless URLs. The 404 solution seems kinda kludgy to me so after some pondering I decided to go with IIRF.
Ok, now let's see how do we test IIRF rules. IIRF is a regular Win32 DLL written in C, so there are no managed hook points that we can use from .Net. Fortunately, it comes with a testing tool called TestDriver, which is a standard .exe that takes a file with source and destination URLs and runs all source URLs against the rules, asserting that each result matches the destination.
All we need now is some glue code to integrate this to our regular tests so if any routes fail it also makes the whole build fail. I also want to see on my TeamCity build log exactly which route failed, and also be able to easily add new route tests. We can do all this with MbUnit's [Factory] and some stdout parsing. Here's the code:
[TestFixture] public class IIRFTests { [Test] [Factory("IIRFTestFactory")] public void IIRFTest(string orig, string dest) { File.WriteAllText(@"SampleUrls.txt", string.Format("{0}\t{1}", orig, dest)); var r = RunProcess(@"TestDriver.exe", @"-d ."); if (r.ErrorLevel != 0) { var actual = Regex.Replace(r.Output.Replace("\r\n", " "), @".*actual\((.*)\).*", "$1"); Assert.AreEqual(dest, actual); } } public IEnumerable<object[]> IIRFTestFactory() { yield return new object[] { "/questions/167586/visual-studio-database-project-designers", "/Question/Index.aspx?id=167586" }; yield return new object[] { "/users/21239/mauricio-scheffer", "/User/Index.aspx?id=21239" }; } public ProcessOutput RunProcess(string fileName, string arguments) { var p = Process.Start(new ProcessStartInfo(fileName, arguments) { CreateNoWindow = true, RedirectStandardOutput = true, RedirectStandardError = true, UseShellExecute = false, }); p.WaitForExit(); return new ProcessOutput(p.ExitCode, p.StandardOutput.ReadToEnd() + p.StandardError.ReadToEnd()); } public class ProcessOutput { public int ErrorLevel { get; private set; } public string Output { get; private set; } public ProcessOutput(int errorLevel, string output) { ErrorLevel = errorLevel; Output = output; } } }
Just released SolrNet 0.2.3 beta1. Here's the changelog:
These are the interesting new features:
This is a very cool feature that isn't even included in the Solr trunk. It's currently only available as a patch, but hopefully it will make its way to trunk soon. It allows you to filter query results based on a document field, thus making a flexible duplicate detection.
This one is a Solr 1.4 feature (currently only available from trunk or nightly builds). Like the name says, it gives you statistics about your numeric fields within your query results. The statistics are: min, max, sum, count, missing (i.e. no value), sum of squares, mean, standard deviation. The cool thing about this is that you can facet it, thus getting separate stats for each value of the field.
This allows you to trigger faceting based on date ranges, i.e. you can create a facet for each day from 8/1/2009 to 9/1/2009.
This is similar to the NHibernate.Search project. It synchronizes a database with Solr (if the Solr document fields are similar to a NHibernate entity fields) and it allows you to issue Solr queries from a regular NHibernate ISession (well, actually a ISession wrapper). You can see more details about its usage in the wiki.
Contributors to this release: Derek Watson, Matt Mondok, Juuso Kosonen.
Get it here:
I'll probably call this a GA release in a couple of weeks if there aren't any serious bugs and once I get the wiki updated.
P.S.: I'll take this opportunity to clear some things up about the project. SolrNet started, like many open source projects, as a way to scratch my own itch. But in the last few months, it has grown beyond that. As a result, I don't have a use for many new features so I am not so motivated to implement them and I don't have any chance to test them in the wild to iron out any bugs and to make sure they are release-quality. This means that I need help from the community (yes, that includes you! ;-) in the form of:
Trunk is very stable, right now there are 368 tests that cover around 80% of the code. I strongly encourage you to get new builds from the build server (see artifacts links) and let me know how it works out for you (both positive and negative constructive feedback are useful).
Finally, I could offer some basic commercial support if you need some feature urgently and don't have the resources to code it.
I've been working for the last month on a library that implements API bindings for OpenX in .Net. The OpenX API is quite big and I recently finished mapping all available methods so I decided to call it an early alpha release.
With this you can do:
It relies on Charles Cook's XML-RPC.NET library (which is the de-facto standard xml-rpc library for .Net) which I had to modify slightly due to some irregularities in the OpenX XML-RPC server. Hopefully Charles will make xml-rpc.net more tolerant to these glitches soon so I can kill off my fork and use the standard xml-rpc.net.
Anyway here's a sample snippet querying for an advertiser's daily stats:
[Test] public void AdvertiserDailyStats() { using (ISession session = new SessionImpl("user", "password", "http://localhost:10002/openx/api/v2/xmlrpc/")) { DailyStats[] r = session.GetAdvertiserDailyStatistics(1, DateTime.Now.AddYears(-1), DateTime.Now); Console.WriteLine("got {0} stats", r.Length); foreach (DailyStats stat in r) { Console.WriteLine("Stats for day {0}:", stat.Day.ToShortDateString()); Console.WriteLine("Impressions: {0}", stat.Impressions); Console.WriteLine("Clicks: {0}", stat.Clicks); Console.WriteLine("Requests: {0}", stat.Requests); Console.WriteLine("Revenue: {0}", ((decimal) stat.Revenue).ToString("C")); } } }
I wanted to release this as as early as possible because it's feature-complete but barely tested so I need people to try it and help me stabilize it by reporting any bugs and hopefully also contribute patches!
You can browse the code and fork it on github.
Binaries are available here.
PS: It seems that both Roger Dickey and I have been working independently on this at the same time, a situation that always sucks. I pinged him when I discovered this and he agreed to work together with me on this one.
I generally advise to keep the IoC container out of the tests, but sometimes it is a convenient tool to have. Take for example the auto-mocking container concept. It's just too useful so I make an exception to the previous rule sometimes.
Here's another use for a container in a test: a profiler. All IoC containers have some proxying abilities (or rather, they delegate to one of the proxying libraries), which can be leveraged to build a basic profiler. A profiler is especially useful in integration tests where there's something that's taking too much time but you can't put your finger on the culprit simply because there are too many components involved.
A real-world example: an application which uses SolrNet was taking too long when getting lots of documents from Solr. There was nothing weird in Solr's log so it was a client issue. Let's profile it!
We create a ProfilingContainer and add all the components we want to profile:
IWindsorContainer container = new ProfilingContainer(); container.AddComponent<ISolrDocumentResponseParser<Document>, SolrDocumentResponseParser<Document>>(); container.AddComponent<ISolrResponseParser<Document>, ResultsResponseParser<Document>>("resultsParser"); container.AddComponent<IReadOnlyMappingManager, AttributesMappingManager>(); container.Register(Component.For<ISolrQueryResultParser<Document>>().ImplementedBy<SolrQueryResultParser<Document>>() .ServiceOverrides(ServiceOverride.ForKey("parsers").Eq(new[] { "resultsParser" }))); container.AddComponent<ISolrFieldParser, DefaultFieldParser>(); container.AddComponent<ISolrDocumentPropertyVisitor, DefaultDocumentVisitor>();
We add the code that exercises the components:
var parser = container.Resolve<ISolrQueryResultParser<Document>>(); for (int i = 0; i < 1000; i++) { parser.Parse(responseXMLWithArrays); }
Finally, we get the profile data:
Node<KeyValuePair<MethodInfo, TimeSpan>> rawProfile = container.GetProfile();
This returns a tree of method calls, each with its logged duration. We just want a list of aggregated durations, so first we flatten the tree:
IEnumerable<KeyValuePair<MethodInfo, TimeSpan>> profile = Flatten(rawProfile);
And now we can query and write the profile:
var q = from n in profile group n.Value by n.Key into x let kv = new { method = x.Key, count = x.Count(), total = x.Sum(t => t.TotalMilliseconds)} orderby kv.total descending select kv; foreach (var i in q) Console.WriteLine("{0} {1}: {2} executions, {3}ms", i.method.DeclaringType, i.method, i.count, i.total);
Which prints something like this (prettified):
| Method | Executions | Total own time |
|---|---|---|
| AttributesMappingManager.GetFields(Type) | 21000 | 2803.23ms |
| DefaultDocumentVisitor.Visit(Object, String, XmlNode) | 10000 | 432.19ms |
| DefaultFieldParser.Parse(XmlNode, Type) | 10000 | 122.77ms |
| SolrQueryResultParser<Document>.Parse(String) | 1000 | 93.52ms |
| DefaultFieldParser.CanHandleType(Type) | 10000 | 44.71ms |
| SolrDocumentResponseParser<Document>.ParseResults(XmlNode) | 1000 | 43.11ms |
| ResultsResponseParser<Document>.Parse(XmlDocument, SolrQueryResults<Document>) | 1000 | 30.28ms |
| DefaultFieldParser.CanHandleSolrType(String) | 10000 | 23.78ms |
Well, it's obvious that AttributesMappingManager.GetFields() is to be blamed here, it's taking a lot of time... which is only logical since it uses a lot of reflection...
But wait a minute, I already thought about that when I wrote that MemoizingMappingManager (which is a memoizer decorator for the AttributesMappingManager)... only I forgot to add it to the container!
After adding the MemoizingMappingManager, the profile comes out like this:
| Method | Executions | Total own time |
|---|---|---|
| DefaultDocumentVisitor.Visit(Object, String, XmlNode) | 10000 | 358.64ms |
| SolrQueryResultParser<Document>.Parse(String) | 1000 | 166.64ms |
| DefaultFieldParser.Parse(XmlNode, Type) | 10000 | 87.13ms |
| MemoizingMappingManager.GetFields(Type) | 21000 | 50.5ms |
| DefaultFieldParser.CanHandleType(Type) | 10000 | 42.03ms |
| SolrDocumentResponseParser<Document>.ParseResults(XmlNode) | 1000 | 34.93ms |
| ResultsResponseParser<Document>.Parse(XmlDocument, SolrQueryResults<Document>) | 1000 | 30.83ms |
| DefaultFieldParser.CanHandleSolrType(String) | 10000 | 22.22ms |
| AttributesMappingManager.GetFields(Type) | 1 | 19.82ms |
This cheap profiler has a number of caveats:
All in all, I found this to be quite useful. ProfilingContainer is just a thin wrapper around WindsorContainer that adds a ProfilerFacility, which in turn sets up a ProfilingInterceptor. So you could, for example, add the ProfilingInterceptor to your application container with a per-web-request lifestyle and code a HttpModule to profile an entire http request.
So I finally hopped on the Git wagon, and I gotta say it's awesome! It takes some learning, but once you grok it, it's quite a simple model really.
I'm currently using Git via git-svn at work at the moment since I don't want to disturb the rest of the team just yet. Git-svn allows me to take most of the advantages of DVCS and when I'm done I commit to svn to publish my changes. Well, I'm not going to do a tutorial here, you can easily find lots of them on the net. I'll just say that if you're considering using a DVCS, go for it. Don't listen to those who say it's more complex or it has many more commands. Just go for it, it takes some learning but it's totally worth it. But please, do it with an open-minded approach. It's a different paradigm, try not to judge it from centralized version control concepts.
In my opinion, within three years from now almost everyone will be using a DVCS. Even the svn team is considering implementing an "hybrid distributed/centralized model" so if you stick with svn you'll probably use decentralized features in a couple of years.
The adoption rate of DVCS is pretty fast. Take a look at these graphs of Git Survey 2008 responses, and GitHub traffic:
%5B4%5D.png) |
And that's just Git. Mercurial and Bazaar usage are growing too.
However, it isn't all roses with Git on Windows. Linux people are used to the command line so it's not much of a problem for them to use the git CLI, but we Windows users are too spoiled by the ubiquitous GUI, and the excellent TortoiseSVN, AnkhSVN and VisualSVN. I think I never had to drop down to the command line to do something in svn. Even when I had to script something that involved svn, I just used SharpSvn and Boo.
With Git, I'm currently using the following mix of interfaces:
There are even more GUIs, like git-cola, qgit and others. So the problem isn't really a lack of GUIs. The problem is that none of them gets everything right, like TortoiseSVN does. Or at least I haven't found one. So I have to keep switching between them, which obviously isn't optimal. Now I don't mean to bash them, to their credit, GUIs and Windows support have improved hugely in the last 12 months or so. In particular, TortoiseGit has gone from nil to very usable in about 6 months by taking advantage of the TortoiseSVN codebase.
Another issue is the speed. Everyone says git is fast. Not so much on Windows, because Windows doesn't have fork() so it has to be emulated (which is slow) and exec() is also much slower than Linux. Git-svn is particularly slow because of this.
Now I don't like to rant aimlessly. I think the way to truly address these issues is to first build a solid foundation, a solid git library for Windows. And that foundation is GitSharp, which is a port of the Java jgit. The port is currently only 50% complete so it needs as much help as possible. Are you interested? Fork away! ;-)
= Reliability.
Today, one of the biggest payment gateways, Authorize.Net, went down because of a big power outage caused by a fire. Everyone is twittering like crazy, ranting and asking customers to order by phone.
Authorize.Net is also our payment gateway at work, but this wasn't a major issue for us. We just had to swap the payment processing implementation from Authorize.Net with the one that uses a Verisign Payflow Pro account, and that's it. We even didn't have to shut down the site because we have this set up via dynamic.web.config.
Really, you gotta love dependency injection in times like these!
I've been using NDepend off and on for the last couple of years. Mainly to diagnose build issues and help me write build scripts for complex legacy applications, since legacy applications with no automated build often fall into cyclic dependencies between different solutions. NDepend's assembly build order feature is great for this. But I've never had the time to really dive into the myriad other features this product offers.
A couple of weeks ago Patrick Smacchia kindly sent me a Pro license for NDepend (thanks Patrick!), so I thought it would be a great opportunity to use it to analyze SolrNet. Now, SolrNet is by all measures a tiny project (ohloh stats are inflated due to Sandcastle, SHFB, etc being in the trunk), but that doesn't mean it can't benefit from NDepend.
First of all, if you're analyzing a library, I recommend that you include your tests in the analysis, so that NDepend can see how the library is actually used. Otherwise, NDepend will suggest that you mark some things as private when it shouldn't. Don't worry about the tests raising warnings in the analysis, you can filter them out as I'll explain later. Plus, having the ability to analyze the tests can be pretty handy too.
For example, we can easily issue some CQL queries to get the code/test ratio:
SELECT ASSEMBLIES WHERE NameLike "Tests" | 1972 LOC (as defined here) |
SELECT ASSEMBLIES WHERE !NameLike "Tests" | 1550 LOC |
So the code:test ratio is 1:1.27, more LOC of tests than code! However, keep in mind that this metric alone doesn't imply a correct coverage.
Browsing the default queries, I find that "Fields that could be declared internal" caught a
public readonly string query;
Oops, fixing right away!
Under "Purity / Immutability / Side-Effects", "Fields should be marked as ReadOnly when possible" showed 29 results, which seemed strange since I always try to make my objects immutable. 24 of these were <>l__initialThreadId fields, which is one of the fields of the iterator that the C# compiler builds when you use yield return. This also happened with the "Methods too big", "Methods too complex" and "Potentially unused methods" metrics.
Of course, you can edit or delete the default CQL queries. For example, the "Potentially unused methods" is defined by default as:
// <Name>Potentially unused methods</Name> WARN IF Count > 0 IN SELECT TOP 10 METHODS WHERE MethodCa == 0 AND // Ca=0 -> No Afferent Coupling -> The method is not used in the context of this application. !IsPublic AND // Public methods might be used by client applications of your assemblies. !IsEntryPoint AND // Main() method is not used by-design. !IsExplicitInterfaceImpl AND // The IL code never explicitely calls explicit interface methods implementation. !IsClassConstructor AND // The IL code never explicitely calls class constructors. !IsFinalizer // The IL code never explicitely calls finalizers.
We can easily add another condition so that these methods don't bother us: AND !FullNameLike "__"
One of the most useful features of NDepend is comparing two versions of a project. I compared the latest release of SolrNet against trunk. Here's a chart of added methods (83 in total, in blue):
34 methods changed:
API breaking changes (no graphic here, just a list):
Don't worry, these are all internal breaking changes, they won't affect the library consumer...
From these graphics you can immediately see that there aren't really many changes, and they aren't focused. The reason is that most of the changes are minor bugfixes and a couple of minor added features.
I only scratched the surface of what's possible with NDepend, but as you can see, small projects can also profit from it, so check it out!
Whenever you have to use some piece of legacy code that's badly designed (i.e. static, no coding to interfaces, god classes), your own code gets tainted with some unwanted dependencies or it gets harder to test because of that unmockable legacy code.
The usual cure for this is to isolate the legacy code with adapters, factories, etc. But what if the legacy code isn't so bad? What if we only need to extract an interface? The resulting adapter would be just trivially forwarding calls to the real object... which is boring code. Can't we do better?
After my last post I've been playing a bit more with David Meyer's Duck Typing Project (aka DeftTech) and thought it would be nice to have the ability to expose a registered Windsor component under one or more duck-typed interfaces. Kind of like forwarded types, but without having the service actually implement those interfaces.
So I came up with a couple of simple extension methods that allow this:
public class Duck { public bool HasQuacked { get; private set; } public bool HasSwum { get; private set;} public void Quack() { HasQuacked = true; } public void Swim() { HasSwum = true; } } public interface IQuack { void Quack(); } public interface ISwimmer { void Swim(); } [Test] public void WindsorDuckTyping() { var container = new WindsorContainer(); DuckComponentExtensions.Kernel = container.Kernel; container.AddFacility<FactorySupportFacility>(); container.Register(Component.For(typeof(Duck)) .Duck<IQuack>() .Duck<ISwimmer>()); ISwimmer swimmer = container.Resolve<ISwimmer>(); swimmer.Swim(); IQuack quack = container.Resolve<IQuack>(); quack.Quack(); Duck duck = container.Resolve<Duck>(); Assert.IsTrue(duck.HasQuacked); Assert.IsTrue(duck.HasSwum); }
So the Duck can be resolved as IQuack even though it doesn't implement IQuack. Here's the code that enables this:
public static class DuckComponentExtensions { public static IKernel Kernel { get; set; } public static ComponentRegistration<object> Duck<TDuck>(this ComponentRegistration<object> reg) { var targetType = reg.Implementation ?? reg.ServiceType; if (!DuckTyping.CanCast<TDuck>(targetType)) throw new ApplicationException(string.Format("Can't duck type '{0}' to type '{1}'", targetType, typeof(TDuck))); Kernel.Register(Component.For<TDuck>() .UsingFactoryMethod(k => DuckTyping.Cast<TDuck>(k.Resolve(reg.Name, reg.ServiceType)))); return reg; } }
Full source code with tests available here.
Apparently the proxies generated by DeftTech can't be proxied by DynamicProxy... so don't try to define any interceptors for these components! Again, the optimal solution would be to implement proper duck-typing with DynamicProxy...
I was trying to come up with some duck typing solution using C# 4.0, based on this, and this, when I realized that DynamicProxy could do this without any new language features:
using System; using System.Collections.Generic; using System.Linq; using System.Text; using Castle.Core.Interceptor; using Castle.DynamicProxy; using Microsoft.VisualStudio.TestTools.UnitTesting; namespace DuckTypingTests { public class Duck { public void Quack() { Console.WriteLine("Quack Quack!"); } public void Swim() { Console.WriteLine("Swimming..."); } } public interface IQuack { void Quack(); } public interface ISwimmer { void Swim(); } [TestClass] public class DuckTypingTests { [TestMethod] public void DuckTyping() { var duck = new Duck(); duck.As<IQuack>().Quack(); duck.As<ISwimmer>().Swim(); } } public static class DuckTypingExtensions { private static readonly ProxyGenerator generator = new ProxyGenerator(); public static T As<T>(this object o) { return generator.CreateInterfaceProxyWithoutTarget<T>(new DuckTypingInterceptor(o)); } } public class DuckTypingInterceptor : IInterceptor { private readonly object target; public DuckTypingInterceptor(object target) { this.target = target; } public void Intercept(IInvocation invocation) { var methods = target.GetType().GetMethods() .Where(m => m.Name == invocation.Method.Name) .Where(m => m.GetParameters().Length == invocation.Arguments.Length) .ToList(); if (methods.Count > 1) throw new ApplicationException(string.Format("Ambiguous method match for '{0}'", invocation.Method.Name)); if (methods.Count == 0) throw new ApplicationException(string.Format("No method '{0}' found", invocation.Method.Name)); var method = methods[0]; if (invocation.GenericArguments != null && invocation.GenericArguments.Length > 0) method = method.MakeGenericMethod(invocation.GenericArguments); invocation.ReturnValue = method.Invoke(target, invocation.Arguments); } } }
Again, this is just a spike, it only matches methods, not properties, and even the method matching in the interceptor will break with method overloading. And let's not mention performance.
It would be interesting to somehow merge David Meyer's Duck Typing Project into Castle DynamicProxy, as it is the most complete duck-typing solution for .net that I know...
Here's another practical use for dynamic in C# 4: testing private methods. I won't discuss here whether this should or shouldn't be done, that's been argued to death for years. Instead, I'll show how it can be done easier with C# 4, without resorting to reflection (at least not directly) or stubs, while also reducing some of the friction associated with doing TDD in statically-typed languages like C#.
The answer is very simple: we just walk away from static typing using C# dynamic:
public class Service { private int Step1() { return 1; } } [TestClass] public class TransparentObjectTests { [TestMethod] public void PrivateMethod() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual(1, s.Step1()); } }
When going dynamic, it's kind of like turning off the compiler, so, for example, it won't bother you at compile-time if a method isn't present. Instead, you'll just get an exception when you run your test, which is more like the dynamic language way of testing things.
Here's the code that enables this. Just like my last post, it's just a spike, it's incomplete and I didn't test it properly, but it could serve as a base for a more complete implementation:
using System; using System.Dynamic; using System.Linq; using System.Reflection; using Microsoft.CSharp.RuntimeBinder; using Microsoft.VisualStudio.TestTools.UnitTesting; namespace DynamicReflection { public class Service { private int Step1() { return 1; } protected int Step2() { return 2; } internal int Step3() { return 3; } private int Prop { get; set; } public int Execute(int a) { return Step1() + Step2() + Step3() + a; } private string ExecuteGeneric<T>(T a) { return a.ToString(); } } [TestClass] public class TransparentObjectTests { [TestMethod] public void PublicMethod() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual(10, s.Execute(4)); } [TestMethod] public void PrivateMethod() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual(1, s.Step1()); } [TestMethod] public void ProtectedMethod() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual(2, s.Step2()); } [TestMethod] public void InternalMethod() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual(3, s.Step3()); } [TestMethod] public void PrivateProperty() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual(0, s.Prop); } [TestMethod] [ExpectedException(typeof(ArgumentException))] public void GenericPrivateMethod_type_mismatch() { dynamic s = new Service().AsTransparentObject(); s.ExecuteGeneric<int>("lalal"); // type param mismatch } [TestMethod] public void GenericPrivateMethod() { dynamic s = new Service().AsTransparentObject(); Assert.AreEqual("1", s.ExecuteGeneric<int>(1)); } } public static class TransparentObjectExtensions { public static dynamic AsTransparentObject<T>(this T o) { return new TransparentObject<T>(o); } } public class TransparentObject<T> : DynamicObject { private readonly T target; public TransparentObject(T target) { this.target = target; } public override bool TryGetMember(GetMemberBinder binder, out object result) { var members = typeof(T).GetMember(binder.Name, BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance); var member = members[0]; if (member is PropertyInfo) { result = (member as PropertyInfo).GetValue(target, null); return true; } if (member is FieldInfo) { result = (member as FieldInfo).GetValue(target); return true; } result = null; return false; } public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result) { var csBinder = binder as CSharpInvokeMemberBinder; var method = typeof(T).GetMethod(binder.Name, BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance); if (method == null) throw new ApplicationException(string.Format("Method '{0}' not found for type '{1}'", binder.Name, typeof(T))); if (csBinder.TypeArguments.Count > 0) method = method.MakeGenericMethod(csBinder.TypeArguments.ToArray()); result = method.Invoke(target, args); return true; } } }
UPDATE 5/17/2010: Igor Ostrovsky recently got the same idea outlined here and has written a proper implementation of this.
Now that VS2010 Beta 1 is out, I figured I'd take another look and try to actually do something with it. So I spiked a little proof of concept for what I mentioned about six months ago: Rails-like finders for NHibernate. In RoR's ActiveRecord you can write:
Person.find_by_name("pepe")without really having a find_by_name method, thanks to Ruby's method_missing. Now, with a few lines of code, you can write this in C# 4.0, thanks to DynamicObject:
ISession session = ...
Person person = session.AsDynamic().GetPersonByName("pepe");which behind the scenes will parse the method name "GetPersonByName" (using a Pascal-Casing convention) into a DetachedCriteria and execute it.
Note that I wrote "Person person = ..." instead of "var person = ...", that's because the result of GetPersonByName() is a dynamic so it has to be cast to its proper type to jump back to the strongly-typed world.
As I said before, this is only a proof of concept. It doesn't support finders that return collections, or operators (as in FindPersonByNameAndCity("Pepe", "Boulogne-sur-Mer")), etc. It would certainly be interesting to really implement this, but I wonder if people would use it, with IQueryable being more strongly-typed, way more flexible, and with better language integration.
Anyway, here's the code that makes this possible. Pay no attention to Castle ActiveRecord, I just used it because... well, geographical convenience, really.
using System; using System.Collections; using System.Collections.Generic; using System.Dynamic; using System.Linq; using System.Text.RegularExpressions; using Castle.ActiveRecord; using Castle.ActiveRecord.Framework.Config; using log4net.Config; using NHibernate; using NHibernate.Criterion; using NUnit.Framework; namespace NHDynamicTests { [TestFixture] public class DynamicTests { [Test] public void DynamicGetPersonByName() { using (ISession s = ActiveRecordMediator.GetSessionFactoryHolder().GetSessionFactory(typeof(object)).OpenSession()) { dynamic ds = s.AsDynamic(); Person person = ds.GetPersonByName("pepe"); } } [ActiveRecord] public class Person { [PrimaryKey] public int Id { get; set; } [Property] public string Name { get; set; } } [TestFixtureSetUp] public void FixtureSetup() { BasicConfigurator.Configure(); var arConfig = new InPlaceConfigurationSource(); var properties = new Dictionary<string, string> { {"connection.driver_class", "NHibernate.Driver.SQLite20Driver"}, {"dialect", "NHibernate.Dialect.SQLiteDialect"}, {"connection.provider", "NHibernate.Connection.DriverConnectionProvider"}, {"connection.connection_string", "Data Source=test.db;Version=3;New=True;"}, }; arConfig.Add(typeof(ActiveRecordBase), properties); ActiveRecordStarter.ResetInitializationFlag(); var arTypes = GetType().GetNestedTypes() .Where(t => t.GetCustomAttributes(typeof(ActiveRecordAttribute), true).Length > 0) .ToArray(); ActiveRecordStarter.Initialize(arConfig, arTypes); ActiveRecordStarter.CreateSchema(); } } public static class ISessionExtensions { public static dynamic AsDynamic(this ISession session) { return new DynamicSession(session); } } public class DynamicSession : DynamicObject { private readonly ISession session; public DynamicSession(ISession session) { this.session = session; } public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result) { result = null; if (!binder.Name.StartsWith("Get")) return false; var tokens = Regex.Replace(binder.Name, "([A-Z])", " $1").Split(' ').Skip(1).ToArray(); if (tokens.Length < 4 || args.Length < 1) return false; // some parameter is missing var typeName = tokens[1]; var type = session.SessionFactory.GetAllClassMetadata() .Cast<DictionaryEntry>() .Select(e => e.Key).Cast<Type>() .FirstOrDefault(t => t.Name == typeName); if (type == null) throw new ApplicationException(string.Format("Type '{0}' is not mapped in NHibernate", typeName)); var fieldName = tokens[3]; var criteria = DetachedCriteria.For(type).Add(Restrictions.Eq(fieldName, args[0])); result = criteria.GetExecutableCriteria(session).UniqueResult(); return true; } } }
BTW: I just can't stand WPF's font rendering, it's so blurry! I don't care if it uses Ideal Width or Compatible Width or whatever you want to call it, it just looks very bad. Please vote for this issue so they fix it as soon as possible!
Welcome to another chapter of "Abusing Using"! IDisposable has been used or abused (depending on who you ask) to create scopes/contexts of all sorts, from database testing to HTML generation to clipboard overriding. Here's another little (ab)use: thread priority setting, for all those background tasks that eat our CPUs:
public static class ThreadSettings { private class DisposableThreadPriority: IDisposable { private readonly ThreadPriority originalPriority; public DisposableThreadPriority(ThreadPriority p) { originalPriority = Thread.CurrentThread.Priority; Thread.CurrentThread.Priority = p; } public void Dispose() { Thread.CurrentThread.Priority = originalPriority; } } public static IDisposable LowestPriority { get { return new DisposableThreadPriority(ThreadPriority.Lowest); } } }
I like to keep this in an interceptor so I can easily apply it to different components just by adding a couple of lines in my web.config:
public class LowPriorityInterceptor : IInterceptor { public void Intercept(IInvocation invocation) { using (ThreadSettings.LowestPriority) { invocation.Proceed(); } } }
