Wednesday, May 2, 2012

Moroco: a minimal mocking library for C# / VB.NET

The more I learn about functional programming, the more I come to question many widely used and accepted practices in mainstream programming.

This time it's the turn of mocks and mocking libraries.

First, since there are so many different definitions for stubs, mocks, fakes, etc, here's my own definition of a mock: an entity (in an object-oriented language, usually an object) used to test an interaction between the entity under test and an external entity (again, in OO languages, these entities are objects).

So mocks are used for interaction-based testing which means testing for side-effects. The original paper on mock objects says this explicitly: "Test code should communicate its intent as simply and clearly as possible. This can be difficult if a test has to set up domain state or the domain code causes side effects."

Side effects are code smells, or more precisely, they should be few and isolated. Even the creators of mock object say that side effects make testing difficult! We should minimize the need for mocking. Code without side-effect doesn't need mocks. You still may need stubs or fakes, but those should be trivial to build.

Quoting Daniel Cazzulino (author of Moq): "The sole presence of a 'Verify' method on the mock is a smell to me, one that will slowly get you into testing the interactions as opposed to testing the observable state changes caused by a particular behavior.". Make those states immutable (i.e. a state change create a new state) and you're half-way to side-effect-free code.

Remember that discussion a few years ago where people said that Typemock was too powerful? The argument was that Typemock "doesn't force you to write testable code". What this really means is "it doesn't make you isolate side-effects". I'm thinking that all current mocking libraries are actually too powerful: instead of making you think how to write pure (side-effect free) code, it encourages you to just use a mock to replace your impure code with pure code in tests.

There's also the matter of library complexity. .NET mocking libraries are big and complex: Moq: 17000 LoC, NSubstitute: 12000 LoC, Rhino Mocks: 87000 LoC, FakeItEasy: 17000 LoC. Not counting the embedded runtime proxy library (usually Castle DynamicProxy).

Many have issues running mocks in parallel (1, 2, 3, 4). It's 2012 and most developers have at least a 4-core workstation, using a mock library that limits my ability to run tests in parallel is getting ridiculous.

In summary, I think mock libraries support an undesirable practice, are not worth their code and have to go, except for very specific scenarios. But I still have a lot of existing side-effecting code I have to test, I can't just wish it away. Refactoring to pure code is not trivial. So I decided that

To which I got an encouraging reply:

By the way I'm not the first or the only one that thinks that mocking libraries aren't worth it. Uncle Bob also prefers manual mocking (even in Java, which is much more verbose than any .NET language), though he mostly stresses the argument of simplicity.

In F# it's easy to do manual mocking thanks to object expressions (1, 2), so you don't have to actually create a new class for each mock. In C#/VB.NET we're not so lucky but we can get 80% there with a little boilerplate, making a "semi-manual", reusable mock class with settable Funcs to define behavior. Example:

interface ISomething {
    void DoSomething(int a);
}

class MockSomething: ISomething {
    public Action<int> doSomething;

    public void DoSomething(int a) {
        doSomething(a);
    }
}

class Test {
    void test() {
        var r = new List<int>();
        var s = new MockSomething {
            doSomething = r.Add
        };
        // etc
    }
}

This isn't anything new, people have been doing this for years. Downsides: doesn't play well with overloaded and generic methods, but still works.

Also, as Uncle Bob explains, manual mocks are prone to break whenever you change the mocked interface, but if you hit this often it could be revealing you that perhaps you should have used an abstract base class instead.

I added some code to track the call count of a Func and named the resulting library Moroco. So I wanted to get away from mocking libraries and ended up writing one, talk about hypocrisy! The difference between Moroco and other mocking libraries is that it's really minimal: less than 400 lines of pretty trivial code, fitting in a single file, with no dependencies. And I'm still against mocks: I get to count mocks to measure mock smell. And run tests in parallel.

I'm already using it in SolrNet, where I simply dropped Moroco's source code in the test project and replaced Rhino.Mocks in all tests. Here's the 'before vs after' of one of the tests:

Rhino.Mocks

[Test]
public void Extract() {
    var mocks = new MockRepository();
    var connection = mocks.StrictMock<ISolrConnection>();
    var extractResponseParser = mocks.StrictMock<ISolrExtractResponseParser>();
    var docSerializer = new SolrDocumentSerializer<TestDocumentWithoutUniqueKey>(new AttributesMappingManager(), new DefaultFieldSerializer());
    var parameters = new ExtractParameters(null, "1", "test.doc");
    With.Mocks(mocks)
        .Expecting(() => {
            Expect.On(connection)
                .Call(connection.PostStream("/update/extract", null, parameters.Content, new List<KeyValuePair<string, string>> {
                    new KeyValuePair<string, string>("literal.id", parameters.Id),
                    new KeyValuePair<string, string>("resource.name", parameters.ResourceName),
                }))
                .Repeat.Once()
                .Return(EmbeddedResource.GetEmbeddedString(GetType(), "Resources.responseWithExtractContent.xml"));
            Expect.On(extractResponseParser)
                .Call(extractResponseParser.Parse(null))
                .IgnoreArguments()
                .Return(new ExtractResponse(null));
        })
        .Verify(() => {
            var ops = new SolrBasicServer<TestDocumentWithoutUniqueKey>(connection, null, docSerializer, null, null, null, null, extractResponseParser);
            ops.Extract(parameters);
        });
}


Moroco

[Test]
public void Extract() {
    var parameters = new ExtractParameters(null, "1", "test.doc");
    var connection = new MSolrConnection();
    connection.postStream += (url, contentType, content, param) => {
        Assert.AreEqual("/update/extract", url);
        Assert.AreEqual(parameters.Content, content);
        var expectedParams = new[] {
            KV.Create("literal.id", parameters.Id),
            KV.Create("resource.name", parameters.ResourceName),
        };
        Assert.AreElementsEqualIgnoringOrder(expectedParams, param);
        return EmbeddedResource.GetEmbeddedString(GetType(), "Resources.responseWithExtractContent.xml");
    };
    var docSerializer = new SolrDocumentSerializer<TestDocumentWithoutUniqueKey>(new AttributesMappingManager(), new DefaultFieldSerializer());
    var extractResponseParser = new MSolrExtractResponseParser {
        parse = _ => new ExtractResponse(null)
    };
    var ops = new SolrBasicServer<TestDocumentWithoutUniqueKey>(connection, null, docSerializer, null, null, null, null, extractResponseParser);
    ops.Extract(parameters);
    Assert.AreEqual(1, connection.postStream.Calls);
}

Yes, I do realize this uses an old Rhino.Mocks API, but it's necessary to get thread-safety.

Conclusion

No, I don't expect anyone to use Moroco instead of Moq, Rhino.Mocks, etc. Yes, I know this means more code (though it's not as much as you might think), and I agree that .NET needs another mock library like I need a hole in my head. But I think we should think twice before using a mock and see if we can find a side-effect-free alternative for some piece of code.
Even when you do use mocks, don't just blindly reach for a mocking library. Consider the trade-offs.

4 comments:

Daniel Yokomizo said...

I never got mock objects and used to be a heavy user of manual stubs for testing. Even after I tried to use mocks primarily for a couple of years I failed to see them improving my tests. But then I decided to read the GOOS book, from the creators of Mock Objects and suddenly understood how we are supposed to use them.

They have an excellent approach to design that uses mocks to expose collaborators. Most of their tests are small with very few mock calls and almost never mock third party interfaces/classes. The book is very good and essential to understand mock objects IMHO.

It's also interesting to notice that they value functional objects (i.e. side-effect free and immutable) and never mock those.

Paige Cook said...

Nice article. However, I am not see the after version of the test using Moroco. I can see it in the source of the page, but it is not displaying in my browser (tried with both Chrome and Firefox).

Mauricio Scheffer said...

@Daniel : thanks for the pointer, I heard many good comments about that book, will have to check it out!

Mauricio Scheffer said...

@Paige: I just fixed it, thanks!