.net, Unit Testing, Visual Studio, Visual Studio Plugin, xUnit

VS2013 Extension (#2) – Xunit Test Runner (plus a few issues)

Normally I use NUnit to create my unit tests.  But after looking through the ASP.net core source code for vNext, I noticed that Microsoft are now using Xunit as their unit testing tool. I’d like to follow Microsoft coding standards and practices as much as possible, so I decided to try out a move from NUnit to Xunit.

Changing the Syntax

First, the [TestFixture] decoration is no longer needed – you don’t need to decorate your test classes. This is pretty nice – I can’t think of a good reason why I should be sorry this isn’t in Xunit.

But now you don’t use [Test] as the decoration for your unit tests – these are now decorated with [Fact]. This blog gives a good explanation of the arguments for and against the revised terminology – it comes down firmly on the side of staying with [Test], but it does present the Xunit team’s reasons for the change as well.

I don’t tend to use [Setup] or [Teardown], which is lucky, because they’re not in Xunit. There are few options if you really need an equivalent – the constructor to set things up, IDispoable implementation to tear things down, and the IUseTestFixture interface which allows you to pass in a data from a your custom type and process it.

Finally, the Assert method names have changed slightly. So, for example, rather than IsTrue from NUnit, the Xunit method is just True. You can read a full list of changes here, and most of the changes are pretty intuitive.

But I did find a few wrinkles in the transition – this might be issues with my particular set up, but I’ll document them in case it helps anyone else.

Running Xunit Tests through VS2013 Test Explorer

Installing Xunit using Nuget was simple – however, I wasn’t able to run the tests out of the box in VS2013. After some searching, I found this link which correctly told me I needed to use a test runner extension. But the details on that site are now out of date so don’t install the VS2013 extension.

The preferred solution is to install the Xunit runner as a Nuget package.  I found another issue here – when I tried to install this through the Nuget tool in Visual Studio, nothing (observable) happened. I certainly didn’t see any of my unit tests appearing in the built-in VS2013 Test Explorer.  Eventually the steps I took to get this to work were:

1. Make sure that the tests are in a project which has been created using the built in “Unit Test Project” template.

2. Install the runner through the Nuget Package Manager Console, with the command:

Install-Package xunit.runner.visualstudio -Pre

After doing these things, I saw my Xunit tests appearing in the VS2013 Text Explorer.

In case the package comes out of pre-release mode, here’s the direct link to Nuget so you can get the most recent information.

Running Xunit Tests through JetBrains dotCover v2.6

I’ve got dotCover v2.6 installed on my development machine, and it can’t see any Xunit tests. Ok – this version is a bit old now, but this is a big problem for me and really makes me want to go back to NUnit or MSTest. I’ve read some posts on xunitcontrib, but this is only up to v2.5 of dotCover. I’ve also read some posts saying that Xunit support comes with an extension from the Nuget gallery – but presently the gallery URL which is supplied with dotCover v2.6 is returning a 404 error.

So presently this is not working for me.

[Update: After writing this post, I contacted JetBrains and they got right back to me to say they had addressed the 404 issue from above. So now I can download the Xunit extension to allow dotCover 2.6 see my Xunit tests.]

(I should say I’ve also tried it with dotCover v3 and it works fine out of the box).

Final Thoughts

I don’t have any compelling reason to switch to Xunit right now. I agree with the Xunit team’s reasons for not implementing [Setup]/[Teardown], but I’m able to replicate that omission in NUnit by just not using that feature.

If I need/choose to switch to Xunit at some point, I think it’ll be trivial to change.

Primarily the reason why I’m staying with NUnit is economic – I’m licensed to use dotCover v2.6 and NUnit just works. I don’t want to pay for an expensive upgrade (just yet anyway, and ReSharper Ultimate looks great).

[Update – as mentioned above, with a little help from JetBrains, I’ve managed to get dotCover v2.6 to work with Xunit. So this has reduced a major barrier for me to the move across to Xunit.

So my updated conclusion is that for new projects, I will give Xunit another try, but I’m not going to retrospectively re-factor stable and mature tests written using the NUnit framework.]