I’ve been using Bamboo CI Server for the last few months to automate builds and deployments. I like the tool because it has good integration with Jira (both tools are from Atlassian), it’s easy enough to configure new builds and deployments, triggers, notifications, etc.
But I realised that something important was missing: Bamboo allows you to add a test runner task in a build project but not in a deployment project! This means that you can’t run tests after a successful deployment (smoke tests, integration tests, …), at least not without a workaround.
The trick is to configure your test runner as an executable in Bamboo. These are the steps in order to configure NUnit and run tests in a deployment project (it should work for any other test runner):
1. Add a new executable for NUnit
Go to Bamboo Administration and click on “Executables” on the left panel.
Click on “add an executable as a server capability”
Add the path to NUnit Console and a label for the new executable. It is important to set the type to “Command” in order to use it in a Deployment project:
Click on the “Add” button to save the new command.
2. Add a new deployment task to run the tests
You can either add a new task for the tests to an existing deployment or add a new deployment project that will only run the tests.
I decided to add a new deployment project that will be triggered after a successful deployment because it’s easier to understand if there is actually a problem with the deployment itself or if the integration tests are failing. Also, this way I am able to run the tests at any time without having to deploy the application.
Whatever your choice is, add a new “Command” task to the deployment project:
In the “Executable” dropdown you should be able to find the command you configured for NUnit. Add arguments and environment variables if necessary:
Save the task and run the deployment. This is an excerpt of the generated log that contains the test results:
NUnit-Console version 2.6.4.14350
Copyright (C) 2002-2012 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Microsoft Windows NT 6.2.9200.0
CLR Version: 2.0.50727.8009 ( Net 3.5 )
ProcessModel: Default DomainUsage: Default
Execution Runtime: net-4.0
..F.F.F.F
Tests run: 5, Errors: 0, Failures: 4, Inconclusive: 0, Time: 6.8491962 seconds
Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
Errors and Failures:
1) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Home.aspx")
Expected: True
But was: False
2) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Services/Activate.aspx")
Expected: True
But was: False
3) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Administration/LostPassword.aspx")
Expected: True
But was: False
4) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Shop/Product/List.aspx")
Expected: True
But was: False
Failing task since return code of [C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe integration-tests-uat.nunit --config="release"] was 4 while expected 0
Finished task 'Run integration tests' with result: Failed
Finalising the build...
Stopping timer.
Build 12484609-16973828-16613398 completed.
Finished processing deployment result Deployment of 'release-16' on 'UAT - Integration Tests'
That’s it! The output is not nicely formatted as in the build tasks but it does the job – you can see how many tests were run and how many have failed (if any).