I’ve been using Bamboo CI Server for the last few months to automate builds and deployments. I like the tool because it has good integration with Jira (both tools are from Atlassian), it’s easy enough to configure new builds and deployments, triggers, notifications, etc.
But I realised that something important was missing: Bamboo allows you to add a test runner task in a build project but not in a deployment project! This means that you can’t run tests after a successful deployment (smoke tests, integration tests, …), at least not without a workaround.
The trick is to configure your test runner as an executable in Bamboo. These are the steps in order to configure NUnit and run tests in a deployment project (it should work for any other test runner):
1. Add a new executable for NUnit
Go to Bamboo Administration and click on “Executables” on the left panel.
Click on “add an executable as a server capability”
Add the path to NUnit Console and a label for the new executable. It is important to set the type to “Command” in order to use it in a Deployment project:
Click on the “Add” button to save the new command.
2. Add a new deployment task to run the tests
You can either add a new task for the tests to an existing deployment or add a new deployment project that will only run the tests.
I decided to add a new deployment project that will be triggered after a successful deployment because it’s easier to understand if there is actually a problem with the deployment itself or if the integration tests are failing. Also, this way I am able to run the tests at any time without having to deploy the application.
Whatever your choice is, add a new “Command” task to the deployment project:
In the “Executable” dropdown you should be able to find the command you configured for NUnit. Add arguments and environment variables if necessary:
Save the task and run the deployment. This is an excerpt of the generated log that contains the test results:
NUnit-Console version 2.6.4.14350 Copyright (C) 2002-2012 Charlie Poole. Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov. Copyright (C) 2000-2002 Philip Craig. All Rights Reserved. Runtime Environment - OS Version: Microsoft Windows NT 6.2.9200.0 CLR Version: 2.0.50727.8009 ( Net 3.5 ) ProcessModel: Default DomainUsage: Default Execution Runtime: net-4.0 ..F.F.F.F Tests run: 5, Errors: 0, Failures: 4, Inconclusive: 0, Time: 6.8491962 seconds Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0 Errors and Failures: 1) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Home.aspx") Expected: True But was: False 2) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Services/Activate.aspx") Expected: True But was: False 3) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Administration/LostPassword.aspx") Expected: True But was: False 4) Test Failure : GivenAnUrl_WhenGettingPage_ShouldreturnSuccessStatusCode("/Shop/Product/List.aspx") Expected: True But was: False Failing task since return code of [C:\Program Files (x86)\NUnit 2.6.4\bin\nunit-console.exe integration-tests-uat.nunit --config="release"] was 4 while expected 0 Finished task 'Run integration tests' with result: Failed Finalising the build... Stopping timer. Build 12484609-16973828-16613398 completed. Finished processing deployment result Deployment of 'release-16' on 'UAT - Integration Tests'
That’s it! The output is not nicely formatted as in the build tasks but it does the job – you can see how many tests were run and how many have failed (if any).
How do you add a trigger after a deployment? Since when in Add Trigger screen, it’s from the build plan and set a trigger to deploy, I only see 3 different ways to trigger a deployment, one is after successful build plan, 2 after successful stage, and 3 scheduled. How do you set trigger after a deployment is complete?
Hi Dk,
I’m afraid I don’t remember how to do it, I stopped using Bamboo a long time ago (I now use Visual Studio Team Services). Sorry about that!
Regards,
Rui
Thanks man, really appreciate your quick response on this. Yes, this wasn’t an easy find even on the web and finally found it. They just added this at the beginning of this year 2017, called the “After Deployment Trigger” as an “add-on” which can be found here:
https://marketplace.atlassian.com/plugins/com.atlassianlab.bamboo.plugins.bamboo-after-deployment-trigger-plugin/server/installation
So maybe you did yours with some work-around or used a scheduled trigger.
Thanks.
I have a question that is prior to this tutorial. How do I build the executable? I have my tests written in VS as a test project. Do I need to somehow add a Console Application to my solution that can then execute the tests? Could you point me in the right direction about how I should correctly do this? Thank you!
Hi Jeremy,
I’m not sure if I understand your question. Your unit tests should be in a class library project, and then you can use Visual Studio Test Runner or a tool like Resharper to run your unit tests from Visual Studio. This might help: How to: Run Tests from Microsoft Visual Studio
The answer to this was NUnit Console Runner. Leave the tests as a testing project, then call the compiled testing dll using the console runner.
Cucumber for Bamboo plugin (available in Atlassian marketplace) allows you add Cucumber Parser tasks in Build as well as Deployment plans.
you can run your integration test with maven and add Cucumber task to parse the tests. It also gives options to integrate reaults with JIRA and provides multiple options to fail or pass the deployment depending on tests results.
https://marketplace.atlassian.com/plugins/com.mdb.plugins.cucumberforbamboo/server/overview
Thanks for this. I would like to know how to make a deployment project to deploy on other environments once it was found ok on a test environment. Is there a way to configure the environment to which to deploy?
Hi Ahmad,
This might be useful to you:
https://confluence.atlassian.com/bamboo/triggers-for-deployment-environments-342098622.html
Regards,
Rui
Hey, I managed to make it work in bamboo cloud as well. Do u have a way to parse the results that Nunit creates like explained above in the deployment project? since deployment project also doesnt have Nunit_parser task inbuilt?
Hi Sundeep,
Unfortunately I wasn’t able to parse the NUnit test results in the deployment project.
Rui
BTW., I forgot to tell you that I am using Bamboo cloud and not standalone Bamboo. So, will this workaround work for Bamboo cloud as well? If so, are there any other steps that I need to follow?
Regards,
Sundeep.
Hi Sundeep,
I never used the cloud version of Bamboo but I think it works the same way as the other version.
Rui
Hi, this seems to be exactly what I was looking for. I wanted to run a set of tests after the deployment was successful. So, I wanted to trigger this from the deployment project rather than from the build task. Problem is, I dont see a link to executables in Bamboo administration page. Why is it? Am I lacking any permission for not seeing that? Also, is there any way to report the results by email in a better formatted way similar to build plan result? I’d have to send it to my manager.
Thanks,
Sundeep.
Hi Sundeep,
I think you need Admin permission to see the Executables on the Administration page.
Rui