Tuesday, October 20, 2009

A look at test planning with Microsoft Test and Lab Manager. Part 5,Test Impact Analysis

In part five of my series on using Microsoft Test and Lab Manager we are going to take a look at test impact analysis.

I find the test impact analysis feature to be one of the most exciting new features of VSTS.  As you read through this blog entry keep in mind this is based on beta 1 (beta 2 was released to MSDN subscribers)

From a high level, following are the steps we need to take to enable test impact analysis.

• Add the test project to source control.

• Create the build definition.

• Make sure “Analyze Test Impacts” in the Process tab is set to “True”.

• Queue a new build.

• Make sure the “Test Impact Collector” option is set in test settings file.

• Run the tests in the test project (this collects the test impact data)

• Change the code.

• Run a new build. (the test impact analysis runs)

• Test manager – Assign the new build

• Test Manager - Track/Recommended tasks

Here is a very good walk through: http://blogs.msdn.com/densto/archive/2009/06/29/test-impact-walk-through.aspx

Once we get things lined up there are a couple places we can see the recommended tests:

1) In the test and lab manager we can see the impacted tests in the Testing Center area - Track tab under the recommended tests section.  By comparing the build in use to the previous build, VSTS recommends what tests to run based on the code changes.

image

2) In Visual Studio the developer can use the test impact view to get a list of tests that will be impacted by a code change before the change is checked in.  The idea here is rather than running all unit tests, the developer can run a subset (the impacted tests) of the tests and be confident that the change is safe.

image

3) The build report will include an interactive list of tests that had been impacted by the code changes in the build.  Anybody that gets the build report can get a list of all tests impacted by the changes in the build.

image

The new test impact analysis is a must use feature.  As a former development manager I can’t tell you how many times I was forced to make a decision about how to hot fix a bug while minimizing the possibility of braking anything and all I had to go on was a gut feel and experience.  This new feature gives the developer a way to fully understand the potential impact of a change before making it.  It also gives a clear set of tests to perform to insure that the released code is of the highest quality possible.

No comments:

Post a Comment