Queries

Mar 28, 2011 at 5:31 PM
Edited Mar 29, 2011 at 10:10 AM

I have a build in TFS 2008 that gets triggered for every check-in, and what I need is following

  • If the source code checkedin by the developers has any errors, then the code changes should not be checked in
  • If any of the tests fails in the test project, then the code changes should not be checked in
  • If the build script fails for any reason then then the code changes should not be checked in

I have the following queries:

  • Does the Buddybuild solves this problem?
  • I run some unit tests as part of my build script (standard way), however when the unit tests fails, the buddy build still checkin the code ? How can i make sure that the checkins are not applied when the test fails or if the build fails for any reason? Is there a variable that I can set to make sure that the buddy build does not checkin the code? After looking at the buddy build targets file, it seems like setting $(CheckInOnBuddyBuildSuccess) to false would do it ? Is this the correct approach? 

Awaiting
Nabeel Farid

Coordinator
Mar 29, 2011 at 3:52 PM

Hi Nabeel,

Regarding the scenario you are describing, when you run tests as part of Team Build 2008, the default behavior upon failure of the tests is to mark the build as partially failed, not fully failed. When the build gets to the gated checkin step, it will check in unless the build is "fully" failed. This behavior is by design and depends on the behavior of Team Build 2008. In order for your scenario to work as you desire it to work, you will need to fail the build upon failure of the tests. Aaron Hallberg describes this in this blog post: http://blogs.msdn.com/b/aaronhallberg/archive/2007/11/05/how-to-fail-a-build-when-tests-fail.aspx.

As for the bevaior of the build not showing up in Build Explorer when triggered through the checkin validation tool, this is a limitation of the current public version of the tool. Unfortunately, you will need to refresh the Queued tab in Build Explorer manually to see the build show up in there. I have made this change in a non-public version and if it helps, I will port it soon to the public version of the tool.

Hope this helps.

Mar 29, 2011 at 7:24 PM

Thanks for the response. Just so that you know I am actually making the build completely fail (not partially failed) on any of the tests failure. The way I am doing this is setting up the 

<TreatTestFailureAsBuildFailure>true</TreatTestFailureAsBuildFailure>

This is TFS 2008 SP1 feature and I am assuming this should bypass/fail gated checkin step? However the buddy build still checks in the code although the build gets failed.

I have a couple of more queries:

  • Firstly, I have come across a problem. Lets say I checked out a file in my visual studio solution, made some changes to it and then ran the buddy build, which in turn checks-in the file on successful build. At this stage, when I looked at my file in the solution in visual studio solution explorer/tfs source control explorer, the file still appears to be checked out. And if i make further changes to it and re-run the buddy build, then it comes up with the policy errors that "the file is in conflict state"? I do not understand why would it say that?  And if I further force the build to run, it gets failed on "Getting Source" step and after that it goes into a limbo state and keeps getting failed at the same "Getting Source" step no matter I do get latest or revert the file back to the previous version. I end up creating a brand new build definition. On the other hand, if I do get latest on the solution, before making any changes to the file, then the buddy build does not comes up with any conflict error and runs fine. But i don't see any point of doing get latest after running buddy build each time.
  • Secondly if I want my developers to use gated checkin feature via buddy build, then Does it mean they should never use the standard tfs checkin feature (right clicking the file and check in the file)? Because that would put the changes in the source control without any validation?

 

Awaiting

Nabeel

Coordinator
Mar 31, 2011 at 8:34 AM
Edited Mar 31, 2011 at 8:36 AM


Sorry for the delayed reply.

So, I was able to perform some testing to figure out why breaking the build upon test failure was not preventing the checkin from occurring, and I found out the following:

  • When tests fail during a build, Team Build 2008 sets the TestSuccess property to false. However, the Buddy Build infrastructure, i.e. the BuddyBuildExtensions.targets script, does not use that property. It only evaluates the BuildBreak property, which is the one set to true by Team Build upon failure in compilation. Now rather than going and changing BuddyBuildExtensions.targets so that it takes into consideration the TestSuccess property, I suggest doing the following:
    • Override the AfterTestConfiguration target that Team Build 2008 furnishes for you, like so:

  <Target Name="AfterTestConfiguration">

    <PropertyGroup Condition=" '$(TestSuccess)' != 'true' ">
      <BuildBreak>true</BuildBreak>
    </PropertyGroup>

  </Target>

  •  
    • This way you force BuildBreak to be true whenever TestSuccess is not true. 

 As for the other two questions:

  • When you queue a gated checkin using the Buddy Build tool, you are given the option of shelving the changes removing the local changes. If you choose to keep the local changes in your workspace, when the build succeeds and checks in, this will create a new version in TFS for those particular files. Now since you still have pending changes against the files in your workspace, and since your base version is not the latest version in TFS anymore (since there was a checkin), you will need to reconcile the changes. So, when you perform a get, TFS will notice the discrepency and offer you to resolve it using the conflict resolution tool (the merge tool). In this case, if you haven't made any other changes to those files after you've queued the build, then you should take the server changes. If you've made any changes locally after queuing, then you will need to probably merge the changes visually using the merge tool. This is the same case with the built in gated checkin tool when you use TFS 2010. However, in TFS 2010, when the build finishes and checks in, you will be notified and presented with the option of reconciling the changes. As for your scenario, if you force a gated checkin of files in that state without resolving the conflicts in your workspace, then the build will run into the same issue. Now since the gated checkin build is designed so that it fails when it cannot resolve changes automatically (since it assumes they need manual intervention to resolve them), then it will fail the build. Now it seems from your description that it fails before that, during getting the source files. In that case, I am not sure what could be the issue unless you can share the error message(s) you see in the build log. You should not though have to create a new build definition to fix this issue. If your build is recreating the workspace whenever a build is triggered, this should fix any workspace corruption that might have happened. If not, i.e. if you are reusing the workspace between builds (like in a CI scenario where you do incremental gets), then that can cause issues. In fact, you should try and avoid running gated checkins against CI build definitions or ones that perform incremental gets or incremental builds.
  • For your second question, the answer is: it depends. Gated checkins are not suitable for each and every scenario. For example, if your build takes more than 20-30 minutes, then gated checkins can probably hurt you more than they would help you. Typically, you would want to restrict the usage of gated checkins to build definitions that are short and sweet. Another scenario where they can be useful is for integrations (merges) from one branch to another, where for example you don't want the merged files committed unless they pass your test suite. In that case, gated checkins are ideal.