Team Foundation Server - Sharing Binaries and Class Libraries Across Multiple Projects

Update 16 July 2008 - this post has been receiving an increased amount of pageviews in recent months, which is nice, but I should point out that it no longer reflects my thinking on how best to structure a TFS source tree. Having lived with the structure described below for several months, and a few project iterations, it became apparent that having only one (shared) copy of common class libraries sometimes made it impossible to build bugfix releases of applications if the common libraries on which they depended had subsequently changed. I'm now sold on the branch and merge philosophy.and heartily recommend reading Eric Sink's excellent Source Control HOWTO guide for more information about SCC best practices.

That caveat aside, the post below may still contain some useful information if you really want to do this, or are struggling with some similar MSBuild-related scenario!

I returned to work after Tuesday's MSDN Roadshow fired up and full of ideas, eager to try out ASP.NET AJAX in anger, and start playing with LINQ on my VS Orcas VPC.  But before all that, I was determined to improve our source control practices and end the branching madness which has become the one downside of using Visual Studio Team Foundation Server.

Like any good development team, we don't believe in reinventing the wheel or duplicating code.  So naturally we have a set of shared class libraries which contain business logic and object models which are common to multiple applications.  Also, we make use of the MS Enterprise Library, and other third party assemblies such as Wintellect PowerCollections.

Now, what I wanted to achieve was this:

1. Share our common code across multiple (ASP.NET) applications, without resorting to branching between TFS projects (which has in the past led to multiple versions of the shared codebase getting out of sync with each other).
2. Also share the third-party DLLs, again without branching.
3. Create Team Builds for each application which will automatically get the latest versions of the shared class libraries and third-party assemblies.

That doesn't sound like a lot to ask for, does it?  But it was surprisingly difficult to achieve - in fact it took me eighty-one attempted builds before I got the desired effect!  So, before I forget how I achieved it, and in case there's anyone else out there trying to do the same thing, let me explain all...

The first thing to do is sort out your source control tree structure, and ensure that all developers in the team are mapping onto their local workspace in the same way (otherwise you'll get into problems with different relative paths to references).
We have a TFS Team Project for each application, then place the solution within a "trunk" folder - this makes it much easier to branch the application at a later date (for example, when you want to start developing a major new version whilst still making smaller bugfix releases to the current live version).
Then, we have a separate Team Project called "Shared Resources", within which is a folder for our class libraries, and another folder for third-party binaries.
So, our tree looks something like this:

$
|-WebApp1
| |-Trunk
| |-VersionX
|-SharedResources
  |-Binaries
  | |-Microsoft
  | | |-EnterpriseLibrary
  | |-Wintellect
  |   |-PowerCollections
  |-ClassLibraries
    |-CommonUtilities
    |-CommonBusinessLogic

You get the idea.  This step was straightforward enough.

The next trick is to use Web Application Projects rather than web sites.  This alternative web project model (which is similar to the model used in VS2003) was released as an optional add-on to VS2005 shortly after its release, and has been baked into VS2005 Service Pack 1, so if all developers on the team are fully patched, there should be no problems with using this project model.  It's required because in this model, all assembly references are defined within a project file (no such project file exists in the standard VS2005 web project model).  This allows us to define a reference to shared third-party binaries which will be recognised by MSBuild.

Scott Guthrie has written a tutorial on upgrading VS2005 Web Site Projects to be VS2005 Web Application Projects, which may come in useful at this point.  It's pretty straightforward, if a little tedious (especially if your existing site has many pages and controls, which you wish to wrap in namespaces).

The suite of shared class libraries should be given its own solution.  For each class library project, add a post-build event to copy its output to a \binaries folder (thanks to this blog post from Vertigo Software for highlighting this crucial step).  The command line required is:

xcopy /Y /S /F "$(TargetPath)" "$(SolutionDir)binaries\"

Our web application projects should then reference these compiled binaries as required, with CopyLocal set to true.  This is vital for the Team Build to work - if project references are used for assemblies belonging to different team projects you'll have no joy.  The downside of this for the development team is that they have to Get Latest Version and recompile the shared class libraries periodically to pick up any changes made by coworkers.  But I think that is more straightforward than if the shared libraries had been branched (so periodic merging would be necessary), and at least with this set-up the automated builds (which we are now in a position to create) will perform a full integration and flag up any issues.  It's just one more good reason to set up scheduled builds :-)

So, at this point I've achieved goals (1) and (2) - eliminated branching of shared class libraries and third party assemblies, and everything is building just fine on the desktop.  Now comes the tricky bit - getting it to work on the build server through MSBuild.  In the dying years of the last century I spent my days coding Assembler on IBM mainframes, and I have to say that the trial-and-error of MSBuild reminds me somewhat of creating JCL (Job Control Langauge) scripts in those dark days!  Fortunately I had this excellent article by Manish Agarwal on which to base my build - he explains in great detail how to persuade MSBuild to first compile the shared assemblies, then the client application, and I encourage you to go check out his post and the other tips on his site.  The only additional step I needed to add was a Get of the third-party binaries.

To cut a long story short, here's how my override of the "BeforeGet" target ended up looking: 

<!-- This task executes prior to Getting the sources -->
  <Target Name="BeforeGet">
    
    <!-- BEGIN GETTING COMPILED THIRD-PARTY BINARIES -->
    <!-- delete default workspace -->
    <DeleteWorkspaceTask
    TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
    Name="$(WorkspaceName)" />
    
    <!-- delete any temporary workspace from last time -->
    <DeleteWorkspaceTask
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
Name="$(WorkspaceName) SR" />
    <!-- make some directories-->
    <Exec WorkingDirectory="$(SolutionRoot)" Command="mkdir 
          SharedResources" />
    <Exec WorkingDirectory="$(SolutionRoot)/SharedResources" 
          Command="mkdir Binaries" />
    <!-- make a temporary workspace -->
    <Exec 
      WorkingDirectory="$(SolutionRoot)/SharedResources/Binaries"
      Command=""$(TfCommand)" workspace /
        new "$(WorkspaceName) SR" 
        /server:$(TeamFoundationServerUrl)" />
   
    <!-- map the workspace to our local folder -->
    <Exec
      WorkingDirectory="$(SolutionRoot)"
      Command=""$(TfCommand)" workfold 
        /map "/workspace:$(WorkSpaceName) SR" 
        /server:$(TeamFoundationServerUrl) 
        "$/SharedResources/Binaries" 
        "$(SolutionRoot)\SharedResources\Binaries""/>
    <!-- get the binaries -->
    <Exec 
      WorkingDirectory="$(SolutionRoot)/SharedResources/Binaries"
      Command=""$(TfCommand)" get " />
    <!-- be tidy and delete the temporary workspace -->
    <DeleteWorkspaceTask
TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
Name="$(WorkspaceName) SR" />
    <!-- END GETTING COMPILED THIRD-PARTY BINARIES -->
    
    
    <!-- BEGIN MAPPING WORKSPACES FOR EACH TEAM PROJECT -->
    <!-- delete default workspace -->
    <DeleteWorkspaceTask
    TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
    Name="$(WorkspaceName)" />
    <!-- create new workspace -->
    <Exec
      WorkingDirectory="$(SolutionRoot)"
      Command=""$(TfCommand)" workspace 
        /new "$(WorkspaceName)" 
        /server:$(TeamFoundationServerUrl)"/>
    <!-- remove default mapping -->
    <Exec
     WorkingDirectory="$(SolutionRoot)"
     Command=""$(TfCommand)" workfold 
       /unmap "/workspace:$(WorkSpaceName)" 
       "$(SolutionRoot)""/>
    <!-- add desired mapping (see itemgroup maps below) -->
    <Exec
      WorkingDirectory="$(SolutionRoot)"
      Command=""$(TfCommand)" workfold 
        /map "/workspace:$(WorkSpaceName)" 
        /server:$(TeamFoundationServerUrl) 
        "%(Map.Identity)" "%(Map.LocalPath)""/>
    <!-- END MAPPING WORKSPACES FOR EACH TEAM PROJECT -->
  </Target>

Whilst my folder mappings and SolutionsToBuild item groups look like this:

<!-- herewith all the solutions we want compiling - shared libs first -->
  <ItemGroup>
    <SolutionToBuild 
      Include="$(SolutionRoot)\SharedResources\ClassLibraries\Libs.sln" />
    <SolutionToBuild 
      Include="$(SolutionRoot)\WebApp1\WebApp1.sln" />
  </ItemGroup>

  <!-- Describes a mapping between team projects and local folders -->
  <ItemGroup>
    <Map Include="$/SharedResources/ClassLibraries/">
      <LocalPath>$(SolutionRoot)\SharedResources\ClassLibraries</LocalPath>
    </Map>
    <Map Include="$/WebApp1">
      <LocalPath>$(SolutionRoot)\WebApp1</LocalPath>
    </Map>
  </ItemGroup>

Voila!  After much hair-pulling and cursing, I had achieved my three goals and facilitated truly shared code and binaries across multiple TFS team projects, with automated Team Builds for all applications.  As a result our solutions open and compile faster (because they don't include the shared libraries), our overnight application builds always include the latest copy of shared libraries (and hence highlight any integration issues), and most importantly we don't have to mess about with periodic merging of branched libraries.

While I'm on the subject of Team Foundation Server, here's an incredibly useful download from Noah Coad - a Team System add-in to allow searching of work items - very useful if you have as many outstanding tasks as I currently do :-)