The wonder of MSBuild

On Discovering the Machinery of MSBuild SDKs

While working on Chimelisp, my (not so serious) Lisp implementation for .NET, I stumbled upon something that felt like finding a hidden room in a manor you've lived in for years: you somewhat expect it to be there, after all, a ghost has to live somewhere. The discovery was simple in retrospect, almost trivial even, yet it opened a door for something with so much potential.

I had been content with the command-line compiler, invoking it manually, watching IL instructions flow into assemblies. The rhythm was familiar: write code, compile, run. But there was friction, the kind that doesn't announce itself until you've spent enough time with it to notice the absence of what could be.

Then I came across Feersum, a Scheme compiler for .NET. Buried in its source was something remarkable: it had integrated itself into MSBuild as a first-class SDK. Projects could reference Sdk="Feersum.Sdk" and suddenly .scm files compiled as naturally as .cs files. The machinery was there, waiting to be understood.

The Structure of MSBuild SDKs

According to Microsoft's documentation¹, MSBuild 15.0 introduced the concept of project SDKs to simplify how build logic is injected into projects. When you write <Project Sdk="MyCompiler.Sdk">, MSBuild adds implicit imports at evaluation time:

<Project>
  <Import Project="Sdk.props" Sdk="MyCompiler.Sdk" />
  
  <!-- Your project content -->
  
  <Import Project="Sdk.targets" Sdk="MyCompiler.Sdk" />
</Project>

The timing is precise. The Sdk.props file is imported before any project properties are evaluated. The Sdk.targets file is imported after everything else. This ordering allows the SDK to establish defaults that the project can override, then define build logic that operates on the final property values.

The NuGet Package Structure

An MSBuild SDK distributed via NuGet must follow a specific structure². At minimum, it requires:

Chimelisp.Sdk/
└── Sdk/
    ├── Sdk.props
    └── Sdk.targets

Both files are mandatory. Unlike regular NuGet packages that use build/<package-id>.props and build/<package-id>.targets, SDK packages use this distinct Sdk/ folder. The NuGet-based SDK resolver in MSBuild (introduced in MSBuild 15.6) queries configured feeds for packages matching the SDK name and version, then returns the path to extract these files.

The package can contain additional elements:

Chimelisp.Sdk/
├── Sdk/
│   ├── Sdk.props
│   └── Sdk.targets
├── build/
│   └── Chimelisp.Sdk.targets    # Additional build logic
├── tools/
│   └── compiler.exe              # The actual compiler
└── lib/
    └── net8.0/
        └── Chimelisp.Runtime.dll # Runtime support

Implementing Sdk.props

The Sdk.props file establishes the compilation model. Understanding the distinction between PropertyGroups and ItemGroups is essential here; as Dan Siegel explains in his article on SDK projects, "A PropertyGroup, is exactly what it sounds like. It's an area where you can declare Properties (think variable declarations), that will be used in the build process"³. ItemGroups, conversely, define collections of items that require processing; files to compile, packages to reference, resources to embed.

For Chimelisp, this means declaring both properties (configuration variables) and items (what constitutes a compilable file):

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  
  <PropertyGroup>
    <MSBuildAllProjects Condition="'$(MSBuildToolsVersion)' != 'Current'">
      $(MSBuildAllProjects);$(MSBuildThisFileFullPath)
    </MSBuildAllProjects>
    
    <!-- Default language version -->
    <LispLanguageVersion Condition="'$(LispLanguageVersion)' == ''">latest</LispLanguageVersion>
    
    <!-- Ensure we have an output type -->
    <OutputType Condition="'$(OutputType)' == ''">Library</OutputType>
    
    <!-- Disable default compile items; we define our own -->
    <EnableDefaultCompileItems>false</EnableDefaultCompileItems>
  </PropertyGroup>

  <!-- Define item types for Lisp source files -->
  <ItemGroup>
    <LispCompile Include="**/*.lisp" 
                 Exclude="$(DefaultItemExcludes);$(DefaultExcludesInProjectFolder)" />
  </ItemGroup>

  <!-- Reference the runtime library -->
  <ItemGroup>
    <PackageReference Include="Chimelisp.Runtime" 
                      Version="1.0.0" 
                      IsImplicitlyDefined="true" />
  </ItemGroup>

</Project>

The EnableDefaultCompileItems property set to false prevents MSBuild from automatically including **/*.cs files, since we define our own compilation items (**/*.lisp). The DefaultItemExcludes and DefaultExcludesInProjectFolder properties handle the glob pattern exclusions that SDK-style projects expect; bin/, obj/, and other build artifacts⁴.

Implementing Sdk.targets

The Sdk.targets file defines the actual compilation process. This is where the compiler is invoked:

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <PropertyGroup>
    <MSBuildAllProjects Condition="'$(MSBuildToolsVersion)' != 'Current'">
      $(MSBuildAllProjects);$(MSBuildThisFileFullPath)
    </MSBuildAllProjects>
  </PropertyGroup>

  <!-- Locate the compiler -->
  <PropertyGroup>
    <ChimelispCompilerPath Condition="'$(ChimelispCompilerPath)' == ''">
      $(MSBuildThisFileDirectory)../tools/ChimelispCompiler.exe
    </ChimelispCompilerPath>
  </PropertyGroup>

  <!-- Define the core compilation target -->
  <Target Name="CoreCompile"
          DependsOnTargets="$(CoreCompileDependsOn)"
          Inputs="@(LispCompile)"
          Outputs="$(TargetPath)">
    
    <PropertyGroup>
      <_LispResponseFile>$(IntermediateOutputPath)$(TargetName).lisp.rsp</_LispResponseFile>
    </PropertyGroup>

    <!-- Generate response file with all source files -->
    <WriteLinesToFile File="$(_LispResponseFile)"
                      Lines="@(LispCompile)"
                      Overwrite="true"
                      Encoding="UTF-8" />

    <!-- Invoke the compiler -->
    <Exec Command='"$(ChimelispCompilerPath)" @"$(_LispResponseFile)" /out:"$(TargetPath)" /target:$(OutputType) /langversion:$(LispLanguageVersion)"'
          WorkingDirectory="$(MSBuildProjectDirectory)" />

    <ItemGroup>
      <FileWrites Include="$(_LispResponseFile)" />
    </ItemGroup>
    
  </Target>

  <!-- Hook into the standard build process -->
  <PropertyGroup>
    <CoreCompileDependsOn>
      $(CoreCompileDependsOn);
      PrepareResources
    </CoreCompileDependsOn>
  </PropertyGroup>

</Project>

The CoreCompile target is the standard MSBuild target for compilation. By defining it here, we replace whatever compilation behavior might come from other imported files. The DependsOnTargets attribute ensures resource preparation and other prerequisites run first.

Version Management with global.json

MSBuild SDK versions can be specified in global.json at the solution root, allowing centralized version management:

{
  "msbuild-sdks": {
    "Chimelisp.Sdk": "1.0.0"
  }
}

With this in place, projects can omit the version from their SDK reference:

<Project Sdk="Chimelisp.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>
</Project>

Centralized Configuration with Directory.Build.props

For solution-wide configuration, MSBuild automatically imports a file named Directory.Build.props from the solution directory if it exists³. This allows common properties to be defined once rather than repeated in every project:

<Project>
  <PropertyGroup>
    <!-- Common metadata for all projects -->
    <Authors>mmagueta</Authors>
    <Copyright>© $([System.DateTime]::Now.Year) mmagueta</Copyright>
    <PackageLicenseUrl>https://github.com/mmagueta/chimelisp/blob/main/LICENSE</PackageLicenseUrl>
    <RepositoryUrl>https://github.com/mmagueta/chimelisp</RepositoryUrl>
    <RepositoryType>git</RepositoryType>
    
    <!-- Version control -->
    <VersionPrefix>1.0.0</VersionPrefix>
    
    <!-- Build configuration -->
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  
  <!-- CI/CD integration -->
  <PropertyGroup>
    <PackageOutputPath>$(MSBuildThisFileDirectory)Artifacts</PackageOutputPath>
    <PackageOutputPath Condition=" '$(BUILD_ARTIFACTSTAGINGDIRECTORY)' != '' ">
      $(BUILD_ARTIFACTSTAGINGDIRECTORY)
    </PackageOutputPath>
  </PropertyGroup>
</Project>

This pattern proves particularly valuable for CI/CD pipelines; build systems like Azure DevOps can set environment variables (such as BUILD_ARTIFACTSTAGINGDIRECTORY) that automatically redirect package output to the appropriate staging location.

Publishing to NuGet

The SDK package itself is a standard NuGet package. The .nuspec file specifies the package metadata:

<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2013/05/nuspec.xsd">
  <metadata>
    <id>Chimelisp.Sdk</id>
    <version>1.0.0</version>
    <authors>mmagueta</authors>
    <description>MSBuild SDK for Chimelisp projects</description>
    <packageTypes>
      <packageType name="MSBuildSdk" />
    </packageTypes>
  </metadata>
  <files>
    <file src="Sdk/**" target="Sdk/" />
    <file src="tools/**" target="tools/" />
    <file src="lib/**" target="lib/" />
  </files>
</package>

The <packageType name="MSBuildSdk" /> element is important. While not strictly required for functionality, it signals to NuGet that this package serves as an MSBuild SDK, which can enable better tooling support.

On SDK Resolution

MSBuild resolves SDKs through a plugin architecture. The NuGet-based resolver is one such plugin, querying configured feeds for packages. This resolution happens at evaluation time, before restore even runs. What David Federman calls "a restore before the restore" in his blog on authoring MSBuild SDKs⁵.

This has implications: SDK packages must be lightweight. They're downloaded during project evaluation, not during a typical NuGet restore. Heavy dependencies or large compiler binaries in the SDK package itself will slow down every build. Better to reference the compiler as a separate tool or package dependency.

Integration with the Build System

What makes this approach powerful is that it makes Chimelisp native to the .NET build system. Projects can:

• Reference Chimelisp libraries with <ProjectReference> like any other .NET project
• Mix .lisp and .cs files in the same solution
• Use standard MSBuild properties: TargetFramework, OutputPath, Configuration
• Integrate with continuous integration systems that understand MSBuild
• Work with Visual Studio and other IDEs that load MSBuild projects

The compiler becomes a first-class participant in the build process, not an afterthought requiring custom scripts or manual invocation.

A Note on Tooling Quality

It strikes me as remarkable that Microsoft, often the target of justified criticism from the free software community (particularly regarding their historical relationship with open source), has produced tooling of this caliber. The MSBuild SDK system is not merely functional; it is well-designed. The separation of concerns between props and targets, the plugin architecture for SDK resolution, the integration with NuGet and etc are not accidents. They reflect careful thought about extensibility and composition.

This is not to absolve Microsoft of their various transgressions against software freedom. Their embrace-extend-extinguish tactics, their vendor lock-in strategies, their historical hostility toward GPL; these are all documented and deserving of criticism. But in the narrow domain of build system design, they have produced something genuinely good. The .NET SDK's approach to project files, the tooling around MSBuild, the documentation (scattered though it may be), these represent engineering of a quality that many FOSS projects struggle to achieve.

Perhaps this is because build systems, by their nature, must be extensible. A closed build system is useless; no one can adapt it to their needs. Microsoft was forced, by the very nature of the problem, to create something open to extension. And in doing so, they produced a system that respects the developer's need to customize and control their build process.

This does not mean we should uncritically adopt Microsoft tooling everywhere. But it does mean we should acknowledge quality where we find it, and perhaps learn from it. The MSBuild SDK pattern could inform how we design build systems in other ecosystems. The principle of "props for configuration, targets for execution" is not .NET-specific. The idea of SDK resolution through package managers could work elsewhere.

We can admire the engineering while maintaining our vigilance about corporate control of our tools. Microsoft clearly knows how to use XML properly (evidently the best format around).

A Final Observation

The MSBuild SDK mechanism is not perfect. The documentation is scattered; some in Microsoft Learn, some in blog posts from the team, some only discoverable by examining existing SDK implementations. The timing of when things are evaluated can be subtle. The error messages when something goes wrong are often hieroglyphs.

But the fundamental design is sound. The pattern of props-then-targets, of early binding for defaults and late binding for execution, aligns with how build systems must actually work. It allows for composition, for overriding, for extension.

What began as a practical problem: how to make Chimelisp integrate better with .NET tooling; became an exercise in understanding the machinery that already existed. The answer was not to build something new, but to align with what was already there.

Sometimes the best solution is to recognize the structure that exists and work within it.

References