The Foundry Local C# SDK

A .NET team evaluating on-device inference usually arrives with a reasonable expectation: there will be a local server, and the application will call it over HTTP. That is how nearly every local model tool works, and it is a shape every developer already knows how to reason about.

Foundry Local is not that shape, and the difference is the reason it is worth a team's attention. The SDK wraps a native library loaded in-process — a .dll on Windows, a .so on Linux, a .dylib on macOS. Calls into the model are function calls. The optional REST endpoint exists, and it is genuinely useful for tools that speak HTTP, but it is not the primary path and nothing in an application needs it.

There are two, and picking the wrong one is the first available mistake.

# Windows — integrates with the Windows ML runtime
dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

# Cross-platform — macOS, Linux, and Windows without Windows ML
dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

The API surface is the same either way. The difference is hardware reach: the Windows package routes execution-provider acquisition and registration through Windows ML, which sources hardware-matched plugins from the operating system and Windows Update and handles driver compatibility negotiation. The cross-platform package registers providers directly and bundles what it needs.

For a shop shipping a Windows desktop product, the Windows package is the default and the wider acceleration surface is the reason. For a shop shipping cross-platform, or developing on macOS and deploying to Windows, the project file handles both — which is the next section, and it is more interesting than it sounds.

The documented reference configuration puts the platform branching in MSBuild rather than in code, which is the right place for it:

<PropertyGroup Condition="$([MSBuild]::IsOSPlatform('Windows'))">
  <TargetFramework>net8.0-windows10.0.18362.0</TargetFramework>
  <Platforms>ARM64;x64</Platforms>
  <WindowsPackageType>None</WindowsPackageType>
</PropertyGroup>

<PropertyGroup Condition="!$([MSBuild]::IsOSPlatform('Windows'))">
  <TargetFramework>net8.0</TargetFramework>
</PropertyGroup>

<ItemGroup Condition="$([MSBuild]::IsOSPlatform('Windows'))">
  <PackageReference Include="Microsoft.AI.Foundry.Local.WinML" />
</ItemGroup>

<ItemGroup Condition="!$([MSBuild]::IsOSPlatform('Windows'))">
  <PackageReference Include="Microsoft.AI.Foundry.Local" />
</ItemGroup>

Three settings in there are load-bearing and easy to skip past. WindowsPackageType set to None builds an unpackaged desktop application rather than an MSIX package. Platforms declares ARM64 alongside x64, which matters more every quarter as Windows on ARM hardware reaches more desks. And RuntimeIdentifier, defaulted to the SDK's own identifier, is what ensures the correct native binaries are selected — the native library at the centre of all this has to match the machine it lands on.

Linux with an NVIDIA GPU needs two additional references, Microsoft.ML.OnnxRuntime.Gpu and Microsoft.ML.OnnxRuntimeGenAI.Cuda, conditioned on the linux-x64 runtime identifier.

None of this is exotic. It is the ordinary work of shipping a .NET application that carries native dependencies, and any team that has shipped one will recognize every line. Worth saying plainly, because "AI in the application" tends to summon expectations of novelty where the actual work is familiar.

The model lifecycle is download, load, infer, unload, and it is identical across all four supported languages. Initialization comes first, and the manager is a singleton:

using Microsoft.AI.Foundry.Local;
using Microsoft.Extensions.Logging;

var config = new Configuration
{
    AppName = "app-name",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information,
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
});
var logger = loggerFactory.CreateLogger<Program>();

await FoundryLocalManager.CreateAsync(config, logger);
var manager = FoundryLocalManager.Instance;

var catalog = await manager.GetCatalogAsync();
var models = await catalog.ListModelsAsync();

That last call returns the models available for this machine's hardware, not a universal list. Hardware detection has already happened by the time the catalog answers.

The rest reads the way resource management reads everywhere else in .NET. Download if not cached — the call is a no-op when the files are already on disk. Load, which initializes the ONNX Runtime session and selects the execution provider. Create a client. Run inference. Unload to free memory, leaving the cached files in place for the next launch.

Two properties, IsCached and IsLoaded, exist for the checks a real application needs to make before deciding whether the next call is instant or involves a multi-gigabyte download over a customer's connection. Any application that skips that distinction will eventually produce a first-run experience nobody tested.

One detail deserves emphasis because it determines architecture: the Core API supports thread-safe, session-based inference with concurrent requests from multiple threads and no external synchronization. For a desktop application with a responsive UI, that removes the need to build a request queue by hand.

Chat completions stream through an async enumerable, which is exactly the idiom a C# developer would expect and hope for:

var chatClient = await model.GetChatClientAsync();

var streamingResponse = chatClient.CompleteChatStreamingAsync(messages, ct);
await foreach (var chunk in streamingResponse)
{
    Console.Write(chunk.Choices[0].Delta.Content);
}

Less widely known is that chat is not the only workload. The SDK includes a native audio client for on-device transcription using Whisper models, running in-process without the REST server:

var audioClient = await model.GetAudioClientAsync();
audioClient.Settings.Language = "en";
audioClient.Settings.Temperature = 0.0f;

await foreach (var chunk in audioClient.TranscribeAudioStreamingAsync(
    "recording.mp3", CancellationToken.None))
{
    Console.Write(chunk.Text);
}

That capability changes the calculus for a whole class of application. Transcription is one of the workloads where sending data to a cloud service is most likely to trip a policy objection — recorded meetings, support calls, clinical notes, anything with a voice in it. A local transcription path with no network hop is not a performance optimization. It is often the only version of the feature that survives review.

The architectural argument for an in-process library is usually made on latency, and latency is the least of it.

The SDK does not require the Foundry Local CLI to be installed on an end user's machine. The application is self-contained. Which means the following support tickets never get written:

• The background service is not running.
• The background service is running on a different port than the application expects.
• Another application changed the shared model store.
• The installer for the second product failed on a locked-down machine.
• A version upgrade of the runtime broke an application that did not ship it.

Every dependency an application asks a customer to install is a support ticket with a delayed fuse.

That last one is worth dwelling on. A shared local runtime is a shared dependency, and shared dependencies drift. An application that bundles its own runtime has a version it controls and can test against. For internal tooling, drift is an annoyance. For software shipped to customers who will never file a useful bug report, it is the difference between a feature that works and a feature that works on the machines the team happened to try.

The trade is real and should be stated: bundling means the application carries the weight and owns the update path. Roughly 20 MB of package growth is a small price, but "owns the update path" is a commitment that outlasts the first release.

That is the whole shape of it. Two packages, a project file that branches by platform, four verbs, and a native library that ships inside the product. For a .NET shop, the unfamiliar part is not the API. It is the idea that the model is now part of the build.