.NET AI Essentials – The Core Building Blocks Explained

Every few months Microsoft ships another piece of the .NET AI story, and it gets confusing fast if you are not following the DevBlogs closely. Jeremy Likness recently laid out the foundational pieces that make up the .NET AI stack, and this post walks through them with working code, along with the things I would flag to you as an architect before you wire this into a production app.

The building blocks he covers are Microsoft.Extensions.AI (MEAI) for talking to language models, Microsoft.Extensions.VectorData for embeddings and semantic search, the Microsoft Agent Framework for agentic workflows, and the Model Context Protocol (MCP) for interoperability between tools. This particular post, and this recreation of it, focuses only on MEAI. The other three get their own posts in the original series.

MEAI: one API instead of four SDKs

If you have shipped anything with Semantic Kernel, think of Microsoft.Extensions.AI as the layer that replaced its core primitives. It gives you a single abstraction, IChatClient, that sits in front of OpenAI, Azure OpenAI, Ollama, and other providers. You also get the dependency injection, builder, and middleware patterns that are already familiar from ASP.NET Core and minimal APIs, which is deliberate. Microsoft wants AI code to feel like the rest of your .NET code, not like a separate universe bolted on top.

Here is what talking to a local Ollama model looks like using OllamaSharp directly, without MEAI in the picture yet:

var uri = new Uri("http://localhost:11434");
var ollama = new OllamaApiClient(uri)
{
    SelectedModel = "mistral:latest"
};
await foreach (var stream in ollama.GenerateAsync("How are you today?"))
{
    Console.Write(stream.Response);
}

This streams a response token by token from a local mistral model and writes it straight to the console. Nothing wrong with it, but it is OllamaSharp specific syntax. Compare it with calling OpenAI directly using their SDK:

OpenAIResponseClient client = new("o3-mini", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
 
OpenAIResponse response = await client.CreateResponseAsync(
            [ResponseItem.CreateUserMessageItem("How are you today?")]
  );
foreach (ResponseItem outputItem in response.OutputItems)
{
    if (outputItem is MessageResponseItem message)
    {
        Console.WriteLine($"{message.Content.FirstOrDefault()?.Text}");
    }
}

Same idea, completely different shape. Different client type, different method names, different way of pulling the text back out of the response. If your app needs to support more than one provider, or you want the option to switch providers later without a rewrite, you end up maintaining two mental models side by side. This is exactly the pain MEAI exists to remove.

The OllamaSharp chat client already implements the universal IChatClient interface, so it works with MEAI as is. The OpenAI SDK does not implement it natively, but the OpenAI adapter package gives you an extension method to bridge the gap:

IChatClient client =
    new OpenAIClient(key).GetChatClient("o3-mini").AsIChatClient();

Once you have wrapped the client, both providers expose the same surface. Streaming a response now looks identical regardless of which model is behind it:

await foreach (ChatResponseUpdate update in client.GetStreamingResponseAsync("How are you today?"))
{
    Console.Write(update);
}

This is the real payoff. You write this loop once, register whichever IChatClient you need through DI, and swapping providers later becomes a configuration change instead of a rewrite. I have seen teams hardcode a specific provider’s SDK deep into business logic early on, and then pay for it later when a client wants Azure OpenAI instead of OpenAI directly, or wants to add a local model option for cost reasons. Coding against IChatClient from day one avoids that trap.

What you get for free once you adopt MEAI

Beyond the common interface, MEAI also manages retries, enforces token limits, plugs into your existing DI container, and supports middleware, which we will get to shortly. None of this is exotic, it is the same set of concerns ASP.NET Core solved for HTTP requests years ago, just applied to model calls instead. If you are already comfortable with the options pattern and middleware pipelines in ASP.NET Core, this will feel familiar rather than new.

Structured output without hand rolled JSON parsing

Getting a model to return well formed JSON that matches a C# type is one of those things that sounds trivial and is not, especially once you factor in models that occasionally add commentary before or after the JSON, or get a field name slightly wrong. Structured output support solves this by giving the model an explicit schema to follow. Here is the raw OpenAI SDK approach:

class Family
{
    public List<Person> Parents { get; set; }
    public List<Person>? Children { get; set; }
 
    class Person
    {
        public string Name { get; set; }
        public int Age { get; set; }
    }
}
 
ChatCompletionOptions options = new()
{
    ResponseFormat = StructuredOutputsExtensions.CreateJsonSchemaFormat<Family>("family", jsonSchemaIsStrict: true),
    MaxOutputTokenCount = 4096,
    Temperature = 0.1f,
    TopP = 0.1f
};
 
List<ChatMessage> messages =
[
    new SystemChatMessage("You are an AI assistant that creates families."),
    new UserChatMessage("Create a family with 2 parents and 2 children.")
];
 
ParsedChatCompletion<Family?> completion = chatClient.CompleteChat(messages, options);
Family? family = completion.Parsed;

This works, but notice how much of it is OpenAI specific: ChatCompletionOptions, ChatMessage, CompleteChat, ParsedChatCompletion. None of that carries over if you switch providers. Here is the same result using MEAI:

class Family
{
    public List<Person> Parents { get; set; }
    public List<Person>? Children { get; set; }
 
    class Person
    {
        public string Name { get; set; }
        public int Age { get; set; }
    }
}
 
var family = await client.GetResponseAsync<Family>(
    [
        new ChatMessage(
            ChatRole.System,
            "You are an AI assistant that creates families."),
        new ChatMessage(
            ChatRole.User,
            "Create a family with 2 parents and 2 children."
        )]);

The generic GetResponseAsync<Family> call generates the JSON schema for you behind the scenes and deserializes the response straight into your type. You lose the fine grained control over Temperature and TopP that the first example had explicitly set, since those live on ChatOptions rather than being baked into this particular overload, so check the ChatOptions overload if you need that control alongside structured output. For most CRUD style extraction tasks though, the shorter version is what I would reach for first, and only drop down to provider specific options when you actually need a knob that MEAI has not exposed yet.

Temperature, tokens, and ChatOptions

Temperature comes up in almost every conversation about model behaviour, so it is worth being precise about what it actually does. A lower temperature makes the model favour its highest probability next token more consistently, which produces more predictable and generally more factual sounding output. A higher temperature lets lower probability tokens win more often, which produces more varied, sometimes more creative, but also more ungrounded output where the model states something inaccurate with full confidence.

As a rough rule, classification, summarization, and data extraction tasks do better at low temperature, somewhere around 0.1 to 0.3, because you want the same input to produce the same output reliably. Brainstorming, marketing copy, or creative writing tasks tolerate and often benefit from a higher temperature, closer to 0.7 or above. MEAI standardizes this and every other tunable, including max token count, through the ChatOptions class, so you set it once per request regardless of which provider is underneath.

On the response side, every ChatResponse carries a UsageDetails object with input and output token counts. If you are running this in production, log this from day one. Token usage is your actual cost driver, and retrofitting usage tracking after a surprise bill is a worse experience than building it in up front.

Middleware: the same pattern, applied to model calls

.NET developers already understand middleware from ASP.NET Core: it is a pipeline you can intercept to run logic before or after the actual request goes through. MEAI applies the identical idea to chat clients. Typical uses include:

• Blocking malicious or unsafe content before it reaches the model
• Throttling or rate limiting outbound requests
• Adding telemetry and tracing across every call, regardless of provider

MEAI ships logging and OpenTelemetry middleware out of the box, applied through the same builder pattern .NET developers already use elsewhere. Here is a helper method that wraps any IChatClient with both:

public IChatClient BuildEnhancedChatClient(
            IChatClient innerClient,
            ILoggerFactory? loggerFactory = null)
        {
            var builder = new ChatClientBuilder(innerClient);
 
            if (loggerFactory is not null)
            {
                builder.UseLogging(loggerFactory);
            }
 
            var sensitiveData = false; // true for debugging
 
            builder.UseOpenTelemetry(
                configure: options =>
                    options.EnableSensitiveData = sensitiveData);
            return builder.Build();
        }

This wraps innerClient regardless of which provider created it, since the middleware operates on IChatClient rather than a concrete type. The EnableSensitiveData flag is worth calling out specifically: leave it false in production. Turning it on logs full prompt and completion content into your telemetry pipeline, which is convenient while debugging locally but becomes a compliance problem the moment real user data or proprietary business context flows through those prompts. Flip it on for a local debugging session, never for a deployed environment.

The OpenTelemetry events this generates can flow to Application Insights, or if you are running under .NET Aspire, straight into the Aspire dashboard. Aspire has added specific support for surfacing model interactions, and you can spot them by the small sparkle icon next to relevant trace entries, as shown below.

DataContent and multi-modal conversations

Text is not the only thing models exchange anymore. Plenty of current models accept images and audio as input and can return similar content back. MEAI models this through an AIContent base type, and while most getting started examples only show TextContent, there are several other built in content types worth knowing about:

• ErrorContent for detailed error information with error codes
• UserInputRequestContent to request user input, including FunctionApprovalRequestContent and FunctionApprovalResponseContent
• FunctionCallContent to represent a tool request
• HostedFileContent to reference data hosted by an AI specific service
• UriContent for a plain web reference

That is not the complete list, but DataContent is the one you will reach for most often. It is essentially a byte array paired with a media type string, which means it can represent an image, an audio clip, or effectively any binary payload you want to hand to a multi-modal model. Here is an example that sends a local photo to a model and asks for a description plus tags:

var instructions = "You are a photo analyst able to extract the utmost detail from a photograph and provide a description so thorough and accurate that another LLM could generate almost the same image just from your description.";
 
var prompt = new TextContent("What's this photo all about? Please provide a detailed description along with tags.");
 
var image = new DataContent(File.ReadAllBytes(@"c:\photo.jpg"), "image/jpeg");
 
var messages = new List<ChatMessage>
            {
                new(ChatRole.System, instructions),
                new(ChatRole.User, [prompt, image])
            };
 
record ImageAnalysis(string Description, string[] tags);
 
var analysis = await chatClient.GetResponseAsync<ImageAnalysis>(messages);

The user message here carries two content items, the text prompt and the image, packaged together in a single ChatMessage. The GetResponseAsync<ImageAnalysis> call combines the multi-modal input with structured output from earlier, so you get a strongly typed result back instead of a paragraph of free text you would otherwise have to parse yourself. One practical warning: File.ReadAllBytes loads the entire image into memory synchronously, which is fine for a console sample but not something you want on a hot path in a web API serving concurrent requests. Use the async file APIs and watch your payload sizes, since large images translate directly into large request bodies and higher token counts on models that tokenize image data.

A few other capabilities worth knowing about

The original post only scratches the surface of what the base extensions provide, since going deep on every feature would need its own article. Worth knowing they exist:

• Cancellation token support so long running model calls stay responsive to user cancellation
• Built-in error handling and resilience for transient failures
• Primitives for working with vectors and embeddings, which lead into Microsoft.Extensions.VectorData
• Image generation support alongside chat completion

Where this fits in a real project

If you are starting a new .NET project that talks to any LLM, coding against IChatClient from the start costs you almost nothing and buys you provider flexibility later. The pattern will already feel familiar since it borrows directly from ASP.NET Core conventions, and your team will not need to learn a second mental model just for the AI parts of the codebase.

The trade-off worth knowing about is that a universal abstraction, by nature, cannot always expose every provider specific feature the day that provider ships it. If OpenAI or Anthropic release a bleeding edge capability, there can be a lag before it surfaces through MEAI, and you may need to drop to the native SDK temporarily for that one feature. That is a reasonable trade for the maintainability you get everywhere else, but budget for the occasional escape hatch rather than assuming MEAI covers a hundred percent of every provider’s surface area at all times.

This is the first of a four part series, with vector data, the Agent Framework, and MCP still to come. Worth following if you are building anything beyond a single provider chatbot.