Generative AI became the fastest growing consumer technology in history. It crossed 100 million users in under two months, faster than Instagram or TikTok managed the same milestone. That growth started at the end of 2022, when OpenAI released a free preview of GPT-3.5 as a conversational chat client called ChatGPT.
The model behind ChatGPT was fine tuned using Reinforcement Learning from Human Feedback, known as RLHF. This is the point where generative AI moved from a research topic into something ordinary users interacted with every day. Microsoft responded quickly, launching the Azure OpenAI Service in early 2023 so developers could provision OpenAI compatible models behind Azure managed endpoints, with the enterprise controls Azure customers expect.
Why .NET developers need a shared vocabulary
Soon after Azure OpenAI Service launched, Microsoft introduced a set of tools built specifically for .NET developers. Semantic Kernel gives you a way to orchestrate prompts, memories, and plugins in C# or Python. Microsoft.Extensions.AI provides a unified abstraction for talking to models through a common IChatClient interface, and Microsoft.Extensions.VectorData standardizes how you talk to vector databases in RAG systems.
Jeremy Likness, from the .NET team at Microsoft, wrote a foundational post that steps back from the pace of AI releases to cover the core concepts every .NET developer working with Microsoft Foundry, GitHub Models, or a local runtime like Ollama should know. This piece walks through that terminology, adds practical commentary from a production standpoint, and looks at where the .NET tooling landscape actually stands today.
Artificial intelligence, generative AI, and what GPT means
AI covers any technique that lets a computer perform tasks that normally require human intelligence, such as reasoning, language, planning, or perception. AI as a field is decades old, but when most people say AI today, they specifically mean generative AI.
Generative AI refers to systems capable of producing text, images, audio, or other content, rather than just classifying input or predicting a number. GPT, the acronym behind most models developers use daily, stands for Generative Pre-trained Transformer. Breaking that down helps: generative means it produces content, pre-trained means it was trained on enormous datasets before you ever touched it, and transformer is the neural network architecture that makes high quality language modeling possible.
Tokens, embeddings, and why translation is hard
Large language models are trained on billions of tokens and can generate text, images, code, or reasoning steps depending on how they were trained and fine tuned. Their ability to work across multiple languages does not come from a dictionary lookup. It comes from learning relationships between words in a shared semantic space.
Translation is hard for traditional software because words carry multiple meanings depending on context. Think about the word pass: you pass a car on the highway, you hike through a mountain pass, you pass on an opportunity, and you keep a park pass on your dashboard. Rule based systems struggle with this kind of ambiguity, while LLMs handle it well because they do not really work with words as strings, they work with meaning.
Models do not read raw text directly. Text gets broken into tokens, which can be whole words, word fragments, or individual characters, and each token is converted into a numeric vector called an embedding. An embedding is a mathematical representation of meaning, not of spelling.
Consider two sentences: the actor was a star, and they loved the stars. The word star shows up in both, but the two uses mean different things, and embeddings capture that difference by placing the word in a different region of semantic space depending on the surrounding context.
The image above is a simplified way to visualize this. The meaning of star gets plotted based on its distance to celestial body on one side and actor on the other. Now scale that up to billions of points across an enormous vocabulary, and you get a picture of how a model actually generates text. It navigates this space and predicts the next most likely vector, one token at a time.
This distance based thinking is also how semantic search works. School and schol sit close together in embedding space, which is why a misspelling still gets corrected. Cat and dog sit close together because they are similar animals. Cat and laptop sit far apart. Semantic search relies on this distance rather than string matching, which is exactly why it finds relevant results even when the query does not share a single word with the matching document.
Parameter counts and what they actually tell you
You will often see LLMs described by parameter count, such as 7B, 14B, 70B, or 123B. Parameters are the trained weights inside the network, and more parameters generally mean deeper reasoning, richer world knowledge, and better nuance in the output. GPT-1, released in 2018, had 117 million parameters, while modern frontier models sit somewhere between 100 billion and 400 billion plus parameters.
Raw parameter count is not the only factor in quality, though. Training data quality and fine tuning matter just as much, so treat parameter count as one signal among several rather than a benchmark on its own when you are choosing a model for a specific project.
Prompts, system instructions, and tools
The previous sections describe the model itself. This section covers what actually goes in and out of it. A prompt is the user input to the model, something like what is the best way to peel a mango. System instructions are the hidden blueprint that shapes how the model behaves before it ever sees the user prompt, for example you are a mango peeling expert.
Getting the balance right between system instructions and prompt design is one of the more underrated skills in building production AI features. A weak system prompt tends to produce inconsistent output even when the underlying model is strong, and teams often spend more time tuning this balance than picking the model itself.
Tools, sometimes called functions, let a model reach past its training data to get current or authoritative information. A weather API, a database lookup, a search engine, or a company knowledge index are all examples. This pattern of giving a model access to outside data is called Retrieval-Augmented Generation, or RAG.
Take a concierge agent that has access to a restaurant API and a weather API. A user asks, can you book me a dinner this week at a restaurant with outdoor seating. The model first calls the weather API to work out which evenings are likely to be dry and warm, then calls the restaurant API to check what is open with available outdoor seating, and returns a shortlist that actually fits the request. Neither API call is hardcoded into the conversation flow. The model decides when and how to call each tool based on the prompt.
Now take a customer service agent for a retail store with its product catalog uploaded. A user types, what kind of batteries does the traveling wonder cube take. The model extracts the product name, vectorizes the query text, and calls the product API with both the name and the vector. Semantic search then finds the section of the product manual closest in meaning to the query, and returns the battery requirement if that information exists in the manual. This is a good illustration of why RAG systems still need decent source data behind them. No amount of prompt engineering fixes a product manual that never mentions batteries in the first place.
Model Context Protocol and agents
Model Context Protocol, or MCP, is a set of standards for interoperability between agents and tools. It gives models a consistent way to discover what tools are available and how to call them, which means you can build a toolbox once and reuse it across different models and agents instead of wiring custom integrations for each one.
An agent is a specialized solution built from a model, a set of tools, and context. A concierge agent, for instance, might pair a reasoning model with tools for weather, events, and local businesses, plus a separate model that generates turn by turn maps. Agents deserve a closer look on their own, particularly how to build them in C#, and that is worth a dedicated post rather than a side note here.
Where to run and manage your models in .NET
Picking a model is only part of the job. Many teams want to host their own models for reasons of trust, security, or cost, and some need fine tuning and custom training on top of a base model. The good news for .NET developers is that this is reasonably well supported today, not a niche capability bolted onto the ecosystem as an afterthought.
GitHub Models gives you a hosted catalog of open and frontier models behind an OpenAI compatible API, with no infrastructure to manage. You can switch between models with minimal code changes, which makes it a solid starting point for prototyping, evaluation work, automation scripts, and CI/CD pipelines where you do not want to commit to a single provider early.
Microsoft Foundry, formerly Azure AI Studio, is the enterprise platform for taking AI into production at scale. It gives you model catalogs spanning OpenAI, Meta, DeepSeek, Cohere, and Mistral among others, agentic workflows through Foundry Agent Service, security and content safety controls, monitoring and tracing, evaluations, and fine tuning. If your organization needs governance around AI usage, this is where that governance actually lives.
Foundry Local brings the same developer experience offline, for on premise, air gapped, or edge environments, using the same agents, tools, and evaluations as the cloud version. It supports a develop local then deploy cloud workflow, which is genuinely useful for testing new models or new code without burning through a cloud budget, and for CI/CD pipelines that should not depend on a hosted account to succeed.
Ollama is a popular open source runtime for running lightweight and mid sized models locally, including Mistral, Llama 3, and Phi-3, through a simple CLI and local server. It integrates with Microsoft.Extensions.AI through the IChatClient interface using the OllamaSharp package, which makes it a reasonable choice for privacy sensitive workflows or for developers who want to experiment without any cloud dependency at all. The trade off is hardware. Running anything beyond a small model locally needs real GPU memory, and performance on a laptop will not match a hosted endpoint.
A unified abstraction across providers
As a .NET developer, you should not have to lock into a single provider to stay productive. This is exactly why Microsoft.Extensions.AI exists, providing consistent APIs for working with models regardless of where they run, and enabling middleware for logging, tracing, and custom behavior injection along the way.
IChatClient client = new OllamaApiClient(
new Uri("http://localhost:11434"), "llama3.1");
var response = await client.GetResponseAsync(
"Summarize the RAG pattern in two sentences.");
Console.WriteLine(response.Text);This is the shape of code you would write against Ollama running locally. Swap OllamaApiClient for an Azure OpenAI client or a GitHub Models client, and the rest of your application code does not change, because everything still talks to the same IChatClient interface. In practice this means you can prototype against GitHub Models or Ollama for free, then swap in Microsoft Foundry for production without rewriting your business logic, provided you kept provider specific configuration out of the calling code. The one thing to watch is that not every provider supports every capability behind IChatClient consistently, so check function calling and streaming support before you commit to a provider for a feature that needs it.
What this means for your next project
If you are new to this space, do not try to learn everything at once. Start with GitHub Models for free experimentation, understand the difference between a prompt and a system instruction, and get comfortable with the idea that RAG is just giving the model a way to look things up rather than something mysterious. Once you have a feature that works locally against GitHub Models or Ollama, moving it to Microsoft Foundry for production is a configuration change, not a rewrite, as long as you built against Microsoft.Extensions.AI abstractions from the start.
This article covers the vocabulary. Actually wiring up Microsoft Foundry with Microsoft.Extensions.AI for a working quickstart, complete with authentication and a real RAG pipeline, deserves a hands on walkthrough of its own. That is a natural next piece for readers who are comfortable with ASP.NET Core and dependency injection but new to AI development, and it is exactly the kind of companion post this foundational article sets up.
Leave a Reply