Semantic Kernel is a lightweight SDK from Microsoft that lets you combine regular code, in C# or Python, with prompts sent to a large language model. One of its more interesting pieces is the Planner, which takes a goal written in plain English and works out which registered functions to call and in what order to reach that goal. Early versions of the Planner had one obvious limitation: it handed the LLM a manual listing every single registered function, and that list only got longer and messier as more skills were added to the kernel.
Why dumping every function into the prompt breaks down
When Planner builds a plan, it first writes out a natural language description of every available function so the model can pick the right ones. This works fine with ten or twenty functions. Once an application registers fifty or a hundred functions across multiple skills, the prompt grows past what is useful, token costs go up, and the model starts making worse choices because it is wading through mostly irrelevant options. This is the same problem you run into with search: dumping an entire catalogue in front of a user rarely helps them find what they actually want.
Semantic filtering narrows the list before the model sees it
The fix Microsoft shipped for the Planner Skill was semantic filtering. Instead of listing every function, Planner now stores an embedding for each registered function’s description in a Semantic Memory store, then searches that store using the plan’s goal text as the query. Only the functions that come back as relevant get included in the manual sent to the model. You can also cap how many functions get pulled in, force specific functions to always be included regardless of relevance, or exclude entire skills from consideration.
Registering an embeddings backend and memory store
Semantic filtering needs two things wired into the kernel before you can use it: an embedding generation backend and an implementation of IMemoryStore to hold those embeddings. The example below uses Azure OpenAI for both text completion and embeddings, with an in-memory VolatileMemoryStore backing the memory.
var kernel = new KernelBuilder()
.Configure(
config =>
{
config.AddAzureOpenAITextCompletion(...);
config.AddAzureOpenAIEmbeddingGeneration(...);
})
.WithMemoryStorage(new VolatileMemoryStore())
.Build();VolatileMemoryStore keeps everything in process memory, so it disappears the moment the app restarts. That is fine for a demo or a quick prototype, but for anything running in production you would swap it for a persistent store. Azure Cognitive Search or Qdrant are common choices, so function embeddings do not need to be recomputed on every cold start.
Turning filtering on when you create a plan
With the kernel configured, enabling semantic filtering for a specific plan is a matter of setting a relevancy threshold on the ContextVariables before calling CreatePlan. The threshold is a similarity score between 0 and 1, and a higher number means Planner will only pull in functions that are closely related to the goal.
var planner = kernel.ImportSkill(new PlannerSkill(kernel), "planning");
// import any other skills or functions ....
var context = new ContextVariables("Create a book with 3 chapters about a group of kids in a club called 'The Thinking Caps.'");
// Set the RelevancyThreshold to enable semantic filtering
context.Set(PlannerSkill.Parameters.RelevancyThreshold, "0.78");
var results = await kernel.RunAsync(context, planner["CreatePlan"]);A threshold around 0.78, as used here, is a reasonable starting point, but the right value depends on how similar your function descriptions are to each other. Set it too low and you are effectively back to stuffing in nearly every function. Set it too high and Planner might miss a function it genuinely needed because its description was not worded closely enough to the goal text.
Fine-tuning which functions get in
Relevance scoring will not always get the selection exactly right, so Planner exposes a few extra knobs. You can cap the total number of functions returned, explicitly exclude functions or whole skills you never want considered, and force certain functions to be included no matter what the embedding search returns.
// To limit the number of relevant functions to include
context.Set(PlannerSkill.Parameters.MaxRelevantFunctions, "5");
// To exclude certain functions or entire Skills from being included in plan creation requests
context.Set(PlannerSkill.Parameters.ExcludedFunctions, "NovelChapter");
context.Set(PlannerSkill.Parameters.ExcludedSkills, "email");
// Or to ensure specific functions are included in plan creation requests regardless of semantic filtering
context.Set(PlannerSkill.Parameters.IncludedFunctions, "DadJoke");ExcludedSkills is worth using deliberately rather than as an afterthought. If your kernel has a skill for sending email alongside skills for generating content, you probably do not want Planner even considering the email skill for a content generation goal, regardless of what the embeddings say. IncludedFunctions is the escape hatch for the opposite case: a function you always want available even when its description is not a close semantic match to the goal.
What Planner does internally
Two steps happen behind the scenes: saving function descriptions to memory, then searching that memory using the goal as the query. When CreatePlan runs, Planner loops through every registered function and writes it into the memory collection, using the function’s description, or its fully qualified name if no description was set, as the lookup key.
// Loop through available functions and save them to memory.
foreach (var function in availableFunctions)
{
var functionName = function.ToFullyQualifiedName();
var key = string.IsNullOrEmpty(function.Description) ? functionName : function.Description;
await context.Memory.SaveInformationAsync(PlannerMemoryCollectionName, key, functionName, function.ToManualString(),
context.CancellationToken);
}This save step runs on every CreatePlan call in the version described here, so with VolatileMemoryStore you are recomputing and re-saving embeddings for every function each time a plan is created. That is fine for a handful of functions but becomes wasteful as the function count grows, which is another reason a persistent, cacheable memory store matters more once you move past prototyping.
Once functions are saved, Planner searches that same collection using the goal text as a semantic query, retrieving results up to MaxRelevantFunctions and filtered by RelevancyThreshold.
// Search for functions that match the semantic query.
var memories = context.Memory.SearchAsync(PlannerMemoryCollectionName, semanticQuery, config.MaxRelevantFunctions, config.RelevancyThreshold.Value,
context.CancellationToken);The result is a shorter, more targeted manual handed to the LLM instead of the full function catalogue. Fewer irrelevant options generally means better plan quality and a smaller prompt, which also helps with token costs on longer running applications.
Where this fits today
This sequential Planner design, write a plan up front by picking from a filtered function list, was Semantic Kernel’s original approach to agentic behaviour. Semantic Kernel has since moved a lot of its planning logic toward native function calling, where the model is given tool definitions directly and decides which to call as part of the completion, closer to how OpenAI’s function calling and later assistant style APIs work. If you are picking up Semantic Kernel today, it is worth checking the current documentation for the Agents and function-calling based planners, since the PlannerSkill and RelevancyThreshold approach described here reflects the Alpha stage SK was in during early 2023.
Even so, the underlying idea, use embeddings to cut down what you show a model before asking it to pick from a list, holds up well beyond Planner specifically. It applies just as much to tool selection in any agent framework, or to narrowing down documents for retrieval augmented generation before they reach the context window.
Leave a Reply