Articles \ agents \ sterling

The tools do the math. The model does the judgment.

Agent Sterling - a .NET code quality reviewer on Microsoft Agent Framework that separates deterministic Roslyn analysis from LLM editorial judgment. Four tools, one system prompt, a workflow executor, and the line between agent and service.

Project Continuum Engine Authors @Tristan & @Claude Topic Agents
Created 2026-04-07 Updated 2026-09-22 Version 3.0

The Thesis

Sterling hands everything countable in a code review to Roslyn.

Cyclomatic complexity and method length are numbers, and Roslyn counts both. Anti-patterns like .Result on async code, async void, and empty catch blocks are syntax. A parser finds them or it doesn't.

But "this factory method does validation, mapping, and persistence in one body", "this interface has one implementation and no reason to exist", and "the complexity is in the plumbing, not the business logic" are editorial. That's the work a staff engineer does in a PR review - reading code and deciding what matters.

Sterling, a C# code-quality agent in the agent-tools companion repo[7], keeps the two apart. Roslyn tools produce the metrics. The model reads the metrics, reads the code it chooses to open, and writes the judgment. It runs as a Microsoft Agent Framework[4] ChatClientAgent with four tools, and a review is one RunAsync call that ends in one QUALITY.md.

The Distinction

The agent is whichever component picks the next tool call.

The companion repo[7] has two other agents, and both do useful work. CrimeSceneInvestigator scans a repository and writes context documentation: six scanners (Markdown, Rules, Structure, Quality, Journal, Done), a ScannerPlanner that asks a model to assign loaded models to scanners, and a ScannerRunner with timeouts, retry, and fallback validation. ModelBoss benchmarks local LLMs with deterministic accuracy scoring, an LLM-as-judge pass, and percentile-ranked scorecards.

In both, C# decides the order of work. AgentInCommand calls CSI's scanners in a fixed sequence, and BossAgent runs resolve, validate, benchmark, judge, score, and report in that order. The models fill in the steps they are handed. That makes them services that call an LLM, which is a perfectly good thing for a benchmark harness to be.

CSI's Quality scanner is the closest comparison, because it writes the same QUALITY.md from the same QualityTools.AnalyzeCSharpFile. It gets eight tools, a four-minute timeout, and a fixed slot after the Structure scanner. Sterling gets four tools and no sequence: the model decides which files to analyze, which to open, what to flag, and when it is finished.

Counted as non-blank lines in the tracked .cs files of each project directory at agent-tools 96b7b6f, tests excluded, Sterling is 356 lines across 7 files, CSI is 1,200 across 14, and ModelBoss is 3,264 across 19. All three reference Agent.SDK, which is another 4,095 lines across 26 files and is where Sterling's Roslyn analysis and file handling actually live.

dotnet run -- ./src --config-key default --headless

The Pattern

ChatClientAgent adds the tool loop Sterling used to configure by hand.

Every agent framework runs the same loop: call the model, check for a tool call, invoke it, append the result, call again. M.E.AI[1] ships it as FunctionInvokingChatClient[2], a delegating IChatClient that intercepts tool-call responses, invokes the matching function, and loops until the model answers in text.

At 0040a44, the commit that introduced Sterling, that loop was four lines of builder config:

var agent = new ChatClientBuilder(chatClient)
    .UseOpenTelemetry(loggerFactory, sourceName: SterlingTrace.Instance.Source.Name)
    .UseFunctionInvocation(loggerFactory, c => c.MaximumIterationsPerRequest = 50)
    .Build();

Commit 8686f2b deleted the builder. SterlingAgent.BuildAgent now hands the plain OpenAI chat client to a ChatClientAgent, which wraps it in FunctionInvokingChatClient by default unless ChatClientAgentOptions.UseProvidedChatClientAsIs is set. The system prompt moved from a ChatRole.System message to the agent's instructions. This is the whole method:

public static AIAgent BuildAgent(IChatClient chatClient, string targetPath, string outputPath)
{
    var fileTools = new FileTools(targetPath);
    var qualityTools = new QualityTools(fileTools);
    var sterlingTools = new SterlingTools(fileTools, qualityTools);

    AITool[] tools =
    [
        AIFunctionFactory.Create(sterlingTools.ListSourceFiles),
        AIFunctionFactory.Create(sterlingTools.AnalyzeFile),
        AIFunctionFactory.Create(sterlingTools.ReadFile),
        AIFunctionFactory.Create(sterlingTools.WriteReport),
    ];

    return new ChatClientAgent(
        chatClient,
        name: "Sterling",
        instructions: SystemPrompt.Build(targetPath, outputPath),
        tools: tools);
}

AIFunctionFactory.Create reads each method's signature and [Description] attributes and generates the JSON schema the model sees. The method is the tool definition.

The CLI path then makes one call. This excerpt from SterlingAgent.RunAsync is trimmed to construction and the call; the try/catch, metrics, and status output around it are cut:

var chatClient = CreateChatClient(modelOptions);
var agent = BuildAgent(chatClient, targetPath, outputPath);

var stopwatch = Stopwatch.StartNew();
using var span = SterlingTrace.Instance.StartSpan("agent-run", ActivityKind.Client);
span?.WithTag("sterling.target", targetPath);

await agent.RunAsync(
    $"Review the C# codebase in: {targetPath}",
    cancellationToken: ct);

The model calls ListSourceFiles, calls AnalyzeFile on the paths it gets back, decides which files need a closer look, calls ReadFile on those, and ends with WriteReport. RunAsync returns when the model stops asking for tools. Sterling discards the response object, because the report is already on disk.

The move cost two things the builder had. The explicit cap of 50 round-trips is gone, so Sterling runs on the FunctionInvokingChatClient default, which is 40 in M.E.AI 10.4.1; a model that analyzes one file per round-trip reaches it on a codebase of about 40 files. The UseOpenTelemetry middleware is gone too, and the only span Sterling opens is agent-run around the whole call. In exchange, BuildAgent is a static function of an IChatClient and two paths, which is what a workflow executor needs to construct Sterling without the CLI.

The Tools

The model decides when each tool runs and how often.

ListSourceFiles

Recursive *.cs discovery that drops any path with a bin or obj segment and returns relative paths, sorted, with forward slashes. For a 50-file project the model sees 50 paths and no source. Its [Description] also tells the model that generated files are excluded, which the code does not do.

AnalyzeFile

A one-line delegation to QualityTools.AnalyzeCSharpFile in Agent.SDK. It parses the file with CSharpSyntaxTree.ParseText and returns per-method line count, cyclomatic complexity, and parameter count; file-level usings, namespaces, and types; anti-patterns (.Result, .Wait(), async void, empty catch blocks) with line numbers; and a health grade. It makes no LLM call.

The grade counts seven triggers: over 500 lines, over 1,000 lines, a method over complexity 10, a method over complexity 20, any anti-pattern, more than three anti-patterns, and more than 20 methods. Zero issues is an A, one a B, two a C, three or four a D, and five or more an F.

It is a syntax tree with no compilation and no semantic model, so the .Result check matches any member access named Result, including a record property that has nothing to do with tasks. The alternative is loading the project through MSBuild to get types, which puts a restore and a build in front of the first tool call. Parsing one file needs only the file, and its false positives are the kind ReadFile exists to catch.

ReadFile

A one-line delegation to FileTools.ReadFileContent, which resolves the path through ResolveSafePath, a prefix check against the target root, before reading. The prompt tells the model to use it on files graded C or worse, files with anti-patterns, and files named Program.cs, *Agent*.cs, *Service*.cs, or *Handler*.cs. This is where the model does work the tools can't: reading code and forming an opinion about it.

WriteReport

A one-line delegation to FileTools.WriteOutput, behind the same path check. It writes QUALITY.md in the target directory unless --output names another path. The prompt requires it as the final action, so the model composes the entire report and writes it in one call.

ListSourceFiles is the only one of the four with a body of its own: 40 lines by AnalyzeFile's count, which includes the attribute.

The Prompt

The system prompt is the only orchestration Sterling has.

CSI has a ScannerPlanner that assigns models to scanners and a ScannerRunner that runs each one with a timeout and retries. Sterling has SystemPrompt.Build, a 60-line method that returns one interpolated string. This excerpt keeps the role, the workflow, and the rules; the judgment categories and report format between them are cut:

You are Sterling, a staff engineer conducting a code quality review of a C# codebase.

## Workflow

1. Call ListSourceFiles with the target directory: {targetPath}
2. Call AnalyzeFile on every .cs file to collect metrics and health grades.
3. For files graded C or worse, or files with anti-patterns, call ReadFile to see the source.
   Also read architecturally important files (Program.cs, *Agent*.cs, *Service*.cs, *Handler*.cs).
4. Compose your report combining hard metrics with editorial observations.
5. Call WriteReport to write the report to: {outputPath}

## Rules

- Name specific methods, classes, and patterns. No vague observations.
- If a file is clean, skip it. Don't invent problems to fill space.
- Don't suggest rewrites. Suggest the smallest change that improves the code.
- Lead with metrics (these are facts), then add your editorial judgment.
- Your final action MUST be WriteReport. Do not end without writing the report.

The cut sections name the judgment categories - naming, single responsibility, hidden coupling, abstraction value, complexity budget, error handling, allocation patterns - and fix the report layout: executive summary, metrics table, file-by-file review, cross-cutting patterns, and recommendations ordered by impact.

The prompt stands in for three classes in the larger agents: CSI's ScannerPlanner (the model decides what to do), CSI's ScannerRunner (ChatClientAgent loops until the model stops calling tools), and ModelBoss's ReportFormatter (the model composes the markdown). It does not replace the runner's timeout and retry. A failed Sterling run logs the exception and exits with code 1, and the only time bound is the OpenAI client's 10-minute NetworkTimeout on each request.

The Output

AnalyzeFile grades every file in Sterling an A, including a 76-line method.

This is the tool's actual output for SterlingAgent.cs at 96b7b6f, produced by calling QualityTools.AnalyzeCSharpFile directly, with no model involved:

## SterlingAgent.cs

- Lines: 145
- Usings: 14
- Namespaces: 1
- Types: 1

### Methods

| Method | Lines | Complexity | Params |
|--------|-------|------------|--------|
| RunAsync | 76 | 7 | 2 |
| BuildAgent | 20 | 1 | 3 |
| CreateChatClient | 12 | 2 | 1 |

**Health Grade: A**

None of the seven triggers fire, and the other six files grade A as well. The tool reports 145 lines for a 144-line file because it counts Split('\n') segments and the file ends in a newline.

RunAsync resolves CLI arguments, prints status, checks the endpoint, builds the agent, runs it, and records the duration and the span. That is what the prompt's single-responsibility category is for, and no threshold in CalculateFileGrade can see it. The model only opens the file because the name rule sends *Agent*.cs to ReadFile regardless of grade.

The same prompt says to skip clean files in the file-by-file review. Pointed at its own source, the model has to decide whether an A with a 76-line method counts as clean.

The Graph

SterlingExecutor makes Sterling a node in a workflow graph.

The test for whether something is an agent or a service: remove the orchestrator. If the model can still finish the task with only its tools and system prompt, it's an agent. If it needs the planner, the runner, or the scorer driving it, it's a service that calls an LLM. Sterling passes that test. Point it at a directory with an LM Studio[3] endpoint and it writes a quality report.

Microsoft Agent Framework's workflow engine[5] composes agents as executors joined by typed edges. Commit 8686f2b added one for Sterling. The code below is the whole file with its doc comments and usings removed:

public sealed record SterlingRequest(string TargetPath, string OutputPath);

internal sealed partial class SterlingExecutor(IChatClient chatClient) : Executor("Sterling")
{
    [MessageHandler]
    private async ValueTask<string> HandleAsync(SterlingRequest request, IWorkflowContext context)
    {
        var agent = SterlingAgent.BuildAgent(chatClient, request.TargetPath, request.OutputPath);

        await agent.RunAsync($"Review the C# codebase in: {request.TargetPath}");

        return request.OutputPath;
    }
}

The executor takes a SterlingRequest, builds the same agent through BuildAgent, runs it, and emits the report path as a string for whatever edge comes next. The Microsoft.Agents.AI.Workflows.Generators package, referenced in Sterling.csproj, generates the handler routing from [MessageHandler] methods on partial executor classes.

No WorkflowBuilder in the repo constructs SterlingExecutor yet; the only workflow is the one-node example in its doc comment, and the class is internal, so nothing outside the Sterling assembly can build one. Its two tests check that SterlingRequest round-trips and deconstructs, and nothing calls HandleAsync. The handler also passes no cancellation token to RunAsync, so cancelling a workflow would not stop a review in progress.

The next step is the graph in that doc comment, then a second node. CSI and ModelBoss have no executors, so composing Sterling with CSI's context scan starts with writing one for CSI. Sterling itself stays one BuildAgent call and four tools, and the edges and any checkpointing belong to the workflow.

← Back to Articles