Tech watch
← All linksEvaluating models and agents against explicit tasks, criteria and test datasets.
Agent loops combine action, observation, and evaluation to refine an answer or retry an operation until it meets a criterion. Jesse Liberty demonstrates the pattern in C# with Microsoft Agent Framework: a loop agent, an evaluator, and an iteration limit.
The example contrasts a deterministic marker such as DONE with a model-based judge, then outlines safeguards to add: timeouts, idempotent calls, logging, and cost tracking. Each iteration adds latency and token usage.
Found via C# Digest
This repository brings together tools for working with the WebMCP API: a Chrome extension inspects the tools exposed by a page, their input schemas, and connection issues, while a CLI evaluates whether LLMs call the right tools against defined test cases.
Demos also show declarative and imperative integrations in practical interfaces. The collection helps move from experimentation toward more structured testing, while remaining a project intended to support adoption of the WebMCP API.
SkillOpt treats the skill itself as an artifact to improve: an agent runs tasks, receives evaluation, and its procedure is revised. Keeping the model and harness fixed makes instruction changes comparable.
Mastra combines tools, workflows, traces and evaluations for building TypeScript agents. The framework supports observing execution and improving behavior through experiments.
