Red-team your AI agents
before an attacker
does it for you.

Project Moonshot is the open LLM testing toolkit from the AI Verify Foundation, Singapore’s AI governance testing body. It provides a catalogue of benchmarks — organised as cookbooks and recipes covering safety, bias, jailbreak resistance, privacy leakage, and capability — plus adversarial red-teaming attack modules. Obiguard runs that catalogue inside your own project, against the AI Agents you have registered.

A benchmark run sends a fixed catalogue of prompts — a cookbook or a single recipe — and grades the answers, so results are comparable between runs and between models. Red-teaming uses attack modules that probe the agent adaptively; each module owns its own termination logic rather than a fixed prompt count, so it explores rather than measures. Most teams use benchmarks to track regression over time and attack modules to find something new.

Grades come from each recipe’s own grading scale, and those scales are not all the same. Most recipes are higher-is-better, where A is 80–100 and E is 0–19. The AdvGLUE and toxicity recipes invert that, because their raw score measures attack success or toxicity rather than performance. The MLCommons recipes do not use letters at all and band into Low Risk through High Risk instead. Obiguard always renders the grade alongside its scale, because a raw score without its scale is meaningless.

Any AI Agent registered in the project. Obiguard resolves the agent to its provider connector and model and injects the credential server-side per run, so the portal never handles a provider token. You choose whether to test the agent’s current system prompt, another prompt from the registry, or a custom one — which is how teams benchmark a prompt change before it reaches production.

No. Runs are dispatched as separate benchmarking or red-teaming sessions against the target model; they do not sit on your live request path and do not consume your policy enforcement budget. The Evaluations page reads posture from the latest completed run per target, so an old run against a since-replaced model cannot drag the headline grade around.

Yes. Each run is retained against the project with its target, the prompt under test, the catalogue version, the resulting grade, and any failing recipes. That record is what framework categories covering adversarial testing ask for — NIST AI RMF MEASURE 2.7, AISCF CR06 security assessment and testing, and the testing side of ISO/IEC 42001 Clause 9.