AdTech · MCP Governance · AdOps Agents
SignalOps AI
MCP Governance Control Plane for AdOps Agents
Strategic context
Why this matters
SignalOps AI is a product concept and working prototype for AdTech operations teams exploring how MCP-style governed tool access can support safer campaign decisions. The platform acts as a governance control plane where campaign signals, tool calls, recommendations, approvals, and audit trails are structured inside a human-in-the-loop operating model.
The problem
Product challenge
AdOps teams often manage campaign delivery, VAST errors, publisher quality, brand-safety checks, pacing risk, targeting constraints, and manual approvals across fragmented systems. This creates slow diagnosis, unclear ownership, inconsistent review, and limited auditability.
Who this is for
Target users
Campaign Operations Manager
Owns delivery and pacing health across the campaign book and needs a governed view of which campaigns carry delivery, VAST, or brand-safety risk today.
AdOps Specialist
Runs day-to-day campaign diagnosis and needs an explainable, evidence-backed recommendation before acting on a flagged campaign.
Publisher Operations Lead
Manages publisher-side delivery and creative quality and needs VAST validation and inventory findings surfaced alongside campaign risk.
Revenue Operations / Yield Manager
Tracks delivery-at-risk and makegood exposure across the portfolio and needs governed tool access without direct write control.
Product / Compliance Reviewer
Approves or rejects high-risk agent recommendations and needs a full audit trail of tool calls, evidence, and rationale before signing off.
Product bet
Product strategy
SignalOps AI does not position AI as the product. The product is the control plane for campaign operations decisions. MCP and agents are used as controlled leverage inside the workflow: exposing approved tools, retrieving campaign signals, preparing recommendations, and routing high-risk actions to human reviewers.
How it works
Product workflow
Campaign risk prioritization
A delivery-risk queue ranks campaigns by pacing exposure, VAST errors, and brand-safety findings, so operators start from the highest-risk case.
Governed evidence retrieval
An MCP-enabled agent calls a bounded set of read-only tools — campaign health, pacing, VAST validation, brand-safety findings, and policy search — before any reasoning begins.
Structured diagnosis
Retrieved evidence and AdOps playbook context are synthesized into a ranked root-cause diagnosis with confidence and cited evidence.
Deterministic risk scoring
Every proposed action is scored by an additive risk function, not model judgment, to classify it as low, medium, high, or critical.
Human-in-the-loop approval
High-risk actions open an approval request; only authorized reviewer roles can approve or reject, and every decision requires a written rationale.
Audit-ready decision trail
Tool calls, evidence, risk scores, and reviewer decisions are logged so a governed run is fully reconstructable after the fact.
AI leverage
Where AI is used — and where it isn't
AI is scoped in as one execution lever inside the product strategy, not the strategy itself. Every recommendation is explainable and every high-risk action is reviewable.
MCP-enabled governed tool access
An agent calls a bounded, MCP-shaped set of read-only tools for campaign health, pacing, VAST validation, and brand-safety findings — never a raw database connection.
RAG over AdOps playbooks
Retrieved operating-playbook context grounds the diagnosis in documented practice rather than unsourced model output.
Structured LLM reasoning
The reasoning step returns a strict, schema-validated diagnosis; any cause that can't cite valid evidence is discarded.
Deterministic risk scoring
Risk classification is computed by a rule-based scoring function over pacing, creative rejection, VAST errors, and brand-safety severity — not left to model judgment.
Grounded fallback for availability
If the model provider is unavailable, the same evidence and tool-call pipeline still returns a clearly labeled deterministic diagnosis rather than failing silently.
Human approval for high-risk actions
Automation stops at the recommendation. Any action scored high-risk requires an authorized reviewer's approval before it proceeds.
Where AI is deliberately not used
AI does not receive database credentials, write access, or an action-execution tool, and it cannot approve or escalate its own output. Critical-risk actions are blocked outright rather than routed for approval, and every recommendation remains a proposal until a named reviewer signs off.
Product system
Core Product Modules
The product broken into its core operating modules — what each one does, who it's built for, and why it matters, before the architecture behind it.
Campaign Health Dashboard
Surfaces portfolio-level delivery, pacing, and risk status across the campaign book.
User: Campaign Operations Manager
Value: Starts investigation from portfolio risk instead of a blank workspace.
MCP Tool Registry
Enumerates the governed, read-only tools an agent is allowed to call, with input and output contracts.
User: AdOps Specialist / Platform Owner
Value: Makes agent tool access inspectable and governable instead of implicit.
Governed Tool Access
Restricts agent tool calls to a bounded, read-only surface with no direct database or write access.
User: AdOps Specialist
Value: Keeps agent leverage controlled while still useful for diagnosis.
AdOps Agent Interface
Lets an operator trigger a governed diagnosis run against a flagged campaign.
User: AdOps Specialist / Campaign Operations Manager
Value: Converts a manual, cross-system investigation into one governed workflow.
Delivery Risk Review
Reviews pacing, targeting, inventory, and delivery-gap evidence behind a flagged campaign.
User: Publisher Operations Lead
Value: Separates delivery risk from creative or brand-safety risk for faster triage.
VAST Validation
Checks creative runtime quality and VAST errors independently of creative approval state.
User: Publisher Operations Lead
Value: Surfaces runtime creative issues an approval-status view alone would miss.
Approval Queue
Routes high-risk recommendations to an authorized reviewer and requires a written rationale on every decision.
User: Product / Compliance Reviewer
Value: Keeps high-risk campaign actions accountable to a named human.
Policy / Brand-Safety Review
Checks campaign and creative signals against brand-safety and policy findings.
User: Product / Compliance Reviewer
Value: Makes policy-sensitive decisions consistent and evidence-backed.
Recommendation Explanation Layer
Translates evidence and tool output into a client-safe, plain-language brief.
User: Revenue Operations / Yield Manager
Value: Lets teams communicate a diagnosis without exposing internal auction mechanics.
Audit Logs
Records every tool call, risk score, policy check, and reviewer decision tied to a run.
User: Product / Compliance Reviewer
Value: Makes any governed run fully reconstructable after the fact.
These modules define the product surface before the architecture and roadmap decisions.
Governed AI access
MCP Governance Architecture
How the decision workspace, backend APIs, MCP layer, and human review fit together. Agents propose; they do not execute high-risk actions on their own.
Decision workspace (frontend)
The Next.js frontend is the decision workspace for AdOps users — campaign health, risk queues, diagnosis, approvals, and governance records in one governed surface.
Governance APIs (backend)
A FastAPI backend exposes campaign, risk, VAST validation, and approval APIs behind JWT authentication and role-based access control.
MCP layer
A standalone MCP server exposes a bounded set of read-only tools that an agent or MCP-compatible host can call — nothing beyond what's registered.
Tool registry
The tool registry defines each available tool's name, description, input schema, and output contract, so agent access stays enumerable and governable.
Agent boundary
The agent has no database credentials, no write access, and no action-execution tool — it proposes a diagnosis, it doesn't act on it.
Human approval
High-risk proposed actions open an approval request; only authorized reviewer roles can approve or reject, and every decision requires a written rationale.
Audit-ready decision trail
Every tool call, risk score, policy check, and reviewer decision is persisted, so a governed run is fully reconstructable after the fact.
How decisions flow
Product Architecture & Decision Pipeline
The architecture is built around one question a reviewer asks under time pressure: why is this campaign flagged, what evidence backs the recommendation, and who is accountable for the tool calls behind it? Each stage exists to answer that question through governed, inspectable steps — not to automate the decision itself.
Stage 1
Inputs / Signals
Campaign delivery, pacing, VAST, and brand-safety signals for a flagged campaign
Stage 2
Processing / Decision Logic
Deterministic risk scoring plus retrieval against AdOps playbooks and the MCP tool registry
Stage 3
AI / Automation Leverage
Governed, read-only MCP tool calls feed a structured LLM diagnosis with cited evidence
Stage 4
Human Review / Governance
High-risk recommendations open an approval request requiring an authorized reviewer's written rationale
Stage 5
Product Action
Approve, reject, or — for critical-risk actions — block outright before anything reaches the campaign
Stage 6
Measurement / Outcome
Audit-ready decision trail: every tool call, risk score, and reviewer decision reconstructable after the fact
Under the hood
Technology Behind the Product
A compact view of what supports the product story — not a developer stack showcase.
Frontend / Product UI
Next.js, TypeScript
Serves the authenticated decision workspace: campaign health, risk queues, diagnosis, and approvals.
Backend / API
FastAPI, JWT auth, RBAC
Serves campaign, risk, VAST validation, and approval APIs behind role-gated access.
Data / Storage
PostgreSQL, pgvector
Stores campaign, governance, and audit records alongside the vector index used for playbook retrieval.
AI / Retrieval
RAG over AdOps playbooks, structured LLM output, deterministic fallback
Grounds diagnosis in retrieved playbook evidence, with a labeled fallback path when no model provider is available.
MCP / Agent Layer
Standalone MCP server (FastMCP) with seven read-only tools; embedded MCP governance API
Exposes governed, enumerable tool access instead of open agent permissions.
Governance
Deterministic risk scoring, human approval queue, audit logging
Keeps every high-risk recommendation reviewable and every governed run auditable.
Deployment / Runtime
Docker, Docker Compose, Vercel, Render, GitHub Actions CI
Matches how the frontend, backend, and MCP server actually run and ship.
Product judgment
Key product decisions
The decisions below are the ones that actually shape whether a system like this earns trust in production.
Expose agent tool access through an MCP-shaped, enumerable tool registry instead of ad hoc function calling.
- Why
- Ungoverned function-calling makes it impossible to answer what can this agent call, and what does each call return — MCP's standard shape (name, input schema, output contract) makes that answerable and reusable across future agents.
- Tradeoff
- More upfront investment in a registry and contract layer than wiring a single agent's tools directly.
- Impact
- Turns tool governance into infrastructure that scales across agents instead of bespoke code per agent.
Score risk with a deterministic, additive function instead of model judgment.
- Why
- A risk score that gates human approval has to be explainable and reproducible, not a probabilistic guess.
- Tradeoff
- The scoring model needs calibration against real pacing, creative, and brand-safety thresholds to stay accurate.
- Impact
- Makes the approval gate defensible to a reviewer and consistent across every run.
Block critical-risk actions outright instead of routing everything to human approval.
- Why
- Not every risk level deserves the same response; a single approval queue for all risk would bury genuinely urgent cases.
- Tradeoff
- Requires a well-calibrated critical threshold, since a false block delays a legitimate action.
- Impact
- Keeps the approval queue focused on the decisions that actually need a human judgment call.
Keep the agent's tool surface strictly read-only, with no database credentials or write access.
- Why
- An agent that can both diagnose and act removes the human checkpoint the governance model depends on.
- Tradeoff
- Every remediation still requires a separate, human-initiated action outside the agent's reach.
- Impact
- Makes the system safe to expose to a wider set of operators without expanding what could go wrong.
Ship a labeled deterministic fallback instead of failing when no model provider is available.
- Why
- A governance workflow that breaks whenever a provider is down isn't trustworthy enough to depend on.
- Tradeoff
- The fallback path needs to stay meaningfully useful, not just a placeholder, since it's a real operating mode.
- Impact
- Keeps the diagnosis and approval workflow usable regardless of model-provider availability.
Require a written rationale on every approval or rejection, with no override on a re-decided request.
- Why
- An audit trail without a stated reason, or one that can be silently overwritten, doesn't hold up to compliance review.
- Tradeoff
- Adds friction to the reviewer's workflow compared to a single-click approve/reject.
- Impact
- Produces a decision trail a compliance or revenue stakeholder can actually trust after the fact.
Scope discipline
MVP scope
What shipped in v1, and what was deliberately left out.
Included
- Governed, read-only MCP tool access to campaign health, pacing, VAST validation, and brand-safety signals
- RAG-grounded diagnosis with cited evidence and a labeled deterministic fallback
- Deterministic risk scoring with human approval for high-risk actions
- Full audit trail of tool calls, risk scores, and reviewer decisions
Excluded from v1
- Autonomous execution of any campaign-changing action
- MCP resources and prompts (only tools are exposed today)
- Automated remediation without a human-initiated approval step
Impact model
Measurement focus
What this system's output would be measured against once in operation — a measurement framework, not results from a live enterprise deployment.
Diagnosis speed
How quickly a flagged campaign moves from risk signal to an evidence-backed diagnosis.
High-risk approval coverage
Share of high-risk recommendations that reach an authorized reviewer with full evidence before any action.
Audit completeness
Whether every tool call, risk score, and reviewer decision on a run is reconstructable after the fact.
Governed tool adoption
How much of agent tool access runs through the governed MCP registry rather than ad hoc integration.
Risk signal distribution
Roadmap
What comes next
MCP governance MVP
- Read-only MCP tool registry and governed tool access
- RAG-grounded diagnosis with deterministic fallback
- Deterministic risk scoring and human approval queue
Governance at scale
- MCP resources and prompts, not just tools
- Cross-campaign risk correlation
- Reviewer performance and SLA analytics
Guarded automation
- Policy-gated automation for proven low-risk, reversible actions
- Predictive delivery-risk alerts ahead of SLA breach
- Client-facing transparency reporting
Product leadership takeaway
What this case study demonstrates
Product judgment
Recognized that agent tool access is only valuable if it's governable — enumerable, attributable, and auditable — and designed the tool registry and MCP layer around that requirement instead of wiring tools directly to the agent.
Tradeoff accepted
Chose a strictly read-only agent boundary and human-gated high-risk approval over broader automation, accepting slower remediation in exchange for a defensible audit trail.
Business relevance
Directly addresses what AdTech compliance, revenue, and platform stakeholders ask for before trusting an agent near campaign delivery: enumerable tool access, deterministic risk scoring, and a reconstructable decision trail.
Stakeholder complexity
Required designing one governance layer — tool registry, risk scoring, approval queue, audit log — that works the same way regardless of which agent or workflow calls it, rather than bespoke governance per feature.
See how this pattern applies across other domains.
Every case study follows the same discipline: real problem, real users, explainable AI, and a measurable impact model.