AdTech / Campaign Operations
AdOps Signal
Campaign operations product for AdTech teams managing delivery risk, policy ambiguity, brand-safety checks, and approval workflows.
Technical walkthrough available via GitHub profile or on request.
Strategic context
Why this matters
AdOps teams sit at the intersection of revenue, brand safety, and policy compliance, but they typically make high-stakes calls from scattered dashboards and tribal knowledge. AdOps Signal is a campaign-operations case study that shows how a senior product leader reduces decision latency without removing accountability: campaign signals are triaged automatically, policy is checked against a governed knowledge base, and every recommendation carries a confidence score and an audit trail. Nothing ships without a human approval on high-risk calls. This case study demonstrates product judgment on where AI should assist versus decide inside a media-operations workflow, and how to design for enterprise trust from the first version.
The problem
Product challenge
AdOps teams operate with scattered signals: campaign delivery gap, spend variance, publisher policy, brand safety, targeting setup, and creative or VAST tag issues. When these signals aren't connected, small problems compound into revenue loss, policy violations, client dissatisfaction, and manual escalation overload. Reviewing a single flagged campaign today means pulling data from a delivery dashboard, a policy wiki, a spend report, and a ticket queue, then applying judgment that varies by reviewer. The risk isn't just operational delay — it's inconsistent decisions on the exact area where AdOps organizations answer directly to legal, finance, and client trust teams.
Who this is for
Target users
AdOps Manager
Owns day-to-day campaign health and needs a single, trustworthy view of which campaigns require attention today.
Campaign Manager
Responsible for delivery and pacing on individual accounts, needs fast, explainable answers on why a campaign is flagged.
Product Manager for AdTech Platform (internal stakeholder)
Owns the roadmap for internal tooling and needs to justify where automation reduces cost without increasing risk.
Trust & Safety / Policy Reviewer (internal stakeholder)
Approves or rejects high-risk decisions and needs evidence, not just a verdict, to stay accountable to policy.
Revenue Operations Lead (internal stakeholder)
Cares about revenue at risk and SLA exposure across the full campaign portfolio, not a single flagged case.
Product bet
Product strategy
The product bet is that AdOps teams don't need more dashboards — they need a governed decision layer that turns scattered signals into one explainable, reviewable recommendation per flagged campaign, so decision quality improves without removing human accountability for revenue and brand-safety calls.
How it works
Product workflow
Campaign signal ingestion
Delivery pacing, spend variance, targeting configuration, and creative metadata are normalized into a single campaign signal record.
Policy / RAG check
Signals are checked against a retrieval-augmented index of publisher policy, brand-safety guidelines, and internal SLA documentation.
Risk classification
Deterministic rules and LLM reasoning combine to classify the issue by type and severity, with a confidence score attached.
Recommended action
The system proposes a specific next action — pause, adjust pacing, escalate to policy, or no action — with a plain-language rationale.
Human approval
Medium- and high-risk recommendations route to the right reviewer before any change is applied.
Audit log
Every signal, recommendation, reviewer decision, and outcome is recorded for governance and post-hoc review.
Impact tracking
Resolved cases feed back into a portfolio-level view of revenue protected, SLA exposure, and reviewer throughput.
AI leverage
Where AI is used — and where it isn't
AI is scoped in as one execution lever inside the product strategy, not the strategy itself. Every recommendation is explainable and every high-risk action is reviewable.
RAG over campaign and policy documents
Used only where explainability and traceability matter: publisher policy, brand-safety rules, and SLA terms are retrieved so recommendations cite the specific clause behind a decision.
Rules for deterministic checks
Hard thresholds (pacing bands, spend variance, SLA deadlines) are enforced with rules, not left to model judgment.
LLM for explanation and recommendation
The LLM synthesizes rule output and retrieved policy into a reviewer-readable explanation and a recommended next action.
Confidence scoring
Each recommendation carries a confidence score derived from signal completeness and policy-match strength, shown directly to the reviewer.
Human approval for high-risk decisions
Automation stops at the recommendation. Anything touching spend, brand safety, or client-facing delivery requires sign-off.
Audit trail for governance
Full decision provenance — inputs, retrieved policy, model output, reviewer action — is stored for compliance review.
Where AI is deliberately not used
AI is not used to execute pause, spend, or brand-safety actions autonomously, and it does not override hard policy or SLA thresholds — those stay deterministic rules. The model's role is confined to explanation, synthesis, and recommendation.
Product system
Core Product Modules
The product broken into its core operating modules — what each one does, who it's built for, and why it matters, before the architecture behind it.
Campaign Risk Queue
Prioritizes campaigns with pacing, delivery, policy, or brand-safety risk.
User: AdOps Manager / Campaign Manager
Value: Reduces manual triage and highlights revenue-risk campaigns.
Policy & Brand-Safety Check
Checks campaign setup, creative signals, targeting, and publisher constraints against policy logic.
User: Policy Reviewer / AdOps Manager
Value: Makes approval decisions more consistent and explainable.
Recommendation & Action Panel
Shows recommended next actions such as approve, hold, adjust targeting, review creative, or escalate.
User: Campaign Manager
Value: Converts risk signals into operational action.
Human Approval Workflow
Keeps high-risk decisions accountable through review, approval, and audit trail.
User: AdOps Lead / Reviewer
Value: Balances speed with brand, revenue, and compliance risk.
Campaign Impact View
Links risk decisions to delivery gap, spend variance, SLA pressure, and affected campaigns.
User: Revenue Operations / Product Lead
Value: Connects AdOps decisions to business impact.
These modules define the product surface before the architecture and roadmap decisions.
How decisions flow
Product Architecture & Decision Pipeline
The architecture is built around one question a reviewer asks under time pressure: why is this campaign flagged, and what evidence backs the recommendation? Each stage exists to answer that question with less manual digging, not to automate the decision itself.
Stage 1
Inputs / Signals
Delivery pacing, spend variance, targeting, and policy/brand-safety signals
Stage 2
Processing / Decision Logic
Risk classification against policy and SLA thresholds
Stage 3
AI / Automation Leverage
RAG-grounded policy explanation with confidence scoring
Stage 4
Human Review / Governance
Reviewer approval required for medium- and high-risk actions
Stage 5
Product Action
Pause, adjust pacing, escalate, or approve the campaign
Stage 6
Measurement / Outcome
Revenue-at-risk reduction, SLA exposure, reviewer throughput
Under the hood
Technology Behind the Product
A compact view of what supports the product story — not a developer stack showcase.
Frontend / Product UI
Next.js, TypeScript
Keeps the reviewer-facing decision surface fast and type-safe as the workflow grows.
Backend / API
FastAPI
Serves signal ingestion and recommendation logic behind the review queue.
Data / Storage
PostgreSQL, pgvector
Stores structured campaign and audit data alongside the vector index used for policy retrieval.
AI / Retrieval
RAG, LLM abstraction
Grounds recommendations in retrieved policy text instead of unsourced model output.
Governance
Audit logs, approval workflows, risk scoring
Keeps every high-risk recommendation reviewable and attributable.
Deployment / Runtime
Vercel, Docker where applicable
Matches how the product UI and services would actually ship.
Product judgment
Key product decisions
The decisions below are the ones that actually shape whether a system like this earns trust in production.
Keep every medium- and high-risk recommendation behind human approval instead of full automation.
- Why
- An incorrect automated action on spend or brand safety is more costly and harder to reverse than a slower, reviewed one.
- Tradeoff
- Slower resolution than full automation, and it requires sustained reviewer capacity.
- Impact
- Builds the trust with operations and policy teams that adoption actually depends on.
Use RAG only where explainability and traceability matter, and keep deterministic rules for policy and SLA thresholds.
- Why
- Hard thresholds need to be enforced exactly; the model's value is explaining and synthesizing, not doing arithmetic on pacing bands.
- Tradeoff
- More implementation complexity than a single-model approach.
- Impact
- A more defensible system than a pure-LLM design, which matters when a reviewer has to justify a call to legal or finance.
Choose revenue-at-risk and SLA exposure reduced as the north-star metrics, not raw automation volume.
- Why
- AdOps and revenue leaders act on dollars and SLA exposure; a high 'cases auto-handled' number says nothing about whether the right calls were made.
- Tradeoff
- Requires explicit, labeled impact estimates rather than a simpler throughput count.
- Impact
- Keeps the roadmap pointed at the outcome the business actually cares about.
Design auditability into the schema from day one, not as a later add-on.
- Why
- AI-influenced decisions touching revenue and brand safety must be reviewable after the fact.
- Tradeoff
- More upfront data modeling effort before any feature work ships.
- Impact
- Removes a rework cycle later and signals enterprise-readiness to compliance stakeholders early.
Route decisions through the reviewer roles that already own the outcome, instead of introducing a new approval authority.
- Why
- Policy and revenue teams were unlikely to adopt a system that changed who was accountable for a call.
- Tradeoff
- Slower initial rollout while trust in the recommendations is established.
- Impact
- Lowers adoption risk by fitting into an existing accountability structure rather than fighting it.
Exclude autonomous execution of any campaign-changing action from MVP scope.
- Why
- Proving recommendation quality has to come before removing the human step, not after.
- Tradeoff
- The MVP moves slower than a fully automated pipeline would.
- Impact
- Keeps the first release low-risk enough to actually get approved for a real campaign book.
Scope discipline
MVP scope
What shipped in v1, and what was deliberately left out.
Included
- Rule-based signal classification for pacing, spend, and SLA thresholds
- RAG-backed policy explanation with citations
- Manual reviewer approval queue for medium/high-risk cases
- Portfolio-level revenue-at-risk and SLA dashboard
Excluded from v1
- Autonomous execution of pause, adjust, or spend actions
- Cross-campaign predictive risk modeling
- Client-facing transparency reporting
Impact model
Measurement focus
What this system's output would be measured against once in operation — a measurement framework, not results from a live enterprise deployment.
Campaign review speed
How quickly a flagged campaign moves from signal to a reviewed, actionable recommendation.
High-risk decision coverage
Share of high-risk campaign signals that reach a reviewer with evidence attached, not just a raw alert.
Approval throughput
How many flagged campaigns a reviewer can move through once triage and evidence are pre-assembled.
Escalation reduction
Fewer tickets escalated without a clear next action already attached.
Risk signal distribution
Roadmap
What comes next
Governed triage MVP
- Rule-based signal classification
- RAG-backed policy explanation
- Manual approval queue
Portfolio-level decisioning
- Cross-campaign risk correlation
- Reviewer performance and SLA analytics
- Configurable escalation policies per client
Adaptive governance
- Feedback loop from reviewer overrides into rule tuning
- Predictive delivery-risk alerts before pacing breaks SLA
- Client-facing transparency reporting
Product leadership takeaway
What this case study demonstrates
Product judgment
Recognizing that AdOps' real bottleneck was inconsistent judgment across reviewers, not a lack of data — and designing a system that standardizes the explanation, not just the alert.
Tradeoff accepted
Chose slower, human-approved decisions over faster full automation, accepting reviewer-capacity cost in exchange for defensibility on revenue- and brand-safety-critical calls.
Business relevance
Ties directly to revenue protection, SLA exposure, and the audit trail enterprise buyers and compliance teams require before they'll adopt any AI-assisted approval workflow.
Stakeholder complexity
Required aligning AdOps, campaign management, policy/trust & safety, and revenue operations stakeholders around one shared, explainable decision record instead of four separate views of the same campaign.
See how this pattern applies across other domains.
Every case study follows the same discipline: real problem, real users, explainable AI, and a measurable impact model.