Meta AI Business Assistant on Instagram

Project overview

Context: Meta launched the AI Business Assistant inside Instagram Boost — a lightweight ad tool used by small business owners to promote posts directly from the app. To answer complex ad, policy, and billing questions in real time, the assistant needed fresh, highly structured source documentation it could reliably retrieve via RAG. Without it, the model hallucinated or defaulted to generic responses that sent users straight to support.

Challenge: No dedicated knowledge infrastructure existed for the IG Boost surface. The AI was querying general Help Center content not written for machine parsing — missing edge cases, misrouting intents, and surfacing answers that didn't match the actual product experience small business users were having.

Approach: Build the ground-truth content layer the AI needed to work: structured knowledge nodes mapped explicitly to IG Boost workflows, a query dataset to scale intent routing, and a systematic failure mode catalog to drive model reliability improvements.

~1,581,606

overall impressions on IG Boost content

-59.2%

reduction in human agent contact rate

reduction in support cases on troubleshooting content

-73.7%

New Product Entry Integration

Improved AI retrieval quality, completely consolidating overlapping data structures into a unified, machine-readable repository.

Single Source of Truth Node

Successfully authored and deployed specialized knowledge nodes, scaling automated entry points for global merchants.

My role

  • Launch Content Architecture: Authored a new library of support and education content explicitly mapped to IG Boost user workflows — ad setup, budget optimization, account troubleshooting — providing the structured source documentation the AI queries to generate accurate, context-specific answers.

  • RAG Structural Alignment: Restructured content into machine-readable modules so the LLM could cleanly parse complex ad parameters — targeting limits, Apple service fee workarounds, policy constraints — and surface accurate answers during live sessions rather than falling back on generic responses.

  • Intent Mapping & Scaling: Built comprehensive user query datasets that documented linguistic ambiguities and edge cases, directly improving how the AI recognizes and routes merchant intents across the Instagram advertising flow.

  • Failure Mode Analysis: Monitored early beta model responses against Help Center metadata, diagnosing and cataloging edge-case retrieval errors — and bringing specific structural fixes back to the content layer to address them at the source.

  • Ground-Truth Evaluation: Conducted multi-dimensional quality evaluations producing ground-truth labels across retrieval accuracy, conversational voice consistency, and taxonomy precision, establishing the benchmarks used to validate automated LLM judges.

Key Collaborators

Embedded with Product, Engineering, and a Content Designer to push for architecture decisions that directly improved RAG retrieval quality — consolidating duplicate content structures and authoring the knowledge nodes the AI needed to stop hallucinating.

Next project: Creator Payouts at Meta