Skip to content

Agent Intelligence Enhancements

Date: November 18, 2025 Branch: claude/review-webinar-campaign-01Jce2Emqdr9LsiNuSA5Etgf

Overview

This document describes comprehensive enhancements to the Vell agent system to transform it from a basic content generator into an intelligent, strategic GTM platform that:

  1. Makes knowledge usage visible - Explicit RAG in execution traces
  2. Evaluates strategically - Analysis tied to business goals, not just generic scores
  3. Refines iteratively - Automatic improvement loops to hit quality targets
  4. Generates coordinated series - True multi-piece campaigns, not single outputs
  5. Structures for handoff - Outputs ready for different team members

Problem Statement

The original webinar campaign execution (ID 35) showed: - ❌ No evidence of knowledge base usage (generic inputs) - ❌ Analysis disconnected from business goals (SEO: 40 but why?) - ❌ No refinement loop (generate → analyze → stop) - ❌ "Series" was a single email, not a sequence - ❌ No structured handoff (just text blobs)

Result: Indistinguishable from a basic "AI content generator" - not Vellocity-level intelligence.

Enhancements Made

1. Enhanced Workflow Planning (WorkflowPlanner)

File: app/Extensions/ContentManager/System/Services/AgentCore/WorkflowPlanner.php

Changes: - Added instruction #4: Explicit KB queries - Plan query_knowledge_base steps BEFORE generate_text to make RAG transparent - Added instruction #5: Refinement loops - Plan analyze_content → refine_content sequences for high-quality outputs - Added instruction #6: Series generation - Use generate_content_series for coordinated sequences - Added comprehensive examples showing KB queries, refinement, and series workflows

Impact: Agents now plan workflows that: - Query knowledge base explicitly (visible in trace) - Refine content iteratively until quality targets are met - Generate true series (3 emails, not 1 email)

Example Planned Workflow (webinar campaign): 

{
  "steps": [
    {
      "capability": "query_knowledge_base",
      "parameters": {
        "query": "webinar details event information product features"
      }
    },
    {
      "capability": "generate_content_series",
      "parameters": {
        "content_type": "social_post",
        "series_count": 3,
        "series_type": "social_campaign"
      },
      "depends_on": [0]
    },
    {
      "capability": "analyze_content",
      "parameters": {
        "content": "social media series",
        "target_seo_score": 70,
        "target_brand_alignment": 80
      },
      "depends_on": [1]
    }
  ]
}

2. Strategic Context in Company Model

File: app/Models/Company.php Migration: database/migrations/2025_11_18_000001_add_strategic_context_to_companies_table.php

New Fields Added: - target_keywords (JSON array) - SEO keywords for content optimization - Example: ["AWS Marketplace", "co-sell", "B2B SaaS"] - key_personas (JSON array of objects) - ICPs with pain points and goals - Example: [{"title": "Alliance Manager", "pain_points": [...], "goals": [...]}] - gtm_goals (JSON array) - Business outcomes to align content with - Example: ["Drive webinar registrations", "Generate 50 MQLs"] - market_positioning (text) - Positioning statement - Example: "The only unified co-sell platform for AWS Marketplace sellers" - competitive_differentiators (JSON array) - Unique advantages - Example: ["Native AWS integration", "Automated partner matching"] - value_propositions (JSON array of objects) - Value by segment - Example: [{"segment": "ISVs", "value": "Close deals 3x faster"}]

Impact: Brand Voice now includes strategic GTM context, not just basic company info.

3. Enhanced Brand Voice Context Builder

File: app/Extensions/ContentManager/System/Services/AgentCore/BrandVoiceContextBuilder.php

Changes: - Added strategic fields to buildBrandVoiceContext() method - Enhanced formatForPrompt() to include: - Market positioning - Target keywords with SEO instruction - Personas with pain points/goals - GTM goals tied to business outcomes - Competitive differentiators - Value propositions by segment

Impact: All AI prompts now include strategic context, making generated content: - Aligned with market positioning - Optimized for target keywords - Addressing persona pain points - Driving toward GTM goals

Example Formatted Context: 

# Brand Voice Context
Company: Vellocity
Market Position: The only unified co-sell platform for AWS Marketplace sellers

## Target Keywords (SEO)
AWS Marketplace, co-sell, B2B SaaS, cloud partnerships
IMPORTANT: Incorporate these keywords naturally into content for SEO optimization.

## Target Personas
- Alliance Manager
  Pain Points: Manual partner tracking, No co-sell visibility
  Goals: Close more co-sell deals, Automate partner workflows

## GTM Goals
IMPORTANT: Align content with these business outcomes:
- Drive webinar registration from AWS partners
- Generate 50 qualified leads from Marketplace sellers
- Close 3 co-sell deals by Q1

4. Strategic Content Analysis

File: app/Extensions/ContentManager/System/Services/Capabilities/AnalyzeContentCapability.php

Changes: - Accept target quality scores as parameters (target_seo_score, target_brand_alignment, etc.) - Evaluate content against strategic context: - Which target keywords are present/missing - Does content address persona pain points? - Does content drive GTM goals? - Are competitive differentiators highlighted? - Return enhanced analysis with: - meets_targets - Boolean flags for each metric - strategic_evaluation - Keyword coverage, persona alignment, GTM alignment - Action-oriented recommendations tied to business goals

Impact: Analysis is no longer generic scores - it's strategic evaluation against business outcomes.

Example Analysis Output: 

{
  "scores": {
    "seo": 45,
    "brand_alignment": 60
  },
  "meets_targets": {
    "seo": false,
    "brand_alignment": false,
    "overall": false
  },
  "strategic_evaluation": {
    "keywords_used": ["AWS Marketplace"],
    "keywords_missing": ["co-sell", "B2B SaaS"],
    "persona_alignment": "Does not address Alliance Manager pain points",
    "gtm_goal_alignment": "Weak connection to webinar registration goal"
  },
  "recommendations": [
    "Add 'co-sell' keyword naturally in CTA",
    "Address pain point: manual partner tracking",
    "Strengthen registration urgency to drive GTM goal"
  ]
}

5. New Capability: Refine Content

File: app/Extensions/ContentManager/System/Services/Capabilities/RefineContentCapability.php

Purpose: Iteratively refine content based on analysis feedback until quality targets are met.

Parameters: - original_content - The content to refine - analysis_results - Analysis feedback with scores and recommendations - refinement_instructions - Optional guidance

How It Works: 1. Takes original content + analysis feedback 2. Builds refinement prompt with: - Strategic context (keywords, personas, GTM goals) - Analysis issues (missing keywords, low scores, strategic gaps) - Specific recommendations 3. Generates improved version addressing all identified issues 4. Returns refined content with improvements_applied list

Impact: Content is automatically improved until it meets quality standards.

Example Usage (in workflow): 

{
  "capability": "refine_content",
  "parameters": {
    "original_content": "from step 1",
    "analysis_results": "from step 2",
    "refinement_instructions": "Improve to meet target: SEO >= 80, Brand Alignment >= 85"
  },
  "depends_on": [1, 2]
}

6. New Capability: Generate Content Series

File: app/Extensions/ContentManager/System/Services/Capabilities/GenerateContentSeriesCapability.php

Purpose: Generate coordinated series of content pieces that work together as a campaign.

Parameters: - content_type - Type of content (email, social_post, blog_post) - topic - Subject matter - series_count - Number of pieces (e.g., 3) - series_type - Strategy type (email_sequence, social_campaign, webinar_sequence, blog_series) - length - short | medium | long

Built-in Series Strategies:

  1. Email Sequence (e.g., 3 emails):
  2. Email #1: Announcement - Build awareness and interest
  3. Email #2: Value/Education - Deepen engagement, address objections

  4. Email #3: Urgency/CTA - Create urgency, strong call-to-action

  5. Social Campaign (e.g., 3 posts):

  6. Vary angles: educational → emotional → social proof → CTA
  7. Each post self-contained but consistent messaging

  8. Progressive engagement

  9. Webinar Sequence:

  10. Build anticipation and value
  11. Address different pain points/personas
  12. Progressive urgency as date approaches

Output Structure: 

{
  "success": true,
  "series": [
    {"piece_number": 1, "content": "...", "word_count": 150},
    {"piece_number": 2, "content": "...", "word_count": 145},
    {"piece_number": 3, "content": "...", "word_count": 160}
  ],
  "series_count": 3,
  "series_type": "email_sequence"
}

Impact: True multi-piece campaigns, not single outputs. Each piece has distinct angle but coordinated strategy.

7. Capability Registration

File: app/Extensions/ContentManager/System/Services/AgentCore/CapabilityRegistry.php

Added to Registry: 1. query_knowledge_base - Now explicit and visible in traces 2. refine_content - Iterative refinement 3. generate_content_series - Coordinated series generation

Impact: These capabilities are now available to all agents for intelligent workflows.

Migration Instructions

1. Run Database Migration

php artisan migrate

This adds strategic context fields to companies table.

2. Bootstrap New Capabilities

use App\Extensions\ContentManager\System\Services\AgentCore\CapabilityRegistry;

CapabilityRegistry::bootstrapDefaults();

This registers the new capabilities in the system.

3. Configure Company Strategic Context (Optional)

To enable full strategic intelligence, populate strategic fields for your company:

$company = Company::find(1);
$company->update([
    'target_keywords' => ['AWS Marketplace', 'co-sell', 'cloud partnerships'],
    'key_personas' => [
        [
            'title' => 'Alliance Manager',
            'pain_points' => ['Manual partner tracking', 'No co-sell visibility'],
            'goals' => ['Close more co-sell deals', 'Automate workflows']
        ],
        [
            'title' => 'PMM',
            'pain_points' => ['Generic content', 'No differentiation'],
            'goals' => ['Stand out in marketplace', 'Drive qualified leads']
        ]
    ],
    'gtm_goals' => [
        'Drive webinar registration from AWS partners',
        'Generate 50 qualified leads',
        'Close 3 co-sell deals by Q1'
    ],
    'market_positioning' => 'The only unified co-sell platform built for AWS Marketplace sellers',
    'competitive_differentiators' => [
        'Native AWS integration',
        'Automated partner matching',
        'Deal tracking and attribution'
    ],
    'value_propositions' => [
        ['segment' => 'ISVs', 'value' => 'Close deals 3x faster with automated co-sell'],
        ['segment' => 'Partners', 'value' => 'Discover qualified leads from mutual customers']
    ]
]);

4. Update Agents to Use New Capabilities

Enable new capabilities for agents:

$agent = Agent::find(1); // Product Launch Agent
$agent->capabilities = [
    'query_knowledge_base',
    'generate_text',
    'generate_content_series',
    'analyze_content',
    'refine_content',
];
$agent->save();

Expected Behavior Changes

Before (Execution 35):

generate_text #1
  Parameters: {"topic": "webinar promotion", "length": "short", "content_type": "social_post"}
  Result: Generic post with no specific details

analyze_content #2
  Result: "SEO: 40 - no keywords"
  → Dead end, no action taken

generate_text #3 (calls it "series")
  Result: Single email, not a sequence

After (New Execution):

query_knowledge_base #1
  Parameters: {"query": "webinar details event information product features target audience"}
  Result: 5 relevant KB documents retrieved
  → Visible in trace!

generate_content_series #2
  Parameters: {"content_type": "social_post", "series_count": 3, "series_type": "social_campaign"}
  Result: 3 coordinated posts with distinct angles
  → True series!

analyze_content #3
  Parameters: {"content": "series from #2", "target_seo_score": 70, "target_brand_alignment": 80}
  Result: Strategic evaluation showing:
    - Keywords used: ["AWS Marketplace", "webinar"]
    - Keywords missing: ["co-sell"]
    - Persona alignment: "Partially addresses Alliance Manager goals"
    - GTM alignment: "Strong connection to webinar registration goal"
    - meets_targets.overall: false (SEO: 55, Brand: 75)

refine_content #4 (conditional - only if targets not met)
  Parameters: {"original_content": "series from #2", "analysis_results": "from #3"}
  Result: Improved series with:
    - Missing "co-sell" keyword added naturally
    - Stronger persona pain point addressing
    - Improved SEO: 78, Brand: 85
  → Iterative improvement!

generate_content_series #5
  Parameters: {"content_type": "email", "series_count": 3, "series_type": "email_sequence"}
  Result: 3-email drip sequence:
    - Email #1: Announcement (build awareness)
    - Email #2: Value (address objections)
    - Email #3: Urgency (strong CTA)

Design Partner Validation

This enhancement addresses all 5 gaps identified by the design partner:

✅ A. Knowledge usage is now visible - Explicit query_knowledge_base steps in workflow - KB results shown in trace with source files and relevance scores - Inputs enriched with KB context

✅ B. Analysis is strategically connected - Scores tied to target keywords, personas, GTM goals - Strategic evaluation shows business impact - Recommendations actionable and goal-oriented

✅ C. Refinement loop adds increasing value - refine_content capability improves content iteratively - Targets enforced (SEO >= 80, Brand >= 85) - Analysis → Refinement → Re-analysis loop

✅ D. "Series" is actually a series - generate_content_series creates coordinated sequences - Built-in strategies (email sequence, social campaign, webinar) - Each piece has distinct angle but consistent messaging

✅ E. Structured for handoff (partial) - Series returned as structured array with piece numbers - Each piece includes metadata (piece_number, total_pieces, series_type) - Next step: Export to ICS/task systems (future enhancement)

Future Enhancements

  1. Structured Output Schemas
  2. Define typed schemas (Campaign, Asset, PostSeries)
  3. Export to JSON/CSV for external systems

  4. Integration with project management tools

  5. Multi-Channel Orchestration

  6. Map series pieces to channels (LinkedIn, email, blog)
  7. Assign to team members (PMM, AE, Marketing)

  8. Generate ICS files for scheduling

  9. Performance Feedback Loop

  10. Track actual campaign performance (CTR, conversions)
  11. Feed back into future generations

  12. Learn what works for each persona/goal

  13. A/B Variant Generation

  14. Generate multiple variants for each piece
  15. Test different angles/CTAs
  16. Auto-select top performer

Testing Recommendations

  1. Test Webinar Campaign Again
  2. Run: "Generate a webinar promotion campaign with social posts and email series"
  3. Verify: KB query appears in trace
  4. Verify: Social series has 3 distinct posts
  5. Verify: Email series has 3 coordinated emails
  6. Verify: Analysis shows strategic evaluation

  7. Verify: Refinement improves scores if targets not met

  8. Test Strategic Context

  9. Add target keywords to a Company
  10. Generate content and verify keywords are incorporated

  11. Check analysis shows keyword coverage

  12. Test Refinement Loop

  13. Generate content with high quality targets (SEO: 85, Brand: 90)
  14. Verify workflow includes refinement step
  15. Verify refined content scores higher

Files Changed

Modified:

  1. app/Extensions/ContentManager/System/Services/AgentCore/WorkflowPlanner.php
  2. app/Models/Company.php
  3. app/Extensions/ContentManager/System/Services/AgentCore/BrandVoiceContextBuilder.php
  4. app/Extensions/ContentManager/System/Services/Capabilities/AnalyzeContentCapability.php
  5. app/Extensions/ContentManager/System/Services/AgentCore/CapabilityRegistry.php

Created:

  1. database/migrations/2025_11_18_000001_add_strategic_context_to_companies_table.php
  2. app/Extensions/ContentManager/System/Services/Capabilities/RefineContentCapability.php
  3. app/Extensions/ContentManager/System/Services/Capabilities/GenerateContentSeriesCapability.php
  4. docs/AGENT_INTELLIGENCE_ENHANCEMENTS.md (this file)

Summary

These enhancements transform the agent system from a basic content generator into an intelligent, strategic GTM platform. Agents now:

  • Think strategically - Aligned with business goals, personas, and market positioning
  • Use knowledge intelligently - Visible RAG with relevant context
  • Improve iteratively - Refinement loops until quality targets are met
  • Generate coordinated campaigns - True multi-piece series with distinct angles
  • Evaluate against outcomes - Analysis tied to GTM goals, not just generic scores

The result: Vellocity-level intelligence that design partners can see and trust.