AI Agents Audit & Implementation Plan¶

Executive Summary¶

This document provides a comprehensive audit of the AI agent architecture and a detailed implementation plan for optimizing the dual-layer system (Internal AgentCore + AWS Bedrock AgentCore).

Part 1: Current State Audit¶

Architecture Overview¶

┌─────────────────────────────────────────────────────────────────────┐
│                        AGENT EXECUTION FLOW                         │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│  ┌─────────────┐    ┌──────────────────┐    ┌──────────────────┐   │
│  │   User      │───▶│  AgentController │───▶│ AgentOrchestrator│   │
│  │   Request   │    │                  │    │   (INTERNAL)     │   │
│  └─────────────┘    └──────────────────┘    └────────┬─────────┘   │
│                                                       │             │
│                     ┌─────────────────────────────────┼─────────┐   │
│                     │         INTERNAL AGENTCORE      │         │   │
│                     │  ┌──────────────────────────────▼──────┐  │   │
│                     │  │         WorkflowPlanner             │  │   │
│                     │  │    (Uses Bedrock Claude Models)     │  │   │
│                     │  └──────────────────────────────┬──────┘  │   │
│                     │                                 │         │   │
│                     │  ┌──────────────────────────────▼──────┐  │   │
│                     │  │       CapabilityRegistry            │  │   │
│                     │  │         (35+ Capabilities)          │  │   │
│                     │  └──────────────────────────────┬──────┘  │   │
│                     │                                 │         │   │
│                     │  ┌──────────────────────────────▼──────┐  │   │
│                     │  │      BrandVoiceContextBuilder       │  │   │
│                     │  └─────────────────────────────────────┘  │   │
│                     └───────────────────────────────────────────┘   │
│                                                                     │
│                     ┌───────────────────────────────────────────┐   │
│                     │      AWS BEDROCK SERVICES (External)      │   │
│                     │  ┌─────────────────┐ ┌─────────────────┐  │   │
│                     │  │ BedrockRuntime  │ │ BedrockAgent    │  │   │
│                     │  │ (Model Invoke)  │ │ (Memory Only)   │  │   │
│                     │  └─────────────────┘ └─────────────────┘  │   │
│                     └───────────────────────────────────────────┘   │
│                                                                     │
└─────────────────────────────────────────────────────────────────────┘

Agent Classification Matrix¶

Component	Type	Location	AWS Bedrock Usage
AgentOrchestrator	Internal	`Services/AgentCore/`	None (pure PHP)
WorkflowPlanner	Internal	`Services/AgentCore/`	Bedrock Models (Claude)
CapabilityRegistry	Internal	`Services/AgentCore/`	None (pure PHP)
AgentMemoryService	Hybrid	`Services/AgentCore/`	Optional Bedrock Agent Memory
BrandVoiceContextBuilder	Internal	`Services/AgentCore/`	None (pure PHP)
35+ Capabilities	Internal	`Services/Capabilities/`	Bedrock Models for AI tasks
BedrockAgentService	External	`Services/Bedrock/`	Full Bedrock Agent Runtime

Capabilities by Bedrock Usage¶

Uses Bedrock Foundation Models (Claude/Nova/SD)¶

Capability	Model Used	Purpose
GenerateTextCapability	Claude 3 Sonnet	Text generation
GenerateImageCapability	Stable Diffusion XL	Image generation
AnalyzeContentCapability	Claude 3 Sonnet	Content analysis
CoSellMatchingCapability	Claude 3 Sonnet	Partner matching
JointGTMPlannerCapability	Claude 3 Sonnet	Campaign planning
SeoIntelligenceCapability	Claude 3 Sonnet	SEO analysis
CompetitorAnalysisCapability	Claude 3 Sonnet	Competitor research
PredictPerformanceCapability	Claude 3 Sonnet	Performance prediction

Uses Bedrock Knowledge Base¶

Capability	Service Used	Purpose
QueryKnowledgeBaseCapability	BedrockKnowledgeBaseService	RAG queries
QueryPlatformKnowledgeCapability	BedrockKnowledgeBaseService	Platform docs

Uses Bedrock Agent Runtime (Memory)¶

Capability	Service Used	Purpose
AgentMemoryService	BedrockAgentService	Session memory

No Bedrock Usage (Pure Internal)¶

Capability	Dependencies	Purpose
FetchExternalUrlCapability	HTTP client	Web scraping
PublishToSocialMediaCapability	Social APIs	Publishing
LinkedInGraphCapability	LinkedIn API	Data enrichment
AceOpportunitySyncCapability	Salesforce API	CRM sync
SyncMarketplaceListingsCapability	AWS Marketplace	Listing sync

Part 2: Gap Analysis¶

Current Gaps Identified¶

Gap	Impact	Priority
Memory feature underutilized	Lost context between sessions	High
No unified agent configuration UI	Complex setup for users	Medium
Bedrock Agent creation not automated	Manual AWS Console work	High
Missing agent performance metrics	Can't optimize costs	Medium
No A/B testing for agent configs	Can't compare approaches	Low

Migration Opportunities¶

Consolidate Memory Management - Unify local and Bedrock memory
Automate Bedrock Agent Provisioning - Create agents via API
Implement Agent Analytics Dashboard - Track performance/costs
Add Agent Templates - Pre-configured agent blueprints
Enable Multi-Region Support - Failover and latency optimization

Part 3: Implementation Plan¶

Phase 1: Foundation (Week 1-2)¶

1.1 Create Agent Configuration Service¶

File: app/Extensions/ContentManager/System/Services/AgentCore/AgentConfigurationService.php

<?php

namespace App\Extensions\ContentManager\System\Services\AgentCore;

use App\Extensions\ContentManager\System\Models\Agent;
use App\Services\Bedrock\BedrockAgentService;

class AgentConfigurationService
{
    public function __construct(
        private BedrockAgentService $bedrockAgentService,
        private CapabilityRegistry $capabilityRegistry
    ) {}

    /**
     * Determine optimal agent configuration based on use case
     */
    public function recommendConfiguration(string $useCase): array
    {
        return match ($useCase) {
            'content_generation' => [
                'type' => 'internal',
                'memory_enabled' => false,
                'ai_model' => 'claude-3-sonnet',
                'capabilities' => ['generate_text', 'analyze_content', 'refine_content'],
            ],
            'co_sell' => [
                'type' => 'hybrid',
                'memory_enabled' => true,
                'memory_type' => 'SESSION_SUMMARY',
                'ai_model' => 'claude-3-sonnet',
                'capabilities' => ['cosell_matching', 'joint_gtm_planner', 'partner_intelligence'],
            ],
            'research' => [
                'type' => 'hybrid',
                'memory_enabled' => true,
                'memory_type' => 'SEMANTIC',
                'ai_model' => 'claude-3-opus',
                'capabilities' => ['query_knowledge_base', 'competitor_analysis', 'seo_intelligence'],
            ],
            default => [
                'type' => 'internal',
                'memory_enabled' => false,
                'ai_model' => 'claude-3-haiku',
            ],
        };
    }

    /**
     * Validate agent can use Bedrock Agent features
     */
    public function validateBedrockAgentAccess(Agent $agent): array
    {
        $issues = [];

        if ($agent->memory_enabled && !$agent->bedrock_agent_id) {
            $issues[] = 'Memory enabled but no Bedrock Agent ID configured';
        }

        if ($agent->bedrock_agent_id && !$agent->bedrock_agent_alias_id) {
            $issues[] = 'Bedrock Agent ID set but no Alias ID configured';
        }

        return [
            'valid' => empty($issues),
            'issues' => $issues,
            'recommendations' => $this->getRecommendations($agent, $issues),
        ];
    }

    private function getRecommendations(Agent $agent, array $issues): array
    {
        $recommendations = [];

        foreach ($issues as $issue) {
            if (str_contains($issue, 'no Bedrock Agent ID')) {
                $recommendations[] = [
                    'action' => 'provision_bedrock_agent',
                    'description' => 'Create a Bedrock Agent in AWS Console or use automated provisioning',
                ];
            }
        }

        return $recommendations;
    }
}

1.2 Add Agent Type Enum¶

File: app/Extensions/ContentManager/System/Enums/AgentType.php

<?php

namespace App\Extensions\ContentManager\System\Enums;

enum AgentType: string
{
    case INTERNAL = 'internal';           // Uses only Internal AgentCore
    case BEDROCK_MEMORY = 'bedrock_memory'; // Internal + Bedrock Memory
    case BEDROCK_FULL = 'bedrock_full';     // Future: Full Bedrock Agent delegation

    public function label(): string
    {
        return match ($this) {
            self::INTERNAL => 'Internal AgentCore',
            self::BEDROCK_MEMORY => 'Internal + Bedrock Memory',
            self::BEDROCK_FULL => 'AWS Bedrock Agent (Full)',
        };
    }

    public function requiresBedrockAgent(): bool
    {
        return match ($this) {
            self::INTERNAL => false,
            self::BEDROCK_MEMORY => true,
            self::BEDROCK_FULL => true,
        };
    }

    public function description(): string
    {
        return match ($this) {
            self::INTERNAL => 'All execution handled by Vell AgentCore. Best for stateless workflows.',
            self::BEDROCK_MEMORY => 'Vell AgentCore with AWS Bedrock session memory. Best for multi-session conversations.',
            self::BEDROCK_FULL => 'Execution delegated to AWS Bedrock Agent. Best for AWS-native deployments.',
        };
    }
}

1.3 Database Migration for Agent Type¶

File: database/migrations/2026_01_13_000001_add_agent_type_to_agents_table.php

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up(): void
    {
        Schema::table('ext_content_manager_agents', function (Blueprint $table) {
            $table->string('agent_type')->default('internal')->after('name');
            $table->json('agent_type_config')->nullable()->after('agent_type');
            $table->timestamp('bedrock_agent_synced_at')->nullable()->after('bedrock_agent_alias_id');
        });
    }

    public function down(): void
    {
        Schema::table('ext_content_manager_agents', function (Blueprint $table) {
            $table->dropColumn(['agent_type', 'agent_type_config', 'bedrock_agent_synced_at']);
        });
    }
};

Phase 2: Bedrock Agent Provisioning (Week 2-3)¶

2.1 Automated Bedrock Agent Creation Service¶

File: app/Services/Bedrock/BedrockAgentProvisioningService.php

<?php

namespace App\Services\Bedrock;

use Aws\BedrockAgent\BedrockAgentClient;
use App\Extensions\ContentManager\System\Models\Agent;
use Illuminate\Support\Facades\Log;

class BedrockAgentProvisioningService
{
    private ?BedrockAgentClient $client = null;

    public function __construct(
        private AwsCredentialService $credentialService
    ) {}

    /**
     * Create a Bedrock Agent for memory capabilities
     */
    public function provisionBedrockAgent(Agent $agent, array $options = []): array
    {
        $client = $this->getClient($agent);

        $agentName = $this->generateAgentName($agent);
        $instruction = $this->buildInstruction($agent);

        try {
            // Create the agent
            $createResult = $client->createAgent([
                'agentName' => $agentName,
                'agentResourceRoleArn' => $options['role_arn'] ?? config('services.bedrock.agent_role_arn'),
                'foundationModel' => $this->mapModel($agent->ai_model),
                'instruction' => $instruction,
                'idleSessionTTLInSeconds' => $agent->memory_ttl_days * 86400,
                'memoryConfiguration' => [
                    'enabledMemoryTypes' => [$agent->memory_type ?? 'SESSION_SUMMARY'],
                    'storageDays' => $agent->memory_ttl_days ?? 30,
                ],
                'description' => "Vell AgentCore memory agent for: {$agent->name}",
                'tags' => [
                    'vell_agent_id' => (string) $agent->id,
                    'company_id' => (string) $agent->company_id,
                    'environment' => config('app.env'),
                ],
            ]);

            $bedrockAgentId = $createResult['agent']['agentId'];

            // Prepare the agent
            $client->prepareAgent(['agentId' => $bedrockAgentId]);

            // Create an alias
            $aliasResult = $client->createAgentAlias([
                'agentId' => $bedrockAgentId,
                'agentAliasName' => 'live',
                'description' => 'Production alias',
            ]);

            return [
                'success' => true,
                'bedrock_agent_id' => $bedrockAgentId,
                'bedrock_agent_alias_id' => $aliasResult['agentAlias']['agentAliasId'],
                'agent_arn' => $createResult['agent']['agentArn'],
            ];

        } catch (\Exception $e) {
            Log::error('Failed to provision Bedrock Agent', [
                'agent_id' => $agent->id,
                'error' => $e->getMessage(),
            ]);

            return [
                'success' => false,
                'error' => $e->getMessage(),
            ];
        }
    }

    /**
     * Sync agent configuration to Bedrock
     */
    public function syncToBedrockAgent(Agent $agent): array
    {
        if (!$agent->bedrock_agent_id) {
            return ['success' => false, 'error' => 'No Bedrock Agent ID'];
        }

        $client = $this->getClient($agent);

        try {
            $client->updateAgent([
                'agentId' => $agent->bedrock_agent_id,
                'agentName' => $this->generateAgentName($agent),
                'instruction' => $this->buildInstruction($agent),
                'foundationModel' => $this->mapModel($agent->ai_model),
            ]);

            $client->prepareAgent(['agentId' => $agent->bedrock_agent_id]);

            $agent->update(['bedrock_agent_synced_at' => now()]);

            return ['success' => true];

        } catch (\Exception $e) {
            return ['success' => false, 'error' => $e->getMessage()];
        }
    }

    /**
     * Delete Bedrock Agent when no longer needed
     */
    public function deleteBedrockAgent(Agent $agent): array
    {
        if (!$agent->bedrock_agent_id) {
            return ['success' => true, 'message' => 'No Bedrock Agent to delete'];
        }

        $client = $this->getClient($agent);

        try {
            // Delete alias first
            if ($agent->bedrock_agent_alias_id) {
                $client->deleteAgentAlias([
                    'agentId' => $agent->bedrock_agent_id,
                    'agentAliasId' => $agent->bedrock_agent_alias_id,
                ]);
            }

            // Delete agent
            $client->deleteAgent([
                'agentId' => $agent->bedrock_agent_id,
            ]);

            $agent->update([
                'bedrock_agent_id' => null,
                'bedrock_agent_alias_id' => null,
                'bedrock_agent_synced_at' => null,
            ]);

            return ['success' => true];

        } catch (\Exception $e) {
            return ['success' => false, 'error' => $e->getMessage()];
        }
    }

    private function getClient(Agent $agent): BedrockAgentClient
    {
        if ($this->client) {
            return $this->client;
        }

        $credentials = $this->credentialService->resolveCredentials($agent->company);

        $this->client = new BedrockAgentClient([
            'version' => 'latest',
            'region' => config('services.bedrock.region', 'us-east-1'),
            'credentials' => $credentials,
        ]);

        return $this->client;
    }

    private function generateAgentName(Agent $agent): string
    {
        return sprintf('vell-%s-%d', \Str::slug($agent->name), $agent->id);
    }

    private function buildInstruction(Agent $agent): string
    {
        $capabilities = $agent->capabilities->pluck('name')->join(', ');

        return "You are an AI assistant for {$agent->name}. " .
               "Your capabilities include: {$capabilities}. " .
               "Maintain context across sessions and remember user preferences.";
    }

    private function mapModel(string $model): string
    {
        return match ($model) {
            'claude-3-opus' => 'anthropic.claude-3-opus-20240229-v1:0',
            'claude-3-sonnet' => 'anthropic.claude-3-sonnet-20240229-v1:0',
            'claude-3-haiku' => 'anthropic.claude-3-haiku-20240307-v1:0',
            'claude-3-5-sonnet' => 'anthropic.claude-3-5-sonnet-20240620-v1:0',
            default => 'anthropic.claude-3-sonnet-20240229-v1:0',
        };
    }
}

Phase 3: Agent Analytics & Monitoring (Week 3-4)¶

3.1 Agent Analytics Service¶

File: app/Extensions/ContentManager/System/Services/AgentCore/AgentAnalyticsService.php

<?php

namespace App\Extensions\ContentManager\System\Services\AgentCore;

use App\Extensions\ContentManager\System\Models\Agent;
use App\Extensions\ContentManager\System\Models\AgentExecution;
use Illuminate\Support\Facades\DB;

class AgentAnalyticsService
{
    /**
     * Get comprehensive agent performance metrics
     */
    public function getAgentMetrics(Agent $agent, string $period = '30d'): array
    {
        $startDate = $this->parseStartDate($period);

        $executions = AgentExecution::where('agent_id', $agent->id)
            ->where('created_at', '>=', $startDate)
            ->get();

        return [
            'summary' => $this->calculateSummary($executions),
            'by_status' => $this->groupByStatus($executions),
            'by_capability' => $this->analyzeCapabilityUsage($executions),
            'cost_analysis' => $this->analyzeCosts($executions),
            'performance' => $this->analyzePerformance($executions),
            'memory_usage' => $this->analyzeMemoryUsage($agent, $startDate),
            'recommendations' => $this->generateRecommendations($agent, $executions),
        ];
    }

    /**
     * Compare Internal vs Bedrock execution patterns
     */
    public function compareAgentTypes(int $companyId, string $period = '30d'): array
    {
        $startDate = $this->parseStartDate($period);

        $internalStats = DB::table('ext_content_manager_agent_executions as e')
            ->join('ext_content_manager_agents as a', 'e.agent_id', '=', 'a.id')
            ->where('a.company_id', $companyId)
            ->where('a.agent_type', 'internal')
            ->where('e.created_at', '>=', $startDate)
            ->selectRaw('
                COUNT(*) as total_executions,
                AVG(TIMESTAMPDIFF(SECOND, e.started_at, e.completed_at)) as avg_duration_seconds,
                SUM(e.credits_used) as total_credits,
                SUM(e.tokens_used) as total_tokens,
                SUM(CASE WHEN e.status = "completed" THEN 1 ELSE 0 END) as successful,
                SUM(CASE WHEN e.status = "failed" THEN 1 ELSE 0 END) as failed
            ')
            ->first();

        $bedrockMemoryStats = DB::table('ext_content_manager_agent_executions as e')
            ->join('ext_content_manager_agents as a', 'e.agent_id', '=', 'a.id')
            ->where('a.company_id', $companyId)
            ->where('a.agent_type', 'bedrock_memory')
            ->where('e.created_at', '>=', $startDate)
            ->selectRaw('
                COUNT(*) as total_executions,
                AVG(TIMESTAMPDIFF(SECOND, e.started_at, e.completed_at)) as avg_duration_seconds,
                SUM(e.credits_used) as total_credits,
                SUM(e.tokens_used) as total_tokens,
                SUM(CASE WHEN e.status = "completed" THEN 1 ELSE 0 END) as successful,
                SUM(CASE WHEN e.status = "failed" THEN 1 ELSE 0 END) as failed
            ')
            ->first();

        return [
            'internal' => [
                'stats' => $internalStats,
                'success_rate' => $internalStats->total_executions > 0
                    ? ($internalStats->successful / $internalStats->total_executions) * 100
                    : 0,
            ],
            'bedrock_memory' => [
                'stats' => $bedrockMemoryStats,
                'success_rate' => $bedrockMemoryStats->total_executions > 0
                    ? ($bedrockMemoryStats->successful / $bedrockMemoryStats->total_executions) * 100
                    : 0,
            ],
            'recommendation' => $this->recommendAgentType($internalStats, $bedrockMemoryStats),
        ];
    }

    private function calculateSummary($executions): array
    {
        return [
            'total_executions' => $executions->count(),
            'successful' => $executions->where('status', 'completed')->count(),
            'failed' => $executions->where('status', 'failed')->count(),
            'cancelled' => $executions->where('status', 'cancelled')->count(),
            'success_rate' => $executions->count() > 0
                ? round(($executions->where('status', 'completed')->count() / $executions->count()) * 100, 2)
                : 0,
            'total_credits' => $executions->sum('credits_used'),
            'total_tokens' => $executions->sum('tokens_used'),
        ];
    }

    private function groupByStatus($executions): array
    {
        return $executions->groupBy('status')
            ->map(fn ($group) => $group->count())
            ->toArray();
    }

    private function analyzeCapabilityUsage($executions): array
    {
        $capabilityUsage = [];

        foreach ($executions as $execution) {
            $steps = $execution->step_results ?? [];
            foreach ($steps as $step) {
                $capability = $step['capability'] ?? 'unknown';
                if (!isset($capabilityUsage[$capability])) {
                    $capabilityUsage[$capability] = [
                        'count' => 0,
                        'success' => 0,
                        'failed' => 0,
                        'total_duration_ms' => 0,
                    ];
                }
                $capabilityUsage[$capability]['count']++;
                if (($step['status'] ?? '') === 'completed') {
                    $capabilityUsage[$capability]['success']++;
                } else {
                    $capabilityUsage[$capability]['failed']++;
                }
                $capabilityUsage[$capability]['total_duration_ms'] += $step['duration_ms'] ?? 0;
            }
        }

        return $capabilityUsage;
    }

    private function analyzeCosts($executions): array
    {
        $totalCredits = $executions->sum('credits_used');
        $totalTokens = $executions->sum('tokens_used');

        // Estimate costs (adjust rates as needed)
        $inputTokenCost = ($totalTokens * 0.6) * 0.000003;  // Claude Sonnet input
        $outputTokenCost = ($totalTokens * 0.4) * 0.000015; // Claude Sonnet output

        return [
            'total_credits' => $totalCredits,
            'total_tokens' => $totalTokens,
            'estimated_aws_cost_usd' => round($inputTokenCost + $outputTokenCost, 4),
            'avg_credits_per_execution' => $executions->count() > 0
                ? round($totalCredits / $executions->count(), 2)
                : 0,
            'avg_tokens_per_execution' => $executions->count() > 0
                ? round($totalTokens / $executions->count(), 0)
                : 0,
        ];
    }

    private function analyzePerformance($executions): array
    {
        $completedExecutions = $executions->whereNotNull('started_at')
            ->whereNotNull('completed_at');

        $durations = $completedExecutions->map(function ($e) {
            return $e->completed_at->diffInSeconds($e->started_at);
        });

        return [
            'avg_duration_seconds' => $durations->avg() ?? 0,
            'min_duration_seconds' => $durations->min() ?? 0,
            'max_duration_seconds' => $durations->max() ?? 0,
            'p95_duration_seconds' => $this->percentile($durations->toArray(), 95),
        ];
    }

    private function analyzeMemoryUsage(Agent $agent, $startDate): array
    {
        if (!$agent->memory_enabled || !$agent->bedrock_agent_id) {
            return ['enabled' => false, 'reason' => 'Memory not configured'];
        }

        $sessions = DB::table('ext_content_manager_agent_memory_sessions')
            ->where('agent_id', $agent->id)
            ->where('created_at', '>=', $startDate)
            ->get();

        return [
            'enabled' => true,
            'total_sessions' => $sessions->count(),
            'active_sessions' => $sessions->whereNull('ended_at')->count(),
            'memory_type' => $agent->memory_type,
            'estimated_token_savings' => $sessions->count() * 500, // ~500 tokens saved per session
        ];
    }

    private function generateRecommendations(Agent $agent, $executions): array
    {
        $recommendations = [];

        // Check if should enable memory
        $avgStepsPerExecution = $executions->avg(fn ($e) => count($e->step_results ?? []));
        if ($avgStepsPerExecution > 3 && !$agent->memory_enabled) {
            $recommendations[] = [
                'type' => 'enable_memory',
                'priority' => 'medium',
                'description' => 'Consider enabling Bedrock Agent memory for multi-step workflows',
                'estimated_benefit' => '40-60% token reduction for follow-up queries',
            ];
        }

        // Check failure rate
        $failureRate = $executions->count() > 0
            ? ($executions->where('status', 'failed')->count() / $executions->count()) * 100
            : 0;

        if ($failureRate > 10) {
            $recommendations[] = [
                'type' => 'investigate_failures',
                'priority' => 'high',
                'description' => "High failure rate detected ({$failureRate}%)",
                'action' => 'Review failed execution logs and adjust guardrails',
            ];
        }

        // Check cost optimization
        $avgTokens = $executions->avg('tokens_used') ?? 0;
        if ($avgTokens > 50000 && $agent->ai_model === 'claude-3-opus') {
            $recommendations[] = [
                'type' => 'model_optimization',
                'priority' => 'medium',
                'description' => 'Consider using Claude 3 Sonnet for cost optimization',
                'estimated_savings' => '60-70% cost reduction with minimal quality impact',
            ];
        }

        return $recommendations;
    }

    private function recommendAgentType($internalStats, $bedrockStats): string
    {
        if ($bedrockStats->total_executions === 0) {
            return 'Consider enabling Bedrock memory for agents with multi-session workflows';
        }

        $internalSuccessRate = $internalStats->total_executions > 0
            ? $internalStats->successful / $internalStats->total_executions
            : 0;

        $bedrockSuccessRate = $bedrockStats->total_executions > 0
            ? $bedrockStats->successful / $bedrockStats->total_executions
            : 0;

        if ($bedrockSuccessRate > $internalSuccessRate + 0.05) {
            return 'Bedrock memory agents show higher success rate - consider migrating more agents';
        }

        return 'Current agent type distribution appears optimal';
    }

    private function parseStartDate(string $period): \Carbon\Carbon
    {
        return match ($period) {
            '7d' => now()->subDays(7),
            '30d' => now()->subDays(30),
            '90d' => now()->subDays(90),
            default => now()->subDays(30),
        };
    }

    private function percentile(array $values, int $percentile): float
    {
        if (empty($values)) return 0;
        sort($values);
        $index = ceil(($percentile / 100) * count($values)) - 1;
        return $values[$index] ?? 0;
    }
}

Phase 4: Migration Tooling (Week 4-5)¶

4.1 Agent Migration Service¶

File: app/Extensions/ContentManager/System/Services/AgentCore/AgentMigrationService.php

<?php

namespace App\Extensions\ContentManager\System\Services\AgentCore;

use App\Extensions\ContentManager\System\Models\Agent;
use App\Extensions\ContentManager\System\Enums\AgentType;
use App\Services\Bedrock\BedrockAgentProvisioningService;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Log;

class AgentMigrationService
{
    public function __construct(
        private BedrockAgentProvisioningService $provisioningService,
        private AgentConfigurationService $configService,
        private AgentAnalyticsService $analyticsService
    ) {}

    /**
     * Migrate agent from internal to Bedrock memory type
     */
    public function migrateToBedrockMemory(Agent $agent, array $options = []): array
    {
        $migrationId = uniqid('migration_');

        Log::info("Starting agent migration", [
            'migration_id' => $migrationId,
            'agent_id' => $agent->id,
            'from_type' => $agent->agent_type,
            'to_type' => AgentType::BEDROCK_MEMORY->value,
        ]);

        try {
            // Step 1: Validate prerequisites
            $validation = $this->configService->validateBedrockAgentAccess($agent);
            if (!$validation['valid'] && !$agent->bedrock_agent_id) {
                // Need to provision Bedrock Agent first
                Log::info("Provisioning Bedrock Agent", ['migration_id' => $migrationId]);

                $provisionResult = $this->provisioningService->provisionBedrockAgent($agent, $options);

                if (!$provisionResult['success']) {
                    throw new \Exception("Failed to provision Bedrock Agent: " . $provisionResult['error']);
                }

                $agent->update([
                    'bedrock_agent_id' => $provisionResult['bedrock_agent_id'],
                    'bedrock_agent_alias_id' => $provisionResult['bedrock_agent_alias_id'],
                ]);

                $agent->refresh();
            }

            // Step 2: Enable memory
            $agent->update([
                'agent_type' => AgentType::BEDROCK_MEMORY->value,
                'memory_enabled' => true,
                'memory_type' => $options['memory_type'] ?? 'SESSION_SUMMARY',
                'memory_ttl_days' => $options['memory_ttl_days'] ?? 30,
            ]);

            // Step 3: Sync configuration to Bedrock
            $syncResult = $this->provisioningService->syncToBedrockAgent($agent);

            if (!$syncResult['success']) {
                Log::warning("Bedrock sync failed but agent migrated", [
                    'migration_id' => $migrationId,
                    'error' => $syncResult['error'],
                ]);
            }

            Log::info("Agent migration completed", [
                'migration_id' => $migrationId,
                'agent_id' => $agent->id,
            ]);

            return [
                'success' => true,
                'migration_id' => $migrationId,
                'agent' => $agent->fresh(),
                'bedrock_agent_id' => $agent->bedrock_agent_id,
            ];

        } catch (\Exception $e) {
            Log::error("Agent migration failed", [
                'migration_id' => $migrationId,
                'agent_id' => $agent->id,
                'error' => $e->getMessage(),
            ]);

            return [
                'success' => false,
                'migration_id' => $migrationId,
                'error' => $e->getMessage(),
            ];
        }
    }

    /**
     * Migrate agent back to internal-only type
     */
    public function migrateToInternal(Agent $agent, bool $deleteBedrockAgent = false): array
    {
        $migrationId = uniqid('migration_');

        try {
            // Optionally delete the Bedrock Agent
            if ($deleteBedrockAgent && $agent->bedrock_agent_id) {
                $deleteResult = $this->provisioningService->deleteBedrockAgent($agent);
                if (!$deleteResult['success']) {
                    Log::warning("Failed to delete Bedrock Agent", [
                        'migration_id' => $migrationId,
                        'error' => $deleteResult['error'],
                    ]);
                }
            }

            $agent->update([
                'agent_type' => AgentType::INTERNAL->value,
                'memory_enabled' => false,
            ]);

            return [
                'success' => true,
                'migration_id' => $migrationId,
                'agent' => $agent->fresh(),
            ];

        } catch (\Exception $e) {
            return [
                'success' => false,
                'migration_id' => $migrationId,
                'error' => $e->getMessage(),
            ];
        }
    }

    /**
     * Batch migrate agents based on criteria
     */
    public function batchMigrate(int $companyId, array $criteria, AgentType $targetType): array
    {
        $agents = Agent::where('company_id', $companyId)
            ->where('agent_type', '!=', $targetType->value);

        // Apply criteria filters
        if (isset($criteria['min_executions'])) {
            $agents->whereHas('executions', function ($q) use ($criteria) {
                $q->havingRaw('COUNT(*) >= ?', [$criteria['min_executions']]);
            });
        }

        if (isset($criteria['capabilities'])) {
            $agents->whereHas('capabilities', function ($q) use ($criteria) {
                $q->whereIn('slug', $criteria['capabilities']);
            });
        }

        $agents = $agents->get();

        $results = [
            'total' => $agents->count(),
            'successful' => 0,
            'failed' => 0,
            'details' => [],
        ];

        foreach ($agents as $agent) {
            $result = match ($targetType) {
                AgentType::BEDROCK_MEMORY => $this->migrateToBedrockMemory($agent),
                AgentType::INTERNAL => $this->migrateToInternal($agent),
                default => ['success' => false, 'error' => 'Unsupported target type'],
            };

            if ($result['success']) {
                $results['successful']++;
            } else {
                $results['failed']++;
            }

            $results['details'][] = [
                'agent_id' => $agent->id,
                'agent_name' => $agent->name,
                'success' => $result['success'],
                'error' => $result['error'] ?? null,
            ];
        }

        return $results;
    }

    /**
     * Get migration recommendations for all agents
     */
    public function getMigrationRecommendations(int $companyId): array
    {
        $agents = Agent::where('company_id', $companyId)->get();

        $recommendations = [];

        foreach ($agents as $agent) {
            $metrics = $this->analyticsService->getAgentMetrics($agent, '30d');
            $recommendation = $this->analyzeAgentForMigration($agent, $metrics);

            if ($recommendation) {
                $recommendations[] = $recommendation;
            }
        }

        // Sort by priority
        usort($recommendations, fn ($a, $b) =>
            $this->priorityScore($b['priority']) - $this->priorityScore($a['priority'])
        );

        return $recommendations;
    }

    private function analyzeAgentForMigration(Agent $agent, array $metrics): ?array
    {
        // Skip if already optimal
        if ($agent->agent_type === AgentType::BEDROCK_MEMORY->value && $agent->memory_enabled) {
            return null;
        }

        $summary = $metrics['summary'];
        $performance = $metrics['performance'];

        // Recommend Bedrock memory for multi-step, high-volume agents
        if (
            $agent->agent_type === AgentType::INTERNAL->value &&
            $summary['total_executions'] > 50 &&
            $performance['avg_duration_seconds'] > 30
        ) {
            return [
                'agent_id' => $agent->id,
                'agent_name' => $agent->name,
                'current_type' => $agent->agent_type,
                'recommended_type' => AgentType::BEDROCK_MEMORY->value,
                'priority' => 'high',
                'reason' => 'High volume, long-running executions would benefit from session memory',
                'estimated_benefit' => [
                    'token_reduction' => '40-60%',
                    'context_continuity' => 'Improved',
                ],
            ];
        }

        // Recommend internal for low-volume, simple agents
        if (
            $agent->agent_type === AgentType::BEDROCK_MEMORY->value &&
            $summary['total_executions'] < 10 &&
            $performance['avg_duration_seconds'] < 10
        ) {
            return [
                'agent_id' => $agent->id,
                'agent_name' => $agent->name,
                'current_type' => $agent->agent_type,
                'recommended_type' => AgentType::INTERNAL->value,
                'priority' => 'low',
                'reason' => 'Low volume agent may not need memory overhead',
                'estimated_benefit' => [
                    'cost_reduction' => 'Minimal AWS API calls',
                    'simplicity' => 'Reduced complexity',
                ],
            ];
        }

        return null;
    }

    private function priorityScore(string $priority): int
    {
        return match ($priority) {
            'high' => 3,
            'medium' => 2,
            'low' => 1,
            default => 0,
        };
    }
}

Phase 5: Admin Dashboard & UI (Week 5-6)¶

5.1 Add Agent Audit Controller¶

File: app/Extensions/ContentManager/System/Http/Controllers/AgentAuditController.php

<?php

namespace App\Extensions\ContentManager\System\Http\Controllers;

use App\Extensions\ContentManager\System\Services\AgentCore\AgentAnalyticsService;
use App\Extensions\ContentManager\System\Services\AgentCore\AgentMigrationService;
use App\Extensions\ContentManager\System\Services\AgentCore\AgentConfigurationService;
use App\Extensions\ContentManager\System\Models\Agent;
use Illuminate\Http\Request;

class AgentAuditController extends Controller
{
    public function __construct(
        private AgentAnalyticsService $analyticsService,
        private AgentMigrationService $migrationService,
        private AgentConfigurationService $configService
    ) {}

    /**
     * Display agent audit dashboard
     */
    public function index(Request $request)
    {
        $company = $request->user()->company;

        $agents = Agent::where('company_id', $company->id)
            ->withCount('executions')
            ->get()
            ->map(function ($agent) {
                return [
                    'id' => $agent->id,
                    'name' => $agent->name,
                    'type' => $agent->agent_type,
                    'memory_enabled' => $agent->memory_enabled,
                    'bedrock_agent_id' => $agent->bedrock_agent_id,
                    'executions_count' => $agent->executions_count,
                    'capabilities' => $agent->capabilities->pluck('name'),
                ];
            });

        $typeComparison = $this->analyticsService->compareAgentTypes($company->id);
        $recommendations = $this->migrationService->getMigrationRecommendations($company->id);

        return view('content-manager::agents.audit', [
            'agents' => $agents,
            'typeComparison' => $typeComparison,
            'recommendations' => $recommendations,
            'summary' => [
                'total_agents' => $agents->count(),
                'internal_agents' => $agents->where('type', 'internal')->count(),
                'bedrock_memory_agents' => $agents->where('type', 'bedrock_memory')->count(),
                'with_memory' => $agents->where('memory_enabled', true)->count(),
            ],
        ]);
    }

    /**
     * Get detailed metrics for a single agent
     */
    public function agentMetrics(Agent $agent, Request $request)
    {
        $this->authorize('view', $agent);

        $period = $request->get('period', '30d');
        $metrics = $this->analyticsService->getAgentMetrics($agent, $period);

        return response()->json([
            'agent' => [
                'id' => $agent->id,
                'name' => $agent->name,
                'type' => $agent->agent_type,
            ],
            'metrics' => $metrics,
        ]);
    }

    /**
     * Migrate a single agent
     */
    public function migrate(Agent $agent, Request $request)
    {
        $this->authorize('update', $agent);

        $targetType = $request->input('target_type');
        $options = $request->input('options', []);

        $result = match ($targetType) {
            'bedrock_memory' => $this->migrationService->migrateToBedrockMemory($agent, $options),
            'internal' => $this->migrationService->migrateToInternal(
                $agent,
                $request->boolean('delete_bedrock_agent')
            ),
            default => ['success' => false, 'error' => 'Invalid target type'],
        };

        if ($result['success']) {
            return response()->json([
                'success' => true,
                'message' => "Agent migrated to {$targetType}",
                'agent' => $result['agent'],
            ]);
        }

        return response()->json([
            'success' => false,
            'error' => $result['error'],
        ], 422);
    }

    /**
     * Batch migrate agents
     */
    public function batchMigrate(Request $request)
    {
        $request->validate([
            'target_type' => 'required|in:internal,bedrock_memory',
            'agent_ids' => 'required|array',
            'agent_ids.*' => 'exists:ext_content_manager_agents,id',
        ]);

        $company = $request->user()->company;
        $targetType = AgentType::from($request->input('target_type'));

        $results = $this->migrationService->batchMigrate(
            $company->id,
            ['agent_ids' => $request->input('agent_ids')],
            $targetType
        );

        return response()->json($results);
    }
}

Part 4: Implementation Checklist¶

Phase 1: Foundation¶

Create AgentConfigurationService
Create AgentType enum
Run migration to add agent_type column
Update Agent model with new relationships

Phase 2: Bedrock Provisioning¶

Create BedrockAgentProvisioningService
Test agent creation in sandbox AWS account
Implement sync functionality
Add error handling and retry logic

Phase 3: Analytics¶

Create AgentAnalyticsService
Build comparison queries for agent types
Implement recommendation engine
Add performance tracking metrics

Phase 4: Migration Tooling¶

Create AgentMigrationService
Implement single agent migration
Implement batch migration
Add rollback capabilities

Phase 5: Admin UI¶

Create AgentAuditController
Build audit dashboard view
Add migration wizard UI
Implement real-time metrics display

Phase 6: Testing & Documentation¶

Write unit tests for all services
Write integration tests for Bedrock operations
Update API documentation
Create user guide for migration

Part 5: Risk Mitigation¶

Risk	Mitigation
Bedrock API rate limits	Implement exponential backoff and queuing
Migration data loss	Create backup before migration, implement rollback
Cost overruns	Add cost monitoring and alerts
Downtime during migration	Use feature flags for gradual rollout
IAM permission issues	Pre-validate permissions before provisioning

Part 6: Success Metrics¶

Metric	Target	Measurement
Agent migration success rate	>95%	Migrations completed / attempted
Memory feature adoption	>50% of eligible agents	Agents with memory / total agents
Token cost reduction	40-60% for memory agents	Before/after token usage
Agent failure rate	<5%	Failed executions / total
Migration rollback rate	<2%	Rollbacks / migrations

Appendix: Quick Reference¶

Current Agent Types¶

INTERNAL (default)
├── Uses: Internal AgentCore only
├── Memory: None (stateless)
├── Best for: Simple, stateless workflows
└── AWS Usage: Bedrock models only

BEDROCK_MEMORY
├── Uses: Internal AgentCore + Bedrock Agent Memory
├── Memory: SESSION_SUMMARY or SEMANTIC
├── Best for: Multi-session conversations, research
└── AWS Usage: Bedrock models + Agent Runtime API

Key Files Reference¶

Purpose	File Path
Main Orchestrator	`Services/AgentCore/AgentOrchestrator.php`
Workflow Planning	`Services/AgentCore/WorkflowPlanner.php`
Memory Management	`Services/AgentCore/AgentMemoryService.php`
Bedrock Agent API	`Services/Bedrock/BedrockAgentService.php`
Agent Model	`Models/Agent.php`
Capabilities	`Services/Capabilities/*.php`

Document generated: 2026-01-13 Version: 1.0