Rulix AI
← All sections
§33

Open questions / out of scope

Items deliberately not nailed down here — to be sharpened as we hit them in practice.

  • Multi-agent coordination governance. When agents call other agents, accountability weakens at every step. The framework treats each agent individually; multi-agent chains need a follow-on section.
  • Prompt change management. Are prompts in Dev / Test / Prod gates, or are they versioned in the repo and reviewed via PR? Both shapes work; pick one per company.
  • Cost-allocation model across departments. Showback vs. chargeback vs. central pool. Out of scope here.
  • Training data provenance for RAG over internal data. Need a data lineage record per agent — referenced in §27, not detailed.
  • External-facing agents. Additional disclosure, consent, and complaint-handling beyond what's in §18.
  • LLM-vendor degradation / shutdown. Migration plan per agent when an upstream model is deprecated.
  • Champion onboarding. How to ramp a new Department Champion with no AI background.
  • Concrete templates. Detailed Agent Card, intake form, post-mortem templates — to be added as separate files.
  • Stage-3 promotion review process. What does the actual review look like, who's in the room, what evidence is presented?
  • MCP / A2A security baseline. Concrete allowlist policy, auth, logging requirements as those protocols mature.

Sources

Articles absorbed into v1.0, in the order they appear in articles.md. Each entry is the article's source and what it contributed.

  • Anthropic — "Trustworthy agents in practice" (2026-04-09). Four-component model (model / harness / tools / environment); five trustworthy-agent principles (human control, alignment, security, transparency, privacy); prompt injection as a multi-layer defense problem; sub-agent coordination as an emerging concern; open standards (MCP).
  • Anthropic — "The need for transparency in Frontier AI" (2025-07-07). Secure Development Framework + System Card pattern. Less directly applicable to adopting companies; useful for downstream vendor selection criteria.
  • Microsoft Cloud Adoption Framework — "Governance and security for AI agents across the organization." Four-layer model (data governance / observability / security / development); central agent control plane; the five things leaders must do (identify / determine ownership / limit access / observe / stop); registry, identity, policy-enforcement, observability and cost requirements; vendor-AI inventory; agent development standards including MCP and A2A.
  • Microsoft CAF — "Organizational readiness for AI agents." Platform team / Workload team / AI CoE split. Skills areas (prompt engineering, agent optimization, AI ethics & governance, AI security, data engineering for AI). Change management.
  • Microsoft CAF — "Manage AI agents across your organization." Integrate (embed agents into existing workflows; phased rollout; reusable templates); standardize; asset lifecycle management; continuous compliance and security; quota optimization; centralized administration via an AI gateway.
  • AI21 Labs — "9 Key AI Governance Frameworks in 2025." EU AI Act, UK framework, EO 14110/14179, NIST AI RMF, AI Bill of Rights, US state laws (Colorado AI Act, California proposed Automated Decision Systems Accountability Act), OECD principles, UNESCO ethics framework, G7 Code of Conduct. Common principles (human oversight, transparency, accountability, safety, fairness, privacy, proportionality, human-centric design). Implementation guidance across design / deployment / monitoring / ongoing risk management / explainability.
  • Databricks — "AI Governance Best Practices." Core principles (fairness, transparency, accountability, privacy, security, built-in safeguards, unified access). Cross-functional governance committees; clear RACI; HITL for high-risk; lifecycle integration; risk assessment as the heart of governance; defined approval and escalation paths; continuous monitoring; incident response; standardized governance artifacts; centralized-federated scaling model.
  • Elementum — "AI Governance Explained: Frameworks and Compliance Tips." Single accountable owner with block-deployment authority; centralized AI inventory including vendor-embedded AI; the three risk drivers (personal data / consequential decisions / autonomous behavior); risk tiering aligned with EU AI Act; HITL with explicit trigger criteria; continuous monitoring vs point-in-time audits; AI agents as privileged identities; procurement integration; runtime enforcement as architectural decision; the auto-update problem with vendor AI; the U.S. state-law patchwork.
  • Elementum — "AI Guardrails." Three-layer guardrails (policy / workflow / runtime); architectural patterns (embedded vs data-layer vs centralized orchestration); core production controls (tool allowlists, permission-aware data access, HITL for irreversible actions, audit trails, circuit breakers, rollback); vendor evaluation criteria (data handling / policy enforcement / audit-trail depth / human review controls); implementation steps (inventory / rank / minimum control set / centralized evidence).
  • Elementum — "How to Control and Monitor the Output of AI Agents." Why agents are uniquely hard to monitor (non-determinism, multi-agent chains, prompt injection surface); control vs monitoring distinction; five control mechanisms (confidence thresholds per process, HITL at irreversible points, input validation / prompt-injection defense, deterministic workflow boundaries, least-privilege scoping); five monitoring signals (distribution shift, escalation rate, decision audit trails, cost per step, exception routing); EU AI Act Article 12 logging and Article 19 retention; deterministic orchestration layer as the architecture for both.
  • lakeFS — "Center of Excellence for Enterprise AI: Models & Best Practices." AI CoE definition; five CoE operating models (Centralized / Federated / Hybrid / Platform-Led / Domain-Focused); CoE components (governance, delivery framework, cross-functional team, shared tooling, data foundation); data challenges that limit CoE success; how to establish a successful CoE; data infrastructure requirements (versioning, reproducibility, lineage, unified access, secure access controls).
  • Microsoft CAF — "Establish an AI Center of Excellence." Building the team (executive sponsorship, CoE leader, multidisciplinary team, placement, operating model); CoE responsibilities (strategy, skills, pilots, standards, intake & prioritization, reusable assets, measurement, optional service management); CoE evolution from centralized control to advisory; inflection points (approval delays, knowledge bottlenecks, friction).
  • Microsoft CAF — "Govern AI." NIST AI RMF–aligned. Risk assessment (workload understanding, responsible-AI principles, specific risks, external dependencies, integration risks); governance policy areas (model selection/onboarding, third-party tools/data, model maintenance and monitoring, regulatory compliance, user conduct, AI integration and replacement); policy enforcement (automated + manual + workload-specific); continuous monitoring; independent reviews.
  • Microsoft CAF — "Secure AI." Three steps: Discover (threat modeling with STRIDE + MITRE ATLAS + OWASP GenAI; AI data risk; vulnerability testing; periodic assessments); Protect (complete asset inventory; secure communication channels; managed identities; data boundaries; DLP; artifact protection); Detect (automated risk detection; AI-focused incident response; platform-specific monitoring).
  • Microsoft CAF — "Manage AI." Operations (CoE for strategic guidance, MLOps vs GenAIOps frameworks, standardized SDKs, sandbox environments); deployment (workload-team authority within guardrails; AI policies; CI/CD); models (measurement baseline; root cause analysis; retraining; promotion with quality gates; retirement tracking); costs; data (golden datasets, secure pipelines, sensitivity classification monitoring); business continuity (continuous monitoring, multi-region, DR testing, version control, automated backups, documented procedures).
  • IBM — "What is an AI center of excellence?" CoE as central organizational structure; functions (promote alignment, share knowledge, provide tech enablement, establish oversight and governance, foster talent); origin story (fragmented expertise, scaling difficulty, governance and ethics demand).
  • Automation Anywhere — "What Is An AI Center of Excellence (CoE)?" Five pillars (strategy & prioritization; embedded governance & agentic guardrails; unified architecture & LLMOps; enablement via reusable building blocks; measurement & continuous feedback); APA as the execution layer; runtime governance; CoE roles (Executive Sponsor, AI CoE Lead, Agent Architect, Observability Lead, AI Ethics Lead, Process Analyst); 6-step implementation roadmap; common challenges; four stages of autonomy (Assisted / Human Validation / Autonomous Operations / Strategic Enablement); CoE focus on standards over ownership; embedded governance for citizen developers.

Review notes (v1.0)

This framework was drafted from the consolidated article set. The author then reviewed it for unsupported claims, ambiguities, and gaps. Findings:

Strongly supported across multiple sources

  • Single accountable owner with block authority (Elementum, Databricks, Automation Anywhere, Microsoft).
  • Three-tier risk + three risk drivers (Elementum, EU AI Act, NIST AI RMF, Databricks).
  • Runtime enforcement vs design-time policy (Elementum, Anthropic, Automation Anywhere).
  • AI CoE as the central operating model (Microsoft, lakeFS, IBM, Automation Anywhere).
  • AI agents as privileged identities (Elementum, Microsoft).
  • Continuous monitoring vs point-in-time audits (Databricks, Elementum, Microsoft, AI21).
  • Vendor-embedded AI in inventory (Elementum, Microsoft).
  • Procurement integration (Elementum, Microsoft).
  • Five control mechanisms before action (Elementum, with OWASP / NIST corroboration cited in the article).
  • Five monitoring signals (Elementum).
  • Common principles across frameworks (AI21, OECD, NIST, EU AI Act, Databricks).

Based on a single primary source

These are useful framings, not yet canonical. Treated as one source's structure:

  • Four stages of autonomy — Automation Anywhere only. Functionally equivalent ideas appear elsewhere (Databricks' centralized-federated; Microsoft's centralized → advisory CoE evolution), but the specific four-stage labeling comes from one source.
  • Five pillars of the CoE — Automation Anywhere only. Other sources have different decompositions of the same activities. Not contentious, but not multi-sourced.
  • Audit log retention ≥ 6 months for High-risk — anchored on EU AI Act Article 19 as quoted in Elementum's article. Other regulations have other retention rules; this is the EU baseline, not universal.
  • Specific promotion thresholds for autonomy stages (90% / 95% / 30 days / 60 days) — these numeric thresholds are illustrative starting points, not validated by a primary source. They should be tuned against actual incident data once a portfolio exists.

Composed by us (synthesis beyond any single article)

  • The §3 mapping of seven failure modes → framework sections is original. Composed by synthesizing the failure patterns the articles describe.
  • The §5 readiness gate list of 8 items is composed from Microsoft CAF readiness + Databricks foundational practices + Elementum / Anthropic prerequisites. No single source enumerates these eight.
  • The §32 12–18 month roadmap is composed (Microsoft CAF gives a 6-phase model; Automation Anywhere gives a 6-step strategy; we structured a time-based plan that aligns with both). Treat the specific week/month numbers as illustrative, not prescriptive.
  • §10's exact tier descriptions and approval-path tables combine Elementum's risk-driver thinking with Databricks' tier definitions; the table itself is ours.

Known soft spots / open gaps

  • Multi-agent governance is thin in the source material. Elementum and Anthropic both flag it as an open problem, and we inherit that openness. If we build multi-agent workflows, the framework needs a follow-on section.
  • A2A protocol governance is referenced (Microsoft, Anthropic) but the protocol is still emerging; §23 is intentionally light.
  • Prompt change management (Dev/Test/Prod for prompts vs. code-style PR review) is mentioned by Databricks and Microsoft but not pinned down. Left open in §33.
  • Cost-allocation models (showback / chargeback / central pool) — sources mention cost monitoring but not allocation. Left open.
  • Specific statistics — earlier framework drafts quoted exact percentages (IBM 63% / 97%; Gartner 69% / 3.4×). v1.0 omits them from the body because they appear only in Elementum's articles without primary verification in the article set. If we want those numbers in, we'll cite primary IBM / Gartner reports first.

Adjustments made during the review pass

  • Removed earlier draft's exact percentage claims pending primary-source verification.
  • Clarified that Stage 3 autonomy is the exception, not the goal (multiple sources warn against treating autonomy as a destination).
  • Added explicit framing that CoE evolution from centralized → advisory is recommended but not automatic — it requires platform investment and explicit triggers.
  • Added §31.5 to capture the broader regulatory references (UK, OECD, UNESCO, G7, EOs, AI Bill of Rights) that AI21 covered. Earlier drafts only had EU AI Act / NIST / ISO.
  • Added §23 (open protocols — MCP / A2A) as its own section. Multiple source articles (Anthropic, Microsoft) treat it as architecturally important enough to call out separately rather than burying inside the stack table.
  • Labeled the autonomy-promotion thresholds (90% / 95% / 30 days / 60 days) as illustrative starting points rather than absolute requirements, because no source validates the specific numbers.