MORPHEUS: Skyfall AI’s Persistent Enterprise Simulation to Benchmark Continual RL

Skyfall AI Releases MORPHEUS: A Persistent Enterprise Simulation That Raises the Bar for Continual Reinforcement Learning

A missed data feed today raises latency tonight, which increases queueing tomorrow. A week later, the “good” policy is quietly steering costs into trouble. MORPHEUS is Skyfall AI’s attempt to make that compounding, long-lived reality hard to ignore.

What MORPHEUS aims to test, and why it matters

Skyfall positions MORPHEUS as a persistent enterprise simulation benchmark built to force continual reinforcement learning (CRL) under structured non-stationarity. Unlike episodic RL suites that reset at the end of each episode, MORPHEUS avoids resets so past decisions compound into future dynamics and fixed policies eventually become suboptimal.

Skyfall grounds the design in what it calls the “Big World Hypothesis” (Javed & Sutton, 2024), the idea that an environment’s effective complexity can exceed an agent’s representational capacity so that fixed dynamics appear non‑stationary. The practical result is that adaptation, forgetting, and recovery become the main evaluation concerns for agents intended to run in production.

How MORPHEUS builds structured non‑stationarity

  • Persistence: environments do not reset. Agent actions can alter the world state and that state persists into future steps.
  • Failure injection engine: Failure events are injected between discrete steps in an execution plan. Skyfall reports a taxonomy of eleven failure types (examples listed include missing_data, dependency_failure, and rate_limit) and preset occurrence rates Skyfall calls light (5%), realistic (8%), moderate (15%), and aggressive (30%).
  • Asynchronous configuration shifts: A controller flips failure presets and demand at fixed timestamps that run independently of the agent’s training updates, and shifts never align with gradient updates.
  • Operational verifiers: Native log-derived signals map to three operational KPIs: failure events, financial ledger status, and resource throughput. Those verifiers feed a composite reward.

Technically, Skyfall ships environments as TypeScript world plugins. Each plugin exports Operational Descriptors (ODs), stepwise execution plans for capabilities, plus a scheduler, seed data, and documentation. Failure injections are placed between OD steps so disruptions can be targeted to specific parts of a workflow.

Takeaway: tests that punish short-term gains that lead to long-term debt need persistence, targeted failure modes, and shifts that don’t conveniently follow training boundaries.

Rewarding operational behavior, what Skyfall reports (and what still needs verification)

Skyfall reports a composite reward computed from three native verifiers with default weights: w_f = 0.5 (failure events), w_l = 0.25 (financial ledger status), and w_p = 0.25 (resource throughput). Skyfall also reports a per-configuration theoretical composite reward upper bound of 0.50 under optimistic assumptions (zero failures, minimum cost, full throughput).

That upper-bound number is notable. I could not verify the derivation from the materials available to me. The paper appendix contains the scaling and units that lead to 0.50, so consult Skyfall’s appendix for the exact math and interpretation. Treat the 0.50 figure as Skyfall’s reported construct until the scaling and units are inspected in the repository or appendix.

Takeaway: composite rewards are only useful when their units map to business KPIs, ask for the derivation and a worked example that ties reward deltas to dollars, downtime minutes, or OTIF percentage points.

Bootstrapping policies: frontier traces → SFT → online PPO (Skyfall’s pipeline)

Skyfall reports a two-stage initialization pipeline to make experiments tractable in large action spaces:

  • A frontier model (Gemini 3.1 Pro) collects trajectories using the ReAct framework.
  • Those traces are used to supervised-fine-tune (SFT) Qwen3-14B, producing a shared checkpoint that is the common starting point for online training.
  • Online post‑training uses PPO as the base optimizer.

This pipeline follows a common pattern: use high-quality demonstrated trajectories to initialize behavior, then refine online with RL. It also raises a reproducibility question. Skyfall’s reported runs depend on proprietary frontier-model traces. Unless Skyfall publishes the SFT checkpoint or provides public traces, exact reproduction by outside teams will be difficult.

Takeaway: if you plan to benchmark or adopt methods from MORPHEUS, confirm whether the SFT checkpoint and frontier traces are public or if Skyfall provides a public fallback.

How MORPHEUS measures continual learning

Rather than a single cumulative reward, MORPHEUS uses a six‑metric evaluation protocol Skyfall describes as better suited to persistent, non‑stationary settings:

  • Per‑configuration reward
  • Adaptation speed
  • Forgetting
  • Recovery time
  • Stability
  • Performance gap

Skyfall highlights adaptation speed as the headline measure. They define it as the number of environment steps until a running-average reward reaches half the reported upper bound (the precise averaging window used to compute the running average is specified in the appendix). Supplementary diagnostics include Relative Adaptation Advantage (RAA), which measures how much faster an algorithm adapts versus a baseline, and effective rank, a proxy for representational plasticity that measures the spread or rank of internal representations.

Takeaway: adaptation time and forgetting matter more than peak reward in long‑lived systems. Ask for the appendix definition of adaptation speed and alternative thresholds (25% / 75%) to test robustness.

Baselines and early results, Skyfall’s reported findings (summary)

Skyfall evaluated four algorithm families, PPO, HER, EWC, and LCM, on two tasks (dynamic resource allocation under structured drift; scheduling with delayed effects) inside two environments (process‑outbound and process‑inbound). High‑level reported findings:

  • None of the baselines reach the reported theoretical upper bound across regimes.
  • PPO and HER generally adapt in the first configuration but often fail to adapt in later regimes.
  • Mean performance gaps are reported as “near 1.0” for every method in the release summary; Skyfall’s paper and repo contain the full plots and numeric curves.

Those outcomes underscore the gap between episodic benchmark success and durable, multi‑regime robustness. The community needs full per‑seed curves, confidence intervals, and compute budgets to judge statistical reliability. Skyfall-Research/morpheus-evals is where those artifacts should appear.

How to verify MORPHEUS before you rely on its scores

Benchmarks steer research and purchasing decisions. Before treating MORPHEUS scores as a production readiness signal, ask Skyfall for these reproducibility artifacts (at minimum):

  • Environment code and tagged release for each TypeScript world plugin, plus Operational Descriptors (ODs), scheduler, and seed data.
  • Full enumeration and semantics of the eleven failure types (the paper summary only lists examples).
  • Asynchronous shift schedules and the code that drives presets so the timing of regime changes is reproducible.
  • Reward computation code and the appendix derivation for the reported composite upper bound = 0.50.
  • SFT checkpoint used to initialize RL runs, or downloadable frontier-model traces plus instructions to re-create the checkpoint.
  • Training scripts with hyperparameters, random seeds, number of independent seeds (expect N ≥ 10), GPU/TPU type, wall‑clock hours per run, and raw logs.
  • Statistical reporting: per‑run traces, confidence intervals (95% CI or bootstrap), and compute/sample-efficiency tables.

If key pieces are proprietary (for example, gated frontier-model APIs or closed SFT checkpoints), request alternative public reproductions, for example demonstrate runs from open models or release high‑quality synthetic traces.

Suggested ablations and experiments that would strengthen claims

  • Ablation matrix: run failure‑injection only vs asynchronous shifts only vs both combined (recommend a 3×3 grid over multiple seeds to quantify interactions).
  • Compounding vs externally‑triggered shifts: add variants where agent actions increase future failure rates and compare adaptation, cumulative regret, and required manual rollback frequency.
  • Reward‑weight sweep: test several weight vectors (for example [0.7, 0.2, 0.1], [0.5, 0.25, 0.25], [0.2, 0.5, 0.3]) to measure sensitivity of winners to business priorities.
  • Broader baselines: include replay‑based continual RL, meta‑RL algorithms, online off‑policy methods, and modern foundation‑model fine‑tuning strategies.
  • Compute/payoff analysis: report sample efficiency, wall‑clock time, and cost so practitioners can decide whether a method is feasible at scale.

Operational guidance for enterprise teams

  • Map rewards to KPIs: translate composite reward units into money, downtime minutes, or OTIF percentage points before using benchmark scores to make business decisions.
  • Monitor for drift and plan recovery: adaptation speed and recovery time should map to your SLAs and escalation procedures.
  • Manage forgetting explicitly: use replay buffers, modular policies, or regularized updates to reduce catastrophic forgetting in long‑running services.
  • Avoid opaque bootstraps: if your pipeline depends on proprietary frontier models, demand published checkpoints or equivalent public baselines to avoid vendor‑lock and irreproducible claims.
  • Human‑in‑the‑loop fallbacks: conservative constraints and manual override mechanisms remain critical when policies operate in persistent, high‑stakes systems.

Risks, limitations, and governance notes

Simulations can misrepresent subtle system‑level interactions. Skyfall’s MORPHEUS design deliberately uses externally scheduled regime shifts, a defensible simplification, but that may understate the realism of compounding shifts that arise from the agent’s own past decisions. Anchor any simulation‑based assessment with domain‑expert validation and controlled pilot deployments.

Where to look next

Skyfall’s project page (https://morpheus.skyfall.ai/) and the OpenReview entry (openreview.net/pdf?id=31P1VAfLkJ) are the places Skyfall points readers for the paper and artifacts. Skyfall also lists an evaluation repository Skyfall-Research/morpheus-evals. Check those locations for the reproducibility checklist items above.

Key questions, and short, honest answers

  • What makes MORPHEUS different from episodic RL benchmarks?

    MORPHEUS removes episodic resets and injects structured non‑stationarity so past actions persist and can change future conditions, forcing continual adaptation rather than episodic mastery (Skyfall AI’s stated design goal).

  • Does MORPHEUS publish the exact failure types, preset rates, and reward math?

    Skyfall reports an eleven‑type failure taxonomy and preset rates (light 5%, realistic 8%, moderate 15%, aggressive 30%) and default reward weights (w_f=0.5, w_l=0.25, w_p=0.25). The derivation of the reported composite upper bound (=0.50) and the full failure semantics should be confirmed in the paper appendix and the Skyfall-Research/morpheus-evals repo.

  • Do standard RL algorithms work on MORPHEUS out of the box?

    According to Skyfall’s reported baselines (PPO, HER, EWC, LCM), no single method closes the gap across regimes; many adapt only in the initial regime and fail on later shifts. Full numeric curves and confidence intervals are required to judge robustness.

  • Is the benchmark reproducible if I don’t have access to proprietary frontier models?

    Skyfall’s reported pipeline uses a proprietary frontier model to generate traces. Reproducibility hinges on whether Skyfall publishes the SFT checkpoint or alternative public traces; request those artifacts or ask for public baseline traces when evaluating the repo.

  • How should an enterprise team use MORPHEUS to assess operational risk?

    Use MORPHEUS as a stress test for adaptation speed, forgetting, and recovery time, but only after mapping reward components to concrete KPIs and validating failure modes with domain experts. Combine simulation results with staged pilots and human‑in‑the‑loop safeguards before full deployment.

Publish the code, the traces, the checkpoints, and a full ablation suite, then the community can determine which continual‑learning strategies actually hold up in life without resets.

0