Level-six engineering validation protocol · 04
A sixth-level engineering protocol for turning a robot model that merely loads into a pinned, rebuildable and reviewable simulation evidence baseline—with any physical-transfer claim kept inside measured boundaries.
Protocol scope
For one declared MuJoCo Menagerie model, scene and runtime used to reproduce structural, numerical and behavioural experiments, with a bounded physical comparison only when a matching robot and measured signals exist. It excludes manufacturing release, system safety approval, certification and autonomous deployment.
Configuration baseline
Menagerie commit, exact model directory, XML and mesh checksums, scene file, local changes, upstream source, per-directory README, licence and copyright notices.
Reject whenDo not begin when files, provenance, local edits or applicable licence notices cannot be reconciled.
MuJoCo release or commit, minimum model version, platform, architecture, dependency lock, loader path, compiler options and clean-environment rebuild instructions.
Reject whenReject a result that only runs in an unrecorded workstation state.
Units, timestep, integrator, solver, iteration and tolerance settings, gravity, contacts, friction, equality constraints, actuator dynamics and sensor definitions.
Reject whenReject hidden defaults, unit ambiguity or a time-step and solver combination that has not been declared and stress-tested.
Scene, controller, initial state, random seeds, perturbation distribution, run count, logged signals, comparison metric, tolerance and the exact claim being tested.
Reject whenDo not accept a replay whose success rule or comparison window was chosen after results were seen.
Staged verification
Freeze the upstream commit and exact asset subset; inventory XML, meshes, textures and support files; record modifications separately; preserve the notices that apply to the selected model.
Source manifest, file tree, checksums, modification patch and licence-notice bundle.
A reviewer can resolve every loaded file to an immutable source or declared local change.
Missing source files, unexplained binary assets, incompatible notices or unrecorded substitutions.
Build the declared environment from scratch, load and compile the model and scene, capture warnings, then repeat on a second clean runner where practicable.
Dependency lock, build log, compiler output, warnings register and machine-readable environment report.
The exact asset loads without undeclared files, unresolved warnings or manual workstation fixes.
A missing dependency, silent asset fallback, version-sensitive failure or manual edit that is absent from the record.
Inspect body and joint trees, frames, limits, inertials, masses, collision geometry, contacts, actuator and sensor maps, then run declared static-pose and gravity checks.
Compiled structural report, unit table, parameter dictionary, annotated frame map and exception register.
Every material parameter has a unit, meaning and provenance; static checks match the declared model intent.
Impossible inertia, inverted axes, undeclared scaling, interpenetration, missing collision coverage or actuator/sensor mismatch.
Run the declared experiment while perturbing timestep, integrator, solver, iterations, tolerances and contact parameters inside a predeclared diagnostic matrix.
Sensitivity matrix, solver statistics, energy and constraint traces, divergence cases and retained failure logs.
The result remains inside its declared tolerance or its sensitivity is explicitly bounded and explained.
Unexplained instability, materially different conclusions under reasonable settings, concealed warnings or cherry-picked stable runs.
Replay fixed-state, fixed-seed golden runs, then repeat the declared seed and perturbation distribution; compare raw traces and outcome distributions against frozen tolerances.
Golden traces, replay hashes, seed manifest, run matrix, outcome distribution and all failed replays.
A second operator reproduces the stated result without changing code, thresholds or the trial distribution.
Post-hoc threshold changes, missing seeds, nondeterminism without a statistical account or failure runs removed from the package.
Only when a matching physical system exists, replay a guarded low-energy task and compare predeclared observables such as joint states, contact timing, currents or task outcomes; model residuals rather than hiding them.
Robot configuration identity, calibrated sensor records, time-aligned simulation/physical traces, residual analysis, interventions and unresolved mismatch register.
Transfer evidence is stated only for the measured observables, configuration, task, environment and tolerance.
Unsafe motion, unmatched hardware, unsynchronised data, uncalibrated sensors or an attempt to infer safety from simulation agreement.
Acceptance claims
A clean environment retrieves the pinned sources and compiles the exact model without undeclared intervention.
Claim boundaryRebuildability does not establish correctness, quality or permission for every downstream use.
A reviewer can trace units, frames, inertials, collision, actuator and sensor parameters to declared sources or assumptions.
Claim boundaryA complete parameter table is not proof of physical identification.
A second operator reproduces frozen traces or distributions inside predeclared numerical tolerances.
Claim boundaryA reproducible simulation can still reproduce the same modeling error.
The report names the exact physical configuration, measured observables, residuals, environment and expiry or retest trigger.
Claim boundaryAgreement on selected observables is not a general digital-twin, safety or deployment claim.
Evidence pack & boundaries
FUURAA analysis
A model loading successfully—or producing a persuasive video—is only an entry condition. Credible simulation evidence requires an immutable asset identity, an explicit numerical contract, replayable traces, sensitivity testing and, where physical transfer is claimed, visible residuals tied to one measured system. FUURAA therefore treats simulation as a bounded evidence instrument, not a shortcut to physical truth.
Primary sources