Skip to content

Reproducibility

Dataset generation uses index-derived random streams so results are stable across serial and parallel execution.

Seed hierarchy

flowchart TB
    D[Dataset seed] --> I[Dataset seed + sample index]
    I --> S[Sample generator]
    S -->|case_builder| C[Build random case]
    C --> L[Load generator]
    S -->|explicit case| L
    L --> M1[Sensor 1 generator]
    L --> M2[Sensor 2 generator]
    M1 --> E1[Effect generators]
    M2 --> E2[Effect generators]

For sample index i, DynaResp derives a 64-bit sample seed from SeedSequence([dataset_seed, i]). A random case_builder consumes that sample generator first; the generator then supplies deterministic seeds to stochastic loads and sensor effects. Explicit cases skip the case-builder step. Within either mode, the same seed and builders reproduce the same values.

Worker-count independence

Records are yielded in sample-index order even when simulations run in a thread pool. Because each sample seed depends only on the root seed and index, worker scheduling does not affect values.

import numpy as np
import dynaresp as dr

signal_only = [dr.Record.signal]

serial = dr.generate_dataset(
    cases=cases,
    system_builder=build_system,
    load_builder=build_load,
    sensors=sensor_layout,
    time=time,
    positions=positions,
    record=signal_only,
    seed=42,
    workers=1,
)
parallel = dr.generate_dataset(
    cases=cases,
    system_builder=build_system,
    load_builder=build_load,
    sensors=sensor_layout,
    time=time,
    positions=positions,
    record=signal_only,
    seed=42,
    workers=8,
)

np.testing.assert_array_equal(serial.signal, parallel.signal)

Exact case designs

cases bypasses parameter sampling entirely. Case order determines sample order, while the dataset seed still controls stochastic loads and sensor effects.

cases = dr.parameter_grid(
    stiffness_scale=[0.8, 1.0, 1.2],
    damping_ratio=[0.01, 0.02],
)

Running the same cases with a different worker count preserves both the chosen parameters and the simulated output. Change the seed only when you want new stochastic excitation or measurement noise for the same physical design.

What is recorded

Schema-v4 datasets always store the structural bookkeeping needed to validate the selection: schema version, sample count, selected record paths, storage dtype, parameter/target path mappings, and stored array keys.

Reproduction details are explicit selections:

  • Record.metadata.generation stores the root dataset seed, DynaResp version, builder identities, sensor definitions, variability provenance, and concrete system and load definitions for the first realization;
  • Record.metadata.seed stores one effective uint64 seed per sample;
  • Record.case_parameter stores every numeric value in the final case mapping;
  • Record.time and Record.positions store the numerical axes.

Selecting storage_dtype=np.float32 does not change the simulation or random streams. Casting and field filtering happen only after each float64 simulation has completed.

The builder source code remains part of the experiment and should be kept in version control with the dataset metadata.

Reproducibility boundaries

Exact reproduction assumes:

  • the same DynaResp, NumPy, and SciPy behavior;
  • the same builders, sensors, and axes;
  • deterministic user-provided callbacks;
  • a consistent floating-point platform;
  • unchanged custom component implementations.

Builders can introduce external state, global randomness, file access, or version-dependent behavior. Use the generator passed to case_builder, use ForceInput.rng inside custom loads, and include external inputs in each case mapping.

  1. Set an explicit dataset seed for case builders, stochastic loads, and effects.
  2. Select Record.metadata.generation and Record.metadata.seed for datasets that must carry their generation provenance.
  3. Store the generated Zarr metadata with the dataset.
  4. Record the environment or lock file alongside important datasets.
  5. Use only supplied generators instead of np.random globals.
  6. Compare serial and parallel generation in tests for custom components.
  7. Treat case or builder changes as dataset-version changes.