Reproducibility in quantum computing depends on more than saving a notebook. This checklist shows what to record before, during, and after a simulator or hardware run so another developer can rebuild the circuit, understand the execution context, and compare results over time.
Overview
A quantum result is the product of several layers: application code, circuit construction, compilation or transpilation, backend configuration, execution conditions, measurement, and classical post-processing. If any of those layers is missing from the record, a later run may produce a different result without making the reason obvious.
Reproducible quantum experiments therefore require two related records. The first is the experiment definition: the code, dependencies, circuit, parameters, and intended procedure. The second is the run record: the backend, job settings, calibration context, returned data, and analysis performed for one execution.
Keep both records under version control. Store source code and small text-based metadata in the repository, while placing large result files in an appropriate artifact store. Give every run a unique identifier and link it to the exact code revision used. This makes a benchmark auditable without forcing another developer to guess which notebook cell or local environment produced the reported number.
The same approach works across Qiskit, Cirq, PennyLane, Amazon Braket, and other quantum programming frameworks. Names and APIs change, but the underlying questions remain stable: What circuit ran? Where did it run? Under which software and hardware conditions? How was the output transformed into the reported result?
What to track
1. Identity, purpose, and source revision
Start each experiment with a short description of the question being tested. Record the hypothesis, target metric, expected baseline, and whether the run is a simulator test, noisy simulation, hardware execution, or a comparison between backends.
- Experiment name and unique run identifier
- Date and time, including time zone
- Repository name, branch, and commit hash
- Notebook path or script entry point
- Input dataset, parameter file, and checksums where practical
- Research question, baseline, and success metric
A commit hash is more useful than a general label such as “latest” or “final.” If a notebook was executed interactively, save the executed version or export the relevant circuit and result objects so the record does not depend on hidden notebook state.
2. Software and environment
Record the language version and every package that can affect circuit construction, compilation, execution, or analysis. At minimum, capture the Python version, quantum SDK version, provider or plugin version, simulator version, and operating-system or container information. A lockfile, environment file, container digest, or package export can complement the human-readable log.
- Python version and environment manager
- Quantum SDK and provider package versions
- Simulator, transpiler, compiler, and numerical library versions
- Operating system, container image, or runtime identifier
- Relevant environment variables, excluding credentials and secrets
Do not place access tokens in notebooks, logs, or repositories. Record the provider and account context only to the level needed for reproducibility and auditing.
3. Circuit definition and compilation
Save a machine-readable representation of the logical circuit as well as a readable diagram or text rendering. Include the number of qubits and classical bits, register layout, gate sequence, parameter values, observable definitions, and measurement mapping. For parameterized circuits, record the parameter vector and its ordering.
Compilation can change the experiment substantially. Record the target backend, optimization level or compiler preset, basis gates if specified, coupling or connectivity assumptions, layout and initial mapping, routing method, dynamical-decoupling options, and any custom pass manager. Save the compiled circuit when the platform permits it, not just the original logical circuit.
For a useful resource-estimation baseline, compare logical and compiled qubit counts, gate counts, two-qubit gate counts, and circuit depth. The quantum circuit resource estimation guide provides a practical framework for recording these quantities.
4. Randomness and execution settings
Record every random seed that can affect the result: circuit generation, parameter initialization, train-test splitting, simulator sampling, randomized compilation, and post-processing. A single seed may not control all sources of randomness, so note the random-number libraries and where each seed is applied.
- Number of shots or repetitions
- Batching, job grouping, and maximum-circuit settings
- Measurement order and bit-string conventions
- Timeout, retry, and queue-related behavior if relevant
- Noise model, error rates, custom channels, and simulator options
- Seed values and the random-number generators used
For variational algorithms, also save optimizer name, learning rate or step rule, stopping criteria, initial parameters, parameter-shift or gradient settings, iteration count, and objective values. A VQE result cannot be meaningfully reproduced if only the final energy is reported; the ansatz, optimizer, measurement strategy, and convergence history matter as well. See the VQE workflow guide for the variables that commonly belong in this record.
5. Backend and hardware context
For cloud quantum computing, record the exact backend name as returned by the provider, not only a friendly label. Capture the provider or platform, region when exposed, backend configuration, supported qubits, and any backend-specific options used by the job.
Hardware conditions can change between submissions. Save the job identifier, submission and completion times, queue or execution status, calibration identifier or timestamp when available, and relevant device properties such as connectivity, gate errors, readout errors, and coherence information. If the platform does not expose a field, write “not available” rather than silently omitting it. This distinction helps separate an unavailable measurement from a forgotten one.
6. Results and post-processing
Preserve raw measurement results whenever the provider allows it. Counts alone may not be enough: also retain quasi-probabilities, expectation values, metadata, warnings, job status, and any returned fields used in the analysis. Store the exact post-processing code and its configuration, including filtering rules, normalization, endian conversion, mitigation method, aggregation, and statistical summaries.
Clearly separate raw output from derived output. If readout mitigation, zero-noise extrapolation, symmetry verification, or another correction is applied, record the calibration circuit, extrapolation points, correction matrix or parameters, and the unmitigated baseline. The error mitigation overview can help structure these fields.
Reusable experiment log template
experiment_id:
objective:
baseline_and_metric:
started_at_utc:
repository:
commit:
entry_point:
python_version:
sdk_and_provider_versions:
backend:
backend_region_or_provider:
logical_qubits:
compiled_qubits:
logical_depth:
compiled_depth:
transpilation_or_compilation_settings:
circuit_file_or_hash:
parameter_values:
random_seeds:
shots:
noise_model:
job_id:
submitted_at_utc:
completed_at_utc:
calibration_id_or_timestamp:
raw_result_file:
postprocessing_commit_or_file:
mitigation_settings:
reported_result:
known_limitations:
Cadence and checkpoints
Use the log at three checkpoints rather than trying to reconstruct it after execution.
- Before the run: freeze the code revision, environment, circuit inputs, seeds, backend target, compilation settings, and success metric. Run a small local smoke test to confirm that the saved circuit and measurement mapping are valid.
- At submission: capture the provider job identifier, exact backend name, shot count, submission time, and serialized execution request if available.
- After completion: save raw results, backend metadata, calibration context, warnings, post-processing outputs, and a short interpretation. Verify that the artifact can be opened without the original interactive session.
For active projects, review the experiment format monthly or at the end of each benchmark cycle. A quarterly review is reasonable for slower projects. Update the template whenever an SDK changes its job metadata, a provider introduces a new backend field, a compiler option changes behavior, or the team adds a new mitigation or analysis step.
Use a consistent notebook convention: one notebook for exploration, one script or reproducible entry point for the final run, and a generated results directory containing metadata, raw output, and derived tables. Avoid making a notebook cell the only source of a critical parameter.
How to interpret changes
When a new run differs from an earlier one, compare the records in layers. First check whether the logical circuit, parameters, shots, and seeds match. Next compare compilation: qubit layout, depth, two-qubit gates, routing, and optimization settings. Only then compare backend and hardware context.
A changed result does not automatically indicate a better or worse algorithm. It may reflect a different transpiled circuit, a new calibration window, a changed noise model, measurement conventions, or a post-processing change. Label comparisons as like-for-like only when the relevant variables are held constant. Otherwise, describe them as a new configuration or an exploratory run.
For hardware benchmarks, report both the central result and the conditions that produced it. For stochastic outputs, include repeated-run variation or an appropriate uncertainty summary instead of presenting one sampled value as definitive. For algorithm development, compare against a simulator or ideal baseline where possible, while keeping the baseline's own software and configuration record.
Unit tests can protect the logical circuit while the experiment log protects the execution. Add tests for qubit ordering, measurement mapping, parameter binding, and expected simple cases before scheduling hardware jobs. The cross-SDK quantum circuit testing guide offers patterns for this layer of the workflow.
When to revisit
Revisit this checklist before every hardware benchmark, published comparison, SDK migration, or rerun intended to validate an earlier result. Perform a formal review monthly during active development and quarterly when experiments are less frequent.
Trigger an immediate update when you change the SDK or Python environment, move between simulators or providers, alter transpilation settings, add error mitigation, change measurement or bit-order conventions, or discover that a backend no longer exposes the same metadata. When porting a circuit between frameworks, record both the original and translated representations, because register ordering, parameter binding, and measurement semantics may differ. Use the circuit-porting checklist before treating the outputs as directly comparable.
Before closing each experiment, run this final action list:
- Commit the code, environment specification, circuit, and experiment log.
- Confirm that the log contains exact versions, seeds, shots, backend, job ID, and calibration context.
- Archive raw results separately from processed results and record their file paths or hashes.
- Re-run the post-processing from the saved raw data in a clean environment.
- Write one paragraph explaining what changed from the previous run and why.
- Mark missing provider fields explicitly and schedule the next review if the workflow is evolving.
A complete record turns a one-off quantum result into a reusable engineering artifact. It lets another developer reproduce the circuit, identify meaningful changes, and decide whether a new result reflects an algorithmic improvement, a software change, or the conditions of a particular hardware run.