Validation¶
This page is generated by scripts/generate_validation_report.py from validation/data/validation_cases.csv and a run of pytest -m validation. Do not edit by hand.
The suite contains 66 validation tests covering 63 checked quantities in 17 cases; 0 test function(s) failing in the run that produced this page.
What is validated, and against what¶
Reference types used throughout:
- analytic - a closed-form or textbook formula evaluated independently of the model code (Faraday's law, stoichiometry, exponential step responses, the Carnot bound, the isentropic work formula, ...).
- cross-check - an independent software implementation given identical inputs (
pvlib,pandapowervs a hand-written complex fixed-point solver and the MATPOWER case9 solution,PyPSA). - self-consistency - conservation laws or an equivalence between two code paths (energy and mass balances, world-driven run vs direct library call, re-run determinism).
Where a source is cited it is a physical constant, a textbook formula or a named public data set; numeric reference values that are only approximately known (e.g. the compressibility of hydrogen at 350 bar) are labelled as such and checked with a correspondingly loose tolerance. Models are surrogates for system-level studies, not bankable simulators: the validity ranges below are part of the contract.
Cases¶
| Case | Quantity | Reference | Source / citation | Tolerance | Status | Notes |
|---|---|---|---|---|---|---|
rc_thermal |
max_abs_error_K_dt900 | 0 | analytic: T(t)=T_amb+RQ+(T0-T_amb-RQ)exp(-t/(RC)) | absolute 0.03 | PASS | 24 h step response; explicit Euler error at dt=900 s (tau=37.5 h) |
rc_thermal |
euler_convergence_order | 1 | analytic: explicit Euler is first order | absolute 0.1 | PASS | error ratio between dt=900 s and dt=450 s |
rc_thermal |
fleet_vs_scalar_loop | independent scalar Euler loop | self-consistency | relative 1e-12 | PASS | fleet of size 1 equals an independent scalar implementation |
rc_thermal |
world_run_max_abs_error_K | 0 | analytic closed form | absolute 0.05 | PASS | fleet driven by signals through world(); Jacobi coupling |
pem_electrolyzer |
faraday_h2_mass_flow | Faraday's law | analytic: n_H2 = eta_FNI/(2F) | relative 1e-9 | PASS | kg/h; 2F with CODATA F |
pem_electrolyzer |
stack_power_balance | P = NIV | analytic | relative 1e-6 | PASS | limited by the 2501-point inversion of j*V(j) |
pem_electrolyzer |
water_to_h2_mass_ratio | M(H2O)/M(H2) = 8.936 | analytic stoichiometry H2O -> H2 + 1/2 O2 | relative 1e-9 | PASS | 1 mol H2O per mol H2 |
pem_electrolyzer |
thermoneutral_voltage_V | HHV*M/(2F) = 1.481 | analytic: E_tn = dH/(2F), dH(H2O,l) = 285.83 kJ/mol (HHV 141.8 MJ/kg) | relative 2e-3 | PASS | model constant 1.48 V |
pem_electrolyzer |
heat_plus_hhv_equals_power | P_actual | self-consistency: electricity = waste heat + H2 HHV energy | relative 3e-3 | PASS | HHV 141.8 MJ/kg vs 1.48 V thermoneutral voltage |
constants |
faraday_constant_C_per_mol | 96485.33212 | CODATA 2018 (e*N_A) | relative 1e-8 | PASS | |
constants |
gas_constant_J_per_mol_K | 8.314462618 | CODATA 2018 | relative 1e-6 | PASS | |
sabatier |
mol_co2_per_mol_h2_reacted | n_H2/4 | analytic: CO2 + 4 H2 -> CH4 + 2 H2O | relative 1e-12 | PASS | |
sabatier |
mol_ch4_per_mol_h2_reacted | n_H2/4 | analytic stoichiometry | relative 1e-12 | PASS | CH4 molar mass chosen so that the mass balance closes exactly |
sabatier |
mol_h2o_per_mol_h2_reacted | n_H2/2 | analytic stoichiometry | relative 1e-12 | PASS | |
sabatier |
mass_balance | H2 fed + CO2 consumed | analytic mass conservation | relative 1e-12 | PASS | = CH4 + H2O + unreacted H2 |
sabatier |
reaction_heat_kJ_per_mol_CH4 | 165.02 | standard formation enthalpies at 298 K: CH4 -74.87, H2O(g) -241.83, CO2 -393.51 kJ/mol | relative 1e-3 | PASS | model uses 165 kJ/mol |
heat_pump |
energy_balance_q_equals_p_times_cop | Q_heat | analytic: Q = P*COP | relative 1e-12 | PASS | |
heat_pump |
max_cop_over_carnot | 0.65 | analytic: second-law bound COP <= T_sink/(T_sink-T_source) | relative 1e-9 | PASS | largest observed COP/COP_Carnot equals the Carnot efficiency factor (0.65) over 900 random cases |
heat_pump |
cop_eta_times_carnot | 3.5 | analytic: COP = eta*T_sink/(T_sink-T_source) | relative 1e-12 | PASS | 15 degC source, 75 degC sink |
wind_turbine |
power_curve_interpolation_MW | published curve table | self-consistency: model vs linear interpolation of a manufacturer-style tabulated curve (generic 1.85 MW class, illustrative values) | relative 1e-4 | PASS | standard density 15 degC/sea level reproduces the curve |
wind_turbine |
power_ratio_vs_air_density | rho/rho_0 | analytic: P = 1/2 rho A Cp v^3 (cubic regime) -> P ~ rho | relative 1e-4 | PASS | barometric formula + ideal gas; dense cubic curve |
wind_turbine |
rotor_lag_at_one_tau_MW | 1.896 | analytic first-order lag 1-exp(-1) | relative 2e-3 | PASS | implicit Euler with dt=0.01 s |
battery |
fleet_round_trip_efficiency | eta^2 | analytic: charge and discharge efficiency each eta | relative 1e-3 | PASS | |
battery |
bess_step_energy_balance | E_stored + E_joule + E_inverter | analytic energy conservation (AC in = stored + joule heat + inverter loss) | absolute 1e-9 | PASS | MWh per step |
battery |
bess_cumulative_energy_balance | sum of parts | analytic energy conservation | relative 1e-9 | PASS | |
battery |
bess_round_trip_efficiency | eta_inv^2 * eta_int | analytic: product of inverter and internal-resistance efficiencies at SoC 0.5 | relative 1e-2 | PASS | Voc varies slightly over the SoC excursion |
h2_tank |
mass_at_low_pressure_vs_ideal_gas | pVM/(R*T) | analytic ideal gas law | relative 1e-3 | PASS | Z-1 = 0.0006*p[bar] -> 3e-4 at 0.5 bar |
h2_tank |
Z_at_350bar_293K | 1.2 | self-consistency: model uses Z = 1 + 0.0006 p[bar]; tabulated H2 compressibility at 350 bar is approx 1.2 | relative 5e-2 | PASS | approximate literature value; checks order of magnitude only |
h2_tank |
pressure_roundtrip_bar | 200 | self-consistency: p -> m -> p | relative 1e-9 | PASS | |
h2_compressor |
specific_work_kJ_per_kg | textbook formula | textbook: w = ncpT1(r^((g-1)/(ng))-1)/eta_is with perfect intercooling (e.g. Cengel and Boles) | relative 1e-12 | PASS | n = 1..4 stages; t_intercool = t_in |
h2_compressor |
isothermal_limit_kJ_per_kg | RTln(p2/p1) | analytic: infinite-stage limit is isothermal compression work | relative 2e-3 | PASS | 400 stages, eta_is = 1 |
h2_compressor |
power_kW_equals_flow_times_work | m_dot*w | analytic | relative 1e-12 | PASS | |
thermal_storage |
stored_energy_equals_integral_flow_minus_loss | sum((Q-loss)*dt) | analytic energy conservation | absolute 1e-9 | PASS | MWh over 1500 steps including both SoC limits |
thermal_storage |
standing_loss_after_one_day | 0.8*exp(-0.02) | analytic exponential decay at 2 %/day | relative 1e-5 | PASS | |
emissions |
natural_gas_t_co2_per_MMBtu | 0.0592 | IPCC 2006 default 56.1 kg CO2/GJ (NCV) x 1.05506 GJ/MMBtu | relative 5e-3 | PASS | model constant 0.0592 t/MMBtu |
emissions |
natural_gas_ipcc_56p1_kg_per_GJ | 0.05919 | IPCC 2006 Guidelines vol. 2 table 2.2 natural gas | relative 5e-3 | PASS | |
emissions |
ccgt_fuel_from_heat_rate | 2426.1 | analytic: P*HR/(1.05506 GJ/MMBtu) | relative 1e-3 | PASS | 1.055e6 kJ/MMBtu used by the model |
carbon_capture |
captured_mass_flow_t_h | 90 | analytic: inlet x capture rate | relative 1e-12 | PASS | |
carbon_capture |
reboiler_GJ_per_tonne | 3 | analytic unit conversion MW_th -> GJ/h | relative 1e-12 | PASS | typical amine 2.5-4 GJ/t asserted as a sanity band |
carbon_capture |
parasitic_kWh_per_tonne | 120 | analytic unit conversion MW -> kWh/h | relative 1e-12 | PASS | |
absorption_chiller |
thermal_cop | 0.7 | analytic: Q_cool = COP_th * Q_heat | relative 1e-12 | PASS | |
pv_pvlib |
cell_temperature_vs_noct_sam_K | pvlib.temperature.noct_sam | cross-check: pvlib 0.16.1 (NOCT 45 degC; module efficiency 0.18) | absolute 0.5 | PASS | G 200-1000 W/m2, 20 degC, 1 m/s; models differ in wind and efficiency terms |
pv_pvlib |
cell_temperature_vs_sapm_K | pvlib.temperature.sapm_cell | cross-check: pvlib SAPM open-rack glass/glass parameters | absolute 1.5 | PASS | documented tolerance: SAPM is a different empirical model |
pv_pvlib |
dc_power_vs_pvwatts_dc_MW | pvlib.pvsystem.pvwatts_dc | cross-check: pvlib PVWatts DC model (gamma -0.0038/K) | relative 1e-9 | PASS | identical equation at identical cell temperature |
pv_pvlib |
ac_power_vs_pvwatts_inverter_MW | pvlib.inverter.pvwatts | cross-check: pvlib PVWatts inverter (eta_nom 0.96) | relative 3e-2 | PASS | generic efficiency curve vs PVWatts curve; documented tolerance |
pandapower |
case9_vm_pu_vs_matpower | MATPOWER case9 voltages | MATPOWER manual case9 solved voltage magnitudes (3 decimals) | absolute 1.0e-3 | PASS | pandapower.networks.case9 AC power flow |
pandapower |
energysim_vs_direct_runpp_vm_pu | direct pp.runpp | self-consistency: world-driven run equals direct runpp with the same load | absolute 1e-9 | PASS | includes kW->MW connection unit conversion |
pandapower |
two_bus_vm_pu_vs_fixed_point | fixed-point solution | analytic: independent complex fixed-point iteration | absolute 1e-6 | PASS | |
pandapower |
two_bus_angle_deg_vs_fixed_point | fixed-point solution | analytic: independent complex fixed-point iteration | absolute 1e-5 | PASS | |
pypsa |
two_bus_vm_pu_vs_fixed_point | fixed-point solution | analytic: independent complex fixed-point iteration | absolute 1e-6 | PASS | |
pypsa |
energysim_vs_direct_pypsa_vm_pu | direct n.pf() | self-consistency: world-driven run equals direct PyPSA call | absolute 1e-9 | PASS | xfail if the adapter raises (e.g. a PyPSA version where n.pf() is unavailable); passes on PyPSA 1.1 |
pypsa |
energysim_vs_direct_pypsa_slack_P_MW | direct n.pf() | self-consistency: world-driven run equals direct PyPSA call | absolute 1e-9 | PASS | |
orchestrator |
jacobi_cascade_convergence_order | 1 | analytic: A x'=ax, B y'=cy+dx with exact subsystem integration; only coupling error remains | absolute 0.1 | PASS | observed order from h=0.25 and 0.125; zero-order hold coupling is first order |
orchestrator |
gauss-seidel_cascade_convergence_order | 1 | analytic cascade as above | absolute 0.1 | PASS | Gauss-Seidel is also first order for one-way coupling |
orchestrator |
gauss_seidel_order_minus_jacobi_order | 0 | self-consistency | absolute 0.15 | PASS | difference of observed orders; asserted within +-0.15 (GS must not be worse) |
orchestrator |
cyclic_jacobi_convergence_order | 1 | analytic: x(t)=expm(M t) x0 for a two-way coupled 2x2 linear system | absolute 0.1 | PASS | scipy.linalg.expm reference |
orchestrator |
cyclic_gs_equals_jacobi | Jacobi result | documented behaviour: loops use a one-step delay in both schemes | relative 1e-12 | PASS | |
orchestrator |
zero_order_hold_prediction_order | 1 | analytic: hold error ~ h*|s'| | absolute 0.15 | PASS | prediction of the end-of-step signal value |
orchestrator |
linear_extrapolation_prediction_order | 2 | analytic: linear extrapolation error ~ h^2*|s''| | absolute 0.3 | PASS | |
orchestrator |
chain_bess_energy_residual_MWh | 0 | analytic energy conservation (property test; 4 seeds) | absolute 1e-9 | PASS | max over steps |
orchestrator |
chain_electrolyzer_energy_relative_residual | 0 | self-consistency: P = Q_waste + m_H2/eta_F*HHV (V_tn=1.48 V vs HHV 141.8 MJ/kg) | absolute 3e-3 | PASS | |
orchestrator |
chain_tank_mass_residual_kg | 0 | analytic mass conservation (property test) | absolute 1e-9 | PASS | max over steps |
orchestrator |
chain_hydrogen_end_to_end_kg | stored mass change | analytic mass conservation across electrolyzer and tank | absolute 1e-6 | PASS |
Orchestrator findings¶
- Zero-order-hold Jacobi coupling of a one-way cascade converges with observed order 1 against the analytic solution when each subsystem integrates exactly; Gauss-Seidel is also first order and never worse.
- Inside a strongly connected component Gauss-Seidel falls back to the Jacobi one-step delay, so the two schemes give identical results for genuinely cyclic systems.
extrapolation='linear'extrapolates each exchanged value a full macro step ahead. It therefore predicts the end-of-step value of a smooth signal with second-order accuracy (versus first order for the hold), but because the receiver holds that value over the whole step, the coupled-ODE error is not reduced; extrapolating to the step midpoint would be the second-order choice. Linear extrapolation is only applied by Jacobi coupling.
Documented validity ranges per model¶
Extracted from the model docstrings.
| Model | Validity |
|---|---|
AbsorptionChiller |
Validity: constant thermal COP independent of source and sink temperatures; cooling between 0 and q_cooling_max_mw, limited by the available waste heat; steady state. |
AlkalineElectrolyzer |
Validity: between min_turndown_pct and 100 % of p_nominal_mw (below that the plant is in STANDBY); system-level steady-state specific consumption with no stack temperature, pressure or degradation model. |
BatteryBESS |
Validity: meaningful for roughly 0-45 C pack temperature, SoC between ~5 and 95 %, and |P| <= p_max_mw. SOH is floored at 0.5. Q_heat_kW is DC joule heat only (inverter losses are not included). |
CHP_SteamTurbine |
Validity: steam flow 0-p_max_mw kg/s, heat demand <= 2 x base power. |
CarbonCapture_Amine |
Validity: steady loads, capture rate 0.8-0.95. |
CombinedCycle_CCGT |
Validity: T_amb -20..45 C, loads between MSG and rated. |
DataCenterLoad |
Validity: T_amb -10..40 C, 0 <= IT <= it_capacity_mw. |
ElectricBoiler |
Validity: 0 to p_max_mw. |
ElectricChiller |
Validity: T_amb 5..45 C, constant-COP part-load behaviour. |
FlowBattery |
Validity: an energy-bucket model; no electrolyte imbalance, crossover or temperature dependence. P_actual is the grid-side power including pump parasitics (positive = import). |
Flywheel |
Validity: seconds-to-minutes duty cycles; no speed-dependent power limit or bearing/windage model beyond the lumped self-discharge. |
FuelCell_PEM |
Validity: constant electrical efficiency independent of load, power between 0 and p_nominal_mw, ramp limited to ramp_rate_mw_min; no degradation or stack-temperature model. |
FuelCell_SOFC |
Validity: constant efficiencies independent of load, power between 0 and p_nominal_mw; no ramp limit and no thermal-cycling or start-up model. |
GasTurbine_OCGT |
Validity: T_amb -20..45 C, loads between MSG and rated; no emissions transients, no turbine inertia dynamics, no humidity or altitude effects. |
H2Compressor |
Validity: ideal-gas hydrogen (gamma = 1.41, no compressibility correction), equal stage pressure ratios, steady-state flow up to max_flow_kg_hr; no start-up or surge behaviour. |
H2StorageTank |
Validity: single-node, isothermal tank (gas temperature fixed at t_gas_c) with the linear compressibility Z = 1 + 0.0006 p[bar]; intended for pressures up to about 350 bar at roughly ambient temperature; no thermal effects of filling or discharging. |
IndustrialHeatPump |
Validity: source -10..60 C, sink 30..120 C, lift 10..90 K. |
IndustrialProcessSteam |
Validity: quasi-static demand header; no pressure, temperature or condensate-return dynamics; constant specific enthalpy. |
MethanationReactor |
Validity: steady state with a constant conversion_efficiency independent of load, temperature and pressure; CO2 is assumed available; H2 feed up to max_h2_feed_kg_hr. |
PEMElectrolyzer |
Validity: stack power between min_turndown * p_nominal_mw and p_nominal_mw (the polarization curve ends at 2.5 A/cm2); steady-state electrochemistry with no temperature dependence of the cell voltage and no degradation; dynamics are limited to the ramp limit and the lumped stack thermal mass. |
PhysicsPV |
Validity: G_poa in 0-1300 W/m2, T_amb -20..50 C, v_wind 0..20 m/s; the output is a smooth surrogate for plant-level studies, not for bankable yield. |
ReciprocatingGenset |
Validity: 0 to rated power; the sum of electrical and heat-recovery efficiency should not exceed 1 (not enforced by the constructor). |
SOECElectrolyzer |
validity guard (600-900 degC, ValueError otherwise); it does not enter the calculation. Stateless: dt has no effect. Validity: operating_temp_c between 600 and 900 degC (enforced at construction); steady state, constant specific electricity consumption independent of load and temperature; the heat for steam generation is supplied externally. |
SolarThermalCollector |
Validity: G > 10 W/m2, dT 0-100 K. |
ThermalStorageTank |
Validity: capacity-level studies at hours-to-days timescales. |
WindTurbine |
Validity: hub-height wind 0-30 m/s, T_amb -30..45 C, altitude 0-3000 m; no wake, turbulence, icing or yaw-misalignment effects. |
Reproducing¶
pvlib and pandapower/pypsa cross-checks are skipped when the packages are not installed.