Validation & Verification in Physics of Plasmas

Physics of Plasmas is making a collection of 20 papers on verification and validation available for free download for a limited time.

Theoretical models, both analytical and numerical, are playing an increasingly important role in predicting complex plasma behavior, and providing a scientific understanding of the underlying physical processes.

Since the ability of a theoretical model to predict plasma behavior is a key measure of the model’s accuracy and its ability to advance scientific understanding, it is Physics of Plasmas’ Editorial Policy to encourage the submission of manuscripts whose primary focus is the verification and/or validation of codes and analytical models aimed at predicting plasma behavior.

Quite a nice collection:

• Verification methodology for plasma simulations and application to a scrape-off layer turbulence code
• Validation of plasma shape reconstruction by Cauchy condition surface method in KSTAR
• Wave dispersion in the hybrid-Vlasov model: Verification of Vlasiator
• Experimental validation of Mueller-Stokes theory and investigation of the influence of the Cotton-Mouton effect on polarimetry in a magnetized fusion plasma
• A verification of the gyrokinetic microstability codes GEM, GYRO, and GS2
• Verification of particle simulation of radio frequency waves in fusion plasmas
• Validation of single-fluid and two-fluid magnetohydrodynamic models of the helicity injected torus spheromak experiment with the NIMROD code
• Numerical verification of Orbit and Nemato codes for magnetic topology diagnosis
• Experimental evidence of ion acoustic soliton chain formation and validation of nonlinear fluid theory
• Verification of electromagnetic fluid-kinetic hybrid electron model in global gyrokinetic particle simulation
• Verification and validation of linear gyrokinetic simulation of Alfvén eigenmodes in the DIII-D tokamak
• Validation of electron temperature gradient turbulence in the Columbia Linear Machine
• Evidence cross-validation and Bayesian inference of MAST plasma equilibria
• Advances in validating gyrokinetic turbulence models against L- and H-mode plasmas
• Analysis of plasma instabilities and verification of the BOUT code for the Large Plasma Device
• On the application of electron cyclotron emission imaging to the validation of theoretical models of magnetohydrodynamic activity
• Methodology for turbulence code validation: Quantification of simulation-experiment agreement and application to the TORPEX experiment
• Nonlinear three-dimensional verification of the SPECYL and PIXIE3D magnetohydrodynamics codes for fusion plasmas
• A synthetic diagnostic for validation of electron gyroradius scale turbulence simulations against coherent scattering measurements
• Verification and validation for magnetic fusion

I started at the bottom of the list with "Verification and validation for magnetic fusion." That one gives a bit of philosophy and some high-level review of the state of the art in this particular field. I especially liked this part:

Strictly speaking we do not verify and validate codes. Technically we verify a set of calculations, then draw inferences about the validity of a code. Similarly, one validates a set of simulations, then draws inferences about the underlying model.

A careful grid convergence study can give results that allow an estimate of the order of accuracy of the solution method, and can be a strong bit of evidence that you've implemented things correctly. This estimate is based on an error ansatz (a model of the leading truncation error term). The values of the error functional for each solution in the grid convergence study allow for estimates of the values of the parameters in this (very simple) model. Even in a "purely mathematical" exercise such as verification, we can never escape uncertainty!

I also like this focus on the pragmatic:

The conditional nature of the definitions of both V&V should be noted. To avoid unbounded philosophical questions, V&V are best defined 1. for a targeted class of problems and applications, 2. for a set of specified variables and 3. at a specified level of accuracy. Together, the goal of validation and verification is an assessment of the extent to which a simulation represents true system behavior sufficiently to be useful.

When one says a code is validated, the natural thing to ask is, "for what purpose?"

The other interesting looking one in the collection that I have only skimmed so far is "Evidence cross-validation and Bayesian inference of MAST plasma equilibria." I can heartily commend their first reference.

Please highlight more V&V goodness in the comments, and enjoy!