Simulation-based Engineering Science

gmcrews has a couple interesting posts on model verification and validation (V&V) and his commment on scientific software has links to a couple ’state of the practice’ reports [1] [2]. The reports are about something called Simulation-based Engineering Science (SBES), which is the (common?) jargon they use to describe doing research and development with computational modelling and simulation.

1 Notes and Excerpts from [1]

Below are some excerpts from the executive summary along with a little commentary.

Simulation has today reached a level of predictive capability that it now firmly complements the traditional pillars of theory and experimentation/observation. Many critical technologies are on the horizon that cannot be understood, developed, or utilized without simulation. At the same time, computers are now affordable and accessible to researchers in every country around the world. The near-zero entry-level cost to perform a computer simulation means that anyone can practice SBE&S, and from anywhere.

1.1 Major Trends Identified

1. Data-intensive applications, including integration of (real-time) experimental and observational data with modelling and simulation to expedite discovery and engineering solutions, were evident in many countries, particularly Switzerland and Japan.
2. Achieving millisecond time-scales with molecular resolution for proteins and other complex matter is now within reach using graphics processors, multicore CPUs, and new algorithms.
3. The panel noted a new and robust trend towards increasing the fidelity of engineering simulations through inclusion of physics and chemistry.
4. The panel sensed excitement about the opportunities that petascale speeds and data capabilities would afford.

1.2 Threats to U.S. Leadership

1. The world of computing is flat, and anyone can do it. What will distinguish us from the rest of the world is our ability to do it better and to exploit new architectures we develop before those architectures become ubiquitous.

   Furthermore, already there are more than 100 million NVIDIA graphics processing units with CUDA compilers distributed worldwide in desktops and laptops, with potential code speedups of up to a thousand-fold in virtually every sector to whomever rewrites their codes to take advantage of these new general programmable GPUs.

2. Inadequate education and training of the next generation of computational scientists threatens global as well as U.S. growth of SBE&S. This is particularly urgent for the United States; unless we prepare researchers to develop and use the next generation of algorithms and computer architectures, we will not be able to exploit their game-changing capabilities.

   Students receive no real training in software engineering for sustainable codes, and little training if any in uncertainty quantification, validation and verification, risk assessment or decision making, which is critical for multiscale simulations that bridge the gap from atoms to enterprise.

3. A persistent pattern of subcritical funding overall for SBE&S threatens U.S. leadership and continued needed advances amidst a recent surge of strategic investments in SBE&S abroad that reflects recognition by those countries of the role of simulations in advancing national competitiveness and its effectiveness as a mechanism for economic stimulus.

I don’t know of any engineering curriculums that have a good program for training people in all of the areas (the physics, numerical methods, design of experiments, statistics and software carpentry) to be competent high-performance simulation developers (in scientific computing the users and developers tend to be the same people). It’s requires multi-disciplinary, and deeply technical knowledge at the same time. The groups that try to go broad with the curriculum tend to treat the simulations as a black box. Those sorts of programs tend to produce people who can turn the crank on a code, but don’t have the deeper technical understanding needed to add the next increment of physics, or apply the newer more efficient solver, or adapt the current code to take advantage of new hardware. Right now that sort of expertise is achieved in an ad-hoc or apprenticeship kind of manner (see for example MIT’s program). That works for producing a few experts at a time (after a lot of time), but it doesn’t scale well.

1.3 Opportunities for Investment

1. There are clear and urgent opportunities for industry-driven partnerships with universities and national laboratories to hardwire scientific discovery and engineering innovation through SBE&S.
2. There is a clear and urgent need for new mechanisms for supporting R&D in SBE&S.

   investment in algorithm, middleware, and software development lags behind investment in hardware, preventing us from fully exploiting and leveraging new and even current architectures. This disparity threatens critical growth in SBE&S capabilities needed to solve important worldwide problems as well as many problems of particular importance to the U.S. economy and national security.

3. There is a clear and urgent need for a new, modern approach to educating and training the next generation of researchers in high performance computing specifically, and in modeling and simulation generally, for scientific discovery and engineering innovation.

   Particular attention must be paid to teaching fundamentals, tools, programming for performance, verification and validation, uncertainty quantification, risk analysis and decision making, and programming the next generation of massively multicore architectures. At the same time, students must gain deep knowledge of their core discipline.

The third finding is interesting, but it is a tall order. So we need to train subject matter experts who are also experts in V&V, decision theory, software development and exploiting unique (and rapidly evolving) hardware. Show me the curiculum that accomplishes that, and I’d be quite impressed (really, post a link in the comments if you know of one).

More on validation:

Experimental validation of models remains difficult and costly, and uncertainty quantification is not being addressed adequately in many of the applications. Models are often constructed with insufficient data or physical measurements, leading to large uncertainty in the input parameters. The economics of parameter estimation and model refinement are rarely considered, and most engineering analyses are conducted under deterministic settings. Current modeling and simulation methods work well for existing products and are mostly used to understand/explain experimental observations. However, they are not ideally suited for developing new products that are not derivatives of current ones.

One of the mistakes that the scientific computing community made early on was in letting the capabilities of our simulations be over-sold without stressing the importance of concurrent, supporting efforts in theory and experiment. There are a significant number of consultants who make outrageous claims about replacing testing with modeling and simulation. It is far to easy for our decision makers to be impressed by the really awesome movies we can make from our simulations, and the claims from the consultants begin to get traction. It is our job to make sure the decision makers understand that the simulation is only as real as our empirical validation of it.

2 Notes and Exerpts from [2]

Below are some exerpts from the executive summary along with a little commentary.

Major Findings:

1. SBES is a discipline indispensable to the nations continued leadership in science and engineering. […] There is ample evidence that developments in these new disciplines could significantly impact virtually every aspect of human experience.
2. Formidable challenges stand in the way of progress in SBES research. These challenges involve resolving open problems associated with multiscale and multi-physics modeling, real-time integration of simulation methods with measurement systems, model validation and verification, handling large data, and visualization. Significantly, one of those challenges is education of the next generation of engineers and scientists in the theory and practices of SBES.
3. There is strong evidence that our nations leadership in computational engineering and science, particularly in areas key to Simulation-Based Engineering Science, is rapidly eroding. Because competing nations worldwide have increased their investments in research, the U.S. has seen a steady reduction in its proportion of scientific advances relative to that of Europe and Asia. Any reversal of those trends will require changes in our educational system as well as changes in how basic research is funded in the U.S.

The ’Principle Recommendations’ in the report amount to ’Give the NSF more money’, which is not surprising if you consider the source. It is interesting that their finding about education is largely the same as the other report.

References

[1] A Report of the National Science Foundation Blue Ribbon Panel on Simulation-Based Engineering Science: Revolutionizing Engineering Science through Simulation, National Science Foundation, May 2006, http://www.nsf.gov/pubs/reports/sbes_final_report.pdf

[2] WTEC Panel Report on International Assessment of Research and Development in Simulation-Based Engineering and Science, 2009, http://www.wtec.org/sbes/SBES-GlobalFinalReport.pdf