What would an integrated development environment for numerical algorithms look like? To me it looks a lot like Fedora. That’s right, Fedora Linux is your integrated development environment. Oh, separate programs in an operating system isn't integrated enough for you? They ought to be, Python or Bash are right there waiting to glue it all together.
An implementer or developer of numerical methods ought to have a good computer algebra system(CAS) available. This is where the high-level algorithm design needs to take place. All the rest of the low level coding needs to be automated as much as possible. You as the human in the loop need to spend time thinking about the bigger picture and get Maxima to code Fortran for you.
Numerical software needs testing. Lots of rigorous testing. You can use your (CAS) to generate unit tests by evaluating the expressions you converted to Fortran for specific cases, and then converting these answers to Fortran too. Then you can easily wrap those routines in a unit testing framework using F2Py.
This approach lets you automate the generation of the low-level functions that will be in your inner loops (which need to be in Fortran for performance sake). Then use Python's powerful standard library and additional packages for things like parsing input decks, file reading and creation, setting up parallel jobs and generating graphics. All things which can be a real pain in Fortran. The real benefit is that the human effort is kept focused where it is most profitably employed: at the higher levels of abstraction of the mathematical concepts rather than the low level coding.
There it is, a complete "environment" in three building blocks: Maxima, Fortran and Python. Almost completely automated from governing equations in the CAS to low-level number crunching in Fortran. Some human intervention required to bring it all together into a working program, but that’ll be in Python, which is easy.
This is the basic method used in the second half of the magnetron problem. I'll post some more simple examples of the whole work-flow in use after the holidays.
A Collaborative Code Development Environment for Computational Electromagnetics
ReplyDeleteAbstract:
A Problem Solving Environment (PSE) is a complete, integrated computing environment for composing, compiling and running applications in a specific problem area or domain. We describe a visual code development tool within a PSE, which enables computational scientists to construct applications by connecting components. The granularity of each component can vary from being a complete code, to a mathematical routine such as a matrix or PDE solver. We first outline the requirements of such an environment, illustrating these with our implementation. The implementation of a computational electromagnetic solver is then described using this code development tool, based on a 2D boundary element code. We emphasise lessons learned, and the importance of using such an environment to support new application development.
The software guys don't think Unix is the answer, they might even be right. Check out the 'code bubbles' video, pretty neat.
ReplyDeleteNotes from the Intentional presentation linked from ThirdBit:
- processable representation of domain knowledge
- describe how the program is generated -- not what it is
- domain representation does not need to be computable (Turing complete)
- any graphical, textual or symbolic notation
Notes on Wilson's 'Extensible Programing' article:
- a very Unixy criticism of gcc for being too monolithic (unintentionally ironic I think)
- argues for 'pluggable frameworks' for programing tools rather than text proessing with pipes and regular expressions on the command line, even manages to get a good dig in on Perl die-hards
- separate models from views
Oh, separate programs in an operating system isn't integrated enough for you?
ReplyDeleteApparently it was good enough for Roache back in 1988:
To make the best use of currently-available codes, the user should have a whole toolkit at his disposal. This toolkit would include a "mid-level", "number-crunching" language such as FORTRAN, symbol-manipulation codes, such as MACSYMA, graphics codes such as DISSPLA, and an operating system which allows easy transfer of data from one code to another. Symbol manipulators provide a high-level interface; there are several available... Graphics provide the user with high-level output. Operating systems are the "glue" which connect all these diverse tools.
I think the modern concession is that a scripting language like Python is the glue rather than the "operating system".