How to buy digital engineering and simulation
A model is only evidence if somebody can say how good it is. Buy the credibility assessment with the software, or you have bought pictures.
This category covers model-based systems engineering tools, digital twins, computational fluid dynamics and structural simulation, and the services built around them. Certified airborne software is a separate guide, and so is maintenance software. The buyer here is usually replacing physical testing with analysis, or promising a customer they can, and the whole question is how much weight the analysis can carry.
Credibility is a deliverable
NASA publishes a standard for models and simulations through its technical standards system, which is a public reference for what a mature organisation asks of an analysis before it is used to support a decision.1 Use it as a checklist when evaluating a vendor or a services firm. Ask how verification and validation are performed and documented. Ask what the model's domain of validity is and how anyone is prevented from using it outside that domain. Ask how uncertainty is quantified and reported to the decision maker, and what the reporting looks like when the answer is close to a limit. Ask who is accountable for the result: the analyst, the tool vendor, or nobody. A services firm that answers those questions with process documents is a different proposition from one that answers with case studies.
The data and the rights around it
Simulation work is fed with your definition data and produces more of it. On government-funded work, rights in technical data are defined by clause: unlimited rights, government purpose rights for a five-year or negotiated period on mixed funding, and limited rights that keep data inside the government and prohibit its use for manufacture without written permission.2 Ask which assertions attach to models and results produced under the contract. Commercially, ask who owns the model, the mesh, the scripts and the parameter sets, and what you can take with you. Ask whether the vendor claims any right to use your models to improve its own products.
Handling controlled information
Defense work carries obligations that reach any firm touching the data: contractors must safeguard covered defense information and report cyber incidents against the published federal security requirements,4 and the certification programme sets assessed levels for protecting federal contract information and controlled unclassified information, running from self-assessment to third-party certification and government-led assessment.3 Ask where the compute runs, in which country, and under which accreditation. Ask what happens to your data on a shared cluster and who administers it. Ask what the vendor's assessment status is today, not what it plans.
The commercial shape
Ask how licensing works: named user, token, core-hour, or cloud burst, and what a peak month costs. Ask what the support model is and where the engineers who answer are. Ask what the training requirement is for your team and how long to competence. Ask what happens to your ongoing work if a licence lapses mid-programme.
Questions that sort vendors
Ask for the verification and validation documentation from a comparable project. Ask for the data rights and ownership terms in writing. Ask where compute and storage physically sit. Ask what a realistic first-year cost is at your actual usage.
Sources, in the order cited
- 1NASA Office of the Chief Engineer · NASA-STD-7009, Standard for Models and Simulations
- 2Acquisition.gov · DFARS 252.227-7013, Rights in Technical Data, Noncommercial Items
- 3Electronic Code of Federal Regulations · 32 CFR Part 170, the Cybersecurity Maturity Model Certification Program
- 4Acquisition.gov · DFARS 252.204-7012, Safeguarding Covered Defense Information and Cyber Incident Reporting