Avoiding Modeling Errors: Validation and Verification

This article explores practical steps to avoid modeling errors, focusing on validation and verification processes during the modeling process.

Structural engineering models demand accuracy. The consequences of modeling errors, whether due to software bugs, erroneous inputs, or unverified assumptions, can be severe. A structure’s safety stands compromised should the model upon which its implementation rest is faulty. Given the serious risks, it’s essential to ensure models are robust and reliable. The responsibility falls on structural engineer to prioritize accuracy at every stage of modeling, from initial setup through to final verification.

This article explores practical steps to avoid such pitfalls, focusing on validation and verification processes during the modeling process. It discusses vital methods to validate inputs, verify outputs, and prevent the potential dangers associated with erroneous modeling assumptions.

Also to help engineers achieve safe and accurate designs, this article outlines a systematic approach to avoiding modeling errors. Key concepts such as software selection, rigorous checks, and techniques for validation and verification are covered. The aim is to enable structural engineers to make informed decisions and ultimately design structures with confidence.

Importance of Model Validation and Verification

Imagine the horror of discovering that a structure you designed begins to fail, threatening public safety and potentially leading to lawsuits, all due to a design error caused by an invalidated modeling assumption, an undetected input error, or even a software bug. Numerous engineering failures have been attribute to over-reliance in the use of software, many of which can be attributed to invalidated design and inaccurate assumptions

Validation and verification are critical to the integrity of structural engineering models. Model validation ensures that the correct questions are asked, and that the model is structured to address the intended engineering problem.  Validation involves aligning the modeling approach with real engineering principles and confirming that the chosen software is capable of producing accurate results for the task at hand.

Verification, on the other hand, is the process of checking the model’s outputs for correctness at different stages. This typically includes comparing software-generated results with crude hand calculations, historical data, and alternative software outputs.

To avoid errors, we must prioritize validation early and often throughout a project lifecycle. The following practical questions are necessary for effective model validation:

• What is the purpose of this model, and am I framing the right questions?
• Is the selected software capable of addressing these questions accurately?
• Do I have sufficient expertise with this software, or should I invest time in learning it?

Steps for Validation and Verification

Software Package Selection

Selecting the correct software is the first step in minimizing modeling errors. As a thumb rule, you should consider a software that allows transparency in calculations, from load build up to member forces and then member design, offering easy access to assumptions and intermediate steps. This transparency aids in identifying discrepancies between hand calculations and software results. Additionally, engineers should avoid software “wizards” that automatically generate models without user input. Instead, starting from scratch and progressively building model complexity allows greater control over each aspect of the model, leading to a more reliable design.

See: Selecting the Right Structural Engineering Software Package

Mesh Sensitivity Analysis

Mesh sensitivity analysis ensures that the finite element model is neither overly simplified nor needlessly complex. This involves starting with a basic mesh where hand calculations are feasible and progressively refining the mesh as necessary. Testing different mesh sizes allows you to identify the ideal balance where the model accurately represents the physical structure without overloading computational resources. There are other errors that may result from meshing (See: Avoiding Finite Element Analysis Errors)

Material and Section Property Verification

Accurate input of material and section properties is also vital for realistic modeling. Engineers should verify that values for strength, elasticity, and other material-specific properties are correct. Small discrepancies here can lead to major errors in the final outputs, so cross-referencing these properties with empirical data or industry standards is recommended.

Compliance with Design Codes

Codes serve as benchmarks, and software calculations should align with them. If discrepancies are found, engineers should investigate and resolve them before proceeding further with the design. Manual calculations can be performed to ensure compliance with design codes

Reviewing and Analyzing Model Outputs

Even with proper validation steps, errors may still arise in the output. Engineers must approach model results critically, applying engineering judgment and assessing whether the results align with expectations based on prior knowledge and calculations. This stimulates critical thinking in the design process and enables the engineer to identify and correct subtle errors that might otherwise go unnoticed. Some methods for output verification include:

Comparing Software Outputs

Running a model in multiple software tools can reveal inconsistencies. If different software packages produce similar results, confidence in the accuracy of the model is strengthened. If there are significant differences, you should analyze the assumptions and calculations within each software tool to identify potential issues.

Engineering Judgment

Engineering judgment is essential when interpreting complex model outputs. Drawing on your previous experience, you might be able to determine whether results seem realistic or if further investigation is necessary.

Visualization Tools

Using graphical outputs, such as charts, heat maps, and stress-strain distributions and even contours, can help you identify trends and anomalies. Visualization simplifies the analysis of complex arrangements, aiding in the interpretation of peak stress values and other critical areas.

Case Studies in Model Validation Errors

The Collaborative Reporting for Safer Structures (CROSS) highlights real-world examples where modeling errors had serious consequences. Reports detail cases such as the incorrect application of wind loads, underestimated foundation designs, and misinterpreted material properties. CROSS’s collection of case studies provides engineers with valuable insights into potential pitfalls and emphasizes the importance of stringent validation processes. A study of these examples is necessary as it allows engineers to better understand the risks involved and reinforce their approach to model validation and verification.

Prevention and Correction Strategies

To reduce the likelihood of errors, engineers can follow these prevention strategies:

1. Use of Change Logs: Maintaining a change log allows you to track adjustments to the model, providing valuable context for reviewing assumptions and identifying the rationale behind each change.
2. Peer Collaboration and Reviews: Discussions with colleagues can provide fresh perspectives and uncover overlooked details. Internal reviews and independent checks ensure comprehensive examination of the model from multiple viewpoints, further enhancing its reliability.
3. Open Communication with Software Developers: Software issues can sometimes be addressed directly with developers. Reporting bugs and seeking clarification on unclear outputs can provide insights into software limitations and encourage the development of software patches for known issues.
4. Independent Checks Using Alternate Software: Running models on different software platforms is an effective way to validate results. If two independent tools provide similar results, it increases confidence in the model’s accuracy. If not, it prompts further investigation into the root cause of discrepancies.

The Role of Engineering Judgment in Model Validation

Above all, engineering judgment stand at the top of the list in avoiding modeling errors. Engineering judgment, often misunderstood as intuition, is a systematic method for evaluating model outputs. Experienced engineers have what is called the ‘engineering feel.’ They can analyze the assumptions, boundary conditions, and results critically, rather than relying solely on software-generated outputs. This approach empowers engineers to challenge questionable results and understand the nuances of model behavior.

For example, when an analysis shows unexpectedly high localized stress at specific points, you should investigate whether this is an artifact of modeling (e.g., point loads or idealized supports) or a real issue. Adjustments to the model, such as distributing loads over larger areas, can help engineers determine whether these stress peaks are artifacts or require further structural reinforcement.

Conclusion

Avoiding errors in structural models requires a proactive approach to validation and verification. Engineers must diligently examine each aspect of the modeling process, from software selection to output analysis.

Adhering to these practices enhances structural model reliability, minimizes the risk of failures, and ultimately ensures public safety. In an era where software plays a dominant role in our design, the engineer’s vigilance in validation and verification is the foundation of a safe, resilient structure.

See: Avoiding Disaster – How to Salvage a Collapsing Building

Sources & Citations

• Lockley, Z., & Williams, K. (2024). “Avoiding Modelling Errors: Validation and Verification.” The Structural Engineer, October 2024, pp. 21-23. Available at: thestructuralengineer.org
• Institution of Structural Engineers (2016). Guidelines for the Use of Computers for Engineering Calculations. London: IStructE Ltd.
• CROSS. “Collaborative Reporting for Safer Structures.” Available at: cross-safety.org