This workshop provides you with the basic concepts and terminology you use in Workshop 2. This workshop explains how simulations can be set to use for research purposes. The workshop focuses on simulation accuracy. The first section of this workshop explains the importance of selecting the correct level of detail for the research project. The second section of the of this workshop explains verification, validation and calibration are used to improve the accuracy of simulations. Additionally, there is an explanation on performance gaps and how real-world measurements can be used to calibrate a simulation.  

WV4 Line 2 – Workshop 2: Simulations 1/2

Level of detail link copied

Building components

Level of detail (LOD) refers to defining and representing different levels of complexity and detail within a 3D model. The level of detail can range from very conceptual shapes to highly detailed representations. There are different levels of detail, which help to reduce the amount of detail and manage the computational requirements. The LOD is a crucial factor in various applications such as for simulations and Computer-Aided-Design (CAD). Finding a balance between accuracy, performance and usability is essential for 3D modelling.  

Mainly there are six levels of detail. The ‘LOD 000’ is a quick 2D polygon representation. The ‘LOD 100’ is a conceptual representation to translate the overall design intent, with a basic shape and dimensions without any detailed information. The ‘LOD 200’ is a schematic design of the approximate geometry, with the approximate dimensions, location and orientation. The ‘LOD 300’ is a detailed design and of the approximate geometry, with the approximate dimensions, location and orientation. The ‘LOD 350’ is the construction documentation of the design and include parts such as connections. The ‘LOD 400’ is the fabrication documentation, which includes detailed information on detailing, fabrication assembly and installations. The ‘LOD 500’ is a real-world verified representation of a building and is used as a guide for maintenance as it represents the current as built state.  

Level of detail in Simulations

When running a simulation model, it’s important to critically assess which elements to include, focusing only on those that directly influence the outcome. For example, adding an internal door in a model designed for internal daylight simulations would be unnecessary and irrelevant as it won’t affect the simulation results. Aside from the elements and level of detail of the form generation, the level of detail in the simulation itself also affects computational time. Before running the complete set of simulation, you should determine whether the improvement of results by adding simulation detail is worth the additional computational time. Examples of simulation setting that influence the computational time are time steps, grid size, radiance parameters, etc.   

Impact of level of detail

Within architectural modelling the trade-off between computational time and simulation accuracy should be managed carefully, as it directly impacts the quality of research. Optimization methods and validation techniques are often used to efficiently find the balance between the level of detail and the computational efficiency. 

High level of detail: 

• Requires more computational resources, which results in longer processing time and a higher memory usage.  
• Provides more accurate simulations, which leads to more reliable research findings.  

Low level of detail: 

• Are quicker to process, as there are fewer details  
• Might not be able to capture all the nuances, which could lead to less accurate simulations. Inaccurate simulations could result in flawed research findings, which reduces the credibility and impact of a research project.  

Exercise: level of detail for simulation accuracy

To get a better understanding on how to select the right level of detail for simulations and the impact the level of detail has on simulation accuracy, you should follow this exercise. In the exercise you will work with a shoebox room with 1 façade and run interior daylight simulations.  

Debate A: Selecting level of detail for research

If you are following this tutorial because you are attending the course BK3WV4, after you have learned the theory and practiced the exercises, you will attend a workshop in class at TU Delft. The workshop lasts 3 hours. During the workshop, you will work with the tutors. Two hours are contact hours with your tutor and one hour is for activities without tutors. During the first one hour, you will discuss the tutorial with the tutor and your classmates. During the second hour, you will write a first draft of your workshop deliverable. During the third and fourth hour, you will discuss how the content of the workshop relates to your research question. 

• What performance indicators are you using for the research? (In other words, what type of simulations would you use for the research.)  
• Which design details are mandatory to run the simulation properly for your research and which aspects are mainly resulting in high computational demand but not influencing the results too much?  
• In which situations is a high level of detail outweighing the high computational time which it results in?  

References

• An improved LOD specification for 3D building models. Filip Biljecki, Hugo Ledoux, and Jantien Stoter. Computers, Environment and Urban Systems, 59: 25–37, 2016. 

WV4 Line 2 – Workshop 2: Simulations 2/2

Verification, validation and calibration link copied

If you want to use simulation models for research, you have to ensure the quality of the generated data. As most of us know: trash in is trash out. The assumptions and input values of the model determine the validity of the results. Therefore, for full transparency of your research, all information about the model’s assumptions and input values should be documented with your research. Proper simulation documentation leaves space for other researchers to recreate the simulation and determine, with some level of confidence, whether or not a model’s result can be used in decision making.  

The purpose of model verification and validation is to make the simulation model meaningful in a real-world context. For this reason, the modelling and simulation procedure includes model verification and validation. Both verification and validation require good documentation, and are crucial parts of assessment, credibility and accreditation. This section describes the basic terminology, for further information about the basic principles of simulation verification, validation and calibration the following reading is recommended:  

• Verification and validation of simulation models – Kleijnen  
• Verification and validation of simulation models – Sargent 
• A Framework for Simulation Validation & Verification Method Selection – Verbraeck & Meijer 

Verification:

Ensures the simulation model and the implementation is correct. In other words, the simulation computer program performs as in- tended. The computerized model verification consists of two parts: specification verification, and implementation verification. The specification verification ensures that the software is satisfactory. The implementation verification ensures that the simulation model has been implemented according to the simulation model specification.  

The simulation model can be verified by manually calculations and comparing the results with the simulation model output. Generally, a simplified version of the simulation program with a known analytical solution is used as test case for verification.  

Validation:

Validation ensures that the conceptual simulation model is an accurate representation of the model. The simulation will never result in a perfect model, as any model is a simplified version of reality. However, the model should be ‘good enough’ for the goal of the simulation. Conceptual model validation determines whether the theories and assumptions underlying the conceptual model are correct, and whether the model representation of the problem entity is “reasonable” for the intended purpose of the model. Operational validation determines that the conceptual model behaves accurately based on its intended purpose. Data Validation ensures the accuracy of the raw data necessary for the model building, model evaluation and testing, and conducting the model experiments to solve the problem are adequate and correct.  

Calibration

Model calibration is used to adjust the simulation model’s parameters in such a way that the simulated output deviates minimally from the real output. The purpose of calibration is to ensure that the simulation outcome is as close to the measured values as possible.  

Performance gap between simulated vs measured

There is often a significant difference between the predicted performance of the design and the measured performance of the realised design. Discrepancy between the performance prediction and measurement is inevitable due to numerical errors in the simulation, and experimental variation in any observation. Analytical approaches require strong constraints and therefore often don’t reflect the real world. During the construction process there could have been errors made or adjustments been made compared to the simulated model. In the simulation process details are often left unspecified. The measurement devices should be calibrated as well, otherwise this can cause inaccuracies in the measurements. Real-world measurements require a certain number of data points, as a lack of data points can result in inaccurate result comparison. However, the goal is to get a reasonable agreement between the prediction and measurement.  

Example papers

The following papers clearly explain how the discussed terminology can be used during research. We highly recommend that you look at these papers.  

“A methodology to improve the performance of PV integrated shading devices using multi-objective optimization” by Taveres-Cachat et. al., 2019 

• ScienceDirect Article by Taveres-Cachat et al. 2019 
• Very good documentation of simulation settings 
• Describes sensitivity analysis of the number of ambient bounces and quality setting for the Radiance daylighting analysis 
• Compares optimization results pareto front  

An EnergyPlus whole building energy model calibration method for office buildings using occupant behaviour data mining and empirical data – Khee Poh et. al., 2014 

• Researchgate Article by Khee Poh et al. 2014  
• Explains the root cause of performance gap for energy simulations 
• Executed real-world measurements over a longer period 
• Calibrates energy model based on measurements 
• Compares base model, proposed design case model, calibrated model and measured data 
  • A clear shift in the distribution of how energy is consumed  

Debate B: Simulation differences with real world measurements

If you are following this tutorial because you are attending the course BK3WV4, after you have learned the theory and practiced the exercises, you will attend a workshop in class at TU Delft. The workshop lasts 3 hours. During the workshop, you will work with the tutors. Two hours are contact hours with your tutor and one hour is for activities without tutors. During the first one hour, you will discuss the tutorial with the tutor and your classmates. During the second hour, you will write a first draft of your workshop deliverable. During the third and fourth hour, you will discuss how the content of the workshop relates to your research question. 

• For the simulation type that you’re selecting for the topic, would it be possible to conduct measurements in a real-world scenario? How would you set this up?  
• For the simulation type that you’re selecting for the topic, what would be a possible discussion point on why there could be a difference between the predicted values and the measured values?  
  • What are the possible reasons for a performance gap between the simulation measurement?