Hi Andrew,
That's an interesting project you are investigating! Below are a couple thoughts on how you could go about testing building of your model's building blocks (I will be referring to "units" as ”building blocks”, just to avoid confusion with SimBiology’s unit system or MATLAB’s unit testing framework).
Here are two ideas for how you could test the building blocks:
- Simulation based: For this you would have to do a manual step and split your template model into the building blocks. Just to name an example, if the liver is a building block then delete all reactions, parameters, species that are not part of, feed into, or are an output of the liver block. Note that this requires to remember to make any changes to your template model to the template building blocks as well (you can use your testing framework to ensure that's the case, more on this later). To test the derived model against the template, you could just simulate it and record the input/outputs of building blocks that are expected to be unchanged from the template (e.g. inflow and outflow of liver). Then use the inflow as initial conditions for the template building-block and verify that the outputs match. To ensure that any updates to the template model are also made to the template building-block, you could use the same testing framework: simulate it the template model, then use the inflows/outflows to verify that your template building blocks behave as expected.
- Model based: The simulation results of your template model are determined by the reaction rates, rules, and initial values of the components in the model object (if doses, events, or observables need to be tested, they can be included as well). Just as in the simulation based approach, you’d also need to define the individual building blocks of your template model (e.g. by taging them using the Tag property, e.g. “liver”, “heart”, “lung”, …) or by simply knowing which species, parameters, reactions make up a building block. Then check if key property values like units, reaction rates, kinetic laws, rule strings, etc. match between the respective components of the template model and the derived model. This verification would not require any simulation. I guess this approach is similar to what you are already doing with your model comparisson, but it may be a little more automated.
In either case you could use the Matlab Unit Testing framework for your verification. There are many ways to do this, ranging from writing your own helper methods for the verification to implementing your own constraints. In any case, a general idea would be to write functions for each building block of the template model that take the derived model; this function would do then the comparison between the model parts, either by simulation or by comparting property values. All this can even be automatically run on every submission to Git.
Would either of this approach work for your project?
-Florian
