What will our "Gold Standard Be" in terms of unit tests?

[WilliamIX]

Currently the check to see if things are working is a TODO. My initial impulse is to just use a handcrafted bioscrape script, however, as we begin testing different mechanisms this will not work without hand reprogramming the gold standards, which will then mess up older tests. Any thoughts?

[murrayrm]

A couple of possibilities:

• We could have some unit tests where the "right answer" can be computed analytically and then confirm that the simulation gives the answer to within some tolerance.

• We could develop regression tests where we store previous answers and re-run them to make sure we get the same answers. This won't work for new mechanisms (or more detailed mechanisms), but can at least be used to make sure that changes don't inadvertently give different answers (due to a bug).

• The bulk of our unit tests can be things that don't check to make sure that the simulated results are correct, but instead check to make sure that the model structure is correct (correct species exist, no spurious species/reactions exist, etc).

We should also try to make sure that anyone who adds new functionality adds unit tests that check that functionality. I obviously didn't do that much of that in the initial pass, because I want to have a minimum implementation that at least "ran", but as we add things then we "require" (in the usual way of open source => nagging) unit tests.

Finally, when we get Travis CI running (see issue #8), we can also do automated coverage testing. We should be shooting for 90% or greater coverage.

[WilliamIX]

That all makes sense, although I don't know to know, apriori for different mechanisms, what the reactions and species should look like, in general. That said, I'll start implementing the following:

A set of exact unit tests for "Basic" or "Simple" mechanisms where we can compare to hard coded bioscrape simulations. I think doing this just with constitutive expression will be enough.

A set of qualitative unit tests, for instance a qualitative repression test will compare an arbitrary set of mechanisms with and without the repressor and make sure that the repressed version has less output than the non-repressed version. Besides repression, are there other test of this nature that you think would be a good idea?