molly_cow_model

title: README authors: Pierre Beukes; Simon Woodward; Gil Levy; Jagir Hussan date: 1 March 2020 updated: 24 October 2021 by Hemda Levy

Sharing Molly cow model

Objectives

Migrate from Molly-AcslX (with limited future) to a translated Molly C++ that ensures more collaboration, easier use, programmer resources, and hopefully the future of the model. Create an open repository for the code and associated documentation that promotes wider use and improvement of the model and publication of peer reviewed papers. Capture all relevant improvements to the model for use in modelling studies that address dairy cow system questions, and provide solutions that benefit dairy farmers. Share all improvements to Molly to a wider group of stakeholders, also for informal peer review. Collaborate with key stakeholders and control the integrity of a master Molly version, to form the basis for decisions on incorporating improvements.

Repo description and procedures

Storage and distribution

This is a private GitHub account containing the Master repo with the code for running several alternative implementations of Molly in RStudio.

The link and access to this repo will be distributed free of charge by DairyNZ (Pierre Beukes pierre.beukes@dairynz.co.nz or delegate) and Virginia Tech (Mark Hanigan mhanigan@vt.edu or delegate). These organizations will keep record of who has access to the repo. Downloading and use is subject to the license agreement (see LICENSE.txt in this repo).

Download procedure

Change procedure

  1. create a fork (click on the fork button top right) makes a copy for you to work on in your own private account

  2. clone your private copy to your computer (click green clone button and copy SSH text). Create new project in RStudio “from version control”, enter the SSH line and set the directory you want to put it in.

  3. Now you can edit the code

  4. Commit and push your changes to Github

  5. Create a pull request for changes you want merged in the master

    Note: for a step-by-step guide on how to use Git and Github see Tom Levy’s document

    molly_cow_model\docs\Git Tutorial.md

Contents of the repository

The repo contains several alternative implementations of the model:

*****adapted to NZ conditions

Folder structure under molly_cow_model

The folder structure corresponds to the mentioned above implementation:

folder_structure

with the additional folders:

R_CODE - contains the R scripts along with some accessory data files

MFILES - an M file (inherited from AcslX) contains initialization values of various parameters (“constants” in AcslX). Each implementation needs to run with its appropriate M file.

docs - contains various related documents

TRIALS folder

Each implementation folder contains its own TRIALS folder. Under this folder there should be a separate folder for each trial data. The repo comes with a single folder named Aston.

The trial folder should contain 4 files, whose names start with the trial name followed by an underscore.

For instance, in the Aston case those files are:

Aston_cows.csv - descriptions of the cows that take part in the trial. Parameters: Name, Age, Breed, SameAs, CalvingDate, ConceptionDate, iBW, iBCSUS, kgMS270, RelativeImportance (ignore this one). The SameAs parameter indicates that the cow is the same as another cow (with the specified index).

Aston_treatment.csv - this file specifies the types of feed the cow receives, the quantities and at which point in time (by days-in-milk, which can be negative to indicate the days before calving). Also the milking frequency. The parameters:

Cow, DIM, MF, Feed1Name, Feed1Quant, Feed2Name, Feed2Quant, Feed3Name, Feed3Quant.

You can have up to 3 different feeds which refer to entries in Feeds.csv under R_CODE.

For quantity you can specify ad lib (instead of specific number), which indicates that cow determines how much she wants to eat.

Aston_time_series.csv - contains the observed results of the trial. Cow names and dates are mandatory and then whatever measurement was taken during the trial.

Aston_header.R - This is an R script file which contains a few parameters which you should pay attention to:

trial $ parameters - should repeat the headers in the observed, those that you want to view in the plots.

trial $ dateRange - defines start date and end date of the trial or the observations and the should follow the trial $ daysBetweenObs, so for instance if the it’s 7 days then dateRange should start and end in multiplications of 7 exactly.

Have your own trial data?

If you want to add your own trial data, you need to create a folder with the trial name under the TRIALS folder corresponding to your chosen implementation.

The trial folder needs to contain 4 files to populate, as mentioned above, all filenames starting with the name of the trial (case sensitive) followed by an underscore “_”:

\_cows.csv; \_time_series.csv; _treatment.csv; _header.R; ## How to run Molly in RStudio 1. Open RStudio and go File | Open Project... and select **molly_cow_model.Rproj** under ...\molly_cow_model 2. Install sundialr. First install [RTools](https://cran.r-project.org/bin/windows/Rtools/) (the same version as R), then run the following R commands: ```R install.packages("devtools") devtools::install_github("sn248/sundialr@1ae40874cda818217446c163248391078553753c") ``` For more details, see item 7 in the section **Auckland Bioengineering Institute (ABI) folder**. 3. Choose the implementation you want to use as your working one (ABI / DNZ_ACSLX / DNZ_CPP / NoA) and in the Files pane go to the selected folder and **Set as Working Directory** 4. In the main pane open file **init.R** under the folder R_CODE 5. Click the ***"Source"*** button and wait for RStudio to compile - it takes a while first time in a session. Thereafter it will be quicker. You need to click the ***"Source"*** button any time you change your data or code, for the changes to take effect. 6. To run a trial use: `runTrial("Aston")` 7. To plot results: `plotTrial("Aston")` 8. Alternatively, to run & plot in one go: `rnp("Aston")` 9. To plot more parameters: add them (case insensitive) to the variable **`MoreVariablesToMonitor`** in **main.R**, re-run, re-plot. 10. ###### Comparing different Molly implementations/versions/M files Each implementation has its own **MollyVersions.csv** file which determines the Molly(s) will run. The headers are: "Branch", "VersionNumber", "M file". "Branch" specifies the implementaion (ABI / DNZ_ACSLX / DNZ_CPP / NoA). "VersionNumber" specifies the applicable version number, according to the following table: | Implementation | Version Numbers (no dot, please) | | -------------- | -------------------------------- | | ABI | "911" | | DNZ_CPP | "895" , "915" | | NoA | "5a", "6x" | | DNZ_ACSLX | "895", "915", "6a" | "M file" specifies which M file will be used for initializing parameters in Molly. As a default the R script looks for an M file, which bears the version number, under the **MFILES** folder and loads it (if exists). If you want to add yet another M file, you need to specify it. You can have up to 4 rows in the **MollyVersions.csv**, which implies there will be up to 4 Molly-s running side-by-side for a given trial and their graphs will be overlayed for comparison in the plot pane. To have goodness-of-fit (RMSE) side-by side report: `view(runStats())` > Note: Cow initialization file in trial folder, e.g. **Aston_cows.csv**, might not fit all implementations/versions you use in **MollyVersions.csv**. You can add an alternative file specfically for the version number, e.g. **Aston_915_cows.csv**, which will override the iBW, iBCSUS, kgMS270 parameters of the original file. > Note for programmers: > > In order to allow 2 - 4 different Molly-s run side-by-side, there is one unified interface between R and the models, namely **molly_glue.cpp** (located under **molly_cow_model**). > > Also, all the implementations inherit from **molly_superclass.h** (located under **molly_cow_model**). 11. To change a Molly parameter add ``` aMollyParameter = ``` in the M file you are using OR see set_molly_variables in other documentation files 12. For C++ API, and more information see file **molly_cow_model\docs\How Molly C++ works.docx** 13. For documentation about the R suite see file **molly_cow_model\docs\How Molly R suite for Molly C++ works.docx** ## Auckland Bioengineering Institute (ABI) folder * Molly version included: 9.11 * Features: Initialize, advance model's time, set and get parameters, plot graphs. (R and C++ versions) Run trial simulations using R, compare Molly's predictions against observed data, compared output of three ODE solvers (Runge Kutta 4, BDF and Adams) * Trials available: Aston 1. Jagir Hussan from ABI provided DairyNZ with a new translation to C++ based predominantly on AcslX's automatic translation of Molly .csl to C++. 2. AcslX's translated code was refactored to group parameters, context variables etc. and calls to Acsl solvers and event handlers were replaced. 3. Output of the code closely matched those of AcslX, quantitative differences are attributed to the change from 32-bit (AcslX binary) to 64-bit floating point (translated binary) respresentation, and the sensitivity of exponential function to increased precision. Note few of Molly's equations use the exponential function to model switching behaviour. 4. Documentation is not carried over from the .csl code into this translation as in the translations in DNZ_CPP and NoA. 5. This new translation will work seamlessly as the old one with the R suite provided here. 6. Key coding segments are distributed in a number of header files and compiled to build the model through the use of "#include" compiler directives. The goal is to reuse common code and improve code maintenance. 7. R bindings use R's sundialr package (tested against version 5.2.0 of SUNDIALS, released in March 2020). 7.1 Issues with linking with sundials libraries have been reported. The location of the libsundials_all library is passed using PKG_LIBS environment varible, however, in some R installations this doesnt seem to be passed to the linker and the build fails. 7.2 Developers could store the libsundials_all library at the path that is visible to the linker for it to link, or forgo using the Cvode solvers. Commenting out the line ***#define enableSundials*** in **molly_glue.cpp** will ensure cvode is decoupled. 7.3 As of 2023, sundialr was removed from CRAN as "issues were not corrected in time" (https://CRAN.R-project.org/package=sundialr, https://github.com/sn248/sundialr/issues/7). So sundialr needs to be installed from source. We are using an old commit because the [next commit](https://github.com/sn248/sundialr/commit/d9cfb6247a202377c1b137e1052b88787df1c15a) updates from SUNDIALS 5.8.0 to 6.1.1, but our code doesn't work with SUNDIALS 6.1.1. 8. Follow steps 1 - 7 of the **How to run Molly in RStudio**, to simulate trials in R, making **molly_cow_model/ABI** your working directory 9. Move to the **RTest** folder and make it your working directory to run **test_molly.r** and **test_molly_cvode.r**. You'll get .png file(s) which will display the graphs when double-clicked. ## Oversight * The NC-2040 group will deal with pull requests for mergers with Molly_NoA Master, DairyNZ with pull requests for Molly_DNZ. * When there is a pull request that affects the common part of Molly C++, which is the bulk of the code, then representatives from NC-2014 and DairyNZ can review and make the call for merging with Master, or not. ## Differences between versions Some of these differences are documented in the journal papers that culminated from the work (see docs folder for a list of refs). Basic differences between Molly_DNZ and Molly_NoA is that Molly_DNZ has been adapted and parameterized for a pasture eating New Zealand cow with a seasonal calving pattern. The intakes are predicted compared to Molly_NoA where the largely TMR diet is eaten following user-defined daily intakes. Other differences are the lactation modules where Molly_DNZ can simulate once-a-day milking following mechanistic changes in mammary cell dynamics. The Molly_DNZ model is largely run as an animal model (dll wrapped as a com object) in a whole farm model representing a New Zealand dairy system. ## Voluntary agreements * Clearly document all changes in the code. * Do not change variable names without a VERY good reason. (see docs folder for variable names|abbreviations and definitions. At the time of writing 14 Dec 2018, these lists have not been checked for accuracy). * Test changes in the code with observed|trial|meta data and provide the data (if possible) and the R script with test stats output. There is an idea to develop a standard R script for loading test data and giving standard test statistics for comparing old versus new versions. * If the changes are deemed substantial enough, document it as a peer reviewed publication.