Benchmark timings for mkin

Each system is characterized by the operating system type, the CPU type, the mkin version, and, as in June 2022 the current R version lead to worse performance, the R version. A compiler was available, so if no analytical solution was available, compiled ODE models are used.

Every fit is only performed once, so the accuracy of the benchmarks is limited.

The following wrapper function for mmkin is used because the way the error model is specified was changed in mkin version 0.9.49.1.

if (packageVersion("mkin") > "0.9.48.1") {
  mmkin_bench <- function(models, datasets, error_model = "const") {
    mmkin(models, datasets, error_model = error_model, cores = 1, quiet = TRUE)
  }
} else {
  mmkin_bench <- function(models, datasets, error_model = NULL) {
    mmkin(models, datasets, reweight.method = error_model, cores = 1, quiet = TRUE)
  }
}

Test cases

Parent only:

FOCUS_C <- FOCUS_2006_C
FOCUS_D <- subset(FOCUS_2006_D, value != 0)
parent_datasets <- list(FOCUS_C, FOCUS_D)


t1 <- system.time(mmkin_bench(c("SFO", "FOMC", "DFOP", "HS"), parent_datasets))[["elapsed"]]
t2 <- system.time(mmkin_bench(c("SFO", "FOMC", "DFOP", "HS"), parent_datasets,
    error_model = "tc"))[["elapsed"]]

One metabolite:

SFO_SFO <- mkinmod(
  parent = mkinsub("SFO", "m1"),
  m1 = mkinsub("SFO"))
FOMC_SFO <- mkinmod(
  parent = mkinsub("FOMC", "m1"),
  m1 = mkinsub("SFO"))
DFOP_SFO <- mkinmod(
  parent = mkinsub("FOMC", "m1"), # erroneously used FOMC twice, not fixed for consistency
  m1 = mkinsub("SFO"))
t3 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D)))[["elapsed"]]
t4 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D),
    error_model = "tc"))[["elapsed"]]
t5 <- system.time(mmkin_bench(list(SFO_SFO, FOMC_SFO, DFOP_SFO), list(FOCUS_D),
    error_model = "obs"))[["elapsed"]]

Two metabolites, synthetic data:

m_synth_SFO_lin <- mkinmod(parent = mkinsub("SFO", "M1"),
                           M1 = mkinsub("SFO", "M2"),
                           M2 = mkinsub("SFO"),
                           use_of_ff = "max", quiet = TRUE)

m_synth_DFOP_par <- mkinmod(parent = mkinsub("DFOP", c("M1", "M2")),
                           M1 = mkinsub("SFO"),
                           M2 = mkinsub("SFO"),
                           use_of_ff = "max", quiet = TRUE)

SFO_lin_a <- synthetic_data_for_UBA_2014[[1]]$data

DFOP_par_c <- synthetic_data_for_UBA_2014[[12]]$data

t6 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a)))[["elapsed"]]
t7 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c)))[["elapsed"]]

t8 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a),
    error_model = "tc"))[["elapsed"]]
t9 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c),
    error_model = "tc"))[["elapsed"]]

t10 <- system.time(mmkin_bench(list(m_synth_SFO_lin), list(SFO_lin_a),
    error_model = "obs"))[["elapsed"]]
t11 <- system.time(mmkin_bench(list(m_synth_DFOP_par), list(DFOP_par_c),
    error_model = "obs"))[["elapsed"]]

Results

Benchmarks for all available error models are shown. They are intended for improving mkin, not for comparing CPUs or operating systems. All trademarks belong to their respective owners.

Parent only

Constant variance (t1) and two-component error model (t2) for four models fitted to two datasets, i.e. eight fits for each test.

OS	CPU	R	mkin	t1	t2
Linux	Ryzen 7 1700	NA	0.9.48.1	3.610	11.019
Linux	Ryzen 7 1700	NA	0.9.49.1	8.184	22.889
Linux	Ryzen 7 1700	NA	0.9.49.2	7.064	12.558
Linux	Ryzen 7 1700	NA	0.9.49.3	7.296	21.239
Linux	Ryzen 7 1700	NA	0.9.49.4	5.936	20.545
Linux	Ryzen 7 1700	NA	0.9.50.2	1.714	3.971
Linux	Ryzen 7 1700	NA	0.9.50.3	1.752	4.156
Linux	Ryzen 7 1700	NA	0.9.50.4	1.786	3.729
Linux	Ryzen 7 1700	NA	1.0.3	1.881	3.504
Linux	Ryzen 7 1700	NA	1.0.4	1.867	3.450
Linux	Ryzen 7 1700	4.1.3	1.1.0	1.791	3.289
Linux	Ryzen 7 1700	4.2.1	1.1.0	1.842	3.453
Linux	i7-4710MQ	4.2.1	1.1.0	1.959	4.116
Linux	i7-4710MQ	4.1.3	1.1.0	1.877	3.906
Linux	i7-4710MQ	4.2.1	1.1.1	1.644	3.172
Linux	Ryzen 7 1700	4.2.1	1.1.1	1.770	3.377
Linux	Ryzen 7 1700	4.2.1	1.1.2	1.957	3.633
Linux	Ryzen 7 1700	4.2.2	1.2.0	2.140	3.774
Linux	Ryzen 7 1700	4.2.2	1.2.2	2.187	3.851
Linux	Ryzen 9 7950X	4.2.2	1.2.0	1.288	1.794
Linux	Ryzen 9 7950X	4.2.2	1.2.2	1.276	1.804
Linux	Ryzen 9 7950X	4.2.2	1.2.3	1.370	1.883
Linux	Ryzen 9 7950X	4.2.3	1.2.3	1.406	1.948
Linux	Ryzen 9 7950X	4.3.0	1.2.4	1.386	1.960
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	4.3.1	1.2.5	2.369	3.632
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	4.3.1	1.2.6	2.856	4.960
Linux	Ryzen 9 7950X	4.3.2	1.2.6	1.408	2.041

One metabolite

Constant variance (t3), two-component error model (t4), and variance by variable (t5) for three models fitted to one dataset, i.e. three fits for each test.

OS	CPU	R	mkin	t3	t4	t5
Linux	Ryzen 7 1700	NA	0.9.48.1	3.764	14.347	9.495
Linux	Ryzen 7 1700	NA	0.9.49.1	4.649	13.789	6.395
Linux	Ryzen 7 1700	NA	0.9.49.2	4.786	8.461	5.675
Linux	Ryzen 7 1700	NA	0.9.49.3	4.510	13.805	7.386
Linux	Ryzen 7 1700	NA	0.9.49.4	4.446	15.335	6.002
Linux	Ryzen 7 1700	NA	0.9.50.2	1.402	6.174	2.764
Linux	Ryzen 7 1700	NA	0.9.50.3	1.430	6.615	2.878
Linux	Ryzen 7 1700	NA	0.9.50.4	1.397	7.251	2.810
Linux	Ryzen 7 1700	NA	1.0.3	1.430	6.344	2.798
Linux	Ryzen 7 1700	NA	1.0.4	1.415	6.364	2.820
Linux	Ryzen 7 1700	4.1.3	1.1.0	1.310	6.279	2.681
Linux	Ryzen 7 1700	4.2.1	1.1.0	3.802	21.247	8.461
Linux	i7-4710MQ	4.2.1	1.1.0	3.334	19.521	7.565
Linux	i7-4710MQ	4.1.3	1.1.0	1.578	8.058	3.339
Linux	i7-4710MQ	4.2.1	1.1.1	1.230	5.839	2.444
Linux	Ryzen 7 1700	4.2.1	1.1.1	1.308	5.758	2.558
Linux	Ryzen 7 1700	4.2.1	1.1.2	1.503	6.147	2.803
Linux	Ryzen 7 1700	4.2.2	1.2.0	1.554	6.193	2.843
Linux	Ryzen 7 1700	4.2.2	1.2.2	1.585	6.335	3.003
Linux	Ryzen 9 7950X	4.2.2	1.2.0	0.792	2.378	1.245
Linux	Ryzen 9 7950X	4.2.2	1.2.2	0.784	2.355	1.233
Linux	Ryzen 9 7950X	4.2.2	1.2.3	0.770	2.011	1.123
Linux	Ryzen 9 7950X	4.2.3	1.2.3	0.793	2.109	1.178
Linux	Ryzen 9 7950X	4.3.0	1.2.4	0.779	2.080	1.106
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	4.3.1	1.2.5	1.823	5.555	2.404
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	4.3.1	1.2.6	1.761	5.405	2.462
Linux	Ryzen 9 7950X	4.3.2	1.2.6	0.795	2.228	1.178

Two metabolites

Constant variance (t6 and t7), two-component error model (t8 and t9), and variance by variable (t10 and t11) for one model fitted to one dataset, i.e. one fit for each test.

OS	CPU	R	mkin	t6	t7	t8	t9	t10	t11
Linux	Ryzen 7 1700	NA	0.9.48.1	2.623	4.587	7.525	16.621	8.576	31.267
Linux	Ryzen 7 1700	NA	0.9.49.1	2.542	4.128	4.632	8.171	3.676	5.636
Linux	Ryzen 7 1700	NA	0.9.49.2	2.723	4.478	4.862	7.618	3.579	5.574
Linux	Ryzen 7 1700	NA	0.9.49.3	2.643	4.374	7.020	11.124	5.388	7.365
Linux	Ryzen 7 1700	NA	0.9.49.4	2.635	4.259	4.737	7.763	3.427	5.626
Linux	Ryzen 7 1700	NA	0.9.50.2	0.777	1.236	1.332	2.872	2.069	2.987
Linux	Ryzen 7 1700	NA	0.9.50.3	0.858	1.264	1.333	2.984	2.113	3.073
Linux	Ryzen 7 1700	NA	0.9.50.4	0.783	1.282	1.486	3.815	1.958	3.105
Linux	Ryzen 7 1700	NA	1.0.3	0.763	1.244	1.457	3.054	1.923	2.839
Linux	Ryzen 7 1700	NA	1.0.4	0.785	1.252	1.466	3.091	1.936	2.826
Linux	Ryzen 7 1700	4.1.3	1.1.0	0.744	1.227	1.288	3.553	1.895	2.738
Linux	Ryzen 7 1700	4.2.1	1.1.0	3.018	4.165	5.036	10.844	6.623	9.722
Linux	i7-4710MQ	4.2.1	1.1.0	2.522	3.792	4.143	11.268	5.935	8.728
Linux	i7-4710MQ	4.1.3	1.1.0	0.907	1.535	1.589	4.544	2.302	3.463
Linux	i7-4710MQ	4.2.1	1.1.1	0.678	1.095	1.149	3.247	1.658	2.472
Linux	Ryzen 7 1700	4.2.1	1.1.1	0.696	1.124	1.321	2.786	1.744	2.566
Linux	Ryzen 7 1700	4.2.1	1.1.2	0.861	1.295	1.507	3.102	1.961	2.852
Linux	Ryzen 7 1700	4.2.2	1.2.0	0.913	1.345	1.539	3.011	1.987	2.802
Linux	Ryzen 7 1700	4.2.2	1.2.2	0.935	1.381	1.551	3.209	1.976	3.013
Linux	Ryzen 9 7950X	4.2.2	1.2.0	0.445	0.591	0.660	1.190	0.814	1.100
Linux	Ryzen 9 7950X	4.2.2	1.2.2	0.443	0.586	0.661	1.176	0.803	1.097
Linux	Ryzen 9 7950X	4.2.2	1.2.3	0.418	0.530	0.591	1.006	0.716	0.949
Linux	Ryzen 9 7950X	4.2.3	1.2.3	0.432	0.549	0.609	1.052	0.743	0.989
Linux	Ryzen 9 7950X	4.3.0	1.2.4	0.410	0.526	0.553	1.249	0.712	0.948
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	4.3.1	1.2.5	0.798	1.096	1.217	3.173	1.634	2.271
Linux	Intel(R) Xeon(R) Gold 6134 CPU @ 3.20GHz	4.3.1	1.2.6	0.813	1.136	1.220	3.114	1.598	2.255
Linux	Ryzen 9 7950X	4.3.2	1.2.6	0.439	0.557	0.585	1.338	0.749	0.999

Johannes Ranke

Last change 17 February 2023 (rebuilt 2023-11-16)

Test cases

Results

Parent only

One metabolite

Two metabolites