1 About
The matlm R package (https://github.com/variani/matlm) fits linear models in matrix form and avoids calling lm. That makes computation efficient if many predictors need to be tested, while calling lm for every predictor results in a considerable overhead in computation time.
The development of our package was inspired by the MatrixEQTL R package (Shabalin 2012). The basic idea is that for a simple linear regression:
\(y = \mu + \beta x + e\)
\(e \sim \mathcal{N}(0, \sigma_e^2)\)
the test statistic of association between \(y\) and \(x\) (t-test) can be calculated using the sample correlation \(r = cor(y, x)\):
\(t = \sqrt{N - 2} \frac{r}{\sqrt{1 - r^2}}\)
Further, when testing M predictors \(X = [x_1, x_2, \dots, x_M]\), the computation takes the form of matrix multiplication operations. If a single outcome is tested, then we compute the cross-product of \(y\) and \(X\). If there are many outcomes or the permutation of \(y\) is sought, then we have a matrix \(Y = [y_1, y_2, \dots, y_L]\) and we compute the cross-product of \(Y\) and \(X\).
Features available in MatrixEQTL:
- splitting data in \(X\) into batches;
- orthogonalization of \(y\) and \(x\) due to other covariates (age, gender, population stratification);
- transformation of \(y\) and \(x\) when the residual errors are correlated, \(e \sim \mathcal{N}(0, V)\);
- parallel computation.
Features implemented or to be implemented in matlm (apart from the features in MatrixEQTL):
- support for storage of \(X\) in different format (out-memory netCDF and hdf5);
- interaction tests;
- permutation tests on outcome;
- permutation tests on predictors;
- bootstrapping on the test statistics;
- correction for correlated statistics.
1.1 Is matlm fast?
Here you see a comparison between lm and matlm (the main function of the package):
