Concept
In statistics, Bayesian multivariate linear regression is a Bayesian approach to multivariate linear regression, i.e. linear regression where the predicted outcome is a vector of correlated random variables rather than a single scalar random variable. A more general treatment of this approach can be found in the article MMSE estimator. Consider a regression problem where the dependent variable to be predicted is not a single real-valued scalar but an m-length vector of correlated real numbers. As in the standard regression setup, there are n observations, where each observation i consists of k−1 explanatory variables, grouped into a vector of length k (where a dummy variable with a value of 1 has been added to allow for an intercept coefficient). This can be viewed as a set of m related regression problems for each observation i: where the set of errors are all correlated. Equivalently, it can be viewed as a single regression problem where the outcome is a row vector and the regression coefficient vectors are stacked next to each other, as follows: The coefficient matrix B is a matrix where the coefficient vectors for each regression problem are stacked horizontally: The noise vector for each observation i is jointly normal, so that the outcomes for a given observation are correlated: We can write the entire regression problem in matrix form as: where Y and E are matrices. The design matrix X is an matrix with the observations stacked vertically, as in the standard linear regression setup: The classical, frequentists linear least squares solution is to simply estimate the matrix of regression coefficients using the Moore-Penrose pseudoinverse: To obtain the Bayesian solution, we need to specify the conditional likelihood and then find the appropriate conjugate prior. As with the univariate case of linear Bayesian regression, we will find that we can specify a natural conditional conjugate prior (which is scale dependent).