EM Algorithm for Mixtures of Regressions

Returns EM algorithm output for mixtures of multiple regressions with arbitrarily many components.

Usage

regmixEM(y, x, lambda = NULL, beta = NULL, sigma = NULL, k = 2,
         addintercept = TRUE, arbmean = TRUE, arbvar = TRUE, 
         epsilon = 1e-08, maxit = 10000, verb = FALSE)

Arguments

y: An n-vector of response values.
x: An nxp matrix of predictors. See addintercept below.
lambda: Initial value of mixing proportions. Entries should sum to 1. This determines number of components. If NULL, then lambda is random from uniform Dirichlet and number of components is determined by beta.
beta: Initial value of beta parameters. Should be a pxk matrix, where p is the number of columns of x and k is number of components. If NULL, then beta has standard normal entries according to a binning method done on the data. If both lambda and beta are NULL, then number of components is determined by sigma.
sigma: A vector of standard deviations. If NULL, then 1/sigma^2 has random standard exponential entries according to a binning method done on the data. If lambda, beta, and sigma are NULL, then number of components is determined by k.
k: Number of components. Ignored unless all of lambda, beta, and sigma are NULL.
addintercept: If TRUE, a column of ones is appended to the x matrix before the value of p is calculated.
arbmean: If TRUE, each mixture component is assumed to have a different set of regression coefficients (i.e., the betas).
arbvar: If TRUE, each mixture component is assumed to have a different sigma.
epsilon: The convergence criterion.
maxit: The maximum number of iterations.
verb: If TRUE, then various updates are printed during each iteration of the algorithm.

Value

regmixEM returns a list of class mixEM with items:

x: The set of predictors (which includes a column of 1's if addintercept = TRUE).
y: The response values.
lambda: The final mixing proportions.
beta: The final regression coefficients.
sigma: The final standard deviations. If arbmean = FALSE, then only the smallest standard deviation is returned. See scale below.
scale: If arbmean = FALSE, then the scale factor for the component standard deviations is returned. Otherwise, this is omitted from the output.
loglik: The final log-likelihood.
posterior: An nxk matrix of posterior probabilities for observations.
all.loglik: A vector of each iteration's log-likelihood.
restarts: The number of times the algorithm restarted due to unacceptable choice of initial values.
ft: A character vector giving the name of the function.

References

de Veaux, R. D. (1989), Mixtures of Linear Regressions, Computational Statistics and Data Analysis 8, 227-245.

Hurn, M., Justel, A. and Robert, C. P. (2003) Estimating Mixtures of Regressions, Journal of Computational and Graphical Statistics 12(1), 55–79.

McLachlan, G. J. and Peel, D. (2000) Finite Mixture Models, John Wiley and Sons, Inc.

Examples

## EM output for NOdata.
 
data(NOdata)
attach(NOdata)
#> The following objects are masked from NOdata (pos = 4):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 8):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 9):
#> 
#>     Equivalence, NO
set.seed(100)
em.out <- regmixEM(Equivalence, NO, verb = TRUE, epsilon = 1e-04)
#> iteration= 1 diff= 61.52481 log-likelihood 59.2794 
#> iteration= 2 diff= 15.93577 log-likelihood 75.21517 
#> iteration= 3 diff= 27.66707 log-likelihood 102.8822 
#> iteration= 4 diff= 17.24268 log-likelihood 120.1249 
#> iteration= 5 diff= 1.238034 log-likelihood 121.363 
#> iteration= 6 diff= 0.2725981 log-likelihood 121.6356 
#> iteration= 7 diff= 0.1762554 log-likelihood 121.8118 
#> iteration= 8 diff= 0.1116986 log-likelihood 121.9235 
#> iteration= 9 diff= 0.06262589 log-likelihood 121.9861 
#> iteration= 10 diff= 0.03058315 log-likelihood 122.0167 
#> iteration= 11 diff= 0.0132683 log-likelihood 122.03 
#> iteration= 12 diff= 0.005278647 log-likelihood 122.0353 
#> iteration= 13 diff= 0.001982899 log-likelihood 122.0372 
#> iteration= 14 diff= 0.0007186432 log-likelihood 122.038 
#> iteration= 15 diff= 0.0002548647 log-likelihood 122.0382 
#> iteration= 16 diff= 8.922635e-05 log-likelihood 122.0383 
#> number of iterations= 16 
em.out[3:6]
#> $lambda
#> [1] 0.4897288 0.5102712
#> 
#> $beta
#>            comp.1      comp.2
#> beta.0 0.56495808  1.24713113
#> beta.1 0.08504295 -0.08304106
#> 
#> $sigma
#> [1] 0.04333108 0.02414372
#> 
#> $loglik
#> [1] 122.0383
#>