EM Algorithm for Mixtures-of-Experts — hmeEM • mixtools

Returns EM algorithm output for a mixture-of-experts model. Currently, this code only handles a 2-component mixture-of-experts, but will be extended to the general k-component hierarchical mixture-of-experts.

Usage

hmeEM(y, x, lambda = NULL, beta = NULL, sigma = NULL, w = NULL,
      k = 2, addintercept = TRUE, epsilon = 1e-08, 
      maxit = 10000, verb = FALSE)

Arguments

y: An n-vector of response values.
x: An nxp matrix of predictors. See addintercept below.
lambda: Initial value of mixing proportions, which are modeled as an inverse logit function of the predictors. Entries should sum to 1. If NULL, then lambda is taken as 1/k for each x.
beta: Initial value of beta parameters. Should be a pxk matrix, where p is the number of columns of x and k is number of components. If NULL, then beta has standard normal entries according to a binning method done on the data.
sigma: A vector of standard deviations. If NULL, then \(1/\code{sigma}^2\) has random standard exponential entries according to a binning method done on the data.
w: A p-vector of coefficients for the way the mixing proportions are modeled. See lambda.
k: Number of components. Currently, only k=2 is accepted.
addintercept: If TRUE, a column of ones is appended to the x matrix before the value of p is calculated.
epsilon: The convergence criterion.
maxit: The maximum number of iterations.
verb: If TRUE, then various updates are printed during each iteration of the algorithm.

Value

hmeEM returns a list of class mixEM with items:

x: The set of predictors (which includes a column of 1's if addintercept = TRUE).
y: The response values.
w: The final coefficients for the functional form of the mixing proportions.
lambda: An nxk matrix of the final mixing proportions.
beta: The final regression coefficients.
sigma: The final standard deviations. If arbmean = FALSE, then only the smallest standard deviation is returned. See scale below.
loglik: The final log-likelihood.
posterior: An nxk matrix of posterior probabilities for observations.
all.loglik: A vector of each iteration's log-likelihood.
restarts: The number of times the algorithm restarted due to unacceptable choice of initial values.
ft: A character vector giving the name of the function.

See also

References

Jacobs, R. A., Jordan, M. I., Nowlan, S. J. and Hinton, G. E. (1991) Adaptive Mixtures of Local Experts, Neural Computation 3(1), 79–87.

McLachlan, G. J. and Peel, D. (2000) Finite Mixture Models, John Wiley and Sons, Inc.

Examples

## EM output for NOdata.
 
data(NOdata)
attach(NOdata)
#> The following objects are masked from NOdata (pos = 3):
#> 
#>     Equivalence, NO
set.seed(100)
em.out <- regmixEM(Equivalence, NO)
#> number of iterations= 25 
hme.out <- hmeEM(Equivalence, NO, beta = em.out$beta)
#> number of iterations= 190 
hme.out[3:7]
#> $w
#> [1] -0.7642111  0.4170725
#> 
#> $lambda
#>            [,1]      [,2]
#>  [1,] 0.6891305 0.3108695
#>  [2,] 0.5480934 0.4519066
#>  [3,] 0.4651973 0.5348027
#>  [4,] 0.6076333 0.3923667
#>  [5,] 0.3900181 0.6099819
#>  [6,] 0.6311271 0.3688729
#>  [7,] 0.3779829 0.6220171
#>  [8,] 0.4313768 0.5686232
#>  [9,] 0.5545930 0.4454070
#> [10,] 0.3748501 0.6251499
#> [11,] 0.6826610 0.3173390
#> [12,] 0.4140765 0.5859235
#> [13,] 0.4121552 0.5878448
#> [14,] 0.4336288 0.5663712
#> [15,] 0.4066089 0.5933911
#> [16,] 0.4747550 0.5252450
#> [17,] 0.4972555 0.5027445
#> [18,] 0.5135180 0.4864820
#> [19,] 0.7141768 0.2858232
#> [20,] 0.6338416 0.3661584
#> [21,] 0.5003835 0.4996165
#> [22,] 0.6040479 0.3959521
#> [23,] 0.4153925 0.5846075
#> [24,] 0.3562809 0.6437191
#> [25,] 0.3951899 0.6048101
#> [26,] 0.3524640 0.6475360
#> [27,] 0.6784903 0.3215097
#> [28,] 0.5038243 0.4961757
#> [29,] 0.6031498 0.3968502
#> [30,] 0.6733743 0.3266257
#> [31,] 0.4005855 0.5994145
#> [32,] 0.4518401 0.5481599
#> [33,] 0.3686162 0.6313838
#> [34,] 0.5934270 0.4065730
#> [35,] 0.4344485 0.5655515
#> [36,] 0.4881863 0.5118137
#> [37,] 0.6600301 0.3399699
#> [38,] 0.4793331 0.5206669
#> [39,] 0.4162030 0.5837970
#> [40,] 0.5338020 0.4661980
#> [41,] 0.6542039 0.3457961
#> [42,] 0.4540103 0.5459897
#> [43,] 0.5119550 0.4880450
#> [44,] 0.4102366 0.5897634
#> [45,] 0.3714357 0.6285643
#> [46,] 0.5402311 0.4597689
#> [47,] 0.4570101 0.5429899
#> [48,] 0.6689560 0.3310440
#> [49,] 0.4895412 0.5104588
#> [50,] 0.6394375 0.3605625
#> [51,] 0.5662028 0.4337972
#> [52,] 0.4942320 0.5057680
#> [53,] 0.5764171 0.4235829
#> [54,] 0.7076628 0.2923372
#> [55,] 0.7058475 0.2941525
#> [56,] 0.4744430 0.5255570
#> [57,] 0.4547341 0.5452659
#> [58,] 0.6705243 0.3294757
#> [59,] 0.5384695 0.4615305
#> [60,] 0.3896213 0.6103787
#> [61,] 0.4778761 0.5221239
#> [62,] 0.6814853 0.3185147
#> [63,] 0.6158551 0.3841449
#> [64,] 0.6908255 0.3091745
#> [65,] 0.3813226 0.6186774
#> [66,] 0.6833834 0.3166166
#> [67,] 0.6687713 0.3312287
#> [68,] 0.6474747 0.3525253
#> [69,] 0.4282082 0.5717918
#> [70,] 0.3867488 0.6132512
#> [71,] 0.5774351 0.4225649
#> [72,] 0.4063070 0.5936930
#> [73,] 0.4682076 0.5317924
#> [74,] 0.6567469 0.3432531
#> [75,] 0.5263219 0.4736781
#> [76,] 0.4106403 0.5893597
#> [77,] 0.6660866 0.3339134
#> [78,] 0.3894230 0.6105770
#> [79,] 0.3936956 0.6063044
#> [80,] 0.5241379 0.4758621
#> [81,] 0.6883257 0.3116743
#> [82,] 0.3728975 0.6271025
#> [83,] 0.3704624 0.6295376
#> [84,] 0.3706570 0.6293430
#> [85,] 0.3819131 0.6180869
#> [86,] 0.3520833 0.6479167
#> [87,] 0.3674522 0.6325478
#> [88,] 0.5070562 0.4929438
#> 
#> $beta
#>            comp.1      comp.2
#> beta.0 0.56130021  1.24848739
#> beta.1 0.08786273 -0.08409468
#> 
#> $sigma
#> [1] 0.04456598 0.02298156
#> 
#> $loglik
#> [1] 123.6206
#>