Metropolis-Hastings Algorithm for Mixtures of Regressions

Return Metropolis-Hastings (M-H) algorithm output for mixtures of multiple regressions with arbitrarily many components.

Usage

regmixMH(y, x, lambda = NULL, beta = NULL, s = NULL, k = 2,
         addintercept = TRUE, mu = NULL, sig = NULL, lam.hyp = NULL,
         sampsize = 1000, omega = 0.01, thin = 1)

Arguments

y

An n-vector of response values.

x

An nxp matrix of predictors. See addintercept below.

lambda

Initial value of mixing proportions. Entries should sum to 1. This determines number of components. If NULL, then lambda is random from uniform Dirichlet and number of components is determined by beta.

beta

Initial value of beta parameters. Should be a pxk matrix, where p is the number of columns of x and k is number of components. If NULL, then beta has uniform standard normal entries. If both lambda and beta are NULL, then number of components is determined by s.

s

k-vector of standard deviations. If NULL, then \(1/\code{s}^2\) has random standard exponential entries. If lambda, beta, and s are NULL, then number of components determined by k.

k

Number of components. Ignored unless all of lambda, beta, and s are NULL.

addintercept

If TRUE, a column of ones is appended to the x matrix before the value of p is calculated.

mu

The prior hyperparameter of same size as beta; the means of beta components. If NULL, these are set to zero.

sig

The prior hyperparameter of same size as beta; the standard deviations of beta components. If NULL, these are all set to five times the overall standard deviation of y.

lam.hyp

The prior hyperparameter of length k for the mixing proportions (i.e., these are hyperparameters for the Dirichlet distribution). If NULL, these are generated from a standard uniform distribution and then scaled to sum to 1.

sampsize

Size of posterior sample returned.

omega

Multiplier of step size to control M-H acceptance rate. Values closer to zero result in higher acceptance rates, generally.

thin

Lag between parameter vectors that will be kept.

Value

regmixMH returns a list of class mixMCMC with items:

x

A nxp matrix of the predictors.

y

A vector of the responses.

theta

A (sampsize/thin) x q matrix of MCMC-sampled q-vectors, where q is the total number of parameters in beta, s, and lambda.

k

The number of components.

See also

regcr

References

Hurn, M., Justel, A. and Robert, C. P. (2003) Estimating Mixtures of Regressions, Journal of Computational and Graphical Statistics 12(1), 55–79.

Examples

## M-H algorithm for NOdata with acceptance rate about 40%.

data(NOdata)
attach(NOdata)
#> The following objects are masked from NOdata (pos = 3):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 4):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 5):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 7):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 11):
#> 
#>     Equivalence, NO
#> The following objects are masked from NOdata (pos = 12):
#> 
#>     Equivalence, NO
set.seed(100)
beta <- matrix(c(1.3, -0.1, 0.6, 0.1), 2, 2)
sigma <- c(.02, .05)
MH.out <- regmixMH(Equivalence, NO, beta = beta, s = sigma, 
                   sampsize = 2500, omega = .0013)
#> Acceptance rate: 39.48%
MH.out$theta[2400:2499,]
#>         beta0.1     beta1.1   beta0.2    beta1.2        s.1        s.2
#>   [1,] 1.254410 -0.08923505 0.5623385 0.08605716 0.02933969 0.03344021
#>   [2,] 1.254410 -0.08923505 0.5623385 0.08605716 0.02933969 0.03344021
#>   [3,] 1.254410 -0.08923505 0.5623385 0.08605716 0.02933969 0.03344021
#>   [4,] 1.255458 -0.08967213 0.5626661 0.08685740 0.02910850 0.03356269
#>   [5,] 1.253427 -0.08972491 0.5623568 0.08479636 0.02895941 0.03360472
#>   [6,] 1.253427 -0.08972491 0.5623568 0.08479636 0.02895941 0.03360472
#>   [7,] 1.253457 -0.08491087 0.5625723 0.08694171 0.02900875 0.03367823
#>   [8,] 1.253457 -0.08491087 0.5625723 0.08694171 0.02900875 0.03367823
#>   [9,] 1.253457 -0.08491087 0.5625723 0.08694171 0.02900875 0.03367823
#>  [10,] 1.253457 -0.08491087 0.5625723 0.08694171 0.02900875 0.03367823
#>  [11,] 1.253457 -0.08491087 0.5625723 0.08694171 0.02900875 0.03367823
#>  [12,] 1.252858 -0.08600474 0.5608805 0.08751656 0.02896942 0.03369924
#>  [13,] 1.252858 -0.08600474 0.5608805 0.08751656 0.02896942 0.03369924
#>  [14,] 1.252858 -0.08600474 0.5608805 0.08751656 0.02896942 0.03369924
#>  [15,] 1.252858 -0.08600474 0.5608805 0.08751656 0.02896942 0.03369924
#>  [16,] 1.252868 -0.08644512 0.5612669 0.08885841 0.02900778 0.03372282
#>  [17,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [18,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [19,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [20,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [21,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [22,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [23,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [24,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [25,] 1.253252 -0.08526195 0.5634879 0.08882754 0.02905912 0.03373287
#>  [26,] 1.246130 -0.08419832 0.5631643 0.08726377 0.02908239 0.03262461
#>  [27,] 1.249684 -0.08308411 0.5626647 0.08560663 0.02911028 0.03278386
#>  [28,] 1.249684 -0.08308411 0.5626647 0.08560663 0.02911028 0.03278386
#>  [29,] 1.249684 -0.08308411 0.5626647 0.08560663 0.02911028 0.03278386
#>  [30,] 1.249684 -0.08308411 0.5626647 0.08560663 0.02911028 0.03278386
#>  [31,] 1.247976 -0.08172885 0.5616326 0.08535635 0.02912969 0.03270071
#>  [32,] 1.247976 -0.08172885 0.5616326 0.08535635 0.02912969 0.03270071
#>  [33,] 1.247976 -0.08172885 0.5616326 0.08535635 0.02912969 0.03270071
#>  [34,] 1.247976 -0.08172885 0.5616326 0.08535635 0.02912969 0.03270071
#>  [35,] 1.247976 -0.08172885 0.5616326 0.08535635 0.02912969 0.03270071
#>  [36,] 1.247823 -0.07778124 0.5612692 0.08651632 0.02898857 0.03267645
#>  [37,] 1.247823 -0.07778124 0.5612692 0.08651632 0.02898857 0.03267645
#>  [38,] 1.248280 -0.07697337 0.5518260 0.08895505 0.02902519 0.03264113
#>  [39,] 1.248280 -0.07697337 0.5518260 0.08895505 0.02902519 0.03264113
#>  [40,] 1.246079 -0.07765583 0.5515278 0.08686658 0.02905050 0.03265415
#>  [41,] 1.246079 -0.07765583 0.5515278 0.08686658 0.02905050 0.03265415
#>  [42,] 1.242887 -0.07701484 0.5504370 0.08647371 0.02930680 0.03265534
#>  [43,] 1.242367 -0.07518253 0.5538026 0.08949631 0.02933665 0.03264252
#>  [44,] 1.242367 -0.07518253 0.5538026 0.08949631 0.02933665 0.03264252
#>  [45,] 1.242367 -0.07518253 0.5538026 0.08949631 0.02933665 0.03264252
#>  [46,] 1.242367 -0.07518253 0.5538026 0.08949631 0.02933665 0.03264252
#>  [47,] 1.242367 -0.07518253 0.5538026 0.08949631 0.02933665 0.03264252
#>  [48,] 1.239898 -0.07383224 0.5563218 0.08745264 0.02932332 0.03260546
#>  [49,] 1.239898 -0.07383224 0.5563218 0.08745264 0.02932332 0.03260546
#>  [50,] 1.240000 -0.07525909 0.5582321 0.09352141 0.02924289 0.03260555
#>  [51,] 1.240000 -0.07525909 0.5582321 0.09352141 0.02924289 0.03260555
#>  [52,] 1.240793 -0.07939593 0.5586602 0.09318136 0.02919065 0.03261756
#>  [53,] 1.240793 -0.07939593 0.5586602 0.09318136 0.02919065 0.03261756
#>  [54,] 1.240793 -0.07939593 0.5586602 0.09318136 0.02919065 0.03261756
#>  [55,] 1.240997 -0.07947991 0.5602306 0.09225317 0.02918792 0.03260170
#>  [56,] 1.240997 -0.07947991 0.5602306 0.09225317 0.02918792 0.03260170
#>  [57,] 1.243751 -0.07951700 0.5592401 0.09189214 0.02968812 0.03245697
#>  [58,] 1.243751 -0.07951700 0.5592401 0.09189214 0.02968812 0.03245697
#>  [59,] 1.243751 -0.07951700 0.5592401 0.09189214 0.02968812 0.03245697
#>  [60,] 1.243751 -0.07951700 0.5592401 0.09189214 0.02968812 0.03245697
#>  [61,] 1.242762 -0.08265697 0.5580760 0.09305108 0.02968681 0.03231539
#>  [62,] 1.244050 -0.08187088 0.5599590 0.09081554 0.02971119 0.03211033
#>  [63,] 1.242690 -0.08001063 0.5583832 0.09081747 0.02969886 0.03210837
#>  [64,] 1.242690 -0.08001063 0.5583832 0.09081747 0.02969886 0.03210837
#>  [65,] 1.242690 -0.08001063 0.5583832 0.09081747 0.02969886 0.03210837
#>  [66,] 1.246613 -0.07988764 0.5494018 0.08943392 0.02967819 0.03218494
#>  [67,] 1.246613 -0.07988764 0.5494018 0.08943392 0.02967819 0.03218494
#>  [68,] 1.246613 -0.07988764 0.5494018 0.08943392 0.02967819 0.03218494
#>  [69,] 1.247663 -0.08104239 0.5502755 0.08980198 0.02976820 0.03225688
#>  [70,] 1.231106 -0.07777578 0.5498541 0.09098048 0.02996607 0.03216542
#>  [71,] 1.231106 -0.07777578 0.5498541 0.09098048 0.02996607 0.03216542
#>  [72,] 1.231106 -0.07777578 0.5498541 0.09098048 0.02996607 0.03216542
#>  [73,] 1.230939 -0.07694301 0.5501247 0.09137574 0.02993412 0.03218888
#>  [74,] 1.230939 -0.07694301 0.5501247 0.09137574 0.02993412 0.03218888
#>  [75,] 1.230939 -0.07694301 0.5501247 0.09137574 0.02993412 0.03218888
#>  [76,] 1.229760 -0.07267403 0.5502329 0.09187104 0.02898158 0.03206489
#>  [77,] 1.229760 -0.07267403 0.5502329 0.09187104 0.02898158 0.03206489
#>  [78,] 1.229760 -0.07267403 0.5502329 0.09187104 0.02898158 0.03206489
#>  [79,] 1.228218 -0.07240322 0.5520055 0.09144797 0.02913194 0.03204554
#>  [80,] 1.226748 -0.07162546 0.5515162 0.09184552 0.02882759 0.03204655
#>  [81,] 1.226748 -0.07162546 0.5515162 0.09184552 0.02882759 0.03204655
#>  [82,] 1.225401 -0.07239923 0.5496854 0.09304677 0.02880777 0.03200133
#>  [83,] 1.225401 -0.07239923 0.5496854 0.09304677 0.02880777 0.03200133
#>  [84,] 1.224953 -0.07121243 0.5503352 0.09311510 0.02883843 0.03210230
#>  [85,] 1.224953 -0.07121243 0.5503352 0.09311510 0.02883843 0.03210230
#>  [86,] 1.238173 -0.07337846 0.5494643 0.09098367 0.02862105 0.03207736
#>  [87,] 1.236200 -0.07477091 0.5480105 0.08939434 0.02866408 0.03205082
#>  [88,] 1.235555 -0.07545189 0.5484364 0.08811507 0.02861984 0.03200608
#>  [89,] 1.228880 -0.07558801 0.5448841 0.08902745 0.02867962 0.03232360
#>  [90,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [91,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [92,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [93,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [94,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [95,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [96,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [97,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [98,] 1.229082 -0.07510223 0.5437388 0.09263522 0.02877457 0.03226480
#>  [99,] 1.229891 -0.07685150 0.5416898 0.08806412 0.02882913 0.03225022
#> [100,] 1.223073 -0.07102211 0.5427422 0.08984683 0.02626507 0.03259091
#>         lambda.1  lambda.2
#>   [1,] 0.4682172 0.5317828
#>   [2,] 0.4682172 0.5317828
#>   [3,] 0.4682172 0.5317828
#>   [4,] 0.4679742 0.5320258
#>   [5,] 0.4679174 0.5320826
#>   [6,] 0.4679174 0.5320826
#>   [7,] 0.4673328 0.5326672
#>   [8,] 0.4673328 0.5326672
#>   [9,] 0.4673328 0.5326672
#>  [10,] 0.4673328 0.5326672
#>  [11,] 0.4673328 0.5326672
#>  [12,] 0.4689420 0.5310580
#>  [13,] 0.4689420 0.5310580
#>  [14,] 0.4689420 0.5310580
#>  [15,] 0.4689420 0.5310580
#>  [16,] 0.4674227 0.5325773
#>  [17,] 0.4663676 0.5336324
#>  [18,] 0.4663676 0.5336324
#>  [19,] 0.4663676 0.5336324
#>  [20,] 0.4663676 0.5336324
#>  [21,] 0.4663676 0.5336324
#>  [22,] 0.4663676 0.5336324
#>  [23,] 0.4663676 0.5336324
#>  [24,] 0.4663676 0.5336324
#>  [25,] 0.4663676 0.5336324
#>  [26,] 0.4673251 0.5326749
#>  [27,] 0.4673464 0.5326536
#>  [28,] 0.4673464 0.5326536
#>  [29,] 0.4673464 0.5326536
#>  [30,] 0.4673464 0.5326536
#>  [31,] 0.4706163 0.5293837
#>  [32,] 0.4706163 0.5293837
#>  [33,] 0.4706163 0.5293837
#>  [34,] 0.4706163 0.5293837
#>  [35,] 0.4706163 0.5293837
#>  [36,] 0.4732809 0.5267191
#>  [37,] 0.4732809 0.5267191
#>  [38,] 0.4740313 0.5259687
#>  [39,] 0.4740313 0.5259687
#>  [40,] 0.4743477 0.5256523
#>  [41,] 0.4743477 0.5256523
#>  [42,] 0.4734871 0.5265129
#>  [43,] 0.4733840 0.5266160
#>  [44,] 0.4733840 0.5266160
#>  [45,] 0.4733840 0.5266160
#>  [46,] 0.4733840 0.5266160
#>  [47,] 0.4733840 0.5266160
#>  [48,] 0.4739050 0.5260950
#>  [49,] 0.4739050 0.5260950
#>  [50,] 0.4737910 0.5262090
#>  [51,] 0.4737910 0.5262090
#>  [52,] 0.4732674 0.5267326
#>  [53,] 0.4732674 0.5267326
#>  [54,] 0.4732674 0.5267326
#>  [55,] 0.4736302 0.5263698
#>  [56,] 0.4736302 0.5263698
#>  [57,] 0.4732535 0.5267465
#>  [58,] 0.4732535 0.5267465
#>  [59,] 0.4732535 0.5267465
#>  [60,] 0.4732535 0.5267465
#>  [61,] 0.4734752 0.5265248
#>  [62,] 0.4749180 0.5250820
#>  [63,] 0.4755269 0.5244731
#>  [64,] 0.4755269 0.5244731
#>  [65,] 0.4755269 0.5244731
#>  [66,] 0.4771956 0.5228044
#>  [67,] 0.4771956 0.5228044
#>  [68,] 0.4771956 0.5228044
#>  [69,] 0.4784359 0.5215641
#>  [70,] 0.4761567 0.5238433
#>  [71,] 0.4761567 0.5238433
#>  [72,] 0.4761567 0.5238433
#>  [73,] 0.4782076 0.5217924
#>  [74,] 0.4782076 0.5217924
#>  [75,] 0.4782076 0.5217924
#>  [76,] 0.4786591 0.5213409
#>  [77,] 0.4786591 0.5213409
#>  [78,] 0.4786591 0.5213409
#>  [79,] 0.4785005 0.5214995
#>  [80,] 0.4786760 0.5213240
#>  [81,] 0.4786760 0.5213240
#>  [82,] 0.4801196 0.5198804
#>  [83,] 0.4801196 0.5198804
#>  [84,] 0.4797845 0.5202155
#>  [85,] 0.4797845 0.5202155
#>  [86,] 0.4799580 0.5200420
#>  [87,] 0.4814088 0.5185912
#>  [88,] 0.4810236 0.5189764
#>  [89,] 0.4810940 0.5189060
#>  [90,] 0.4803038 0.5196962
#>  [91,] 0.4803038 0.5196962
#>  [92,] 0.4803038 0.5196962
#>  [93,] 0.4803038 0.5196962
#>  [94,] 0.4803038 0.5196962
#>  [95,] 0.4803038 0.5196962
#>  [96,] 0.4803038 0.5196962
#>  [97,] 0.4803038 0.5196962
#>  [98,] 0.4803038 0.5196962
#>  [99,] 0.4802069 0.5197931
#> [100,] 0.4802070 0.5197930