Skip to contents

Compute Cohen's d Measure of Effect Size

Source: R/cohens_d.R
cohens_d.Rd

Compute the effect size for t-test. T-test conventional effect sizes, proposed by Cohen, are: 0.2 (small effect), 0.5 (moderate effect) and 0.8 (large effect).

Cohen's d is calculated as the difference between means or mean minus mu divided by the estimated standardized deviation.

For independent samples t-test, there are two possibilities implemented. If the t-test did not make a homogeneity of variance assumption, (the Welch test), the variance term will mirror the Welch test, otherwise a pooled estimate is used.

If a paired samples t-test was requested, then effect size desired is based on the standard deviation of the differences.

It can also return confidence intervals for the effect size, either by bootstrap (the default) or by an analytic, deterministic method (see ci.method).

See the Datanovia tutorial Cohen’s d Effect Size in R for a worked walkthrough.

Usage

cohens_d(
  data,
  formula,
  comparisons = NULL,
  ref.group = NULL,
  paired = FALSE,
  mu = 0,
  var.equal = FALSE,
  hedges.correction = FALSE,
  ci = FALSE,
  conf.level = 0.95,
  ci.type = "perc",
  nboot = 1000,
  boot.parallel = getOption("boot.parallel", "no"),
  boot.ncpus = getOption("boot.ncpus", 1L),
  id = NULL,
  ci.method = c("boot", "analytic")
)

Arguments

data

a data.frame containing the variables in the formula.

formula

a formula of the form x ~ group where x is a numeric variable giving the data values and group is a factor with one or multiple levels giving the corresponding groups. For example, formula = TP53 ~ cancer_group.

comparisons

A list of length-2 vectors specifying the groups of interest to be compared. For example to compare groups "A" vs "B" and "B" vs "C", the argument is as follow: comparisons = list(c("A", "B"), c("B", "C"))

ref.group

a character string specifying the reference group. If specified, for a given grouping variable, each of the group levels will be compared to the reference group (i.e. control group).

If ref.group = "all", pairwise two sample tests are performed for comparing each grouping variable levels against all (i.e. basemean).

paired

a logical indicating whether you want a paired test.

mu

the theoretical mean (one-sample test) or the hypothesized difference in means (two-sample test). It is subtracted from the mean difference before standardizing, so a non-zero mu shifts the effect size accordingly. Default is 0.

var.equal

a logical variable indicating whether to treat the two variances as being equal. If TRUE then the pooled variance is used to estimate the variance otherwise the Welch (or Satterthwaite) approximation to the degrees of freedom is used. Used only for unpaired or independent samples test.

hedges.correction

logical indicating whether apply the Hedges correction by multiplying the usual value of Cohen's d by (N-3)/(N-2.25) (for unpaired t-test) and by (n1-2)/(n1-1.25) for paired t-test; where N is the total size of the two groups being compared (N = n1 + n2).

ci

If TRUE, returns confidence intervals by bootstrap. May be slow.

conf.level

The level for the confidence interval.

ci.type

The type of confidence interval to use. Can be any of "norm", "basic", "perc", or "bca". Passed to boot::boot.ci.

nboot

The number of replications to use for bootstrap.

boot.parallel

The type of parallel operation to be used when computing the bootstrap confidence interval. Allowed values are "no" (default), "multicore" and "snow". Passed to boot(). Defaults to getOption("boot.parallel", "no"), so it can also be set globally with options(boot.parallel = "multicore"). Only used when ci = TRUE.

boot.ncpus

Integer. The number of processes to be used in the parallel bootstrap. Defaults to getOption("boot.ncpus", 1L). Note that boot.parallel has no effect unless boot.ncpus > 1. Only used when ci = TRUE.

id

(optional) character string with the name of the column holding the sample/subject identifier, used only for a paired test (paired = TRUE). When supplied, the two groups are matched by id (instead of by row order) before the paired d is computed, as in t_test(). Only complete pairs (subjects measured in both groups) are used.

ci.method

the method used to compute the confidence interval when ci = TRUE. Either "boot" (default) for a percentile bootstrap, or "analytic" for a deterministic interval obtained by inverting the noncentral t distribution (Steiger, 2004; Cumming & Finch, 2001) – the same machinery anova_test(ci = ) uses for partial eta-squared. The analytic interval covers the one-sample, paired and two-sample (equal-variance and Welch) cases, is reproducible across runs (no seed), and matches effectsize::cohens_d(ci = ). With hedges.correction = TRUE the estimate and both bounds are scaled by the documented \((N - 3)/(N - 2.25)\) approximation, so they are not identical to effectsize::hedges_g(ci = ), which applies the exact gamma-function correction at the design's degrees of freedom. The gap is proportional to the effect size and grows as the samples shrink. When the interval is not defined for a given input (for example a degenerate group), the bootstrap is used as a fallback. The default "boot" leaves the returned interval unchanged from previous versions.

Value

return a data frame with some of the following columns:

  • .y.: the y variable used in the test.

  • group1,group2: the compared groups in the pairwise tests.

  • n,n1,n2: Sample counts.

  • effsize: estimate of the effect size (d value).

  • magnitude: magnitude of effect size.

  • conf.low,conf.high: lower and upper bound of the effect size confidence interval.

Details

Quantification of the effect size magnitude is performed using the thresholds defined in Cohen (1992). The magnitude is assessed using the thresholds provided in (Cohen 1992), i.e. |d| < 0.2 "negligible", |d| < 0.5 "small", |d| < 0.8 "medium", otherwise "large".

References

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York:Academic Press.

  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.

  • Hedges, Larry & Olkin, Ingram. (1985). Statistical Methods in Meta-Analysis. 10.2307/1164953.

  • Navarro, Daniel. 2015. Learning Statistics with R: A Tutorial for Psychology Students and Other Beginners (Version 0.5).

  • Steiger, J. H. (2004). Beyond the F test: Effect size confidence intervals and tests of close fit in the analysis of variance and contrast analysis. Psychological Methods, 9(2), 164-182.

  • Cumming, G., & Finch, S. (2001). A primer on the understanding, use, and calculation of confidence intervals that are based on central and noncentral distributions. Educational and Psychological Measurement, 61(4), 532-574.

See also

The Datanovia tutorial: Cohen’s d Effect Size in R.

Examples

# One-sample t test effect size
ToothGrowth %>% cohens_d(len ~ 1, mu = 0)
#> # A tibble: 1 × 6
#>   .y.   group1 group2     effsize     n magnitude
#> * <chr> <chr>  <chr>        <dbl> <int> <ord>    
#> 1 len   1      null model    2.46    60 large    

# Two indepedent samples t-test effect size
ToothGrowth %>% cohens_d(len ~ supp, var.equal = TRUE)
#> # A tibble: 1 × 7
#>   .y.   group1 group2 effsize    n1    n2 magnitude
#> * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
#> 1 len   OJ     VC       0.495    30    30 small    

# Paired samples effect size
df <- data.frame(
  id = 1:5,
  pre  = c(110, 122, 101, 120, 140),
  post = c(150, 160, 110, 140, 155)
)
df <- df %>% gather(key = "treatment", value = "value", -id)
head(df)
#>   id treatment value
#> 1  1       pre   110
#> 2  2       pre   122
#> 3  3       pre   101
#> 4  4       pre   120
#> 5  5       pre   140
#> 6  1      post   150

df %>% cohens_d(value ~ treatment, paired = TRUE)
#> # A tibble: 1 × 7
#>   .y.   group1 group2 effsize    n1    n2 magnitude
#> * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
#> 1 value post   pre       1.75     5     5 large