[Statistics] Theory and Methodology

1. Data Visualization
Discrete data:
Bar Chart
Continuous data:
Box plot
Histogram

2. Mean, Median, and Mode
Mean: affected by extreme values (outliers)
Median: the exact middle value (if even, the average of two middle values); Not affected by extreme values
Mode: the value that occurs most frequently


3. Standard Deviation

Bienaymé-Chebyshev Rule

Regardless of how the data are distributed, a certain percentage of values must fall within K standard deviations from the mean:

4. Probability calculation by count

Count

r draws from n balls

order matters + with replacement: use powers: n^r

order matters + without replacement: use factorials: n!/(n-r)! or n(n-1)(n-2)...(n-r+1)

combination (order doesn't matter) + without replacement: use choosing: n choose r: n!/(r!(n-r)!)

Example:

What is the probability of there is at least two people with the same birthday in a class of 25?

order matters without replacement / order matters with replacement

365!/(365-25)! / 365^25

5. Independence

A and B are independent if and only if P(A&B)=P(A)*P(B)

6. Conditional probability

    Assume P(B)>0, then we have P(A|B)=P(A & B) / P(B)

7. Bayes' Rule

    P(A|B) = P(B|A) P(A) / P(B)

or P(A|B) = P(B|A) P(A) / (P(B|A)P(A) + P(B|~A)P(~A))

8. Odds

Odds = P(A)/P(~A)

Odds Ratio=Odds for group 1 / Odds for group 2

Risk Ratio = P(D|E)/P(D|~E)

For rare outcome, OR =similar RR. For common outcome, OR>RR if RR>1, OR<RR if RR<1

RR = OR/[(1-P0)+(P0*OR)] with P0=P(D|~E)

9. Variance

V(X+Y) = V(X) + V(Y) + 2Cov(X,Y)

If X and Y are independent, V(X+Y) = V(X) + V(Y)

Covariance: cov(X,Y)=E[(X-EX)(Y-EY)]

Sample Covariance: cov(X,Y)=sum[(xi - x_bar)(yi - y_bar)]/(n-1)

10. Multi-nomial distribution

11. Poisson Distribution

mean = lambda

variance = lambda

Poisson process (lambda as rate)

For example, lambda = 2 events per month, while t = month

12. Normal distribution

68% within 1 sigma

95% within 2 sigma

99.7% within 3 sigma

Is my data normally distributed?

1. Look at the histogram! Does it appear bell shaped?

2. Compute descriptive summary measures—are mean, median, and mode similar?

3. Do 2/3 of observations lie within 1 std dev of the mean? Do 95% of observations lie within 2 std dev of the mean?

4. Look at a normal probability plot—is it approximately linear?

5. Run tests of normality (such as Kolmogorov-Smirnov). But, be cautious, highly influenced by sample size!

The Normal Probability Plot

1. Order the data.

2. Find corresponding standardized normal quantile values.

3. Plot the observed data values against normal quantile values. 

4. Evaluate the plot for evidence of linearity.

13 Pitfalls of Hypothesis Testing

(1) over-emphasis on p-value

Clinically unimportant effects may be statistically significant if a study is large (and therefore, has a small standard error and extreme precision). 

Pay attention to effect size and confidence intervals.

(2) association does not equal causation

Statistical significance does not imply a cause-effect relationship.

Interpret results in the context of the study design.

(3) data dredging/multiple comparisons 

Multiple Testing

Treatment:

Control study-wide type I error by planning a limited number of tests. 

Distinguish between planned and exploratory tests in the results. 

Correct for multiple comparisons.

Hallmarks of a chance finding:

Analyses are exploratory

Many tests have been performed but only a few are significant

The significant p-values are modest in size (between p=0.01 and p=0.05)

The pattern of effect sizes is inconsistent

The p-values are not adjusted for multiple comparisons

(4) high type II error (low statistical power)

Studies may miss effects if they are insufficiently powered (lack precision). 

Design adequately powered studies and interpret in the context of study power if results are null.

(5) the fallacy of comparing statistical significance

“the effect was significant in the treatment group, but not significant in the control group” does not imply that the groups differ significantly"

B>A Significantly

C = A 

Does not mean B>C Significantly

14 Within Group vs. Between Group tests

15 Overview of Statistical Test

For categorical data, if sample size is small, use Fisher's exact or McNemar's exact test.

16 Correlation effect on hypothesis testing

Ignoring correlations will:

overestimate p-values for within-person or within-cluster comparisons

underestimate p-values for between-person or between-cluster comparisons