# Bayesian Statistics (a very brief introduction)

Download Bayesian Statistics (a very brief introduction)

## Preview text

Bayesian Statistics (a very brief introduction)

Ken Rice Epi 516, Biost 520

1.30pm, T478, April 4, 2018

Overview

Rather than trying to cram a PhD’sworth of material into 90 minutes...

• What is Bayes’ Rule, a.k.a. Bayes’ Theorem? • What is Bayesian inference? • Where can Bayesian inference be helpful? • How, if at all, is it diﬀerent to frequentist inference?

Note: the literature contains many pro- and anti-Bayesian polemics, many of which are ill-informed and unhelpful. I will try not to rant, and aim to be accurate.

Further Note: There will, unavoidably, be some discussion of epistemology, i.e. philosophy concerned with the nature and scope of knowledge. But...

1

Overview

Using a spade for some jobs and shovel for others does not require you to sign up to a lifetime of using only Spadian or Shovelist philosophy, or to believing that only spades or only shovels represent the One True Path to garden neatness.

There are diﬀerent ways of tackling statistical problems, too. 2

Bayes’ Theorem

Before we get to inference: Bayes’ Theorem is a result in conditional probability, stating that for two events A and B...

P[ A|B ] = P[ A and B ] = P[ B|A ]P[ A ].

P[ B ]

P[ B ]

In this example; • P[ A|B ] = 13//1100 = 1/3 • P[ B|A ] = 15//1100 = 1/5 • And 1/3 = 1/5 × 53//1100 ( )

In words: the conditional probability of A given B is the conditional probability of B given A scaled by the relative probability of A compared to B.

3

Bayes’ Theorem

Why does it matter? If 1% of a population have cancer, for a screening test with 80% sensitivity and 95% speciﬁcity;

Have Cancer

Test Positive

P[ Test +ve|Cancer ] = 80% P[ Test +ve] = 5.75 P[ Cancer ]

P[ Cancer|Test +ve ] ≈ 14%

... i.e. most positive results are actually false alarms

Mixing up P[ A|B ] with P[ B|A ] is the Prosecutor’s Fallacy; a small probability of evidence given innocence need NOT mean a small probability of innocence given evidence.

4

Bayes’ Theorem: Sally Clark

• After the sudden death of two baby sons, Sally Clark (above, center) was sentenced to life in prison in 1999

• Among other errors, expert witness Prof Roy Meadow (above right) had wrongly interpreted the small probability of two cot deaths as a small probability of Clark’s innocence

• After a long campaign, including refutation of Meadow’s statistics, Clark was released and cleared in 2003

• After being freed, she developed alcoholism and died in 2007 5

Bayes’ Theorem: XKCD at the beach

This is roughly equal to # of times I’ve picked up a seashell at the ocean , # of times I’ve picked up a seashell

...which in my case is pretty close to 1, and gets much closer if we’re considering only times I didn’t put it to my ear.

6

Bayes’ Theorem

Bayes’ theorem also applies to continuous variables – say systolic and diastolic blood pressure;

The conditional densities of the random variables are related this way;

f (x) f (x|y) = f (y|x)

f (y) ...which we can write as

f (x|y) ∝ f (y|x)f (x). This proportionality statement is just a re-wording of Bayes’ Theorem.

Note: Like probabilities, densities are ≥ 0, and ‘add up to 1’. 7

Bayesian inference

So far, nothing’s controversial; Bayes’ Theorem is a rule about the ‘language’ of probabilities, that can be used in any analysis describing random variables, i.e. any data analysis.

Q. So why all the fuss? A. Bayesian inference uses more than just Bayes’ Theorem

In addition to describing random variables, Bayesian inference uses the ‘language’ of

probability to describe what is known about parameters.

Note: Frequentist inference, e.g. using p-values & conﬁdence intervals, does not quantify what is known about parameters.∗

*many people initially think it does; an important job for instructors of intro

Stat/Biostat courses is convincing those people that they are wrong.

8

Freq’ist inference (I know, shoot me!)

Frequentist inference, set all a-quiver;

Adapted from Gonick & Smith, The Cartoon Guide to Statistics

9

Ken Rice Epi 516, Biost 520

1.30pm, T478, April 4, 2018

Overview

Rather than trying to cram a PhD’sworth of material into 90 minutes...

• What is Bayes’ Rule, a.k.a. Bayes’ Theorem? • What is Bayesian inference? • Where can Bayesian inference be helpful? • How, if at all, is it diﬀerent to frequentist inference?

Note: the literature contains many pro- and anti-Bayesian polemics, many of which are ill-informed and unhelpful. I will try not to rant, and aim to be accurate.

Further Note: There will, unavoidably, be some discussion of epistemology, i.e. philosophy concerned with the nature and scope of knowledge. But...

1

Overview

Using a spade for some jobs and shovel for others does not require you to sign up to a lifetime of using only Spadian or Shovelist philosophy, or to believing that only spades or only shovels represent the One True Path to garden neatness.

There are diﬀerent ways of tackling statistical problems, too. 2

Bayes’ Theorem

Before we get to inference: Bayes’ Theorem is a result in conditional probability, stating that for two events A and B...

P[ A|B ] = P[ A and B ] = P[ B|A ]P[ A ].

P[ B ]

P[ B ]

In this example; • P[ A|B ] = 13//1100 = 1/3 • P[ B|A ] = 15//1100 = 1/5 • And 1/3 = 1/5 × 53//1100 ( )

In words: the conditional probability of A given B is the conditional probability of B given A scaled by the relative probability of A compared to B.

3

Bayes’ Theorem

Why does it matter? If 1% of a population have cancer, for a screening test with 80% sensitivity and 95% speciﬁcity;

Have Cancer

Test Positive

P[ Test +ve|Cancer ] = 80% P[ Test +ve] = 5.75 P[ Cancer ]

P[ Cancer|Test +ve ] ≈ 14%

... i.e. most positive results are actually false alarms

Mixing up P[ A|B ] with P[ B|A ] is the Prosecutor’s Fallacy; a small probability of evidence given innocence need NOT mean a small probability of innocence given evidence.

4

Bayes’ Theorem: Sally Clark

• After the sudden death of two baby sons, Sally Clark (above, center) was sentenced to life in prison in 1999

• Among other errors, expert witness Prof Roy Meadow (above right) had wrongly interpreted the small probability of two cot deaths as a small probability of Clark’s innocence

• After a long campaign, including refutation of Meadow’s statistics, Clark was released and cleared in 2003

• After being freed, she developed alcoholism and died in 2007 5

Bayes’ Theorem: XKCD at the beach

This is roughly equal to # of times I’ve picked up a seashell at the ocean , # of times I’ve picked up a seashell

...which in my case is pretty close to 1, and gets much closer if we’re considering only times I didn’t put it to my ear.

6

Bayes’ Theorem

Bayes’ theorem also applies to continuous variables – say systolic and diastolic blood pressure;

The conditional densities of the random variables are related this way;

f (x) f (x|y) = f (y|x)

f (y) ...which we can write as

f (x|y) ∝ f (y|x)f (x). This proportionality statement is just a re-wording of Bayes’ Theorem.

Note: Like probabilities, densities are ≥ 0, and ‘add up to 1’. 7

Bayesian inference

So far, nothing’s controversial; Bayes’ Theorem is a rule about the ‘language’ of probabilities, that can be used in any analysis describing random variables, i.e. any data analysis.

Q. So why all the fuss? A. Bayesian inference uses more than just Bayes’ Theorem

In addition to describing random variables, Bayesian inference uses the ‘language’ of

probability to describe what is known about parameters.

Note: Frequentist inference, e.g. using p-values & conﬁdence intervals, does not quantify what is known about parameters.∗

*many people initially think it does; an important job for instructors of intro

Stat/Biostat courses is convincing those people that they are wrong.

8

Freq’ist inference (I know, shoot me!)

Frequentist inference, set all a-quiver;

Adapted from Gonick & Smith, The Cartoon Guide to Statistics

9

## Categories

## You my also like

### Climbing the Hills of Compiled Credal Networks

1.3 MB6.5K2.2K### Bayesian Statistics for Genetics Lecture 1: Introduction

6.2 MB31K5K### Bayesian Based Neural Network Model for Solar Photovoltaic

362.9 KB24.9K9K### Probability, Statistics, and Stochastic Processes

4 MB20.6K7.6K### Lecture 17: What is Statistical Inference?

399 KB9K1.3K### The Objectivity of Subjective Bayesian Inference

124.6 KB7K2.1K### On the Differential Privacy of Bayesian Inference

715.2 KB8.4K3.4K### Design and implementation of a database inference controller

1.8 MB11.1K1.6K### CSE 20: Discrete Mathematics

174.9 KB6K2.9K