Probability of heads

[karthickgururaj]

You are given a coin that has a probability p for heads turning up (and (1-p) that tails will turn up). p is unknown and can be any real number in [0, 1] - all with equal likelihood.

 

You toss the coin 10 times, and 5 times heads turn up.

 

a. (easy) What is your best guess for p (and why)?

b. (tricky) What is the probability that p has the value equal to your best guess?

c. What is the probability that p = 0.5 (+/- 0.05) ?

[plasmid]

There's no need to have Bayes meddling with this problem.

If you have a coin with probability p of landing heads, then the probability of tossing H heads out of N tosses is

f(p) = p^H * (1-p)^N-H * (N choose H)

which is a polynomial in p. Since we're dealing specifically with the case of N=10 and H=5, we have

f(p) = p⁵ * (1-p)⁵ * 252

f(p) = 252 * (p⁵ - 5*p⁶ + 10*p⁷ - 10*p⁸ + 5*p⁹ - p¹⁰)

That function f(p) describes the probability that tossing a p coin 10 times will yield 5 heads. If we're given the fact that 5 heads were tossed out of 10 and we're asked to calculate the probability that p is within a range, then we can integrate f(p) and compare the value within that range to the total integral from 0 to 1.

Integral[f(p)] = 252 * (1/6 * p⁶ - 5/7 * p⁷ + 10/8 * p⁸ - 10/9 * p⁹ + 5/10 * p¹⁰ - 1/11 p¹¹)

The integral from 0.45 to 0.55 is

Integral[f(p)](0.55) - Integral[f(p)](0.45)

= 252 * (0.00461344 - 0.01087454 + 0.01046674 - 0.00511707 + 0.00126648 - 0.00012665)

- 252 * (0.00138396 - 0.00266907 + 0.00210189 - 0.00084076 + 0.00017025 - 0.00001393)

= 0.0575568 - 0.03334968

= 0.02420712

The integral from 0 to 1 is

252 * (1/6 - 5/7 + 10/8 - 10/9 + 5/10 - 1/11)

= 0.09090909

So the probability that p is between 0.45 and 0.55 is approximately

0.02420712 / 0.09090909

= 0.26627832

Well, I suppose you could say that it implicitly relies on Bayesian analysis in a way that I just don't draw attention to in this approach.

[nuurhasan]

I'd say P is 1. Since tails is 1-P, it will be zero (0) - It is logical to assume that on a coin heads and tails would be 1 and 0. 

[k-man]

p=1/2. Since from 10 tosses the coin didn't exhibit any unfairness (heads appeared the same number of times as tails) the best guess is the coin is fair. There is no reason to presume any bias in the coin due to the symmetry of the results.

 

I don't know how to calculate b) and c) yet, but my guess is that b) is no longer 0. Initially, all values of p were equally likely and any particular value of p had probability of 0. After some tosses, the probability distribution of p will take the shape of a bell curve with the mean equal to the ratio of heads to the total number of tosses. The more tosses are made, the higher the peak of the bell curve will be, but even after one toss it's no longer 0.

[bonanova]

a) .5
b) 0. p is said to be a real number, so it's non-zero only over non-zero intervals.
c) It's given by the standard error of estimate of mean (SEEM) based on ten samples. I don't recall the formula.

[karthickgururaj]

@nuurhasan: Not sure how to react to what said The question is to find the probability that a heads will turn up, but you have assigned a number (1) for heads and 0 for tails. For a fair coin, the probability is 0.5.

 

@k-man: your guess for p is right, but not the response to (b), see bonanova's response.

[karthickgururaj]

a) .5

b) 0. p is said to be a real number, so it's non-zero only over non-zero intervals.

c) It's given by the standard error of estimate of mean (SEEM) based on ten samples. I don't recall the formula.

I don't know about SEEM - will look it up. My approach was more from fundamentals.

[bonanova]

Since the interval is small and centered about the mean:

Consider only the probability density function for a fair coin.

For 10 tosses, the mean number of heads is 5, with a standard deviation of (0.25)^1/2 = 1.581.

The interval (.45, .55) describes heads-probabilities for which the expected number of heads

would be 5 +/- 0.5 = 5 +/- .316 standard deviations.

We can calculate the fraction f of values that lie within z standard deviations of the mean:

f = erf (z/sqrt(2)) = erf (.316/1.414) = 0.050405.

Since that makes it highly unlikely that 5 H in ten flips would appear for a random p value,

It hints that p would lie in the specified range with probability approximated by

Probability that p(H) = 0.5 +/- 0.5 ~ 1 - 0.050405 ~ 0.949.

[karthickgururaj]

Since the interval is small and centered about the mean:

Consider only the probability density function for a fair coin.

For 10 tosses, the mean number of heads is 5, with a standard deviation of (0.25)^1/2 = 1.581.

The interval (.45, .55) describes heads-probabilities for which the expected number of heads

would be 5 +/- 0.5 = 5 +/- .316 standard deviations.

We can calculate the fraction f of values that lie within z standard deviations of the mean:

f = erf (z/sqrt(2)) = erf (.316/1.414) = 0.050405.

Since that makes it highly unlikely that 5 H in ten flips would appear for a random p value,

It hints that p would lie in the specified range with probability approximated by

Probability that p(H) = 0.5 +/- 0.5 0.05 ~ 1 - 0.050405 ~ 0.949.

(a small correction above)

 

 

Probability that p is 0.5 +/- 0.05 ~ 0.266245

If I compute how the value changes for 100 tosses of the coin, where 50 heads turn up, it climbs up to: ~ 0.6875

 

 

..is to use conditional probability and Bayes' theorem.

Not sure about the validity, but it is simple enough to understand.

 

 

I'll post the full solution in couple of days if no one else responds.

[gavinksong]

(a) is obviously 0.5

(b) I would have to say 0.

© gah. I should probably know a more efficient way of calculating this but... say F(p) = binom(5, 10) * p^5 * (1 - p)^5. Then it's the integral of F(p) over [.45, .55] divided by the integral of F(p) over [0, 1]. Or something like that?

[gavinksong]

oh yeah, so it looks like plasmid got it before me.

I plugged in my formula, except without the binomial thing cuz you can factor that out. I got .266, same as plasmid.

[k-man]

There's no need to have Bayes meddling with this problem.

If you have a coin with probability p of landing heads, then the probability of tossing H heads out of N tosses is

f(p) = p^H * (1-p)^N-H * (N choose H)

which is a polynomial in p. Since we're dealing specifically with the case of N=10 and H=5, we have

f(p) = p⁵ * (1-p)⁵ * 252

f(p) = 252 * (p⁵ - 5*p⁶ + 10*p⁷ - 10*p⁸ + 5*p⁹ - p¹⁰)

That function f(p) describes the probability that tossing a p coin 10 times will yield 5 heads. If we're given the fact that 5 heads were tossed out of 10 and we're asked to calculate the probability that p is within a range, then we can integrate f(p) and compare the value within that range to the total integral from 0 to 1.

Integral[f(p)] = 252 * (1/6 * p⁶ - 5/7 * p⁷ + 10/8 * p⁸ - 10/9 * p⁹ + 5/10 * p¹⁰ - 1/11 p¹¹)

The integral from 0.45 to 0.55 is

Integral[f(p)](0.55) - Integral[f(p)](0.45)

= 252 * (0.00461344 - 0.01087454 + 0.01046674 - 0.00511707 + 0.00126648 - 0.00012665)

- 252 * (0.00138396 - 0.00266907 + 0.00210189 - 0.00084076 + 0.00017025 - 0.00001393)

= 0.0575568 - 0.03334968

= 0.02420712

The integral from 0 to 1 is

252 * (1/6 - 5/7 + 10/8 - 10/9 + 5/10 - 1/11)

= 0.09090909

So the probability that p is between 0.45 and 0.55 is approximately

0.02420712 / 0.09090909

= 0.26627832

Well, I suppose you could say that it implicitly relies on Bayesian analysis in a way that I just don't draw attention to in this approach.

 

Nice! However, I think either due to the precision or a mistake somewhere your result is a bit off.

 

The integral from 0.45 to 0.55 is exactly 0.0242041. The integral from 0 to 1 is exactly 1/11. The result is therefore 0.0242041*11 or exactly 0.2662451, which is very close to karthik's result, but the solution credit should go to plasmid.

[karthickgururaj]

There's no need to have Bayes meddling with this problem.

If you have a coin with probability p of landing heads, then the probability of tossing H heads out of N tosses is

f(p) = p^H * (1-p)^N-H * (N choose H)

which is a polynomial in p. Since we're dealing specifically with the case of N=10 and H=5, we have

f(p) = p⁵ * (1-p)⁵ * 252

f(p) = 252 * (p⁵ - 5*p⁶ + 10*p⁷ - 10*p⁸ + 5*p⁹ - p¹⁰)

That function f(p) describes the probability that tossing a p coin 10 times will yield 5 heads. If we're given the fact that 5 heads were tossed out of 10 and we're asked to calculate the probability that p is within a range, then we can integrate f(p) and compare the value within that range to the total integral from 0 to 1.

Integral[f(p)] = 252 * (1/6 * p⁶ - 5/7 * p⁷ + 10/8 * p⁸ - 10/9 * p⁹ + 5/10 * p¹⁰ - 1/11 p¹¹)

The integral from 0.45 to 0.55 is

Integral[f(p)](0.55) - Integral[f(p)](0.45)

= 252 * (0.00461344 - 0.01087454 + 0.01046674 - 0.00511707 + 0.00126648 - 0.00012665)

- 252 * (0.00138396 - 0.00266907 + 0.00210189 - 0.00084076 + 0.00017025 - 0.00001393)

= 0.0575568 - 0.03334968

= 0.02420712

The integral from 0 to 1 is

252 * (1/6 - 5/7 + 10/8 - 10/9 + 5/10 - 1/11)

= 0.09090909

So the probability that p is between 0.45 and 0.55 is approximately

0.02420712 / 0.09090909

= 0.26627832

Well, I suppose you could say that it implicitly relies on Bayesian analysis in a way that I just don't draw attention to in this approach.

My method is exactly the same. Both you and gavinksong came to the same conclusion, within few mins of each other.

 

..The best estimate for p has probability 0 of occurring. Of course, this is not surprising or counter-intuitive. The best estimate for p is one that has the least error..

 

@plasmid, it is Bayes' rule indirectly

 

 

Let,

 

A be the outcome that "p is between r and r+dr (where dr is an infinitesimally small increment of r and 0 < r < 1)".

B be the outcome that "5 heads turn up in 10 coin tosses".

 

Then the a priori probability p(A) (probability of A, given no other data) is nothing but 'dr'. Let a priori probability of B be p(B). This is the probability that B will occur ("5 heads turn up") , when we do not know what is p (the probability of heads). We may not know what is p(B), but we can readily agree that p(B) will not depend on r.

 

We also have,

p(B|A) = Probability of B occurring, given that A has occurred = ¹⁰C₅.r⁵.(1-r)⁵

 

By Bayes theorem,

 

p(A|B) = Probability of A occurring, given that B has occurred

     = p(B|A). p(A)/p(B)

     = ¹⁰C₅.r⁵.(1-r)⁵.dr/p(B)

 

Now, if we integrate p(A|B) over the interval (0, 1), we must get "1" - i.e, it is a certainty that r is in the range (0, 1). Using this, we can calculate p(B) = Integral[¹⁰C₅.r⁵.(1-r)⁵.dr] over r = 0..1.

 

Plugging this back in the equation above, and again integrating over the interval [0.45, 0.55], we get the desired answer, which both plasmid and gavinksong have provided already.

 

Now, why I described all this is, I found how p(B) becomes Integral[¹⁰C₅.r⁵.(1-r)⁵.dr] over r = 0..1 to be interesting!

 

It is like, I have this question - what is the chance of rain for next five days, if you do not know what is the "probability that it will rain on any given day"? The question seems slightly absurd, but has a perfectly valid expression.

 

Of course, if we think about the question again, we arrive at the same result directly.. But, I just found it to be interesting