Talk:Confidence interval

I have removed some inaccurate or misleading statements from recent edits. Please do not edit this article unless you understand the difference between a confidence interval and a credible interval. Editors of this article have been working for several years to remove inaccurate statements about the interpretatation of a confidence interval, but people continue to put them back in.

As the article correctly states, when we speak of a 95% confidence interval, the 95% probability relates to the probability of repeated samples yielding intervals that cover the parameter. It is not correct in general that for a given realized interval that this interval has a 95% probability of covering the parameter. In fact, it is straightforward to construct examples where the probability that a given interval covers a parameter is very different from the confidence level.

It would be a good idea to read these articles before making edits relating to the interpretation of confidence intervals.

Morey, Richard D.; Hoekstra, Rink; Rouder, Jeffrey N.; Lee, Michael D.; Wagenmakers, Eric-Jan (2016). "The fallacy of placing confidence in confidence intervals". Psychonomic Bulletin & Review. 23 (1): 103–123. doi:10.3758/s13423-015-0947-8. https://link.springer.com/article/10.3758/s13423-015-0947-8 PMC 4742505. PMID 26450628.

Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E. J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 21(5), 1157–1164. https://link.springer.com/article/10.3758/s13423-013-0572-3

Morey, R., Hoekstra, R. D., Rouder, J. N., Wagenmakers, E. J. (2015). Continued misinterpretation of confidence intervals: response to Miller and Ulrich. Psychon Bull Rev. 2015 Nov 30;23:131–140. doi: 10.3758/s13423-015-0955-8 https://link.springer.com/article/10.3758/s13423-015-0955-8 Dezaxa (talk) 12:23, 13 April 2025 (UTC)

I'm unsure of how helpful the weather forecast analogy—added in this edit by FRuDIxAFLG—given in § Common misunderstandings is. The article currently reads:

A 95% confidence level does not mean that for a given realized interval there is a 95% probability that the population parameter lies within the interval (i.e., a 95% probability that the interval covers the population parameter). This distinction can be understood by analogy: If the weather forecast is accurate 95% of the time, it does not follow that it is accurate today with 95% probability. For instance, maybe forecasts of sun are 99% accurate and forecasts of rain are 80% accurate. Then, having seen today's forecast, one would be either 99% or 80% confident, but never 95% confident.

Does this not miss the point? The statement having seen today's forecast, one would be either 99% or 80% confident, but never 95% confident seems rather misleading to me: on the day for which the weather is forecast, we can simply observe the actual weather. At that point, it is no longer a question of probability or confidence, but of fact: the prediction is either right, or it isn't.

Even if the analogy did map cleanly onto confidence intervals (which I'm not sure it does), it seems to suggest that we can assign a probability to whether a specific realised interval contains the true parameter value, which is precisely the misconception it is trying to correct. Pink Bee (talk) 14:52, 16 April 2025 (UTC)

If this particular analogy isn't to everyone's liking then we can change it. But clearly something is needed to help people understand what is going on here. This talk page is evidence of that.

Anyway, of course you can assign a probability to whether the parameter is in a particular interval; you just need to use a Bayesian approach. That's a weird thing to do, but who cares if it's weird?

FRuDIxAFLG (talk) 17:38, 16 April 2025 (UTC)

I think we should probably avoid weirdness in a section about common misunderstandings, in favour of giving the simplest possible explanation. The primary audience for this section is likely to be people who have these misconceptions, and people who can appreciate the differences between Bayesian and frequentist perspectives may not be well-represented in that group.

I agree that an example or analogy or some other intuitive explanation is necessary here. I like the example given in this source under "Common misunderstanding of the true meaning of confidence intervals". Maybe we could do something along those lines—what do you think? Pink Bee (talk) 18:47, 16 April 2025 (UTC)

I'm inclined to agree that the example is not particularly helpful. The fundamental point is that a confidence interval is a frequentist concept. As such, any use of the term probability must be understood as a long-run frequency, and a frequency requires a reference class. In the case of confidence intervals, the reference class is the class of intervals that would be obtained from repeated experiments. Once a single experiment has been performed and a particular realized interval has been calculated, that interval can be thought of as belonging to many different reference classes, which is why it is potentially misleading to think of it as having a probability of covering the parameter of interest. It is a probability only in the specific sense that it is drawn from a reference class of intervals from hypothetical repeated experiments. Dezaxa (talk) 10:14, 18 April 2025 (UTC)

I have replaced the example with one based on that in the source I mentioned here. I've made it as approachable as I can, but I'd appreciate it if someone with more statistics knowledge than me could check and edit it as needed to make sure it's still technically accurate. Pink Bee (talk) 20:24, 20 April 2025 (UTC)

My only problem with the example is it's too easy. Usually we don't know the true mean, but even then the usual misconceptions can be dangerous.

FRuDIxAFLG (talk) 21:32, 20 April 2025 (UTC)

I take your point, and I agree that if the factory premise were the setup for an exercise, giving the true mean would make it pointless. But this is an example, not an exercise, and I don't see how we can demonstrate the fallacy quite as simply without just giving specific values. The risk of saying "if the true mean is x then ___, or if it is y then ___" after giving a specific CI is that it permits the misunderstanding that the population parameter is somehow random, which would justify making probability statements about it.

That said, this is WP:NOTTEXTBOOK, so maybe I should stop worrying about how best to teach this. I have removed the true mean for now. Pink Bee (talk) 23:07, 20 April 2025 (UTC)

This is a response to the edit by Cheetah17 (talk · contribs). I disagree with this edit for several reasons:

• The phrase "likely to contain the true value" comes with a pile of asterisks. I don't want to be pedantic about it, but I think the older phrase "used to estimate" strikes a good balance between brevity and correctness.

• I don't like putting "frequentist" and "Bayesian" in the introduction. I think the introduction should be as short as possible while summarizing the most important ideas in the article. Maybe we could say "you might want a credible interval instead," but anything more would be too much.

• In the same vein, the "true population mean as a fixed unknown constant" belongs far away from the introduction. Nobody needs to understand this in order to use (or interpret!) confidence intervals.

I'd like to change the introduction back. If anyone disagrees or has other suggestions, this is a good time to speak up.

FRuDIxAFLG (talk) 18:21, 26 December 2025 (UTC)

Thank you for your review of my contribution. The reason I edited the previous version, is because I found it lacking sufficient context and precision. The introduction of Wikipedia article is the most read section of many pages, hence it is essential to provide sufficient context to properly frame the topic of discussion. The previous version of the introduction lacked reference to frequentist and bayesian frameworks, and a comparison to the credible interval, which is often confused with "confidence interval". Therefore, I found it a improvement to make this known from the start.

Regarding your comment on the merit of "a confidence interval (CI) is a range of values which is likely to contain the true value of an unknown statistical parameter", do you believe the sentence to be wrong? If so, under which circumstances? ~2025-43039-86 (talk) 02:27, 27 December 2025 (UTC)

Thank you for your review of my contribution. The reason I edited the previous version is because I found it lacking sufficient context and precision. The introduction of Wikipedia article is the most read section of many pages, hence it is essential to provide sufficient context to properly frame the topic of discussion. The previous version of the introduction lacked reference to frequentist and bayesian frameworks, and a comparison to the credible interval, which is often confused with "confidence interval". Therefore, I found it a improvement to make this known from the start.

Regarding your comment on the merit of "a confidence interval (CI) is a range of values which is likely to contain the true value of an unknown statistical parameter", do you believe the sentence to be wrong? If so, under which circumstances? Cheetah17 (talk) 02:28, 27 December 2025 (UTC)

Yes, in general that sentence will be wrong. A confidence interval procedure can result in an interval where you know (afterward) that it isn't likely to contain the parameter. The easiest example is the trivial 50% procedure that returns either all real numbers or an empty interval. When your interval is empty, obviously you know it won't contain the parameter, but it's still a 50% confidence interval because of "the long-run reliability of the method used to generate the interval," as explained in the article. The Welch interval, also mentioned in the article, is another good example. So "a confidence interval is likely to contain the true value" is improper, whereas "a confidence interval is the result of a procedure that usually produces intervals that contain the true value" would be fine. To avoid such a circumlocution, I think it's better to simply to say an interval is an estimate, then explain further in another part of the article.

As for the rest of the introduction, I see no reason to compare frequentist and Bayesian methods there. If someone wants to learn about that from Wikipedia then there are other pages that discuss it. If someone comes to the confidence interval article then I'm assuming they want to know what is a confidence interval. They might be a student or data analyst who is working on a problem. This sort of person would not benefit at all from learning about frequentists vs Bayesians. What they need is practical advice, simple examples, clear warnings about common pitfalls, and perhaps the suggestion that they try a credible interval instead. Even then you don't have to tell them credible intervals are Bayesian, rather that they're an alternative procedure with different properties.

FRuDIxAFLG (talk) 03:12, 28 December 2025 (UTC)

I have some serious concerns regarding the definition of 'confidence interval' presented in this article. I’ve noticed that the user @FRuDIxAFLG is doing an excellent job improving the entry. However, I disagree with the edit he made on September 28, 2025. While I agree that treating a parameter as a random variable is a matter of interpretation (and also that a probability cannot be 'equal' to a confidence interval), I disagree with the specific change to the article. I checked the Dekking textbook (which I studied from a some years ago), and on page 342, he uses the following phrase to explain that the parameter value is fixed, while it is the identified interval that varies from one experiment to another: '...if I am near the city of Paris, then the city of Paris is near me: the statement T is within 200 of θ is the same as θ is within 200 of T...' From what I read, a 95% confidence interval implies a 95% probability that the parameter falls within that interval (although it is not defined this way). What Dekking wants to clarify is that the parameter value itself is not a random variable; rather, the estimate and its corresponding interval are. Furthermore, I am not certain that such a probability is exclusively associated with the credible interval in Bayesian inference. Therefore, I suggest changing the sentence in the main article from '...A 95% confidence level does not imply a 95% probability...' to its previous version: '...A 95% confidence level is not defined as a 95% probability...' This should make it clearer to the reader that a 95% confidence level is not defined as the probability of the parameter being inside it with a 95% chance, but rather as the 95% probability that the interval contains the parameter. I would appreciate it if someone could clarify if I have misunderstood anything. Otherwise, I would expect a more experienced editor to implement the change I’ve suggested. Thanks, everyone. ~2026-72470 (talk) 18:18, 4 January 2026 (UTC)

Dekking goes on to say:

Evaluating ${\displaystyle T}$ for the Michelson data we find as its realization ${\displaystyle t=299852.4}$, and this yields the statement ${\displaystyle \theta \in (299652.4,300052.4)}$. (23.2) Because we substituted the realization for the random variable, we cannot claim that (23.2) holds with probability at least 75%: either the true speed of light ${\displaystyle \theta }$ belongs to the interval or it does not; the statement we make is either true or false, we just do not know which. However, because the procedure guarantees a probability of at least 75% of getting a "right" statement, we say: ${\displaystyle \theta \in (299652.4,300052.4)}$ with confidence at least 75%.

So: the procedure results in an interval containing the true speed 75% of the time, but it does not follow that a particular interval has a 75% chance of containing the true speed. You can make 50% confidence intervals by flipping a coin and choosing either ${\displaystyle \mathbb {R} }$ or ${\displaystyle \varnothing }$ as your interval, but once you've made your choice you'll be certain whether the interval contains the parameter or not. The introduction could perhaps make this distinction clearer.

Why Dekking added that last sentence is a mystery to me; I can't imagine anything more confusing for a statistics student.

FRuDIxAFLG (talk) 21:34, 4 January 2026 (UTC)

Dear @FRuDIxAFLG, thank you for the clarification. I fully agree with you. Re-reading the text I commented on, I realize that, so as written, it is somewhat confusing. In this context, the true mean is a fixed value and not a random variable. In the sentence '...A 95% confidence level does not imply a 95% probability that the true parameter lies within a particular calculated interval...', the reference is to a 'particular calculated interval', which is a realization of an estimator; therefore, the interval itself (calculated based on an observed sample) is not a random variable either. In other words, once the sample is observed, we can no longer talk in terms of probability.

I think this concept makes it quite complicated for a non-specialized reader seeking an introduction to confidence intervals. I would suggest simplifying the explanation by referring to the estimator (viewed as a random variable) rather than the estimate obtained from an observed sample. I see that the concept I mentioned above is addressed in the 'Interpretation' subsection, and the comparison with credible intervals is covered in the 'Comparison with credible intervals' subsection.

Therefore, if you agree, I would suggest removing these concepts from the introduction and stating more simply: 'A 95% confidence level indicates that the interval, viewed as an estimator, will contain the true parameter with a 95% probability,' replacing the text from 'A 95% confidence...' to '...bayesian inference.' I am not sure if this is sufficiently clear and correct, but I wanted to provide a proposal to suggest a general direction. Thanks again. ~2026-72470 (talk) 23:15, 4 January 2026 (UTC)