Talk:Perplexity

Article currently states:

The lowest perplexity that has been published on the Brown Corpus (1 million words of American English of varying topics and genres) as of 1992 is indeed about 247 per word

I'm really not sure why the best we could do 25 years ago is relevant here. I see perplexity of language models in English (although not on the Brown corpus) that are substantially lower than this (e.g. ~100 for recurrent neural net models). If the Brown corpus is the benchmark, what are more recent figures on that corpus? JulesH (talk) 09:17, 3 June 2017 (UTC)

I haven't seen any recent experiments on the Brown corpus, but there's no reason not to replace that with a more common benchmark. The One Billion Word Benchmark ^[1] is common now, and Exploring the Limits of Language Modeling (2016) ^[2] gets a perplexity of 23.7 using an ensemble of massive LSTM and skipgram models. I'm not aware of any better results on that set. That whole section needs to be rewritten if we switch the example, though; it refers to the Brown corpus and the 247 ppl result all over the place. 130.108.103.115 (talk) 23:01, 26 September 2017 (UTC)

The inverse of the perplexity (which, in the case of the fair k-sided die, represents the probability of guessing correctly), is 1/1.38 = 0.72, not 0.9.

This seems barmy to me. Inverse perplexity seems like a crazy thing. Perplexity is kind of like average list length in an underspecified input system. Inverse list length WTF? I can't make any useful mental picture of this. — MaxEnt 00:22, 30 March 2018 (UTC)

In point of fact, the "perplexity" in "bits" is the average number of "Yes/No" questions that one would need to ask to guess the correct word assuming that you know the vocabulary but not the word-probabilities.

To the best of my knowledge, the "inverse perplexity" doesn't mean _anything_.

(Possibly the writer was thinking about lg(1/p_i) being the "surprisal" corresponding to word "i" in the vocabulary, and got the two concepts confused. "Entropy" is then the "expected surprisal".) Speaker to Wolves (talk) 17:28, 29 November 2023 (UTC)

The addition of the sources from 14:22, 31 July 2023‎ by Intuivo is being undone for the following reasons:

• Relevance: The cited COVID-19 article discusses Perplexity in a specific context (LDA models) that doesn't align with the broader concept in the main article.
• Availability of Authoritative Sources: More suitable references that directly address Perplexity in information theory are likely available.
• Editorial Integrity: The inclusion of more relevant and direct sources would better adhere to Wikipedia's guidelines on reliable sourcing and clarity.

The undoing aims to maintain the article's focus and ensure that the sources are directly relevant to the subject of Perplexity. — Preceding unsigned comment added by Ynwaps (talk • contribs) 20:45, 24 August 2023 (UTC)

Something that might be interesting to discuss on this page is perplexity as used to guide the so-called "scaling laws" of large language models:

2020, Kaplan et al.: Scaling Laws for Neural Language Models

2022, Hoffman et al.: Training Compute-Optimal Large Language Models

It's already discussed on the linked page, so perhaps just a link from here to there would suffice. Still, interesting to see perplexity used in this way. TokenByToken (talk) 22:42, 21 June 2024 (UTC)