Topics that are more on the inference side of things than algorithmic

My (applied computational) statistics professor straight up told me that "nobody reads" those theoretical/mathematical papers published in journals such as Annals of Statistics, Annals of Probability, etc.

Is that true? I mean, I'm sure there is some nuance, and he is being a bit biased, but is it true that theoretical/mathematical statistics papers are barely read? If so, then how are these papers getting the funding to be pursued in the first place?

Been told by multiple people that this is the case.

They say that nothing new is coming out basically and it's a dying field of research.

Do you agree?

Should I reconsider specialising in time series analysis for my honours year/PhD?

Benjamin-Hochberg corrections have been mathematically proved to show the standard Benjamini-Hochberg procedure can fail to control the false discovery rate for two-sided tests when the underlying test statistics follow a correlated multivariate Gaussian distribution.

EDIT: The proof was obtained by GPT-5.6 Pro. The model was asked directly to prove or disprove the conjecture and was provided only with the mathematical definition of the Benjamini–Hochberg procedure. After about 90 minutes of reasoning, the model produced a proof, an example, and code for the numerical certificate, which form the basis of this paper. The author carefully checked the entire argument and the associated numerical certificate. Subsequently, the author asked the model to provide additional simulations, related work, and illustrations for a paper draft, and wrote the final version by editing the AI-generated draft.

More info below:

https://faculty.wharton.upenn.edu/wp-content/uploads/2017/06/bh.pdf

Full paper: https://arxiv.org/abs/2603.12288

This paper attempts to provide a formal explanation for a modern paradox in tabular ML — why do highly flexible models sometimes achieve state-of-the-art performance on high-dimensional, collinear, error-prone data that the dominant paradigm (Garbage in, Garbage Out / GIGO) says should produce inaccurate predictions?

It was discussed previously on r/MachineLearning from a ML theory perspective and crossposted here. Tailored to the ML community, that post focused on the information-theoretic proofs and the connection to Benign Overfitting. As the first author, I'm posting here separately because r/statistics deserves a different conversation. Not a rehash of the ML discussion but a new engagement with what I think this community will find most significant about the work.

The argument I want to make to this community specifically:

Modern machine learning has produced remarkable empirical results. It has also produced a field that, in its rush toward architectural innovation and benchmark performance, has sometimes lost contact with the theoretical traditions that were quietly working on its foundational problems decades before deep learning existed.

The paper is, among other things, an argument that classical quantitative fields (e.g., statistics, psychometrics, measurement theory, information theory) were not made obsolete by the ML revolution. They were bypassed by it. And that bypass has had real costs in how the ML community understands its own successes and failures.

One specific instance of this is the paradox stated above... which lacks a comprehensively satisfying explanation within ML's own theoretical framework.

At a high level, the paper argues that the explanation was always available in the classical statistical tradition. It just wasn't being looked for there.

What the paper does:

The framework formalizes a data-generating structure that classical statistics and psychometrics would immediately recognize:

Y ← S⁽¹⁾ → S⁽²⁾ → S'⁽²⁾

Unobservable latent states S⁽¹⁾ drive both the outcome Y and the observable predictor variables S'⁽²⁾ through a two-stage stochastic process. This is the latent factor model. Spearman formalized it in 1904. Thurstone extended it in 1947. The IRT tradition developed it rigorously for the next seventy years. Every statistician trained in psychometrics, educational measurement, or structural equation modeling knows this structure and its properties intimately.

What the paper adds is a formal information-theoretic treatment of the predictive consequences of this structure... specifically, what it implies for the limits of different data quality improvement strategies.

The proof partitions predictor-space noise into two formally distinct components:

Predictor Error: observational discrepancy between true and measured predictor values. This is classical measurement error. The statistics literature has a rich treatment of it — attenuation bias, errors-in-variables models, reliability coefficients, the Spearman-Brown prophecy formula. Cleaning strategies, repeated measurement, and instrumental variables approaches address this type of noise. The statistical tradition has been handling Predictor Error rigorously for a century.

Structural Uncertainty: the irreducible ambiguity that remains even with perfect measurement of a fixed predictor set, arising from the probabilistic nature of the S⁽¹⁾ → S⁽²⁾ generative mapping. Even a perfectly measured set of indicators cannot fully identify the underlying latent states if the set is structurally incomplete. A patient's billing codes are imperfect proxies of their underlying physiology regardless of how accurately those codes are recorded. A firm's observable financial metrics are imperfect proxies of its underlying economic state regardless of measurement precision. This is not measurement error. It is an information deficit inherent in the architecture of the indicator set itself.

The paper shows that Depth strategies — improving measurement fidelity for a fixed indicator set — are bounded by Structural Uncertainty. On the other hand, breadth strategies — expanding the indicator set with distinct proxies of the same latent states — asymptotically overcome both noise types.

This is the heart of the formal explanation offered for the ML paradox. And every element of it — the latent factor structure, the Local Independence assumption, the distinction between measurement error and structural incompleteness — comes directly from the classical statistical and psychometric tradition.

The connection to classical statistics that the ML community missed:

The ML community's dominant pre-processing paradigm — aggressive data cleaning, dimensionality reduction, penalization of collinearity — emerged from a period when the dominant modeling tools genuinely couldn't handle high-dimensional correlated data. The prescription was practically correct given those constraints. But it was theoretically incomplete because it conflated Predictor Error and Structural Uncertainty into a single undifferentiated noise concept and mainly prescribed a single solution (data cleaning) that only addresses one of them.

The statistical tradition never made this conflation. Reliability theory distinguishes between measurement error and construct coverage. Validity theory asks whether an indicator set captures the full latent construct or only part of it — which is precisely the Structural Uncertainty question in different language. The concept of a measurement instrument's comprehensive coverage of the latent domain is foundational to psychometrics and educational measurement in ways that ML's data quality frameworks simply don't have an equivalent for.

The framework is, in a sense, the formalization of what a broadly-trained statistician or psychometrician may tell an ML practitioner if they are in the room when the GIGO paradigm is being applied to high dimensional, tabular, real-world data: your data quality framework is incomplete because it doesn't distinguish between measurement error and structural incompleteness, and conflating them leads to the wrong prescription in high-dimensional latent-structure contexts.

The relevance argument stated directly:

The ML community has produced impressive modeling tools. Generally, it has not always produced a comparably impressive theoretical understanding of when and why those tools work. The theoretical explanations that do exist treat the data distribution as a fixed input and focus on model and algorithm properties. They are largely silent on the question of what properties of the data-generating structure enable or prevent robust prediction.

Classical statistics, particularly the latent variable modeling tradition, the measurement theory tradition, and the information-theoretic foundations that statisticians like Shannon developed, has been thinking carefully about data-generating structures for decades. The paper argues that this tradition contains the theoretical machinery needed to answer the questions that ML's own theoretical framework struggles with.

This is not an argument that classical statistics is better than modern ML. It is an argument that the two traditions are complementary in ways that have not been recognized. That the path toward a more complete theoretical understanding of modern ML runs through classical statistical foundations rather than away from them.

What it is not claiming:

The paper is not an argument that data cleaning is always wrong or that the GIGO paradigm is universally false. The paper provides a principled boundary delineating when traditional data quality focus remains distinctly powerful, specifically when Predictor Error rather than Structural Uncertainty is the binding constraint, and when Common Method Variance creates specific risks that only outcome variable cleaning can fully address. The scope conditions matter and the paper is explicit about them.

What I'd most value from this community:

The ML community's engagement with the paper has focused primarily on the Benign Overfitting connection and the practical feature selection implications. Both are legitimate entry points.

But this community is better positioned than any other to evaluate the deeper claim:

• Whether the classical measurement and latent factor traditions contain the theoretical foundations that ML's tabular data quality framework is missing, and whether the framework correctly formalizes that connection.

I'd particularly welcome perspectives from statisticians who have thought about the relationship between measurement theory and prediction, the information-theoretic limits of latent variable recovery, or the validity framework's implications for predictor set architecture.

Critical engagement with whether the classical connections are as deep as the paper claims is more valuable than general reception.

Say you are developing some sort of statistical test and you have run simulations and applications, and now you wanna work on its theoretical guarantees (asymptotics, efficiency, consistency, etc.)

Can you not just feed in the test statistics, assumptions, etc. to your favorite intelligent AI and get it to derive everything?

In this instance, is the focus of statistical theory going to be more on the algorithms/computation side and dealing with issues like scalability going forward, while leaving the math to AI?

Didn't AI solve that big unsolved problem in pure mathematics the other day (Navier-stokes or something, I think)

As an honours year (Australia) student I am doing a methodological statistics research topic relating to multivariate time series under structural breaks.

My literature review involves reading, analyzing, and synthesizing the literature regarding all things time series under structural breaks. The issue is that a lot of the papers use methods that I haven't formally learned, so I am reading these papers just barely able to scrape by in terms of understanding.

Then I have to synthesize all of these somewhat-understandable papers and discuss my potential contribution with my topic. How can I do this when the literature is full of methods and lingo that I have little understanding of?

Thankfully I have a full year to execute this project so plenty of time to understand everything fully, but how do I even get started?

Context: I have to send my final dissertation manuscript to my committee tomorrow. My chair called me this morning and told me my stats are all wrong because I ran too many tests with too small of a sample (10 participants). When I asked for guidance on resolving this, they said "You want a PhD? Figure it out". I have spent all morning reading resources and I have gotten myself more confused with each one.

This is a single group pre-post design based on feasibility and exploring preliminary clinical outcomes. I originally had planned paired samples t-tests, but I ended up with such a small sample that it would inflate error rate. Also because of the small sample, any assumptions of underlying distributions can't be met, so I'm using exact sign tests. I modeled my stats after another published pilot study with 7 participants and 15 comparisons. They do not do a correction.

I ran the 7 exact sign tests for my outcome variables, found significance in 6/7, and also presented the data visually. If I did a Bonferroni correction to control for multiple comparisons, my new p-value would be .007 and basically impossible to get with exact sign tests. Is my only option to only present stats for my main variable, and only present descriptive/visuals for the rest?

The aim of my thesis is to examine the determinants of Olympic medal performance across countries.

Specifically number of athletes, GDP, GDP per capita, HDI, Population, Inflation, Urbanisation, Unemployment, country size , host dummy (if they ever organized an olympics) and democracy index as explanatory variables.

Going through the material of my econometics class, I performed a Wald-test in GRETL using OLS with robust standard errors (HC1), and it left me with nr of athletes, GDP, country size ( square meters) and democracy index using a 10% significance level.

Then I performed a Ramsey RESET Test but the results did not indicate significant misspecification. Still, when trying to make scatter or residual plots, there’s barely any linearity for democracy and country size.

There’s heteroskedasticity (I am using robust standard errors), and the distribution of the olympic medals is not normal ( though my sample is quite big, 125 countries, including those who haven’t won any medals in the year 2021.)

Is my method completely wrong, as in using OLS for this

For my undergraduate reaearch project, I was thinking of doing something ambitious.

Model averaging has been shown to decrease the overall variance of forecasts while retaining low bias.

Since bayesian and frequentist methods each have their own strengths and weaknesses, could averaging the forecasts of both types of models provide even more accurate forecasts?

Study says 25% patients reported something, but n=6

Help me understand who is wrong here, me or the author of this abstract yet to be presented in an academic event

They performed a surgery in 6 patients.

After that, 25% reported one thing, and 75% reported another almost unrelated thing. Is this possible? I'd expect the numbers should be 16% or 33% for 1/6 or 2/6 patients reporting that. And 66% or 87% for 4/6 or 5/6.

I don't think each patient can have half a success. Either they reported that thing or they didn't.

But to get 25% makes me think they only considered 4 patients, for some reason, and 1/4 reported that. Is there some statistics that can explain the 25% figure?

Here's the abstract, including nsfw diagrams: https://www.auajournals.org/doi/10.1097/01.JU.0001191384.77563.6d.19

Theme is somewhat funny but the math is what got me.

Edit: nsfw warning

I cannot find any super highly ranked journals in this niche of computational (nonparametric) statistics, where you are developing algorithms and showing their good theoretical properties via simulations (which is what my professor is doing).

Relevant topics in this niche include the backfitting algorithm, bootstrap, monte carlo simulations, EM algorithm. All are simulation based instead of mathematical (for example, you prove the size and power of a proposed test via simulations instead of closed-form mathematical proofs).

All the relevant journals seem lowly ranked (communications in statistics - simulation and compution, journal of statistical computation and simulation) and the top ones (journal of computational and graphical statistics, JASA, computational statistics and data analysis) all have papers with mathematical proofs instead of purely algorithmic development and simulation.

Am I missing something here? My professor tells me computational statistics (this version) is much more lucrative than mathematical statistics, but the evidence doesn't seem to indicate so? The higher the journal the more mathematical it is, is what I'm noticing.

[research]

I have an M.Pharm in Pharmaceutics, and recently I’ve started developing a strong interest in statistics and quantitative methods.

My academic background is primarily in pharmaceutics, so I’m still relatively new to statistics. I started learning statistical concepts through YouTube, Reddit, and online resources, and I’ve become increasingly interested in understanding not just how to use statistical tests, but how statistics can actually be used to solve real problems in pharmaceutical research.

During my M.Pharm work, I have already had some exposure to statistical methods. For example, I have used Design of Experiments (DoE) for formulation optimization and statistical approaches for evaluating/validating pharmaceutical formulations and processes.

However, I don't want my statistical knowledge to remain limited to things like “which statistical test should I use for this dataset?” I’m interested in going deeper and eventually using statistics/computational methods as a tool for pharmaceutics research, formulation development, drug delivery, and potentially drug discovery.

I’m currently thinking about pursuing a PhD in Pharmaceutics/Pharmaceutical Research, and I would ideally like my PhD to remain strongly rooted in experimental pharmaceutics while incorporating meaningful statistical or computational components.

For example, I’m curious about areas such as:

DoE and QbD for formulation and process optimization

Statistical modelling of drug release and dissolution

Pharmacokinetic/pharmacodynamic modelling

Predictive modelling for formulation development

Machine learning/AI in drug delivery or formulation development

Biostatistics applied to pharmaceutical experiments

Multivariate analysis and process monitoring

Process analytical technology (PAT) and data-driven manufacturing

Stability modelling and shelf-life prediction

Statistical approaches to drug discovery and development

Population-based modelling or PBPK

Using experimental data + computational models to guide formulation development

I’m particularly interested in the idea of having a 75%+ pharmaceutics/experimental component, while using statistics/computational methods as a powerful supporting component rather than completely moving into a pure computer science or mathematics PhD..

Give me your insights please

I have size data for multiple species/populations of the same genus. I want to compare the mean size of those with the mean size of some fossil populations to see if the fossils have a significantly different size to modern populations, and if so, to which.

The problem is, most samples are not normally distributed, and the data is heteroscedastic, since some have small sample sizes like n=29 and some have sample sizes in the hundreds.

Which test would be the best to compare the samples? To my understanding, KW wouldn’t be the best because the heteroscedastic nature of my data.

I am quite interested in this research area, but I don't see much active research in (theoretical) robust statistics anymore that is not incorporating AI/machine learning in some way.

I remember that in college (15+ years ago) we modeled random generators using Shannon's entropy, and then we extended that definition to include different types of generators: for example Collatz series can be as an output of a generator, just that instead of being completely random it follows the Collatz map.

Now I would like to use a similar approach to model LLMs, but I cant find anymore any reference to this approach. Does anyone have any academic source that could back up this interpretation of LLMs as entropic generators?

Say there are thousands of trade pair of a bot wallet with all the contextual trade and not a single missing data or order. Every decision by the said expert is done using the same information available to every one. Meme coin markets are short lived and algorithmically depending on outside information is highly unlikely. How much can we understand from those trades data? The exits are highly state dependent. No clustering of any TP/SL. Neither any time constraint except a hard cut off timer around 20 minutes if the algo doesnt trigger the sell. What kind of techniques i can implement? Can we figure out what they are optimizing for? There seems to be too many variables, despite the same observable raw features, what lookback or transformation of that feature, what prediction are they optimizing for almost felt like black box to me. With so many data of their distinct behavior can approximating/knowing what they are optimizing for, what drives the entry what drives the exit is so difficult? Any guidance will be highly appreciated.

I'm sharing this new tabular data diagnostic tool (Entropic Scree). It can be used to estimate these properties of your high-d, real-world, dirty dataset:

• The informational volume of the signal (i.e., helps you assess whether the signal is strong enough to survive the dataset's idiosyncratic volume).
• The overall signal-to-idiosyncratic volume ratio (SNR).
• The intrinsic rank.
• Provides an exploratory map that allows for the identification of decoupled sub-networks of variables.
• The linear sufficiency (i.e., does the dataset align with the linear assumptions of standard PCA?).

Instead of evaluating linear variance, rank order, or Euclidean distance like traditional PCA variants, this new method evaluates a transformed mutual information metric. Relative to these baselines, it is less reliant on strong parametric or distance assumptions, making it appropriate to apply more broadly.

It also serves as a practical diagnostic of the theory explored in the From Garbage to Gold framework, which describes when and why uncurated, error-prone data can be used directly to create accurate prediction models.

There is a preprint that presents the full technical details, and Python and R packages will be released soon. Right now though, the original function is already available in R (see Quick Start R Function Code below).

• Entropic Scree Preprint: https://doi.org/10.5281/zenodo.22028087
• Entropic Scree GitHub: https://github.com/tjleestjohn/entropic-scree
• From Garbage to Gold Preprint: https://arxiv.org/abs/2603.12288

Let me know how it goes if you give it a try... or if you have any questions or comments of course.

############ 
# Quick Start R Function Code.
# To load the function, copy and paste the following into your R console, then hit enter. 
############

# 1. Define the direct URL to the raw function script on GitHub
url <- "https://raw.githubusercontent.com/tjleestjohn/entropic-scree/main/Entropic.Scree.R%20-%20ENLI.R"

# 2. Define what you want to name the file on your computer
file_name <- "Entropic.Scree.R - ENLI.R"

# 3. Download the script to your current working directory
download.file(url, destfile = file_name)

# 4. Source the core function into your R environment
source(file_name)

# 5. Ex. To run the function and extract bipolar modules:
# results <- Entropic.Scree(dt 
#                         , extract_bipolar_modules = TRUE)
#
# View the extracted structural sub-networks for the primary axes:
# results$bipolar_modules

I'm an independent researcher writing a paper on a nineteenth-century historical question that was recently reviewed by the editor of an academic journal.

The editor's feedback was encouraging. She felt the historical premise was reasonable, but recommended that I have the statistical methodology reviewed by statisticians before submitting it elsewhere.

The historical details aren't particularly important for the question I'm asking.

The methodological problem looks something like this:

• There is a finite historical population.
• Within that population is a smaller subgroup independently identified through numerous historical sources.
• I have a separate corpus of legal documents that was created for an entirely unrelated purpose.
• When I examine that legal corpus, a surprisingly large proportion of the individuals belong to the independently identified subgroup.

The question is not whether statistics can prove a historical conclusion.

Rather, it's this:

How should a statistician think about whether this observed clustering is better explained by coincidence or by some underlying historical relationship, given that the data are historical, the sample is not random, and many potentially important variables are unknowable?

I've intentionally tried to avoid overstating the mathematics. My current paper argues that statistics cannot establish causation here, but that it can help evaluate whether the observed clustering is robust across a range of reasonable assumptions.

An editor suggested that I seek feedback from statisticians before publishing. I'm therefore looking for someone with experience in:

• applied statistics
• probability
• hypergeometric distributions
• Bayesian inference
• sensitivity analysis
• historical or observational data

I'm not looking for someone to "prove" my historical conclusion. In fact, I'd prefer someone who is willing to critique my methodology, assumptions, and modeling choices.

If this sounds like something you'd enjoy looking at—or if you know someone who specializes in this type of problem—I would greatly appreciate hearing from you.

Thanks!
Bill Reel

I Want to Prove an Online Roulette Wheel is Rigged

Hi all, I've never posted or commented here before so go easy on me. I have a background in Finance, mostly M&A but I did some statistics and probability stuff in undergrad. Mainly regression analysis and beta, nothing really advanced as far as stat/prob so I'm here asking for ideas and help.

I am aware that independent events cannot be used to predict other independent events; however computer programs cannot generate truly random numbers and I have an aching suspicion that online roulette programs force the distribution to return to the mean somehow.

My plan is to use excel to compile a list of spin outcomes, one at a time, I will use 1 for black, -1 for red and 0 for green. I am unsure how having 3 data points will affect regression analysis and I am unsure how I would even interpret the data outside of comparing the correlation coefficient to a control set to determine if it's statistically significant.

To be honest I'm not even sure if regression analysis is the best method to use for this experiment but as I said my background is not statistical or mathematical.

My ultimate goal is simply to backtest how random or fair a given roulette game is. As an added bonus I'd like to be able to determine if there are more complex patterns occurring, ie if it spins red 3 times is there on average a greater likelihood that it spins black or red on the next spin. Anything that could be a violation of the true randomness of the roulette wheel.

Thank you for reading.

Do you think theory folks ("pure statisticians") are higher in the academic hierarchy than applied statisticians who do not contribute to the development of new models and methods?

One thing is the barrier to entry; it is much harder to be a theoretician than to be an empiricist. In addition, as a theoretician, you have the capability to develop a new model or method that would be used by hundreds and thousands of people, while an empiricist is more confined to his specific domain.

But the other side of this argument is supply and demand. There is a lot more demand for applied research than for theory.

Do you think applied research has a certain ceiling because you are ultimately not going to develop a breakthrough, cutting-edge method?

Hi guys

I’m looking for recommendations for a statistical consultant (perhaps a PhD student or early-career biostatistician?) who could advise on test selection for a very small, unfunded veterinary retrospective study. I’m willing to pay, of course.

I’ve contacted several consulting services, but I suspect the project is too tiny for them. It would probably involve only an hour or two of work.

The study is a retrospective case series of dogs with tibial fractures, with two joint angles measured at three paired time points: pre-operatively, immediately post-operatively, and at follow-up. Measurements were also compared with the contralateral unaffected limb.

I’m happy to perform the analyses myself and mainly need advice on the most appropriate statistical approach.

I originally used prespecified paired t-tests, but the editor has requested that I seek advice from a statistician regarding whether it is preferable to do an omnibus test followed by post-hoc comparisons. So I mainly need advice on whether repeated-measures ANOVA or another omnibus approach is required, or whether Holm-adjusted paired comparisons alone would be defensible. (I'd also like to know if there is any particular way I should handle one case with bilateral fractures since those pairs aren't truly independent?)

Please message me if you might be able to help, or if you can recommend someone suitable. I can provide study summary, raw data, Q-Q plots, and results of paired t-t tests, as well as the whole manuscript + editorial feedback if helpful.

UPDATE

Thank you everyone and thankyou especially _compactsupport_, I've got enough to respond to the reviewers now and give them a couple of options :)

Hello I am an undergrad completing my analysis on my first research project. I created a between subjects experiment and manipulated the gender of actors in a vignette story. Participants were randomly assigned to receive one version of the scenario.

Here is the piece I am boiling my brain on. The vignette was broken into three parts according to narrative progression. At each of the 3 junctions participants responded to the same question using a likert scale. If it matters, the entire narrative progression was available to participants at the same time, just split into paragraphs with my question underneath each section.

I originally thought this should be treated as a between subjects repeated measure as each junction is measuring the same underlying construct. A faculty member who took a look at my work suggested that it was not a repeated measure but actually a multivariate design - so each section is being treated as it's own DV.

Can anyone give some insight as to which analysis is better suited for my study design, or help me understand which design I've created?

I have a hypothesis that my experimental group will be rated lower overall, and I have a hypothesis that my junction C will be rated lower overall in both conditions.

Thank you in advance. I ended up creating a rather complicated experiment for my first go and level of experience, but it is forcing me to learn quickly!

Hi statistics friends this is my first time posting here so I hope this question is okay. I was discussing this paper with peers at my medical school and when I did a deeper dive the statistics look extremely suspect to me. They claim statistical significance of the difference in inflammatory markers relative to particulate inhalation between males and females, but the 95% confidence interval on the two regressions overlap almost entirely. Could someone else take a brief look at this and tell me whether it looks suspicious to you as well? I don't want to be wrong if I accuse this data of being incorrectly analyzed without asking for a second opinion. Thank you so much in advance

https://www.frontiersin.org/journals/psychiatry/articles/10.3389/fpsyt.2025.1588964/full