What is the p-value for outlier detection?

A significance level of 0.05 indicates a 5% risk of concluding that an outlier exists when no actual outlier exists. If the p-value is less than or equal to the significance level, the decision is to reject the null hypothesis and conclude that an outlier exists .

What is the conformal p-value?

The conformal p-value, which promises to provide rigorous finite-sample error rate control under the exchangeability assumption , is a fundamental concept for hypothesis testing based on complex machine-learning algorithms.

How do you statistically test for outliers?

You can choose from four main ways to detect outliers: Sorting your values from low to high and checking minimum and maximum values. Visualizing your data with a box plot and looking for outliers. Using the interquartile range to create fences for your data. Using statistical procedures to identify extreme values. Nov 30, 2021

What is the best test for outliers?

Grubbs' Test - this is the recommended test when testing for a single outlier. Tietjen-Moore Test - this is a generalization of the Grubbs' test to the case of more than one outlier. It has the limitation that the number of outliers must be specified exactly.

What is out of distribution conformal prediction?

Research on Out-Of-Distribution (OOD) detection focuses mainly on building scores that efficiently distinguish OOD data from In Distribution (ID) data. On the other hand, Conformal Prediction (CP) uses non-conformity scores to construct prediction sets with probabilistic coverage guarantees .

What is the p-value of the normal distribution curve?

When thinking about the standard normal distribution (bell curve), the p-value corresponds to the area under the curve where extreme values are not likely to be the result of chance . The p-value can be calculated using a calculator with the test statistic (z-test or t-test).

What does conformal mean in statistics?

Conformal prediction is a relatively new framework for quantifying uncertainty in the predictions made by arbitrary prediction algorithms . Fundamentally, it does so by converting an algorithm's predictions into prediction sets, which have strong finite-sample coverage properties.

What is the T score for outlier detection?

Outlier Detection: T-scores help identify outliers in datasets. When a data point's t-score is significantly higher or lower than the average, it suggests that the data point is an outlier . This information is valuable for detecting anomalies and understanding the overall distribution of the data.

What is the p-value in the Grubbs test?

Grubbs' test statistic (G) is the difference between the sample mean and either the smallest or largest data value, divided by the standard deviation. Minitab uses Grubbs' test statistic to calculate the p-value, which is the probability of rejecting the null hypothesis when it is true .

What is a statistically significant outlier?

In simple terms, an outlier is an extremely high or extremely low data point relative to the nearest data point and the rest of the neighboring co-existing values in a data graph or dataset you're working with . Outliers are extreme values that stand out greatly from the overall pattern of values in a dataset or graph.

What does a P value of 0.06 mean?

A p value of 0.06 means that there is a probability of 6% of obtaining that result by chance when the treatment has no real effect .

Integrative conformal p-values for out-of-distribution testing with labelled outliers (2024)

Article Navigation

Volume 86 Issue 3 July 2024

< Previous
Next >

Journal Article

Get access

Ziyi Liang

Department of Mathematics, University of Southern California

Los Angeles, CA

USA

Search for other works by this author on:

Oxford Academic

Google Scholar

Matteo Sesia

Department of Data Sciences and Operations, University of Southern California

Los Angeles, CA

USA

Address for correspondence: Matteo Sesia, Department of Data Sciences and Operations, University of Southern California, Los Angeles, CA 90089, USA. Email: sesia@marshall.usc.edu

https://orcid.org/0000-0001-9046-907X

Search for other works by this author on:

Oxford Academic

Google Scholar

Wenguang Sun

School of Management and Center for Data Science, Zhejiang University

Hangzhou

China

Search for other works by this author on:

Oxford Academic

Google Scholar

Journal of the Royal Statistical Society Series B: Statistical Methodology, Volume 86, Issue 3, July 2024, Pages 671–693, https://doi.org/10.1093/jrsssb/qkad138

Abstract

This paper presents a conformal inference method for out-of-distribution testing that leverages side information from labelled outliers, which are commonly underutilized or even discarded by conventional conformal p-values. This solution is practical and blends inductive and transductive inference strategies to adaptively weight conformal p-values, while also automatically leveraging the most powerful model from a collection of one-class and binary classifiers. Further, this approach leads to rigorous false discovery rate control in multiple testing when combined with a conditional calibration strategy. Extensive numerical simulations show that the proposed method outperforms existing approaches.

conformal inference, false discovery rate, machine learning, one-class classification, testing for outliers, weighted hypothesis testing

This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/pages/standard-publication-reuse-rights)

You do not currently have access to this article.

Download all slides

Sign in

Get help with access

Personal account

Sign in with email/username & password
Get email alerts
Save searches
Purchase content
Activate your purchase/trial code
Add your ORCID iD

Institutional access

Sign in with a library card
Sign in with username/password
Recommend to your librarian

Institutional account management

Get help with access

Institutional access

Access to content on Oxford Academic is often provided through institutional subscriptions and purchases. If you are a member of an institution with an active account, you may be able to access content in one of the following ways:

IP based access

Typically, access is provided across an institutional network to a range of IP addresses. This authentication occurs automatically, and it is not possible to sign out of an IP authenticated account.

Sign in through your institution

Choose this option to get remote access when outside your institution. Shibboleth/Open Athens technology is used to provide single sign-on between your institutionâ€™s website and Oxford Academic.

Click Sign in through your institution.
Select your institution from the list provided, which will take you to your institution's website to sign in.
When on the institution site, please use the credentials provided by your institution. Do not use an Oxford Academic personal account.
Following successful sign in, you will be returned to Oxford Academic.

If your institution is not listed or you cannot sign in to your institutionâ€™s website, please contact your librarian or administrator.

Sign in with a library card

Enter your library card number to sign in. If you cannot sign in, please contact your librarian.

Society Members

Society member access to a journal is achieved in one of the following ways:

Sign in through society site

Many societies offer single sign-on between the society website and Oxford Academic. If you see â€˜Sign in through society siteâ€™ in the sign in pane within a journal:

Click Sign in through society site.
When on the society site, please use the credentials provided by that society. Do not use an Oxford Academic personal account.
Following successful sign in, you will be returned to Oxford Academic.

If you do not have a society account or have forgotten your username or password, please contact your society.

Sign in using a personal account

Some societies use Oxford Academic personal accounts to provide access to their members. See below.

Personal account

A personal account can be used to get email alerts, save searches, purchase content, and activate subscriptions.

Some societies use Oxford Academic personal accounts to provide access to their members.

Viewing your signed in accounts

Click the account icon in the top right to:

View your signed in personal account and access account management features.
View the institutional accounts that are providing access.

Signed in but can't access content

Oxford Academic is home to a wide variety of products. The institutional subscription may not cover the content that you are trying to access. If you believe you should have access to that content, please contact your librarian.

Institutional account management

For librarians and administrators, your personal account also provides access to institutional account management. Here you will find options to view and activate subscriptions, manage institutional settings and access options, access usage statistics, and more.

Purchase

Subscription prices and ordering for this journal

Purchasing options for books and journals across Oxford Academic

Short-term Access

To purchase short-term access, please sign in to your personal account above.

Don't already have a personal account? Register

Integrative conformal p-values for out-of-distribution testing with labelled outliers - 24 Hours access

EUR €48.00

GBP £41.00

USD $51.00

Rental

This article is also available for rental through DeepDyve.

Advertisem*nt

Citations

Views

423

Altmetric

More metrics information

Metrics

Total Views 423

269 Pageviews

154 PDF Downloads

Since 1/1/2024

Month:	Total Views:
January 2024	155
February 2024	37
March 2024	57
April 2024	47
May 2024	41
June 2024	49
July 2024	37

Citations

Altmetrics

Email alerts

Article activity alert

Advance article alerts

New issue alert

Receive exclusive offers and updates from Oxford Academic

Citing articles via

Google Scholar

Latest
Most Read
Most Cited

Authors’ Reply to the Discussion of ‘Safe Testing’

Graphical criteria for the identification of marginal causal effects in continuous-time survival and event-history analyses

Judith ter Schure’s contribution to the Discussion of ‘Safe Testing’ by Grünwald, De Heide and Koolen

Christine P. Chai’s contribution to the Discussion of ‘Safe Testing’ by Grünwald, De Heide and Koolen

Luigi Pace and Alessandra Salvan’s contribution to the Discussion of ‘Safe Testing’ by Peter Grünwald, Rianne de Heide and Wouter Koolen

Integrative conformal p-values for out-of-distribution testing with labelled outliers (2024)

Cite

Abstract

Sign in

Royal Statistical Society members

Personal account

Institutional access

Institutional account management

Get help with access

Institutional access

IP based access

Sign in through your institution

Sign in with a library card

Society Members

Sign in through society site

Sign in using a personal account

Personal account

Viewing your signed in accounts

Signed in but can't access content

Institutional account management

Purchase

Short-term Access

Rental

Citations

Views

Altmetric

Email alerts

Citing articles via

Latest

Most Read

Most Cited

FAQs

What is the p-value for outlier detection? ›

What does a P value of 0.06 mean? ›