0
DAYS
A global capital of research, innovation, fashion and design.
- Poster deadline: May 17, 2024
- Paper deadline: April 1st, 2024 April 15th, 2024
- Author notification: end of May, 2024
- Camera ready: June 30th, 2024
Prof. Johanna Ziegel (ETH Zurich)
Prof. Matteo Sesia (University of Southern California)
Prof. Johanna Ziegel
ETH Zurich, Switzerland
(Conformal) Isotonic Distributional Regression
Isotonic distributional regression (IDR) is a nonparametric distributional regression approach under a monotonicity constraint. It has found application as a generic method for uncertainty quantification, in statistical postprocessing of weather forecasts, and in distributional single index models. IDR has favorable in-sample calibration and optimality properties, which allow to conformalize it and obtain out-of-sample online guarantees.
Johanna Ziegel is Professor of Statistics at ETH Zurich, Switzerland, since 2024. She obtained her PhD in 2009 at ETH Zurich.
After postdoctoral positions in Melbourne, Australia, and Heidelberg, Germany, she joined the University of Bern from 2012-2023.
She was Scientific Advisory Board Member of the Oeschger Centre for Climate Change Research (OCCR) in Bern from 2018-2023. She has been Council Member of the Bernoulli Society and has served as Associate Editor for several journals including Electronic Journal of Statistics, Mathematical Finance, Bernoulli, JASA: Theory & Methods, International Journal of Forecasting. She won the Credit Swiss Award for Best Teaching at the University of Bern in 2022.
Prof. Matteo Sesia
University of Southern California, USA
Adaptive conformal classification with noisy labels
This talk describes adaptive conformal prediction methods for classification that can automatically adapt to random label contamination in the calibration sample, leading to more informative prediction sets with stronger coverage guarantees compared to state-of-the-art approaches. This is made possible by a precise characterization of the effective coverage inflation (or deflation) suffered by standard conformal inferences in the presence of label contamination, which is then made actionable through new calibration algorithms. This solution is flexible and can leverage different modeling assumptions about the label contamination process, while requiring no knowledge of the underlying data distribution or of the inner workings of the machine-learning classifier. The advantages of the proposed methods are demonstrated through extensive simulations and an application to object classification with the CIFAR-10H image data set.
Matteo Sesia is an Assistant Professor in the Department of Data Sciences and Operations at the University of Southern California (USC), Marshall School of Business, with a courtesy appointment in the Thomas Lord Department of Computer Science. His research is centered on distribution-free inference and conformal prediction.
Before joining USC, he earned his PhD in Statistics from Stanford University in 2020, where he was advised by Emmanuel Candès.
Prof. Henrik Boström
KTH Royal Institute of Technology, Stockholm, Sweden
Conformal prediction in Python with crepes
crepes is a Python package that implements conformal classifiers, regressors, and predictive systems on top of any standard classifier or regressor, turning the original predictions into well-calibrated p-values and cumulative distribution functions, or prediction sets and intervals with coverage guarantees. The crepes package implements standard and Mondrian conformal classifiers as well as standard, normalized and Mondrian conformal regressors and predictive systems. In this tutorial, we will give a brief introduction to the underlying algorithms and show how to generate, apply and evaluate conformal classifiers, regressors and predictive systems using crepes.
Henrik Boström is professor of computer science - data science systems at KTH Royal Institute of Technology. His research focuses on machine learning algorithms and applications, in particular conformal prediction, ensemble learning and explainable machine learning. He has led and worked in projects on machine learning applications within the pharmaceutical industry, healthcare and medicine, automotive industry, and insurance industry.
He has served as action editor/editor for several journals, including Machine Learning, and Data Mining and Knowledge Discovery, and is a frequent area chair/senior program committee member, e.g., at SIGKDD and IJCAI.
Dr. Thibault Cordier
Lab Invent of Capgemini Invent, France
MAPIE: A Versatile Tool for Conformal Prediction in Python - Uncertainty Quantification Made Easy
Conformal prediction (CP) offers a robust theoretical framework for quantifying and managing uncertainties within machine learning models. This tutorial explores MAPIE (Model Agnostic Prediction Interval Estimator), a versatile Python library that facilitates the implementation and use of CP methods. Established as part of the scikit-learn-contrib project since 2021, MAPIE supports a variety of machine learning tasks such as classification, regression, and time-series analysis, and is adept in both split and cross-conformal settings. But it doesn't stop there - MAPIE can also be used to handle more complex tasks like multi-label classification and semantic segmentation in computer vision, ensuring probabilistic guarantees on crucial metrics like recall and precision. Join this tutorial to dive into the world of conformal predictions and how to quickly manage your uncertainties using MAPIE.
Thibault Cordier is a Data and Research Scientist at Capgemini Invent, where he is a member of the Lab Invent team in France and serves as the technical leader of the MAPIE project.
Prior to joining the research team at Capgemini Invent, he earned his PhD in Computer Science in 2023 at Avignon University.
Up to now, his research has focused on distribution-free inference and conformal prediction, with applications in computer vision, natural language processing, and time series analysis.
General Paper
"Split Conformal Prediction under Data Contamination"
Jason Clarkson, Wenkai Xu, Mihai Cucuringu, Gesine Reinert
Student Paper
"Conformal Predictive Systems Under Covariate Shift"
Jef Jonkers, Glenn Van Wallendael, Luc Duchateau, Sofie Van Hoecke
Registration
Welcome speech
Keynote
"Adaptive conformal classification with noisy labels"
Matteo Sesia (University of Southern California, USA)
Chair: Simone Vantini
Tea/Coffee break
Paper session - Theory
Chair: Aldo Solari
"Enhancing Conformal Prediction Using E-Test Statistics"
Alexander A. Balinsky and Alexander David Balinsky
[Download paper] [Download presentation]
"Asymptotic uniqueness in long-term prediction"
Vladimir Vovk
[Download paper] [Download presentation]
"Split Conformal Prediction under Data Contamination"
Jason Clarkson, Wenkai Xu, Mihai Cucuringu and Gesine Reinert
Lunch break
Tutorial
"MAPIE: A Versatile Tool for Conformal Prediction in Python - Uncertainty Quantification Made Easy"
Thibault Cordier (Lab Invent of Capgemini Invent, France)
Chair: Matteo Fontana
Paper session - Classification #1
Chair: Teresa Bortolotti
"Multi-label Conformal Prediction with a Mahalanobis Distance Nonconformity Measure"
Kostas Katsios and Harris Papadopulos
[Download paper] [Download presentation]
"A probabilistic scaling approach to conformal predictions in binary image classification"
Alberto Carlevaro, Sara Narteni, Fabrizio Dabbene, Teodoro Alamo and Maurizio Mongelli
[Download paper] [Download presentation]
"Calibrated Explanations for Multi-class"
Tuwe Löfström, Helena Löfström and Ulf Johansson
Paper session - Classification #2
Chair: Tuwe Lofstrom
"Multi-class Classification with Reject Option and Performance Guarantees using Conformal Prediction"
Alberto García-Galindo, Marcos López-De-Castro and Ruben Armañanzas
[Download paper] [Download presentation]
"Evidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction"
Hamed Karimi and Reza Samavi
[Download paper] [Download presentation]
"Entropy Reweighted Conformal Prediction for Calibrated Neural Networks"
Rui Luo and Nicolo Colombo
Fruit break + Walking Tour of the Historical Campus of Polimi
Keynote
"(Conformal) Isotonic Distributional Regression"
Prof. Johanna Ziegel (ETH Zurich, Switzerland)
Chair: Francesca Ieva
Paper session - Distributional Prediction
Chair: Matteo Fontana
"Conformal Predictive Systems Under Covariate Shift"
Jef Jonkers, Glenn Van Wallendael, Luc Duchateau and Sofie Van Hoecke
[Download paper] [Download presentation]
"Inductive Venn-Abers Predictive Distributions: New Applications & Evaluation"
Ilia Nouretdinov and James Gammerman
[Download paper] [Download presentation]
"Tailoring the Tails: Enhancing the Reliability of Probabilistic Load Forecasts"
Roberto Baviera and Pietro Manzoni
[Download paper] [Download presentation]
Tea/Coffee break
Paper session - Time Series
Chair: Henrik Bostrom
"Adaptive Conformal Inference for Multi-Step Ahead Time-Series Forecasting Online"
Johan Hallberg Szabadváry
[Download paper] [Download presentation]
"ConForME: Multi-horizon conditional conformal time series forecasting"
Aloysio Galvao Lopes, Eric Goubault, Sylvie Putot and Laurent Pautet
"Conformal time series decomposition with component-wise exchangeability"
Derck Prinzhorn, Thijmen Nijdam, Putri Van der Linden and Alexander Timans
Lunch break
Visit of Milan
Paper session - Applications #1
Chair: Lars Carlsson
"The Uncertain Object: Application of Conformal Prediction to Aerial and Satellite Images"
Vicky Copley, Greg Finlay and Ben Hiett
[Download paper] [Download presentation]
"Calibrated Large Language Models for Binary Question Answering"
Patrizio Giovannotti and Alexander Gammerman
[Download paper] [Download presentation]
"Clustered Conformal Prediction for the Housing Market"
Anders Hjort, Jonathan P. Williams and Johan Pensar
[Download paper] [Download presentation]
Paper session - Applications #2
Chair: Alfredo Gimenez Zapiola
"Copula-based conformal prediction for object detection: a more efficient approach"
Bruce Cyusa Mukama, Soundouss Messoudi, Sylvain Rousseau and Sébastien Destercke
[Download paper] [Download presentation]
"Uncertainty Quantification for Metamodels"
Martin Okánik, Athanasios Trantas, Merijn Pepijn de Bakker and Elena Lazovik
[Download paper] [Download presentation]
"Reliable Change Point Detection for ACGH data"
Charalambos Eliades and Harris Papadopoulos
[Download paper] [Download presentation]
Tea/Coffee break
Poster session
Chair: Matteo Fontana
"Distribution-free Uncertainty Quantification for Contour Segmentation"
Wenhui Zhang and Surajit Ray
"Fairness Considerations for Conformal Classification"
Arlan Abzhanov and Brieuc Lehmann
"Collective Outlier Detection and Enumeration with Conformalized Closed Testing"
Chiara Gaia Magnani, Matteo Sesia and Aldo Solari
"Preferent compression for tight generalization bounds"
Marco C. Campi and Simone Garatti
"Anomaly Detection in Multivariate Profiles with Conformal Bayesian Inference"
Nina Deliu and Brunero Liseo
Lunch break
Tutorial
"Conformal prediction in Python with crepes"
Prof. Henrik Boström (KTH Royal Institute of Technology, Stockholm, Sweden)
Chair: Simone Vantini
Paper session - Regression
Chair: Alex Gammerman
"CoPAL: Conformal Prediction in Active Learning An Algorithm for Enhancing Remaining Useful Life Estimation in Predictive Maintenance"
Zahra Kharazian, Tony Lindgren, Sindri Magnusson and Henrik Boström
[Download paper] [Download presentation]
"Distribution-free risk assessment of regression-based machine learning algorithms"
Sukrita Singh, Neeraj Sarna, Yuanyuan Li, Yang Lin, Agni Orfanoudaki and Michael Berger
[Download paper] [Download presentation]
"Conformal Regression with Reject Option"
Ulf Johansson, Cecilia Sönströd and Henrik Boström
[Download paper] [Download presentation]
Tea/Coffee break
Paper session - Testing and Feature Selection
Chair: Volodya Vovk
"Conformal Stability Measure of Feature Selection Algorithms"
Marcos López-De-Castro, Alberto García-Galindo and Rubén Armañanzas
[Download paper] [Download presentation]
"Estimating Quality of Approximated Shapley Values Using Conformal Prediction"
Amr Alkhatib, Henrik Boström and Ulf Johansson
[Download paper] [Download presentation]
"Testing Exchangeability between Real and Synthetic Data"
Helena Löfström, Lars Carlsson and Ernst Ahlberg
[Download paper] [Download presentation]
Closing address with best paper awards
Conference Dinner
The 13th Symposium on Conformal and Probabilistic Prediction with Applications (COPA 2024) will be held from the 9th to the 11th of September 2024, at Politecnico di Milano, Leonardo Campus, Piazza Leonardo da Vinci, Italy. Submissions are invited on original and previously unpublished research concerning all aspects of conformal and probabilistic prediction. The accepted papers will be published in the Proceedings of Machine Learning Research, Volume 230.
Conformal prediction (CP) is a modern machine and statistical learning method that allows to develop valid predictions under weak probabilistic assumptions. CP can be used to form set predictions, using any underlying point predictor, and for very general target variables, allowing the error levels to be controlled by the user. Therefore, CP has been widely used to develop robust forms of probabilistic prediction methodologies, and applied to many practical real life challenges.
The aim of this symposium is to serve as a forum for the presentation of new and ongoing work and the exchange of ideas between researchers on any aspect of conformal and probabilistic prediction, including their application to interesting problems in any field.
Topics of the symposium include, but are not limited to:
Authors are invited to submit original, English-language research contributions or experience reports. Typical papers should not exceed 20 pages, and formatted according to JMLR (Journal of Machine Learning Research) template and style guidelines. The LaTeX package is available here.
All aspects of the submission and notification process will be handled online via the EasyChair Conference System at:
https://easychair.org/conferences/?conf=copa20240
Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the symposium to present the work.
Authors are invited to submit original, English-language research abstracts. The contributions should not be longer than 300 words.
Submitted abstracts will be refereed for quality, correctness, originality, and relevance. Notification and reviews will be communicated via email. All accepted contributions will be presented during the Poster Session at the Symposium.
All aspects of the submission and notification process will be handled online via the EasyChair Conference System at:
https://easychair.org/conferences/?conf=copa20240
Please note that the submission of an abstract should be regarded as a commitment that, in case the poster is accepted, at least one of the authors will register and attend the symposium to attend the poster session.
Submitted papers will be refereed for quality, correctness, originality, and relevance. Notification and reviews will be communicated via email. All accepted papers will be presented at the Symposium and published in the PMLR (Proceedings of Machine Learning Research).
There will be two Alexey Chervonenkis awards for the Best Paper and Best Student Paper, presented at the conference.
Researchers interested in Conformal Prediction may be interested in joining our online discussion group. Future announcements and related materials will be published regularly.
All accepted papers will be presented at the conference and published by the Proceedings of Machine Learning Research (PMLR). Volume 230 at http://proceedings.mlr.press/v230/
Please make sure to use the correct style file; it should be already installed on your computer, or will be installed "on the fly". The camera-ready papers should follow the style of the Proceedings section of JMLR rather than JMLR itself.
In the end you should prepare two files: (1) your paper in PDF format; (2) the copyright form (http://proceedings.mlr.press/pmlr-license-agreement.pdf). Please upload the final version of your paper and the signed copyright form on EasyChair.
The beginning of your file will look like:
\documentclass[wcp]{jmlr} \usepackage{amsmath,amssymb,graphicx,url} \jmlrvolume{230} \jmlryear{2024} \jmlrworkshop{Conformal and Probabilistic Prediction with Applications} \jmlrproceedings{PMLR}{Proceedings of Machine Learning Research} \title{Nonparametric predictive distributions based on conformal prediction} \author{\Name{John Smith}\Email{j.smith@gmail.com}\\ \addr{Royal Holloway, University of London, Egham, Surrey, UK}\\ \Name{Minge Shen}\Email{m.shen@gmail.com}\\ \addr{Rutgers University, New Brunswick, NJ, USA}} \editor{Simone Vantini, Matteo Fontana, Aldo Solari, Henrik Boström and Lars Carlsson} \begin{abstract} This paper applies conformal prediction to derive predictive distributions that are valid under a nonparametric assumption. \end{abstract} \begin{keywords} Conformal prediction, predictive distributions, regression. \end{keywords}
Notice the presence of the command
\jmlrproceedings{PMLR}{Proceedings of Machine Learning Research}
Please let us know if you encounter any problems following these instructions.
For further information (which, however, can be confusing), see http://jmlr.org/proceedings/faq.html (under the heading "What is the Style File for the Proceedings?"); COPA 2024 uses the one column style file. The style file is available from the CTAN (Comprehensive TeX Archive Network) web site at http://ctan.org/tex-archive/macros/latex/contrib/jmlr you can go inside the directory called "sample_papers" and emulate the files jmlr_sample.tex and jmlr_sample.bib (the latter is only needed if you use bibtex).
Some useful advices on the JMLR style can be found at http://jmlr.org/format/format.html (However, please make sure to use the Proceedings style, as described above, rather than the main journal style).
Please notice that non-EU citizens might require a VISA to come to Italy.
To know if you need a VISA and in that case to know how to get a VISA you can refer the indication reported at this link: https://vistoperitalia.esteri.it/home/en
The COPA 2024 talks will take place in classroom 16B.1.1, located on the first floor of Building 16B at the Leonardo Campus of the Politecnico di Milano.
Classroom 16B.1.1 is easily accessible by foot from the Piola M2 Metro Station in less than five minutes. Here are the directions to the classroom from the station:
If you're arriving from neighboring European countries, you might consider taking a train to Milan. The city is well-connected to major European cities via high-speed rail services. International train services terminates either at Garibaldi FS Station or Centrale FS Station.
While not as common as other modes of transport, buses do offer international routes to Milan from nearby countries like Switzerland, France, and Austria.
Milan boasts an efficient metro system with five lines (M1, M2, M3, M4, M5) covering most of the city and reaching outlying areas. The metro is fast, frequent, clean, and usually the quickest way to navigate Milan's urban sprawl.
Milan's extensive tram network and bus routes provide comprehensive coverage throughout the city, including areas not served by the metro. These are convenient for reaching specific neighborhoods and exploring the city at a more leisurely pace.
Milan offers several bike-sharing systems, which allow you to rent bicycles across the city. It's a fantastic way to explore Milan's streets, especially during pleasant weather, and it's eco-friendly too.
Milan is a walkable city, especially in the historic center where many attractions are clustered close together. Walking allows you to soak in the city's vibrant atmosphere, discover hidden gems, and enjoy the beautiful architecture at your own pace.
You can conveniently pay for and utilize your Metro, tram, and bus tickets by simply tapping your credit or debit card on the card readers installed on all buses and trams, as well as on the dedicated turnstiles at metro stations. Always remember to tap your credit or debit card when exiting metro stations to ensure you are charged the lowest fare by the end of the day.
The centerpiece of Milan, the Duomo is one of Europe’s greatest architectural and cultural landmarks.
As Italy’s largest church and one of the largest in the world, it took over 600 years to build.
This extravagant 19th-century glass-topped, barrel-vaulted tunnel serves as a lively, noisy and colorful shopping mall.
The Castello Sforzesco is a medieval fortification located in Milan, Northern Italy.
It was built in the 15th century by Francesco Sforza, Duke of Milan, on the remnants of a 14th-century fortification.
This world-renowned opera house was built in 1778, where many composers wrote and conducted works including such greats as Rossini, Puccini, Verdi and Toscanini.
This busy and vibrant city square features the gothic cathedral, one of the largest in Europe.
The Last Supper is a mural painting by the Italian High Renaissance artist Leonardo da Vinci, dated to c. 1495–1498.
400€
250€
150€
Department of Computer Science
Royal Holloway University of London
United Kingdom
General enquiries: