0
DAYS
A major seaside resort town, just 1 hour away from central London.
- Poster (2-page extended abstract) deadline: July 4, 2022
- Paper deadline: April 24 May 8, 2022 (extended)
- Author notification: June 23 June 27, 2022
- Camera-ready: July 7, 2022
- Prof. Rina Barber (University of Chicago, USA)
- Prof. Eyke Hüllermeier (University of Munich, Germany)
- Dr. Sébastien Destercke (University of Technology of Compiègne, France)
Prof. Rina Foygel Barber
Department of Statistics at University of Chicago, USA
Conformal prediction beyond exchangeability
Conformal prediction is a popular, modern technique for providing valid predictive inference for arbitrary machine learning models. Its validity relies on the assumptions of exchangeability of the data, and symmetry of the given model fitting algorithm as a function of the data. However, exchangeability is often violated when predictive models are deployed in practice. For example, if the data distribution drifts over time, then the data points are no longer exchangeable; moreover, in such settings, we might want to use an algorithm that treats recent observations as more relevant, which would violate the assumption that data points are treated symmetrically.
This talk proposes new methodology to deal with both aspects: we use weighted quantiles to introduce robustness against distribution drift, and design a new technique to allow for algorithms that do not treat data points symmetrically, with theoretical results verifying coverage guarantees that are robust to violations of exchangeability. This work is joint with Emmanuel Candes, Aaditya Ramdas, and Ryan Tibshirani.
Barber is a Professor in the Department of Statistics at the University of Chicago. Before starting at U of C, she was a NSF postdoctoral fellow during 2012-13 in the Department of Statistics at Stanford University. She received her PhD in Statistics at the University of Chicago in 2012.
Her research interests are in developing and analyzing estimation, inference, and optimization tools for structured high-dimensional data problems such as sparse regression, sparse nonparametric models, and low-rank models. She works on developing methods for false discovery rate control in settings where we may have undersampled data or misspecified models, and for scalable optimization techniques for nonconvex problems.
Prof. Eyke Hüllermeier
Department of Computer Science at University of Munich, Germany
Uncertainty Quantification in Machine Learning: From Aleatoric to Epistemic
Due to the steadily increasing relevance of machine learning for practical applications, many of which are coming with safety requirements, the notion of uncertainty has received increasing attention in machine learning research in the recent past.
This talk will address questions regarding the representation and adequate handling of (predictive) uncertainty in (supervised) machine learning. A specific focus will be put on the distinction between two important types of uncertainty, often referred to as aleatoric and epistemic, and how to represent and quantify these uncertainties in terms of suitable numerical measures. Roughly speaking, while aleatoric uncertainty is due to randomness inherent in the data generating process, epistemic uncertainty is caused by the learner’s ignorance about the true underlying model. Some recent proposals specifically tailored to capturing aleatoric and epistemic uncertainty will be presented, and other methods for handling uncertainty, such as conformal prediction, will be interpreted in light of this distinction.
Hüllermeier is a full professor at the LMU Munich, Germany, where he is the Chair of Artificial Intelligence and Machine Learning. He graduated in mathematics and business computing, received his PhD in computer science from the University of Paderborn in 1997, and a Habilitation degree in 2002. He held professorships at the Universities of Marburg (2002-04), Dortmund (2004), Magdeburg (2005-06) and again Marburg (2007-14).
His research interests are centered around methods and theoretical foundations of artificial intelligence, with a specific focus on machine learning and reasoning under uncertainty. He has published more than 300 articles on these topics in top-tier journals and major international conferences, and several of his contributions have been recognized with scientific awards. He is a coordinator of the EUSFLAT working group on Machine Learning and Data Mining and head of the IEEE CIS Task Force on Machine Learning.
Dr. Sébastien Destercke
Heudiasyc Laboratory at University of Technology of Compiègne, France
Uncertain data in learning: challenges and opportunities
How to account for uncertain data in learning and estimation procedures is an old problem, including for example issues such as censored or missing data, the use of soft labels, etc.
In this talk, I will start by discussing the nature of data uncertainty, arguing that this uncertainty can be of non-statistical nature, and that uncertainty models generalising both probabilities and sets are interesting tools to model uncertain data in general. From there, I will start by describing some challenges arising when one has to learn in presence of uncertain data, and will finish on a more positive note by showing some settings where modelling data uncertainty can actually be beneficial to the learning procedure. I will also try to connect such modelling, challenges and opportunities to conformal or Venn-Abers predictors.
Destercke graduated from the Faculté Polytechnique de Mons as an Engineer with a specialization in computer science and applied mathematics. Since October 2011, he is a CNRS researcher in the Heuristique et Diagnostic des Systèmes Complexes research unit, and leading the CID team since September 2020.
Most of his research focuses on reasoning under severe uncertainty, where by severe it is understood incomplete and imprecise information. He has in particular investigated theories using probability sets rather than single probabilities as models of such uncertainty. His work is shared between theoretical issues and more applied considerations.
General Paper
"Robust Gas Demand Forecasting with Conformal Prediction"
Mouhcine Mendil, Luca Mossina, Marc Nabhan and Kevin Pasini
Student Paper
"Cough-based COVID-19 detection with audio quality clustering and confidence measure based learning"
Alice E. Ashby, Julia A. Meister, Khuong An Nguyen, Zhiyuan Luo and Werner Gentzke
Registration
Welcome speech
Tutorial Session (1)
"Conformal Regressors and Predictive Systems - a Gentle Introduction"
Prof. Henrik Boström (KTH Royal Institute of Technology, Sweden)
Chair: Dr. Lars Carlsson
Tea/Coffee break
Tutorial Session (2)
"Applications of Conformal Prediction"
Dr. Lars Carlsson (Universal Prediction AB, Sweden; Royal Holloway University of London, UK)
Dr. Ernst Ahlberg (Universal Prediction AB, Sweden; Uppsala University, Sweden)
Chair: Dr. Tuwe Löfström
Lunch break
Keynote (1)
"Uncertainty Quantification in Machine Learning: From Aleatoric to Epistemic"
Prof. Eyke Hüllermeier (Department of Computer Science at University of Munich, Germany)
Chair: Prof. Henrik Boström
Paper session (1): "Implementations"
Chair: Prof. Ulf Johansson
"Tutorial for using conformal prediction in KNIME"
Tuwe Löfström, Artem Ryasik and Ulf Johansson
[Download paper] [Download presentation]
"Crepes: a Python Package for Generating Conformal Regressors and Predictive Systems"
Henrik Boström
Tea/Coffee break
Paper session (2): "Venn prediction"
Chair: Dr. Ernst Ahlberg
"Assessing Explanation Quality by Venn Prediction"
Amr Alkhatib, Henrik Boström and Ulf Johansson
[Download paper] [Download presentation]
"Calibration of Natural Language Understanding Models with Venn-ABERS Predictors"
Patrizio Giovannotti
[Download paper] [Download presentation]
"Well-Calibrated Rule Extractors"
Ulf Johansson, Tuwe Löfström and Niclas Ståhl
Dinner at The Walrus (10 Ship Street, Brighton, BN1 1AD)
Paper session (3): "Applications"
Chair: Dr. Harris Papadopoulos
"Conformal prediction of small-molecule drug resistance in cancer cell lines"
Saiveth Hernandez-Hernandez, Sachin Vishwakarma and Pedro Ballester
[Download paper] [Download presentation]
"Uncertainty Estimation for Single-cell Label Transfer"
Robin Khatri and Stefan Bonn (online)
[Download paper] [Download presentation]
"Cough-based COVID-19 detection with audio quality clustering and confidence measure based learning"
Alice E. Ashby, Julia A. Meister, Khuong An Nguyen, Zhiyuan Luo and Werner Gentzke
[Download paper] [Download presentation]
"Online Portfolio Hedging with the Weak Aggregating Algorithm"
Najim Al-Baghdadi, Yuri Kalnishkan, David Lindsay and Sian Lindsay
Tea/Coffee break
Paper session (4): "Machine Learning 1"
Chair: Prof. Alex Gammerman
"Conformal testing: binary case with Markov alternatives"
Vladimir Vovk, Ilia Nouretdinov and Alexander Gammerman
[Download paper] [Download presentation]
"A Betting Function for addressing Concept Drift with Conformal Martingales"
Charalambos Eliades and Harris Papadopoulos
[Download paper] [Download presentation]
"On efficiency of Learning Under Privileged Information"
Ilia Nouretdinov (online)
[Download paper] [Download presentation]
"Tensor-Train Kernel Learning for Gaussian Processes"
Max Kirstein, David Sommer and Martin Eigel (online)
Lunch break
Keynote (2)
"Conformal prediction beyond exchangeability"
Prof. Rina Foygel Barber (Department of Statistics at University of Chicago, USA)
Chair: Prof. Henrik Boström
Royal Pavilion tour
Paper session (5): "Machine Learning 2"
Chair: Prof. Vladimir Vovk
"Pruning Neural networks for inductive conformal prediction"
Xindi Zhao and Anthony Bellotti (online)
[Download paper] [Download presentation]
"Ellipsoidal conformal inference for Multi-Target Regression"
Soundouss Messoudi, Sébastien Destercke and Sylvain Rousseau
[Download paper] [Download presentation]
"Robust Gas Demand Forecasting with Conformal Prediction"
Mouhcine Mendil, Luca Mossina, Marc Nabhan and Kevin Pasini
[Download paper] [Download presentation]
"Conformal prediction for hypersonic flight vehicle classification"
Zepu Xi, Xuebin Zhuang and Hongbo Chen (online)
Tea/Coffee break
Keynote (3)
"Uncertain data in learning: challenges and opportunities"
Dr. Sébastien Destercke (Heudiasyc Laboratory at University of Technology of Compiègne, France)
Chair: Prof. Henrik Boström
Poster teaser
Chair: Prof. Ulf Johansson
Lunch & Poster session
Chair: Prof. Ulf Johansson
"Prediction of Energy Consumption with Inductive Venn-Abers Predictive Distribution"
Ilia Nouretdinov and James Gammerman
"Communication-efficient Conformal Prediction for Distributed Datasets"
Nery Riquelme-Granada, Zhiyuan Luo and Khuong An Nguyen
"House Price Prediction with Confidence: Empirical Results from the Norwegian Market"
Anders Hjort
"Conformal Multistep-Ahead Multivariate Time-Series Forecasting"
Filip Schlembach, Evgueni Smirnov and Irena Koprinska
"Conformal Decision Rules"
Husam Abdelqader, Evgueni Smirnov, Marc Pont and Marciano Geijselaers (online)
Closing address with best paper awards
The 11th Symposium on Conformal and Probabilistic Prediction with Applications (COPA 2022) will be held from August 24th to 26th, 2022, at University of Brighton, United Kingdom. Submissions are invited on original and previously unpublished research concerning all aspects of conformal and probabilistic prediction. The symposium proceedings will be published in the Proceedings of Machine Learning Research.
Conformal prediction (CP) is a modern machine learning method that allows to make valid predictions under relatively weak statistical assumptions. CP can be used to form set predictions, using any underlying point predictor, allowing the error levels to be controlled by the user. Therefore, CPs have been widely applied to many practical real life challenges.
Building on the work on CP, various extensions have been developed recently. The aim of this symposium is to serve as a forum for the presentation of new and ongoing work and the exchange of ideas between researchers on any aspect of conformal and probabilistic prediction and their applications to interesting problems in any field.
Topics of the symposium include, but are not limited to:
Authors are invited to submit original, English-language research contributions or experience reports. Papers should be no longer than 20 pages formatted according to the well-known JMLR (Journal of Machine Learning Research) style. The LaTeX package for the style is available here.
All aspects of the submission and notification process will be handled online via the EasyChair Conference System at:
https://easychair.org/conferences/?conf=copa2022
Submission of a paper should be regarded as a commitment that, should the paper be accepted, at least one of the authors will register and attend the symposium (physically or online) to present the work.
Submitted papers will be refereed for quality, correctness, originality, and relevance. Notification and reviews will be communicated via email. All accepted papers will be presented at the Symposium and published in the PMLR (Proceedings of Machine Learning Research).
There will be two Alexey Chervonenkis awards for the Best Paper and Best Student Paper, presented at the conference. Each awardee will receive £100 and a certificate.
Researchers interested in Conformal Prediction may be interested in joining our online discussion group. Future announcements and related materials will be published regularly.
All accepted papers will be presented at the conference and published by the Proceedings of Machine Learning Research (PMLR). Volume 179 at http://proceedings.mlr.press/v179/
Please make sure to use the correct style file; it should be already installed on your computer, or will be installed "on the fly". The camera-ready papers should follow the style of the Proceedings section of JMLR rather than JMLR itself.
In the end you should prepare two files: (1) your paper in PDF format; (2) the copyright form (http://proceedings.mlr.press/pmlr-license-agreement.pdf). Please upload the final version of your paper in PDF via EasyChair and email the signed copyright form to K.A.Nguyen@brighton.ac.uk
The beginning of your file will look like:
\documentclass[wcp]{jmlr} \usepackage{amsmath,amssymb,graphicx,url} \jmlrvolume{179} \jmlryear{2022} \jmlrworkshop{Conformal and Probabilistic Prediction with Applications} \jmlrproceedings{PMLR}{Proceedings of Machine Learning Research} \title{Nonparametric predictive distributions based on conformal prediction} \author{\Name{John Smith}\Email{j.smith@gmail.com}\\ \addr{Royal Holloway, University of London, Egham, Surrey, UK}\\ \Name{Minge Shen}\Email{m.shen@gmail.com}\\ \addr{Rutgers University, New Brunswick, NJ, USA}} \editor{Ulf Johansson, Henrik Boström, Khuong An Nguyen, Zhiyuan Luo and Lars Carlsson} \begin{abstract} This paper applies conformal prediction to derive predictive distributions that are valid under a nonparametric assumption. \end{abstract} \begin{keywords} Conformal prediction, predictive distributions, regression. \end{keywords}
Notice the presence of the command
\jmlrproceedings{PMLR}{Proceedings of Machine Learning Research}
Please let us know if you encounter any problems following these instructions.
For further information (which, however, can be confusing), see http://jmlr.org/proceedings/faq.html (under the heading "What is the Style File for the Proceedings?"); COPA 2021 uses the one column style file. The style file is available from the CTAN (Comprehensive TeX Archive Network) web site at http://ctan.org/tex-archive/macros/latex/contrib/jmlr you can go inside the directory called "sample_papers" and emulate the files jmlr_sample.tex and jmlr_sample.bib (the latter is only needed if you use bibtex).
Some useful advices on the JMLR style can be found at http://jmlr.org/format/format.html (However, please make sure to use the Proceedings style, as described above, rather than the main journal style).
Authors who need a UK Visa, please send an email request to K.A.Nguyen@brighton.ac.uk, including the passport number, name (as appeared on passport), nationality, current institution, and your plan to fund the trip (e.g., by yourself or by university).
We aim to send the invitation letter within 7 days.
Located in the center of Brighton, just steps from the sea, the building's extraordinary peaks and spires look is hard to miss.
Over in the pavilion's former Royal Stables and Riding School is the Brighton Museum & Art Gallery. This first-rate museum is worth visiting for its impressive collection of Art Deco pieces, its costume gallery with fashions from the 18th century.
The pier is populated with its amusement arcades, joke shops, fish-and-chip stands, and other fun things to do, including state-of-the-art thrill rides and game arcades.
Undoubtedly one of the most impressive new attractions on England's south coast, the British Airways i360 Viewing Tower is a must-visit.
The structure's circular observation platform can lift up to 200 people to heights of 453 feet for a spectacular view of the surrounding area and over the English Channel. Other features include a tearoom and gift shop.
Underneath the town's train station, the toy museum contains a vast array of vintage, rare, and unique toys from Britain and Europe.
Highlights of the museum's vast collection include antique model trains by Hornby; stuffed bears by Steiff; die-cast cars by Corgi; and all sorts of dolls, toy soldiers, farmyards, circuses, planes, and puppets.
200£
170£
50£
University of Brighton
United Kingdom