Background Check

This project is maintained by REFRAME

Background Check: A general technique to build more reliable and versatile classifiers

Miquel Perello-Nieto, Telmo de Menezes e Silva Filho, Meelis Kull and Peter Flach

We introduce a powerful technique to make classifiers more reliable and versatile. Background Check equips classifiers with the ability to assess the difference of unlabelled test data from the training data. In particular, Background Check gives classifiers the capability to (i) perform cautious classification with a reject option; (ii) identify outliers; and (iii) better assess the confidence in their predictions. We derive the method from first principles and consider four particular relationships between background and foreground distributions.

Supplementary Material

We have created an appendix with supplementary material containing the mathematical proofs, algorithms and extended explanations for figures and tables (see this pdf file).

Slides ICDM 2016

Tutorials

We have created a series of tutorials in Python and Jupyter Notbook format that introduces the motivation of Background Check and how to apply in different situations.

  1. Introduction to background check
  2. How to perform background check
  3. Cautious classification

Contributors

The code for Background Check and for the experiments submitted to ICDM-2016 has been created by Miquel Perelló Nieto (@perellonieto) and Telmo de Menezes e Silva Filho (@tmfilho).

Citing Background Check

If you want to cite this work, please use the following citation format

M. Perello-Nieto, T. M. Silva Filho, M. Kull and P. Flach. "Background Check: A general technique to build more reliable and versatile classifiers" International Conference on Data Mining (ICDM). 2016. IEEE. to appear

or the following bibtex entry:

@inproceedings{,
 title={Background Check: A general technique to build more reliable and
        versatile classifiers},
 author={Miquel Perello-Nieto and
         Telmo Menezes {Silva Filho} and
         Meelis Kull and
         Peter Flach},
 conference={International Conference on Data Mining (ICDM)},
 organization={IEEE},
 pages={},
 year={2016},
 note={to appear}
}

Support or Contact

If you are having problems executing the experiments or the tutorials, do not hesitate to open an issue or contact us.