A Reference Count Can Stay Correct While Its Identity Labels Go Wrong
Abstract
Five reference photographs per person need not provide five reliable identity labels. We reassigned one of each person's five gallery photographs in a count-preserving cycle and tested two elementary matchers on 200 untouched queries. Nearest-gallery accuracy fell from 87% to 71%; accuracy using class centroids fell from 82.5% to 76%. The centroid rule therefore lost less accuracy and became the better rule under this particular corruption. A fixed reference budget concealed a consequential change in reference integrity. The experiment establishes a controlled sensitivity result, conditional on one historical collection and one reassignment pattern.
Controlled gallery-label experiment on a historical face collection. Photograph labels are changed deliberately; no naturally occurring annotation-error rate or generated-video performance is estimated.
Original publication: November 4, 2026.
References
- Ovadia, Yaniv, et al. “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.” Advances in Neural Information Processing Systems, vol. 32, 2019. https://arxiv.org/abs/1906.02530.
- Kim, Minchul, et al. "AdaFace: Quality Adaptive Margin for Face Recognition." arXiv preprint arXiv:2204.00964, 2022. https://arxiv.org/abs/2204.00964.
- Meng, Qiang, et al. "MagFace: A Universal Representation for Face Recognition and Quality Assessment." arXiv preprint arXiv:2103.06627, 2021. https://arxiv.org/abs/2103.06627.
- Schroff, Florian, et al. "FaceNet: A Unified Embedding for Face Recognition and Clustering" arXiv preprint arXiv:1503.03832, 2015. https://arxiv.org/abs/1503.03832.
- Carroll, Raymond J., David Ruppert, Leonard A. Stefanski, and Ciprian M. Crainiceanu. “Measurement Error in Nonlinear Models.” 2006. https://doi.org/10.1201/9781420010138.
- Deng, Jiankang, et al. "ArcFace: Additive Angular Margin Loss for Deep Face Recognition" arXiv preprint arXiv:1801.07698, 2018. https://arxiv.org/abs/1801.07698.
- Kärkkäinen, Kimmo and Jungseock Joo. "FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age" arXiv preprint arXiv:1908.04913, 2019. https://arxiv.org/abs/1908.04913.
- Gebru, Timnit, et al. "Datasheets for datasets" Communications of the ACM, 2021. https://doi.org/10.1145/3458723.
- Belhumeur, P.N.; Hespanha, J.P.; Kriegman, D.J.. (1997). Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/34.598228.
- Samaria, F.S.; Harter, A.C.. (1994). Parameterisation of a stochastic model for human face identification. Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. https://doi.org/10.1109/acv.1994.341300.
- Meng, Zijie; Liu, Jiwen; Liu, Yufei; Tong, Chengzhuo; Liu, Xiaoqiang; Zhang, Yuanxing; Xu, Yulong; Wan, Pengfei. (2026). ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation. arXiv:2606.11670. https://arxiv.org/abs/2606.11670.
- Cook, R. Dennis. (1977). Detection of Influential Observation in Linear Regression. Technometrics. https://doi.org/10.1080/00401706.1977.10489493.
- Varma, Sudhir and Richard Simon. “Bias in error estimation when using cross-validation for model selection.” BMC Bioinformatics, vol. 7, no. 1, 2006. https://doi.org/10.1186/1471-2105-7-91.
- Lin, Lawrence I-Kuei. (1989). A Concordance Correlation Coefficient to Evaluate Reproducibility. Biometrics. https://doi.org/10.2307/2532051.
- Bland, J. Martin, and Douglas G. Altman. "Statistical Methods for Assessing Agreement between Two Methods of Clinical Measurement." The Lancet, vol. 327, no. 8476, 1986, pp. 307-310. https://doi.org/10.1016/S0140-6736(86)90837-8.
- Cao, Qiong, et al. "VGGFace2: A dataset for recognising faces across pose and age" arXiv preprint arXiv:1710.08092, 2017. https://arxiv.org/abs/1710.08092.
- Roberts, David R., et al. “Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure.” Ecography, vol. 40, no. 8, 2017, pp. 913-929. https://doi.org/10.1111/ecog.02881.
- AT&T Laboratories Cambridge. (2026). Modified Olivetti faces: Sam Roweis MATLAB version [Data archive; mirror accessed 7 November 2026]. File identity and row-label layout verified against the scikit-learn maintainer loader. https://ndownloader.figshare.com/files/5976027.
- Huang, Gary B., Marwan Mattar, Tamara Berg, and Erik Learned-Miller. "Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments." ECCV Workshop on Faces in Real-Life Images, 2008. https://cs.umass.edu/~elm/papers/Huang_eccv2008-lfw.pdf.
- Huber, Peter J.. (1964). Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics. https://doi.org/10.1214/aoms/1177703732.
- Kemelmacher-Shlizerman, Ira, et al. "The MegaFace Benchmark: 1 Million Faces for Recognition at Scale" arXiv preprint arXiv:1512.00596, 2015. https://arxiv.org/abs/1512.00596.
- Turk, M.A.; Pentland, A.P.. (1991). Face recognition using eigenfaces. Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/cvpr.1991.139758.
- Natarajan, Nagarajan; Dhillon, Inderjit S.; Ravikumar, Pradeep; Tewari, Ambuj. (2013). Learning with Noisy Labels. Advances in Neural Information Processing Systems 26. https://papers.nips.cc/paper_files/paper/2013/hash/3871bd64012152bfb53fdf04b401193f-Abstract.html.
- White, Halbert. “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica, vol. 48, no. 4, 1980, pp. 817-838. https://doi.org/10.2307/1912934.
- Sculley, D., et al. "Hidden Technical Debt in Machine Learning Systems." Advances in Neural Information Processing Systems, vol. 28, 2015. https://papers.nips.cc/paper/2015/hash/86df7dcfd896fcaf2674f757a2463eba-Abstract.html
- Xiaoyang Tan; Triggs, Bill. (2010). Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions. IEEE Transactions on Image Processing. https://doi.org/10.1109/tip.2010.2042645.
- Wilkinson, Mark D.; Dumontier, Michel; Aalbersberg, IJsbrand Jan; Appleton, Gabrielle; Axton, Myles; Baak, Arie; Blomberg, Niklas; Boiten, Jan-Willem; da Silva Santos, Luiz Bonino; Bourne, Philip E.; Bouwman, Jildau; Brookes, Anthony J.; Clark, Tim; Crosas, Mercè; Dillo, Ingrid; Dumon, Olivier; Edmunds, Scott; Evelo, Chris T.; Finkers, Richard; Gonzalez-Beltran, Alejandra; Gray, Alasdair J.G.; Groth, Paul; Goble, Carole; Grethe, Jeffrey S.; Heringa, Jaap; ’t Hoen, Peter A.C; Hooft, Rob; Kuhn, Tobias; Kok, Ruben; Kok, Joost; Lusher, Scott J.; Martone, Maryann E.; Mons, Albert; Packer, Abel L.; Persson, Bengt; Rocca-Serra, Philippe; Roos, Marco; van Schaik, Rene; Sansone, Susanna-Assunta; Schultes, Erik; Sengstag, Thierry; Slater, Ted; Strawn, George; Swertz, Morris A.; Thompson, Mark; van der Lei, Johan; van Mulligen, Erik; Velterop, Jan; Waagmeester, Andra; Wittenburg, Peter; Wolstencroft, Katherine; Zhao, Jun; Mons, Barend. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data. https://doi.org/10.1038/sdata.2016.18.
- Stevens, S. S.. (1946). On the Theory of Scales of Measurement. Science. https://doi.org/10.1126/science.103.2684.677.
- Rousseeuw, Peter J.. (1984). Least Median of Squares Regression. Journal of the American Statistical Association. https://doi.org/10.1080/01621459.1984.10477105.
- Wang, Hao, et al. "CosFace: Large Margin Cosine Loss for Deep Face Recognition" arXiv preprint arXiv:1801.09414, 2018. https://arxiv.org/abs/1801.09414.
Published
Issue
Section
License
Copyright (c) 2026 William Hall, Tyler Wells, Ryan King

This work is licensed under a Creative Commons Attribution 4.0 International License.