Machine Vision and Applications (2018) 29:873–890
Kinship veriﬁcation from facial images and videos: human versus
Miguel Bordallo Lopez
· Abdenour Hadid
· Elhocine Boutellaa
· Jorge Goncalves
· Vassilis Kostakos
Received: 1 November 2017 / Revised: 12 March 2018 / Accepted: 26 April 2018 / Published online: 29 May 2018
© Springer-Verlag GmbH Germany, part of Springer Nature 2018
Automatic kinship veriﬁcation from facial images is a relatively new and challenging research problem in computer vision. It
consists in automatically determining whether two persons have a biological kin relation by examining their facial attributes. In
this work, we compare the performance of humans and machines in kinship veriﬁcation tasks. We investigate the state-of-the-
art methods in automatic kinship veriﬁcation from facial images, comparing their performance with the one obtained by asking
humans to complete an equivalent task using a crowdsourcing system. Our results show that machines can consistently beat
humans in kinship classiﬁcation tasks in both images and videos. In addition, we study the limitations of currently available
kinship databases and analyzing their possible impact in kinship veriﬁcation experiment and this type of comparison.
Keywords Kinship veriﬁcation · Face analysis · Biometrics · Crowdsourcing
It is common practice for humans, to visually identify rel-
atives from faces. Relatives usually wonder which facial
attributes do a new born inherit from each parent. The
human ability of kinship recognition has been the object of
many psychological studies [21,24]. Inspired by these stud-
ies, automatic kinship (or family) veriﬁcation [30,84] has
been recently considered as an interesting and open research
problem in computer vision and it is receiving an increasing
attention by the research community.
Automatic kinship veriﬁcation from faces aims to deter-
mine whether two persons have a biological kin relation by
comparing their facial attributes. This is a difﬁcult task that
Electronic supplementary material The online version of this article
(https://doi.org/10.1007/s00138-018-0943-x) contains supplementary
material, which is available to authorized users.
Miguel Bordallo Lopez
Center for Machine Vision and Signal Analysis, University of
Oulu, Oulu, Finland
School of Computing and Information Systems, University of
Melbourne, Melbourne, Australia
Center for Ubiquitous Computing, University of Oulu, Oulu,
sometimes needs to deal with subtle similarities that often
escape the human eye.
Kinship veriﬁcation has a role in numerous applications.
In addition to biological relation veriﬁcation, kinship estima-
tion is an important feature in the automatic analysis of the
huge amount of photographs daily shared on social media,
since it helps understanding the family relationships in these
photographs. It can also be used for automatically organizing
family albums and generating family trees based on present
or historical photographs. In addition to image classiﬁcation,
kinship veriﬁcation proves also useful in cases of missing
children and elderly people with reduced cognitive capabil-
ities, as well as in kidnaping cases.
All these applications assume an automatic kinship veri-
ﬁcation system able to assess kin relationships from limited
input data. However, and despite the recent progress, kinship
veriﬁcation from faces remains a challenging task. It inherits
the research problems of face veriﬁcation from images cap-
tured in the wild under adverse pose, expression, illumination
and occlusion conditions.
In addition, kinship veriﬁcation should deal with wider
intra-class and inter-class variations. Moreover, automatic
kinship veriﬁcation can face new challenges since unbal-
anced datasets naturally exist in a family, and a pair of input
images may be from persons of different sex and/or with a
large age difference.