Data-Driven Research on the Matching Degree of Eyes, Eyebrows and Face Shapes

There is a close relationship between the attractiveness of the face and the facial features. The shape of the facial features determines the level of attractiveness, in which the eyes and eyebrows are particularly vital. In this article, we proposed a method to study the facial attractiveness by combining global face shape and local geometric features of eye and eyebrow and using computer big data analysis for assistance. Firstly, we collected 300 images of East Asian female and use machine learning methods to evaluate the attractiveness scores of face images. Secondly, geometric models were constructed separately for the eyebrows and the eyes to obtain their geometric and shape features. Correlation analysis was performed on the obtained data to study their shape matching of different facial attractiveness rating levels. Finally, the relationship between the shape of face and eyebrows was analyzed by combining the facial ratio and the geometric features of the eyebrows. The research in this article can provide reference for medical and beauty institutions and women’s makeup, and further study in the field of facial aesthetic analysis based on geometric features.

Introduction

Beauty has always been the center of countless topics. Maslow believes that human beings always have a yearning for beauty. Appreciation of beautiful things is an instinctive pursuit of human beings, and beautiful things are refreshing. Face is a special aesthetic object, and people’s evaluation of the attractiveness of others’ faces often occurs in daily life (Aharon et al., 2001). In society, individuals with beautiful appearance have more favorable social living conditions, such as having a better spouse, or a better job opportunity, or being more likely to be assisted (Bashour, 2006; Johnston, 2006). Therefore, with the continuous improvement of people’s living standards, an increasingly number of people are investing a large amount of money and energy to make their appearance more attractive. In cognitive psychology, researchers have overturned the long-held argument that beauty is subjective, and a great number of experiments have discovered that there is a high extent of agreement between what faces are beautiful, which is highly consistent with culture, race, age, gender (Langlois et al., 2000; Aarabi et al., 2001; Koscinski, 2009). This consistency can lead us to believe that the perception of facial attractiveness is data-driven (Etcoff, 1994). Therefore, the use of computer big data to analyze facial attractiveness can further study the facial attractiveness. If we can grasp the rule of the attractiveness of the face, it will be of great application value, such as beauty salons.

Among the factors affecting the attractiveness of the face, the geometric features of the face are essential. There are many kinds of judgment standards for the beauty of the face in ancient and modern China and foreign countries (Perrett et al., 1999; Dobke et al., 2006; Rhodes, 2006; Atiyeh and Hayek, 2008; Holland, 2008; O’Toole et al., 2015), especially the geometric features of the face, which include the geometrical relationship of the position, angle and proportion of each organ of the human face, as well as the appearance of eyebrows, eyes, nose and mouth (Xu, 2008). The aesthetic analysis of face based on geometric features is of great research value in the field of facial aesthetics (Eisenthal, 2006). Yan et al. (2017) tracked the observer’s eye’s gaze position and pupil size changes, ultimately found that participants mainly judge the attractiveness by paying attention to the nose. Penna et al. (2015) studied the effect of the lip on the attractiveness of the face, shows what it is that makes lips attractive and their connection between gender differences. Aarabi et al. (2001) extracts feature points such as face contours, eyes, eyebrows and mouths from face images of two-dimensional positive neutral expressions, and uses a vector composed of the ratios of feature points to represent faces to analyze the attractiveness of the face from local area; Young et al. (2006) studied the ideal distance in the eyes, nose, ears and lips based on a new theory on beauty: The Circles of Prominence (COP), which theorizes that the width of the iris serves as an ideal for multiple distances and shapes within the face. The data supports that the ideal distance for eyebrow height, nasal bridge and tip width, and lower lip height are all 1 iris width as predicted by the COP. The ideal height of the upper lip was statistically found to be 1/2 iris width. Among the facial features, eyebrows and eyes occupy an indispensable proportion. It is often said that the eyes are the windows of the soul, then we can consider the eyebrows as curtains. The eye is a picture of life, and the eyebrow is the picture frame. Female used to pencil eyebrow to make themselves more beautiful in ancient times, which lead to many poets describe female’s eyebrows. In modern times, female also make them more attractive through trimming or tattooing eyebrows. Contrast has been found to be a factor in perceived beauty of female faces. Russell (2009) found that cosmetics may function in part by exaggerating a sexually dimorphic attribute – facial contrast – to make the face appear more feminine and hence attractive. This would be especially true for eyebrows, which are made darker than the surrounding skin by the application of eye-liner and other cosmetics. So by changing the shape of the eyebrows through makeup, choosing the eyebrow shape that suits you can make the face more attractive. Changing the eyebrows can change the overall facial impression (Morikawa, 2012), so eyebrows play an important role in the facial features. Studies have shown that the eye accounts for the largest proportion of the five sense organs that affect the attractiveness of the face (Cunningham, 1986), and a pair of beautiful eyes tends to leave a good impression on people. Therefore, the attractiveness of the face is closely related to the geometric and shape characteristics of the eyebrows and eyes. Mitsuhiro and Kitaoka (2016) studied the connection between facial attractiveness, likability and characteristics of the eye. It was confirmed that eyes were important clues in facial evaluations whereas some of the characteristics of eyes were selectively related with facial beauty and attractiveness, and likability. Matsushita et al. (2015) studied the influence of the shape and position of female eyebrows on eyes. The results show that facial attractiveness is directly affected by eyebrows and indirectly by characteristics of eyes, and large eyes tend to be more attractive.

Female demand more cosmetics than male, which means that compared with male, “beauty” is more important for female, so there are many studies on the attractiveness of female’s faces. Rizvi et al. (2014) use a hybrid approach based on Beauty Mask and Facial Proportions to predict female facial beauty. Dantcheva and Dugelay (2014) studied the relationship between subjective perception of female facial beauty based on anthropometric, non-permanent and acquisition characteristics. Fan et al. (2012) used computer software to generate different ratios of facial images that were not affected by hairstyle, expression, skin color, texture, etc. By analyzing the relationship between these images and attractiveness levels, they finally discovered an optimal proportion of attractive female faces and established a model to predict face attractiveness with good predictability. Research shows that facial beauty is a universal concept which can be learned by a machine (Eisenthal, 2006). Rizvi et al. (2014) presented a hybrid approach to estimate female facial beauty based on Machine Learning techniques. They use a combination of two approaches: Beauty Mask and Facial Proportions, to find the features that constitute Ideal Female facial beauty and thus, develop a female facial beauty scoring system based on the same. Gray et al. (2010) concentrated on fully-automated learning methods. Instead of annotating facial features manually, they just use the original pixels as input to research and develop intelligent systems to learn the female facial aesthetics and generate human-like predictors. In recent years, many existing databases have been used to research face attractiveness (Whitehill and Movellan, 2008), and training people’s cognition of facial beauty with computer technology. Liu et al. (2017) suggested an end-to-end label distributed learning (LDL) framework with deep convolution neural network (CNN) and geometric features, and made a lot of experiments on SCUT-FBP dataset (Liang et al., 2018). Choudhary and Gandhi (2017) proposed a study of facial attractiveness in a machine learning environment and various techniques were applied to SCUT-FBP facial image data set (Liang et al., 2018) to learn facial attractiveness. It is shown that facial beauty is a common concept that machines can learn.

Although there are many methods to study facial features and attractiveness, many studies focus on the distance and proportion between the facial features, without considering the shape features and matching degree of them. Face attractiveness can also be measured more comprehensively by the connection between local and global of the face. Previous studies have concentrated on one of the facial features associated with facial attractiveness, while eyebrows and eyes are very close to each other in the face, so we predict that there is a certain relationship between the combination of the two features and facial attractiveness. In this article, our contributions can be summarized as follows:

Firstly, we collect 300 Asian female images on the Internet and use machine learning to evaluate the attractiveness scores of all face images;

Secondly, construct geometric models of the eyebrows and eyes, and calculate the parameters of the eyebrows and eyes according to the characteristics of them, including the length, area, average width, curvature of the eyebrows, the size of the eyes and pupils, and the proportion of iris to whole eye. Combine the obtained data of geometric and shape features to analyze the relationship between the shape matching of the two and the facial attractiveness; and

Finally, extract the feature points of the face image, and analyze the relationship between the shape of face and eyebrow according to the distance between the feature points, combined with the face proportion and the geometric and shape features of the eyebrows.

In the next section we will introduce the materials and methods used in this work. Then the experimental results and discussion are given. The final section summarizes this article.

Materials and Methods

Data Collection

In this study, we used an image dataset containing 300 East Asian female face images publicly downloaded at https://image.baidu.com/ and http://www.manmankan.com/dy2013/mingxing/neidi/nvmingxing.shtml#. These images have a neutral expression, a simple background, and minimal occlusion, all of which contribute to the facial beauty perception of geometry and appearance. The main reason behind use of this specific dataset and not using any other, is that our research here requires that these female images have clear eyebrows and eyes that are not obscured by the hair. The attractiveness scores of these images are then evaluated by machine learning methods for subsequent research.

Facial Attractiveness Rating Prediction

In today’s image processing field, due to the rapid development of science and technology and artificial intelligence, machine learning is occupying an increasingly important position. To accurately classify the beauty of 300 East Asian female’s faces, we used machine learning to evaluate the attractiveness scores of these facial images. The evaluation model is divided into two modules: training and testing. In the training process, the 68 feature points of the face image are extracted firstly, then the face feature vectors are extracted, and finally the training is performed according to the obtained feature vectors. The label of the average attractiveness score assessed by real-life evaluators cited in the classification model is from the article (Xie et al., 2015), which has been shown to be suitable for assessing facial attractiveness scores of Asian female. During the testing, the final predicted score of facial attractiveness is obtained by inputting the facial image into the predictor. The specific prediction process is shown in Figure 1.

FIGURE 1

Figure 1. Prediction process of facial attractiveness score (Clarification: the training images were obtained from the SCUT-FBP dataset, which is publicly available at http://www.hcii-lab.net/data/SCUT-FBP, written informed consent of the testing image was obtained from the individual for the publication).

Landmark Localization

The frontal_face_detector of dlib library in Python is primarily used as a face extractor and finally the facial landmark is written into the file. The obtained facial marked point image is shown in Figure 2.

FIGURE 2

Figure 2. Facial marked point image (Clarification: written informed consent was obtained from the individual for the publication of this image).

Feature Generation

The four facial ratio features are calculated from each training image by using the feature points, and the obtained features are saved to a file.

Model Training

The input data used in the training model are the features of the images and the artificial score label. In the process of training, PCA algorithm is used to reduce the dimension of the feature, and then the random forest training model is adopted.

Attractiveness Score Prediction

According to the data and model obtained above, a CNN is designed for predicting the attractiveness score of self-selected images. Figure 3 shows the architecture of the network. The network included mainly six convolution layers that feature maps are 50, 100, 150, 200, 250, and 300. The size of the corresponding filter is 5 × 5, 5 × 5, 4 × 4, 4 × 4, 4 × 4, and 2 × 2. In this network, there are also fully connected layers. Finally, we can output the attractiveness score of the input image.

FIGURE 3

Figure 3. Network architecture of CNN.

Building a Model of Eyebrow

The common shapes of eyebrows are shown in Figure 4. According to the shape characteristics of eyebrows, the following geometric parameters such as the length, area, average width and bending degree of eyebrows are used to measure the shape model of eyebrows. These parameters can directly reflect the characteristics of the appearance of eyebrows.

FIGURE 4

Figure 4. Common shapes of eyebrow.

Eyebrow Threshold Segmentation

Firstly, the unilateral eyebrow image is taken from the frontal face image to normalize the size of 100 × 50, and then adaptively thresholded. Since the binary image after the adaptive thresholding has noise, the Blob analysis is used to eliminate the noise and a small amount of hair interference, a binary image of the normalized eyebrow area is obtained. For the value B (i, j) of the gray image at the (i, j) position, it can be obtained from Eq. 1:

$$B(i,j)=\{\begin{array}{c}\hfill 255,gray(i,j)\ge T\hfill \\ \hfill 0,gray(i,j)<T\hfill \end{array}(1)$$

where T is an adaptive closed value, that is, the threshold itself is a variable, and the closed value T is different at each pixel. It is obtained by calculating the weighted average of the neighborhood around the pixel and then subtracting a constant. The formula is as follows:

$$T=\frac{1}{N}\sum _{\mathrm{i}=1}^N{G}_{\mathrm{i}}-C(2)$$

where N is the number of neighboring pixels around the pixel to be calculated; G is the gray value of the i-th neighboring pixel; and C is a given constant.

Through adaptive thresholding, eyebrow regions can be accurately segmented, as shown in Figure 5. The first row is the original image of eyebrows, and the second row is the result of threshold segmentation and denoising.

FIGURE 5

Figure 5. The first line is the original image of the eyebrows, the second line is the binary image of the eyebrows after threshold segmentation and denoising, and the third line is the curve of the eyebrows.

Geometric Parameters to Measure Eyebrow Shape

The area of eyebrows

Since the eyebrow images have been normalized, the area of the eyebrow varies among individuals. It can be obtained by dividing the binary eyebrow image by threshold and counting the number of pixels in the eyebrow area, that is, the number of white pixels in the binary image.

The length of eyebrows

To calculate the length of eyebrows, first extract the curve of eyebrows. The curve diagram of eyebrows can be obtained from the binary graph of eyebrows, and the process is as follows: Firstly after obtaining the eyebrow binaries, the eyebrow area was detected; Secondly take the intermediate value of the maximum and minimum values of the ordinate on each abscissa of the eyebrow region, and traverse all abscissa values to obtain a series of points; Thirdly connect the points to get the curve of the eyebrows. As shown the third row in Figure 5, after the eyebrow curve is obtained, the number of white pixels in the statistical curve is the length L of the eyebrow. Formula is:

$$L=\{\begin{array}{c}\hfill L+1,gray(i,j)=255\hfill \\ \hfill L,gray(i,j)=0\hfill \end{array}(3)$$

Average width of the eyebrows

It is defined as the area of the eyebrows divided by the length of the eyebrows.

The curvature of the eyebrows

The curve of eyebrow is a crucial parameter of eyebrow appearance, it is the important basis that people differentiates eyebrow. We need to get the curve of eyebrows first, and calculate the bending degree of eyebrows through the curve of eyebrows. The steps to calculate eyebrow curvature are as follows: Firstly, calculate the linear equation of two ends of eyebrow curve; Secondly, the distance from the point to the straight line is used to calculate the distance from all points on the eyebrow curve to the line equation except two end points; Thirdly, the furthest distance is defined as the degree of eyebrow curvature.

The calculation formula of distance M from point (x0,y0) to line Ax + By + C = 0 is as follows:

$$M=\frac{\left|A{x}_0+B{y}_0+C\right|}{\sqrt{A^2+B^2}}(4)$$

Building a Model of Eye

When we observe the shape of the eye, we can find that the eye can be approximately elliptical and the iris can be approximately circular. Therefore, in this article, we use least-squares fitting of ellipse and circle to obtain the geometric parameters of eyes and irises. Firstly, the image of the eye area is normalized to the size of 100 × 50, and then the edge detection and processing of the eye area are carried out with sobel operator to obtain the edge of the eye and iris, and the contour lines of the two are obtained by ellipse and circle fitting. Finally, the contour equation coefficient is used to calculate the area of eyes and irises.

Extraction of Eye and Iris Contour

Firstly, the normalized eye image is processed by edge detection with sobel operator, and the edges of eyes and irises are extracted and binarized. In the edge image processed in the previous step, there are both edge points of the eye contour and edge points of the non-eye contour. The true contours of the eye and iris are extracted by searching for the connected area with the largest edge pixel and making interference correction. The result of these two steps is shown in Figure 6.

FIGURE 6

Figure 6. These four are images of eye after normalized, processed by edge detection with sobel operator, searched for the connected area with the largest edge pixel and made interference correction.

Least-Square Fitting Ellipses of Eye Contour

In this article, the direct least-square method is used for fitting. The core of least squares technique is to find parameter sets that minimize the distance measure between data points and ellipses (Fitzgibbon et al., 1999), which have high efficiency and good stability against noise. The basic principle of direct least square method (Fitzgibbon et al., 1999) is explained below.

The general equation of the plane quadratic curve of an ellipse can be expressed by Eq. 5:

$$F(A,X)=A\cdot X$$

$$=a{x}^2+bxy+c{y}^2+dx+ey+f=0(5)$$

where $A={[a,b,c,d,e,f]}^{\mathrm{T}}$ and $X={[x^2,xy,y^2,x,y,1]}^{\mathrm{T}}$. F(A; X_i) is called the “algebraic distance” of a point [x_i, y_i] to the conic F(A; X) = 0. According to geometric knowledge, when the curve coefficient satisfies 4ac - b² = 1, the general quadratic equation represented by the above formula is ellipse. Therefore, the ellipse fitting problem is transformed to find the minimum sum of squared algebraic distances between points and quadratic curves F(A; X) = 0 under the constraint of 4ac - b² = 1:

$$\sum _{\mathrm{i}=1}^{N}F{(A;X_{\mathrm{i}})}^2(6)$$

where N is the number of fitting points, giving

$$D_{\mathrm{N}\times 6}={[X_1{X}_2\mathrm{\dots }{X}_{\mathrm{N}}]}^{\mathrm{T}}(7)$$

$$C=\left[\begin{array}{cccccc}\hfill 0\hfill & \hfill 0\hfill & \hfill 2\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill -1\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 2\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right](8)$$

where D is called the design matrix and C is the matrix representing the constraints. Now, the constrained ellipse fitting problem is reduced to minimizing E = ŞDA޲ subject to the constraint A^TCA = 1. By introducing the Lagrange multiplier and differentiating, we obtain the system of simultaneous equations:

$$\{\begin{array}{c}\hfill 2D^{T}DA-2\mathrm{\lambda }CA=0,\hfill \\ \hfill A^{T}CA=1.\hfill \end{array}(9)$$

This may be rewritten as:

$$\{\begin{array}{c}\hfill SA=\mathrm{\lambda }CA,\hfill \\ \hfill A^{T}CA=1.\hfill \end{array}(10)$$

where S is the scatter matrix D^TD. For the first formula of the Eq. 10, its six sets of solutions can be obtained by considering the generalized eigenvectors. For any μi ∈ R⁺,(λi, μiui) is also a solution, where i = 1,2,...,6, so (λi, μiui) must also satisfy the second formula of the Eq. 10, that is, ${\mathrm{\mu }}_{\mathrm{i}}^2{u}_{\mathrm{i}}^{T}C{u}_{\mathrm{i}}=1$, then

$${\mathrm{\mu }}_{\mathrm{i}}=\sqrt{\frac{1}{u_{\mathrm{i}}^{T}C{u}_{\mathrm{i}}}}=\sqrt{\frac{{\mathrm{\lambda }}_{\mathrm{i}}}{u_{\mathrm{i}}^{T}S{u}_{\mathrm{i}}}}(11)$$

According to the proof in the literature (Fitzgibbon et al., 1999), we know that λi > 0, and the sign of generalized eigenvalue is the same as that of Q^-1CQ^-1 and conditional constraint matrix C, where S = Q².

Since the eigenvalue of C is {–2, –1, 2, 0, 0, 0}, only the unique generalized eigenvalue λi > 0 and the corresponding generalized eigenvector λ_i are the solutions of ellipse fitting. The only solution:

$$A_j={\mathrm{\mu }}_{\mathrm{i}}{u}_{\mathrm{i}},j\in i(12)$$

After obtaining the six parameters of the elliptic equation, the geometric center of the ellipse is:

$$x_c=\frac{be-2cd}{4ac-b^2}(13)$$

$$y_c=\frac{bd-2ae}{4ac-b^2}(14)$$

The long and short half axes are:

$$m=\sqrt{\frac{2(a{x}_c^2+c{y}_c^2+b{x}_c{y}_c-f)}{a+c-\sqrt{{(a-c)}^2+b^2}}}(15)$$

$$n=\sqrt{\frac{2(a{x}_c^2+c{y}_c^2+b{x}_c{y}_c-f)}{a+c+\sqrt{{(a-c)}^2+b^2}}}(16)$$

Then the area of the eye after ellipse fitting is:

$$eyearea=\mathrm{\pi }mn(17)$$

Least-Square Fitting Circles of Iris Contour

The iris is approximately circular, so use the least-square circle to fit the iris profile. The basic principle of least square circle method (Moura and Kitney, 1991) is explained below. A circle in the plane can be determined according to the center (A, B) and radius R. The general formula of the circle equation on the plane is

$$x^2+y^2+ax+by+c=0(18)$$

The center (A, B) and radius R:

$$\{\begin{array}{c}\hfill A=-0.5a\hfill \\ \hfill B=-0.5b\hfill \\ \hfill R=0.5\sqrt{a^2+b^2-4c}\hfill \end{array}(19)$$

The general Eq. 18 of the circle is a linear equation of a, b and c. The mathematical model of circle fitting is established by using the least-square method to obtain the value of parameter a, b and c, and then the actual parameters A, B and R of the circle are according to the Eq. 19. In the N(N ≥ 3) sets of data (xi, yi), i = 1,2,...,N originally measured, according to the general formula and the least squares principle, the minimum value of the objective function is required:

$$F(a,b,c)=\sum _{\mathrm{i}=1}^N{(x_{\mathrm{i}}^2+y_{\mathrm{i}}^2+a{x}_{\mathrm{i}}+b{y}_{\mathrm{i}}+c)}^2(20)$$

Take the partial derivative of F(a, b, c) with respect to a, b, c, set the partial derivative equal to zero to obtain the extreme point and

$$\left[\begin{array}{c}\hfill {\displaystyle \sum _{i=1}^N}{x}_i^3+{\displaystyle \sum _{i=1}^N}{x}_i{y}_i^2\hfill \\ \hfill {\displaystyle \sum _{i=1}^N}{y}_i^3+{\displaystyle \sum _{i=1}^N}{x}_i^2{y}_i\hfill \\ \hfill {\displaystyle \sum _{i=1}^N}{x}_i^2+{\displaystyle \sum _{i=1}^N}{y}_i^2\hfill \end{array}\right]+\left[\begin{array}{ccc}\hfill {\displaystyle \sum _{i=1}^N}{x}_i^2\hfill & \hfill {\displaystyle \sum _{i=1}^N}{x}_i{y}_i\hfill & \hfill {\displaystyle \sum _{i=1}^N}{x}_i\hfill \\ \hfill {\displaystyle \sum _{i=1}^N}{x}_i{y}_i\hfill & \hfill {\displaystyle \sum _{i=1}^N}{y}_i^2\hfill & \hfill {\displaystyle \sum _{i=1}^N}{y}_i\hfill \\ \hfill {\displaystyle \sum _{i=1}^N}{x}_i\hfill & \hfill {\displaystyle \sum _{i=1}^N}{y}_i\hfill & \hfill N\hfill \end{array}\right]\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left[\begin{array}{c}\hfill a\hfill \\ \hfill b\hfill \\ \hfill c\hfill \end{array}\right]=\left[\begin{array}{c}\hfill 0\hfill \\ \hfill 0\hfill \\ \hfill 0\hfill \end{array}\right](21)$$

By solving the above equation a, b, and c can be obtained. The center coordinate and radius of the circle can be obtained by combining formula (19), and the iris area is

$$irisarea=\mathrm{\pi }{R}^2(22)$$

Fitting Results

Based on the above method of least squares ellipse and circle fitting for eyes and irises, the final fitting results are shown in Figure 7. For the eyes with clear outline, the fitting effect is better.

FIGURE 7

Figure 7. The fitting results of eyes and irises.

Facial Ratio Represents the Shape of Face

Face shape is the most intuitive feature of face image, and the information of face shape is more stable, so people pay special attention to the face shape. Common face shapes include heart, round, oval, square and so on. In order to study the relationship between the shape of face and eyebrow, we used the facial transverse/longitudinal ratio D1/D2 shown in Figure 8 to represent the difference in face shape. And we design a custom hairline detector because the height of the face needs to be calculated, which adds a point to the hairline on the basis of the 68 feature points in Figure 2. The transverse to longitudinal ratios of four typical facial shapes are shown in Figure 9.

FIGURE 8

Figure 8. D1/D2 is the transverse/longitudinal ratio of face. The green rectangle marks the point of the hairline after adding hairline detector (Clarification: written informed consent was obtained from the individual for the publication of this image).

FIGURE 9

Figure 9. Examples of four typical facial shapes: from left to right are square, oval, round, and heart. And the facial transverse/longitudinal ratio D1/D2 are 0.754, 0.791, 0.872, and 0.736, respectively (Clarification: written informed consents were obtained from the individuals for the publication of these images).

Results and Discussion

Analysis of Attractiveness Score

The 300 facial attractiveness scores obtained by machine learning method are: 92 images in the range of 2–3 scores, 146 images in the range of 3–4 scores, and 62 images in the range of 4–5 scores (the entire score range is 1–5, with a maximum of 5 points). Faces with higher attractiveness scores accounted for a larger proportion in order to promote effective researching of beautiful faces.

Correlation Analysis of Eyebrows and Eyes

The distributions of eyebrow parameters obtained by using the geometric feature model of eyebrows constructed in the previous section are shown in Figure 10. Analysis of the data shows that the lengths of eyebrows are mostly concentrated in 80–90 pixels, and the eyebrows with the bending degree in the range of 5–13 pixels account for nearly two-thirds. The areas of eyebrows are mostly distributed in the range of 900–1100 pixels, while the average widths are concentrated in 11–13 pixels. The parameters in the middle range of eyebrows accounted for the majority. And the distributions of eye parameters obtained after using the eye geometric feature model constructed in the previous section are shown in Figure 11. The geometric parameters of eyes and irises are analyzed, and it is found that the sizes of eyes are concentrated in 600–1000 pixels, while the proportions of iris to whole eye are concentrated in 0.5–0.6. In the faces with attractiveness scores of 2–3, the images with eyes larger than 800 pixels account for about 24%. In the faces with attractiveness scores of 3–4, the images with eyes larger than 800 pixels account for about 46%. And in the faces with attractiveness scores of 4–5, 56% of the images have eyes larger than 800 pixels. From these data, it can be concluded that eye size is strongly related to facial attractiveness, but the relationship between them is not absolute. Not all faces with high attractiveness scores have large eyes, but small eyes with the appropriate eyebrow shape can also make face increasingly attractive. So we connect the eyes and eyebrows that are visually closely related to the eyes and analyze them with parameters.

FIGURE 10

Figure 10. The four figures are length, curvature, area, average width distribution of eyebrow, respectively. The horizontal axis is the interval of the described parameter. The vertical axis is the number of images in the parameter interval.

FIGURE 11

Figure 11. The three figures are distributions of area of iris, proportion of iris to whole eye, area of eye, respectively. The horizontal axis is the interval of the described parameter. The vertical axis is the number of images in the parameter interval.

Through analyzing the data, it is found that there is a relationship between eyebrow curvature and eye size. We have drawn the scatter diagram of the two when the attractiveness scores of face images are within the range of 2–3, 3–4, and 4–5, respectively, as shown in Figure 12. As can be seen from the figure, the size of eyes decreases with the increase of eyebrow curvature, and as the attractiveness score increases, a growing number of points conform to this rule and the trend becomes obvious. Except for a small number of non-conforming samples, we can draw a conclusion that the combination of large eyes with low-bending eyebrows and small eyes with high-bending eyebrows will lead to a higher score of facial attractiveness. This conclusion can be explained by “Three Foreheads and Five Eyes,” which reflects the traditional Chinese aesthetics. It is the general law of the face structure derived by the ancient Chinese painters based on the position of the facial features. The “three foreheads” are divided into the upper, the atrium and the lower. The upper refers to the area from hairline to eyebrow peak. Atrium means from under the eyebrow to the bottom of the nose, and the lower means from the nose to the lower jaw. Eyebrow with a higher degree of curvature generally have a higher brow ridge, which can increase the proportion of the area occupied by the eyebrows and the eyes, and longitudinally lengthen the eye area to make the eyes look larger. For large eyes, the gentle brow ridge can neutralize the area occupied by the eyebrow on the human face and focus people’s attention on the eyes.

FIGURE 12

Figure 12. The three figures are the scatter diagrams of the eye area and curvature of eyebrow when the attractiveness scores of face images are within the range of 2–3, 3–4, and 4–5, respectively.

Correlation Analysis of Eyebrows and Face Shape

Data analysis of D1/D2 and eyebrow parameters revealed that there is a relationship between the shape of face and eyebrow. D1/D2, which is the ratio of transverse to longitudinal of the face, can be used to characterize the change in face shape. The closer D1/D2 is to 1, the rounder the face shape is, while the smaller D1/D2 is, the longer the face shape is. Figure 13 compares the relationship between D1/D2 and eyebrow curvature in different attractiveness score ratings. According to the analysis of the above data, in the range of 3–4 and 4–5 of attractiveness score, the average transverse to longitudinal ratio raises continuously with the increase of eyebrow curvature parameter interval, and this variation is more pronounced in the range of 4–5 score. This suggests that rounder faces are better matched with higher curved eyebrows, while longer faces are better matched with lower curved ones. In terms of vision, when the shape of face is long, the eyebrows with low ridge are needed to neutralize, which will shorten the face. If the brow ridge is high, it will appear that the face is pulled longer. Brows with lower curvature can visually cause the face to stretch laterally, so such eyebrows are more suitable for long faces. Contrary to the long face, the cheek of the round face is fuller, which is needs to be elongated and slim. The eyebrows that are high in height and relatively obvious in turning can enlarge the distance between the eyebrows and the mandible, so the round face is suitable for drawing eyebrows with high bending. This combination of face and eyebrows will make face increasingly attractive.

FIGURE 13

Figure 13. The D1/D2 of the longitudinal axis is the average value in each eyebrow curvature parameter interval. The three curves show the relationship of the two between the attractiveness score intervals in 2–3, 3–4, and 4–5, respectively.

Conclusion

Facial attractiveness is closely linked to our daily life and social interaction. In this article, we propose an approach to study the attractiveness by combining the geometric and shape features of the eyebrows and the eyes, and the shape of face. The data used are 300 Asian female face images and scored using machine learning methods. The geometric models of eyebrows and eyes are constructed separately, and the proportions of face are calculated. Combine these geometric features in the face of different attractiveness rating levels to research the matching degree of eyebrows, eyes and face shape. The results show that the high-bending eyebrows with small eyes and round face, low-bending eyebrows with large eyes and longer face will make the facial image have a higher attractiveness score. This method has certain value and practical significance for exploring the relationship between the matching degree of facial features and the attractiveness of face, and can be used as a reference for plastic surgery hospitals or beauty institutions.

Author Contributions

JZ proposed the idea. JZ, MZ, and CH analyzed the data. All authors collected all images, designed the experiments, and drafted and revised the manuscript.

Funding

This work was supported by National Natural Science Foundation of China (Grant No. 61701401) and Natural Science Foundation of Shaanxi Province (Grant No. 2019KJXX-061).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Aarabi, P., Hughes, D., Mohajer, K., and Emami, M. (2001). “The automatic measurement of facial beauty,” in Proceedings of the IEEE International Conference on Systems, Tucson, AZ, doi: 10.1109/ICSMC.2001.972963

CrossRef Full Text | Google Scholar

Aharon, I., Etcoff, N., Ariely, D., Chabris, C. F., O’Connor, E., and Breiter, H. C. (2001). Beautiful faces have variable reward value: fmri and behavioral evidence. Neuron 32, 537–551. doi: 10.1016/S0896-6273(01)00491-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Atiyeh, B. S., and Hayek, S. N. (2008). Numeric expression of aesthetics and beauty. Aesthet. Plast. Surg. 32, 209–216. doi: 10.1007/s00266-007-9074-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Bashour, M. (2006). History and current concepts in the analysis of facial attractiveness. Plast. Reconstr. Surg. 118, 741–756. doi: 10.1097/01.prs.0000233051.61512.65

PubMed Abstract | CrossRef Full Text | Google Scholar

Choudhary, G., and Gandhi, T. K. (2017). “Indexing facial attractiveness and well beings using machine learning,” in Proceedings of the Humanitarian Technology Conference, Seattle, WA, doi: 10.1109/R10-HTC.2016.7906813

CrossRef Full Text | Google Scholar

Cunningham, M. R. (1986). Measuring the physical in physical attractiveness: quasi-experiments on the sociobiology of female facial beauty. J. Person. Soc. Psychol. 50, 925–935. doi: 10.1037/0022-3514.50.5.925

CrossRef Full Text | Google Scholar

Dantcheva, A., and Dugelay, J. L. (2014). Assessment of female facial beauty based on anthropometric, non-permanent and acquisition characteristics. Multimedia Tools Appl. 74, 11331–11355. doi: 10.1007/s11042-014-2234-5

CrossRef Full Text | Google Scholar

Dobke, M., Chung, C., and Takabe, K. (2006). Facial aesthetic preferences among asian women: are all oriental asians the same? Aesthet. Plast. Surg. 30, 342–347. doi: 10.1007/s00266-005-0091-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Eisenthal, Y. (2006). Facial attractiveness : beauty and the machine. Neural Comput. 18, 119–142. doi: 10.1162/089976606774841602

PubMed Abstract | CrossRef Full Text | Google Scholar

Etcoff, N. L. (1994). Beauty and the beholder. Nature 368, 186–187. doi: 10.1038/368186a0

PubMed Abstract | CrossRef Full Text | Google Scholar

Fan, J., Chau, K. P., Wan, X., Zhai, L., and Lau, E. (2012). Prediction of facial attractiveness from facial proportions. Pattern Recogn. 45, 2326–2334. doi: 10.1016/j.patcog.2011.11.024

CrossRef Full Text | Google Scholar

Fitzgibbon, A., Pilu, M., and Fisher, R. B. (1999). Direct least square fitting of ellipses. IEEE Trans. Pattern Anal. Mach. Intell. 21, 476–480. doi: 10.1109/34.765658

PubMed Abstract | CrossRef Full Text | Google Scholar

Gray, D., Yu, K., Xu, W., and Gong, Y. (2010). “Predicting Facial Beauty without Landmarks,” in European Conference on Computer Vision, eds K. Daniilidis, P. Maragos, and N. Paragios (Heidelberg: Springer-Verlag), doi: 10.1007/978-3-642-15567-3_32

CrossRef Full Text | Google Scholar

Holland, E. (2008). Marquardt’s phi mask: pitfalls of relying on fashion models and the golden ratio to describe a beautiful face. Aesthet. Plast. Surg. 32, 200–208. doi: 10.1007/s00266-007-9080-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnston, V. S. (2006). Mate choice decisions: the role of facial beauty. Trends Cogn. Sci. 10, 9–13. doi: 10.1016/j.tics.2005.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Koscinski, K. (2009). Current status and future directions of research on facial attractiveness. Anthropol. Rev. 72, 45–65. doi: 10.2478/v10044-008-0015-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Hallam, M., and Smoot, M. (2000). Maxims or myths of beauty? a meta-analytic and theoretical review. Psychol. Bull. 126, 390–423. doi: 10.1037//0033-2909.126.3.390

PubMed Abstract | CrossRef Full Text | Google Scholar

Liang, L. Y., Lin, L. J., Lian, W., Xie, D. R., and Li, M. R. (2018). SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction. Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE.

Google Scholar

Liu, S., Li, B., Fan, Y. Y., Quo, Z., and Samal, A. (2017). “Facial attractiveness computation by label distribution learning with deep CNN and geometric features,” in Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), (Hong Kong: IEEE Computer Society), doi: 10.1109/ICME.2017.8019454

CrossRef Full Text | Google Scholar

Matsushita, S., Morikawa, K., Mitsuzane, S., and Yamanami, H. (2015). Eye shape illusions induced by eyebrow positions. Perception 44, 529–540. doi: 10.1068/p7823

PubMed Abstract | CrossRef Full Text | Google Scholar

Mitsuhiro, K., and Kitaoka, A. (2016). “The role of eye characteristics in facial beauty, likability and attractiveness,” in Proceedings of the 31st International Congress of Psychology (ICP2016), PACIFICO Yokohama, Japan.

Google Scholar

Morikawa, K. (2012). New directions in research on visual illusions of shape and size related to the human face and body: illusions caused by makeup and clothing. Jpn. Psychol. Rev. 55, 348–361.

Google Scholar

Moura, L., and Kitney, R. (1991). A direct method for least-squares circle fitting. Comput. Phys. Commun. 64, 57–63. doi: 10.1016/0010-4655(91)90049-q

PubMed Abstract | CrossRef Full Text | Google Scholar

O’Toole, A., Price, T. V. T., Bartlett, J., and Blanz, V. (2015). 3d shape and 2d surface textures of human faces: the role of “averages” in attractiveness and age. Image Vis. Comput. 18, 9–19. doi: 10.1037/0022-3514.50.5.925

CrossRef Full Text | Google Scholar

Penna, V., Fricke, A., Iblher, N., Eisenhardt, S. U., and Stark, G. B. (2015). The Attractive Lip: A Photomorphometric Analysis. Statistics, Data Analysis, and Decision Modeling. Upper Saddle River, NJ: Prentice Hall, doi: 10.1016/j.bjps.2015.03.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrett, D. I., Burt, D. M., Pentonvoak, I. S., Lee, K. J., Rowland, D. A., and Edwards, R. (1999). Symmetry and human facial attractiveness. Evol. Hum. Behav. 20, 295–307. doi: 10.1016/S1090-5138(99)00014-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Rhodes, G. (2006). The evolutionary psychology of facial beauty. Annu. Rev. Psychol. 57, 199–226. doi: 10.1146/annurev.psych.57.102904.190208

PubMed Abstract | CrossRef Full Text | Google Scholar

Rizvi, Q. M., Karawia, A. A., and Kumar, S. (2014). “Female facial beauty analysis for assessment of facial attractiveness,” in Proceedings of the International Conference on Information Management in the Knowledge Economy, (Piscataway, NJ: IEEE).

Google Scholar

Russell, R. (2009). A sex difference in facial contrast and its exaggeration by cosmetics. Perception 38, 1211–1219. doi: 10.1068/p6331

PubMed Abstract | CrossRef Full Text | Google Scholar

Whitehill, J., and Movellan, J. R. (2008). “Personalized facial attractiveness prediction,” in IEEE International Conference on Automatic Face & Gesture Recognition, (Amsterdam: IEEE), doi: 10.1109/AFGR.2008.4813332

CrossRef Full Text | Google Scholar

Xie, D., Liang, L., Jin, L., Xu, J., and Li, M. (2015). “SCUT-FBP: A Benchmark Dataset for Facial Beauty Perception,” in 2015 IEEE International Conference on Systems, Man, and Cybernetics (SMC), (Kowloon: IEEE), doi: 10.1109/SMC.2015.319

CrossRef Full Text | Google Scholar

Xu, Y. (2008). A Research on Face Recognition Method based on Geometric Features. Wuhan Shi: China University of Geosciences.

Google Scholar

Yan, Z., Xiaoying, W., Juan, W., Lili, Z., and Yu, X. (2017). Patterns of eye movements when observers judge female facial attractiveness. Front. Psychol. 8:1909. doi: 10.3389/fpsyg.2017.01909

PubMed Abstract | CrossRef Full Text | Google Scholar

Young, P. A., Sinha, U., Rice, D. H., and Stucker, F. (2006). Circles of prominence: a new theory on facial aesthetics. Arch. Facial. Plast. Surg. 8, 263–267. doi: 10.1001/archfaci.8.4.263

PubMed Abstract | CrossRef Full Text | Google Scholar