TY - JOUR
AU1 - Maruyama, Yasuhisa
AU2 - Nakamura, Ryuto
AU3 - Tsuji, Shota
AU4 - Xuan, Yingli
AU5 - Mizutani, Kunio
AU6 - Okaze, Tsubasa
AU7 - Yoshimura, Natsue
AB - Introduction Our daily productivity is largely influenced by the thermal comfort of our environment, and prolonged exposure to an uncomfortable environment can often be harmful to our health [1]. Especially, air temperature and relative humidity greatly affect our comfort levels; hence, they are usually controlled with air-conditioning systems in indoor spaces. Airflow (or wind) is used to increase the efficiency of thermal control and is highly effective in that it can provide instantaneous pleasantness. Moreover, since wind can be delivered promptly, it is considered a very important factor for achieving and maintaining an optimal environment immediately [2] and with high energy efficiency [3]. Natural ventilation systems also have the potential for improvement of thermal comfort with energy-saving even though comprehensive control is more effective depending on the outside weather condition [4]. Accordingly, algorithms for the automatic control of electrical devices have been studied to realize optimal environments without manual control. However, there is a growing need for objective measurement methods for thermal comfort and pleasantness. One of the widely recognized indices to quantify thermal comfort is the predicted mean vote (PMV) [5]. PMV estimates how hot or cold people feel in the environment based on the air temperature, humidity, air velocity, mean radiant temperature, clothing, and metabolic rate. However, because the PMV estimates the heat balance under steady or quasi-steady states, it is difficult to consider people’s instantaneous physiological and psychological states, which are transient. Accordingly, other measures need to be employed to recognize personal transient thermal comfort to achieve and maintain personally optimized and energy-efficient environments. One solution is to use physiological signals, such as electroencephalography (EEG), skin temperature, galvanic skin response, and heart rate [6, 7]. For example, Yao et al. investigated the relationship between thermal comfort and physiological signals, such as EEG, skin temperature, and electrocardiogram, by manipulating the ambient temperature [8, 9]. Some other studies have predicted thermal comfort levels based on physiological signals using machine learning methods [10–13]. Wu et al. performed classification analyses between comfortable and uncomfortable states using EEG [10]. They also performed an online experiment to control an air conditioner based on participants’ thermal comfort predicted using their EEG [11]. In addition, Shan et al. and Shan and Yang used machine learning methods to distinguish between several thermal conditions [12, 13]. These studies focused on EEG because it directly measures the brain activity, which may reflect the personal thermal comfort. Although wind can provide instantaneous pleasantness, most previous studies have focused on the relationship between EEG and ambient temperature and/or relative humidity, and only a few studies have investigated the relationship between EEG and airflow. Okamoto et al. and Tamura et al. performed indoor experiments and reported that beta and gamma bands of EEG showed different activities between different airflow conditions and were related to comfort levels [14–16]. Furthermore, Raheel et al. utilized hot and cold air to increase the emotional experience in tactile-enhanced multimedia and reported improved prediction accuracy in EEG-based emotion recognition compared with no-airflow conditions [17]. Although these studies showed that some EEG characteristics reflect the emotional components related to airflow, it remains unclear whether EEG can be used to recognize the instantaneous pleasantness of fluctuations in airflow. Accordingly, the present study investigated whether EEG is useful in recognizing the instantaneous pleasantness of wind. If the pleasantness of wind that people were exposed to can be recognized through EEG, it would lead to the realization of an optimal and energy-efficient environment through automatic control of electrical devices producing airflow in the future. We conducted an experiment in an artificial climate chamber where wind could be generated (Fig 1) and performed an EEG-based classification of the pleasantness of the ambient airflow reproducing the natural wind fluctuations using machine learning techniques. Participants experienced ambient airflows of one of four velocity levels on their anterior side for 10 s and reported their pleasantness levels after the wind stopped. Because the brain regions that show activities related to the instantaneous pleasantness of thermal stimuli were identified in a previous functional magnetic resonance imaging (fMRI) study [18], we estimated the source-level activity from the channel EEG, and only the signals in those brain regions were used in the classification analysis. Comparison between different machine learning methods was also performed in this study. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 1. Design of the artificial climate chamber. (a) Floor plan. Arrows indicate airflow (wind). (b) Internal view of the measurement room. This photograph was captured in the preparation room. Airflow generators (electric fans) are installed on a wall. (c) Internal view of the airflow path. This photograph was captured from around the preparation room. Participants sat on a chair placed in the middle of the path. https://doi.org/10.1371/journal.pone.0299036.g001 Materials and methods Participants Participants were recruited from the Tokyo Institute of Technology and Tokyo Polytechnic University. Eighteen healthy undergraduate and graduate students between 21 and 30 years of age (8 females, 10 males; mean age (±standard deviation), 23.5 (±2.0) years) participated in this study. All participants were right-handed and had no history of neurological or psychiatric disorders. Prior to the experiment, all participants provided written informed consent. This study was approved by the Institutional Review Boards of the Tokyo Institute of Technology (Approval No. 2018170) and Tokyo Polytechnic University (Approval No. 2019–05), and the experiment was conducted in accordance with the Declaration of Helsinki. Experiment The experiment was performed in an artificial climate chamber at the Atsugi Campus of Tokyo Polytechnic University (Fig 1) in September 2019. During the experiment, the air temperature and relative humidity were maintained at 30°C and 70%, respectively. The chamber was equipped with 48 electric fans that produced wind on a wall, and the wind velocity could be controlled by a computer. During the experiment, the participants sat on a chair in the airflow path such that they could not see the fans. To adjust their body to hot and humid conditions, the participants entered the chamber approximately 1 h before the experiment commenced, during which time EEG electrodes and skin temperature sensors were attached. The flow of the trial in the experiment is shown in Fig 2. One trial consisted of a rest period of 2 s, a wind-blowing period of 10 s, and a pleasantness-reporting period of 6 s. During the wind-blowing period, the participants kept their eyes closed, and the wind blew onto the anterior side of the participants. To eliminate the sound produced by the fans, the participants wore noise-canceling earbuds through which pink noise was played continuously during the experiment. The start and end of the wind-blowing periods were notified to the participants by a beep sound transmitted through the earbuds. After the beep sound denoting the end of the wind-blowing period, the participants opened their eyes and reported their pleasantness using a computerized visual analog scale ranging from 1.0 to 9.0 in 0.1-point steps. Values of 1.0, 5.0, and 9.0 represented unpleasant, neutral, and pleasant states, respectively. Each session consisted of 40 trials, and each participant completed three sessions (120 trials in total). In this study, four wind velocity levels were used to induce different levels of pleasantness. In particular, participants were exposed to winds of 0.44, 1.0, 2.0, and 4.0 m/s measured at the participant seat located at 1.1 m height in the trial. The thermal sensation of participants with PMV was calculated to be slightly warm when the participants were exposed to wind speeds of 1.0 or 2.0 m/s at an air temperature of 30°C and relative humidity of 70%, assuming the outside environment in the summer of Tokyo, for enough a long time. Then, almost half of the 1.0 m/s and twice of 2.0 m/s wind speed conditions were introduced. The turbulence intensity of the exposed wind was approximately 30% for all wind velocities to reproduce the natural wind velocity fluctuations in an outdoor environment by devising a flow channel in the chamber. The wind velocity was presented in a pseudo-random order such that the same wind velocity was not assigned to two or more consecutive trials. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Trial flow. During 10 s of wind-blowing period, participants were exposed to airflows with velocities of 0.44, 1.0, 2.0, and 4.0 m/s. During 6 s of the pleasantness-reporting period, the participants reported their pleasantness using a computerized visual analog scale. The participants wore noise-canceling earbuds throughout the experiment, and the start and end of the wind-blowing periods were notified to them through the earbuds. The experiment consisted of 120 trials. https://doi.org/10.1371/journal.pone.0299036.g002 EEG data were recorded with 64 electrodes using eego sports 64 (ANT Neuro b.v., Hengelo, The Netherlands) at a sampling frequency of 512 Hz. EEG electrodes were placed according to the 10/10 system. The skin temperature was measured at the left chest, upper arm, thigh, and lower leg during the experiment by using a Z2015 heat flow sensor (HIOKI E.E. CORPORATION, Nagano, Japan) at a sampling frequency of 0.1 Hz, although these temperature data were not analyzed in the current study. EEG processing We preprocessed the EEG data using EEGLAB version 2020 [19] and MATLAB R2020b (MathWorks, Inc., Natick, MA, USA). After applying a high-pass filter (0.5 Hz) and a low-pass filter (45 Hz) to remove low-frequency drift and high-frequency noise, we extracted the epochs consisting of 1 s of pre-wind period and 10 s of wind-blowing period from the continuous EEG data. We then removed noisy channels (mean across participants: 1.0), interpolated them using the spherical spline method, and rejected epochs containing large artifacts, such as muscle activity (mean across participants: 4.2), by visual inspection. Subsequently, independent component analysis was applied to the EEG data, and the independent components that appeared to represent activities related to eye movements were removed. The EEG data were then re-referenced to the average of all EEG channels. We excluded the first 2 s of the wind-blowing period in each trial; thus, we analyzed only the periods during which the wind velocity was stable. To minimize the influence of other neural activities in the brain on the classification of the pleasantness of wind, the pleasantness-related signals were extracted and used as the features for classification in this study. Based on a previous fMRI study that identified brain regions showing activities related to the instantaneous pleasantness of thermal stimuli [18], we estimated the source-level activity from the channel EEG, and only signals from those brain regions were included in the analysis. The estimation of the source activity from the channel EEG data was performed using the exact low-resolution brain electromagnetic tomography (eLORETA) method [20–22] after baseline correction of the channel EEG on MNE-Python [23, 24]. eLORETA is a linear imaging method for estimating the distributed source activity and achieves zero localization error. The noise covariance matrix was calculated using 1 s of pre-wind periods. We estimated the source activity at 4098 vertices per hemisphere in the cerebral cortex of the template brain. The orientation of each dipole was fixed perpendicularly to the cortical surface. The power spectral density was calculated using Welch’s method for each vertex [25]. The window length and overlapping segments were 512 (1 s) and 256 (0.5 s), respectively, and the Hann window was applied as the window function. To reduce the computational cost, the vertices were assigned to 68 brain regions based on the Desikan–Killiany atlas [26], and the power at all the corresponding vertices was averaged for each brain region. Subsequently, the powers of the theta (4–7 Hz), alpha (8–13 Hz), beta (14–30 Hz), and gamma (31–45 Hz) bands were calculated by averaging the powers at the corresponding frequency bins. To reduce the feature dimension and prevent irrelevant brain activity from affecting the classification, the brain regions in the cerebral cortex that were reported in a previous fMRI study related to the instantaneous pleasantness of thermal stimuli were used in the classification analysis [18]. Specifically, the rostral anterior cingulate cortex, medial orbitofrontal cortex, and lateral orbitofrontal cortex were included in both hemispheres. Classification analysis We performed a within-participant binary classification of wind pleasantness based on the reported pleasantness scores. One class, labeled as the pleasantness class, included trials with reported pleasantness scores greater than 5.0, and the other class, which consisted of trials with reported pleasantness scores less than 5.0, was labeled as the unpleasantness class. Data from two participants (Participants 17 and 18), where either class had fewer than 30 trials, were excluded from the classification analysis. The power values of the four frequency bands in six regions (three regions per hemisphere) were used as the features for the classification analysis after logarithmic transformation. In this study, we tested four classifiers: logistic regression, support vector machine (SVM) with the linear kernel, SVM with the radial basis function (RBF) kernel, and artificial neural network (ANN) with one hidden layer. In the ANN, AdamW was used as the optimizer [27], and the sigmoid function and rectified linear unit (ReLU) were used as the activation functions in the output and hidden layers, respectively. Logistic regression and SVM were performed using the scikit-learn package [28], and ANN was performed using the TensorFlow package [29] in Python. To eliminate the effect of wind velocity on the classification of pleasantness, trials of the same wind velocity conditions were used only in either the training or test data in the classification analysis. Specifically, the classification accuracy was calculated by training the classifiers using trials from three out of four wind velocity conditions and testing them using trials from the remaining wind velocity condition. Fig 3 shows the schematic of this step. For example, when the test data consisted of trials with wind velocity of 0.44 m/s, the training data consisted of trials with wind velocities of 1.0, 2.0, and 4.0 m/s. This procedure was repeated four times such that each wind velocity condition was assigned to the test data once, and the mean accuracy was reported as the participant’s classification accuracy. We performed the analysis in this manner to prevent the classification accuracy from reflecting the wind velocity rather than pleasantness, because the wind velocity and pleasantness scores were closely related. We did not perform the classification when the data of either class contained less than 30 trials in the original training data, as a shortage of training data can lead to overfitting. Before training the classifiers, trials of a class with larger samples were randomly undersampled in the training data such that both classes had equal numbers of trials, and the classifiers were not biased to either class. We repeated these procedures five times and reported the mean classification accuracies. In each repetition, trials used for training the classifiers were randomly selected in the undersampling step so that the same dataset was not used for training more than once. Before training and testing the classifiers, all the features were normalized using the means and standard deviations of the undersampled training data. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Schematic illustration of classification analysis. Training data consisted of trials of 3 out of 4 wind velocity conditions, and trials of the remaining wind velocity condition were used as test data. In the actual experiment, one session consisted of 40 trials, and in the analysis, trials to be excluded were randomly selected in the undersampling step. https://doi.org/10.1371/journal.pone.0299036.g003 Hyperparameters, including the regularization parameter in logistic regression; the regularization parameter and gamma in SVM; and the number of units in the hidden layer, batch size, learning rate, weight decay parameter, and number of epochs in ANN, were determined using the undersampled training data, where 20% of them were used as the validation data. Hyperparameter tuning was performed based on the tree-structured Parzen estimator algorithm [30] in Optuna software [31]. A one-tailed one-sample t-test was used to determine whether the classification accuracy of each classifier was significantly higher than the chance level (i.e., 0.5). Holm–Bonferroni correction was applied to the p-values to account for the multiple comparison problem [32]. In addition, we compared the classification accuracies of all four classifiers. One-way repeated-measures analysis of variance (ANOVA) was used to examine whether the classification accuracy was different among the classifiers, and a p-value with Greenhouse–Geisser correction was reported. To investigate the frequency bands that largely contributed to the classification, we calculated the feature importance of each frequency band by calculating the decrease in accuracy when the features at the frequency band were removed. In this method, all the features in each frequency band were removed from both the training and test data, and the decrease in the classification accuracy relative to the baseline setting using all features was calculated. Because the feature set consisted of the log power values at four frequency bands in six brain regions, this procedure was repeated four times by altering the excluded frequency band. SVM with the RBF kernel had the best classification accuracy among all tested classifiers; hence, it was used to calculate the feature importance scores. This procedure was executed for each frequency band, and we reported the mean feature importance scores of the participants. To determine whether the feature importance score in each frequency band was significantly higher than 0, a one-tailed one-sample t-test was used and the p-values were corrected with Holm–Bonferroni method [32]. Moreover, the difference in the feature importance score among the frequency band was examined with one-way repeated-measures ANOVA. Participants Participants were recruited from the Tokyo Institute of Technology and Tokyo Polytechnic University. Eighteen healthy undergraduate and graduate students between 21 and 30 years of age (8 females, 10 males; mean age (±standard deviation), 23.5 (±2.0) years) participated in this study. All participants were right-handed and had no history of neurological or psychiatric disorders. Prior to the experiment, all participants provided written informed consent. This study was approved by the Institutional Review Boards of the Tokyo Institute of Technology (Approval No. 2018170) and Tokyo Polytechnic University (Approval No. 2019–05), and the experiment was conducted in accordance with the Declaration of Helsinki. Experiment The experiment was performed in an artificial climate chamber at the Atsugi Campus of Tokyo Polytechnic University (Fig 1) in September 2019. During the experiment, the air temperature and relative humidity were maintained at 30°C and 70%, respectively. The chamber was equipped with 48 electric fans that produced wind on a wall, and the wind velocity could be controlled by a computer. During the experiment, the participants sat on a chair in the airflow path such that they could not see the fans. To adjust their body to hot and humid conditions, the participants entered the chamber approximately 1 h before the experiment commenced, during which time EEG electrodes and skin temperature sensors were attached. The flow of the trial in the experiment is shown in Fig 2. One trial consisted of a rest period of 2 s, a wind-blowing period of 10 s, and a pleasantness-reporting period of 6 s. During the wind-blowing period, the participants kept their eyes closed, and the wind blew onto the anterior side of the participants. To eliminate the sound produced by the fans, the participants wore noise-canceling earbuds through which pink noise was played continuously during the experiment. The start and end of the wind-blowing periods were notified to the participants by a beep sound transmitted through the earbuds. After the beep sound denoting the end of the wind-blowing period, the participants opened their eyes and reported their pleasantness using a computerized visual analog scale ranging from 1.0 to 9.0 in 0.1-point steps. Values of 1.0, 5.0, and 9.0 represented unpleasant, neutral, and pleasant states, respectively. Each session consisted of 40 trials, and each participant completed three sessions (120 trials in total). In this study, four wind velocity levels were used to induce different levels of pleasantness. In particular, participants were exposed to winds of 0.44, 1.0, 2.0, and 4.0 m/s measured at the participant seat located at 1.1 m height in the trial. The thermal sensation of participants with PMV was calculated to be slightly warm when the participants were exposed to wind speeds of 1.0 or 2.0 m/s at an air temperature of 30°C and relative humidity of 70%, assuming the outside environment in the summer of Tokyo, for enough a long time. Then, almost half of the 1.0 m/s and twice of 2.0 m/s wind speed conditions were introduced. The turbulence intensity of the exposed wind was approximately 30% for all wind velocities to reproduce the natural wind velocity fluctuations in an outdoor environment by devising a flow channel in the chamber. The wind velocity was presented in a pseudo-random order such that the same wind velocity was not assigned to two or more consecutive trials. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 2. Trial flow. During 10 s of wind-blowing period, participants were exposed to airflows with velocities of 0.44, 1.0, 2.0, and 4.0 m/s. During 6 s of the pleasantness-reporting period, the participants reported their pleasantness using a computerized visual analog scale. The participants wore noise-canceling earbuds throughout the experiment, and the start and end of the wind-blowing periods were notified to them through the earbuds. The experiment consisted of 120 trials. https://doi.org/10.1371/journal.pone.0299036.g002 EEG data were recorded with 64 electrodes using eego sports 64 (ANT Neuro b.v., Hengelo, The Netherlands) at a sampling frequency of 512 Hz. EEG electrodes were placed according to the 10/10 system. The skin temperature was measured at the left chest, upper arm, thigh, and lower leg during the experiment by using a Z2015 heat flow sensor (HIOKI E.E. CORPORATION, Nagano, Japan) at a sampling frequency of 0.1 Hz, although these temperature data were not analyzed in the current study. EEG processing We preprocessed the EEG data using EEGLAB version 2020 [19] and MATLAB R2020b (MathWorks, Inc., Natick, MA, USA). After applying a high-pass filter (0.5 Hz) and a low-pass filter (45 Hz) to remove low-frequency drift and high-frequency noise, we extracted the epochs consisting of 1 s of pre-wind period and 10 s of wind-blowing period from the continuous EEG data. We then removed noisy channels (mean across participants: 1.0), interpolated them using the spherical spline method, and rejected epochs containing large artifacts, such as muscle activity (mean across participants: 4.2), by visual inspection. Subsequently, independent component analysis was applied to the EEG data, and the independent components that appeared to represent activities related to eye movements were removed. The EEG data were then re-referenced to the average of all EEG channels. We excluded the first 2 s of the wind-blowing period in each trial; thus, we analyzed only the periods during which the wind velocity was stable. To minimize the influence of other neural activities in the brain on the classification of the pleasantness of wind, the pleasantness-related signals were extracted and used as the features for classification in this study. Based on a previous fMRI study that identified brain regions showing activities related to the instantaneous pleasantness of thermal stimuli [18], we estimated the source-level activity from the channel EEG, and only signals from those brain regions were included in the analysis. The estimation of the source activity from the channel EEG data was performed using the exact low-resolution brain electromagnetic tomography (eLORETA) method [20–22] after baseline correction of the channel EEG on MNE-Python [23, 24]. eLORETA is a linear imaging method for estimating the distributed source activity and achieves zero localization error. The noise covariance matrix was calculated using 1 s of pre-wind periods. We estimated the source activity at 4098 vertices per hemisphere in the cerebral cortex of the template brain. The orientation of each dipole was fixed perpendicularly to the cortical surface. The power spectral density was calculated using Welch’s method for each vertex [25]. The window length and overlapping segments were 512 (1 s) and 256 (0.5 s), respectively, and the Hann window was applied as the window function. To reduce the computational cost, the vertices were assigned to 68 brain regions based on the Desikan–Killiany atlas [26], and the power at all the corresponding vertices was averaged for each brain region. Subsequently, the powers of the theta (4–7 Hz), alpha (8–13 Hz), beta (14–30 Hz), and gamma (31–45 Hz) bands were calculated by averaging the powers at the corresponding frequency bins. To reduce the feature dimension and prevent irrelevant brain activity from affecting the classification, the brain regions in the cerebral cortex that were reported in a previous fMRI study related to the instantaneous pleasantness of thermal stimuli were used in the classification analysis [18]. Specifically, the rostral anterior cingulate cortex, medial orbitofrontal cortex, and lateral orbitofrontal cortex were included in both hemispheres. Classification analysis We performed a within-participant binary classification of wind pleasantness based on the reported pleasantness scores. One class, labeled as the pleasantness class, included trials with reported pleasantness scores greater than 5.0, and the other class, which consisted of trials with reported pleasantness scores less than 5.0, was labeled as the unpleasantness class. Data from two participants (Participants 17 and 18), where either class had fewer than 30 trials, were excluded from the classification analysis. The power values of the four frequency bands in six regions (three regions per hemisphere) were used as the features for the classification analysis after logarithmic transformation. In this study, we tested four classifiers: logistic regression, support vector machine (SVM) with the linear kernel, SVM with the radial basis function (RBF) kernel, and artificial neural network (ANN) with one hidden layer. In the ANN, AdamW was used as the optimizer [27], and the sigmoid function and rectified linear unit (ReLU) were used as the activation functions in the output and hidden layers, respectively. Logistic regression and SVM were performed using the scikit-learn package [28], and ANN was performed using the TensorFlow package [29] in Python. To eliminate the effect of wind velocity on the classification of pleasantness, trials of the same wind velocity conditions were used only in either the training or test data in the classification analysis. Specifically, the classification accuracy was calculated by training the classifiers using trials from three out of four wind velocity conditions and testing them using trials from the remaining wind velocity condition. Fig 3 shows the schematic of this step. For example, when the test data consisted of trials with wind velocity of 0.44 m/s, the training data consisted of trials with wind velocities of 1.0, 2.0, and 4.0 m/s. This procedure was repeated four times such that each wind velocity condition was assigned to the test data once, and the mean accuracy was reported as the participant’s classification accuracy. We performed the analysis in this manner to prevent the classification accuracy from reflecting the wind velocity rather than pleasantness, because the wind velocity and pleasantness scores were closely related. We did not perform the classification when the data of either class contained less than 30 trials in the original training data, as a shortage of training data can lead to overfitting. Before training the classifiers, trials of a class with larger samples were randomly undersampled in the training data such that both classes had equal numbers of trials, and the classifiers were not biased to either class. We repeated these procedures five times and reported the mean classification accuracies. In each repetition, trials used for training the classifiers were randomly selected in the undersampling step so that the same dataset was not used for training more than once. Before training and testing the classifiers, all the features were normalized using the means and standard deviations of the undersampled training data. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 3. Schematic illustration of classification analysis. Training data consisted of trials of 3 out of 4 wind velocity conditions, and trials of the remaining wind velocity condition were used as test data. In the actual experiment, one session consisted of 40 trials, and in the analysis, trials to be excluded were randomly selected in the undersampling step. https://doi.org/10.1371/journal.pone.0299036.g003 Hyperparameters, including the regularization parameter in logistic regression; the regularization parameter and gamma in SVM; and the number of units in the hidden layer, batch size, learning rate, weight decay parameter, and number of epochs in ANN, were determined using the undersampled training data, where 20% of them were used as the validation data. Hyperparameter tuning was performed based on the tree-structured Parzen estimator algorithm [30] in Optuna software [31]. A one-tailed one-sample t-test was used to determine whether the classification accuracy of each classifier was significantly higher than the chance level (i.e., 0.5). Holm–Bonferroni correction was applied to the p-values to account for the multiple comparison problem [32]. In addition, we compared the classification accuracies of all four classifiers. One-way repeated-measures analysis of variance (ANOVA) was used to examine whether the classification accuracy was different among the classifiers, and a p-value with Greenhouse–Geisser correction was reported. To investigate the frequency bands that largely contributed to the classification, we calculated the feature importance of each frequency band by calculating the decrease in accuracy when the features at the frequency band were removed. In this method, all the features in each frequency band were removed from both the training and test data, and the decrease in the classification accuracy relative to the baseline setting using all features was calculated. Because the feature set consisted of the log power values at four frequency bands in six brain regions, this procedure was repeated four times by altering the excluded frequency band. SVM with the RBF kernel had the best classification accuracy among all tested classifiers; hence, it was used to calculate the feature importance scores. This procedure was executed for each frequency band, and we reported the mean feature importance scores of the participants. To determine whether the feature importance score in each frequency band was significantly higher than 0, a one-tailed one-sample t-test was used and the p-values were corrected with Holm–Bonferroni method [32]. Moreover, the difference in the feature importance score among the frequency band was examined with one-way repeated-measures ANOVA. Results Ratings of pleasantness Fig 4 shows the distribution of the reported pleasantness scores for all 18 participants under each wind velocity condition. The scores of 1.0, 5.0, and 9.0 represent unpleasant, neutral, and pleasant states, respectively. The mean reported pleasantness scores for the 18 participants were 4.3, 5.5, 6.9, and 5.3 during exposure to wind with velocities of 0.44, 1.0, 2.0, and 4.0 m/s, respectively. These results indicate that the participants found the second-strongest wind (2.0 m/s) most pleasant, while the weakest wind (0.44 m/s) induced unpleasantness in most participants. Because pleasantness scores for the strongest wind condition (4.0 m/s) were lower than those for the second- and third-strongest wind conditions (1.0 and 2.0 m/s), the pleasantness scores had a nonlinear relationship with the wind velocity. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Distribution of reported pleasantness scores. Horizontal axis depicts the pleasantness scores, where values of 1.0, 5.0, and 9.0 denote unpleasant, neutral, and pleasant states, respectively. Vertical axis indicates each participant. The color of the dots represents the number of reports. Wind velocity of: (a) 0.44 m/s, (b) 1.0 m/s, (c) 2.0 m/s, and (d) 4.0 m/s. https://doi.org/10.1371/journal.pone.0299036.g004 Classification The mean classification accuracies of the binary classifiers are shown in Fig 5. The mean classification accuracies (±standard deviation) were 0.544 (±0.135), 0.571 (±0.125), 0.573 (±0.105), and 0.564 (±0.100) for logistic regression, SVM with the linear kernel, SVM with the RBF kernel, and ANN, respectively. SVM with the linear kernel, SVM with the RBF kernel, and ANN showed significantly higher classification accuracy than chance (p<0.05 in the one-sample t-test with Holm–Bonferroni correction [32]). Statistically significant differences in classification accuracy among the classifiers were not observed in the repeated-measures ANOVA (p = 0.39). Table 1 shows the feature importance scores for the four frequency bands. The SVM with the RBF kernel was used for the computation. None of the feature importance scores were significantly higher than 0 in any frequency band, and the scores did not exhibit differences among the frequency bands (p = 0.65 in repeated-measures ANOVA). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Mean classification accuracy of the binary classifiers. Error bars represent standard deviations. *p<0.05 in t-test with Holm–Bonferroni correction. https://doi.org/10.1371/journal.pone.0299036.g005 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Feature importance of each frequency band. https://doi.org/10.1371/journal.pone.0299036.t001 Ratings of pleasantness Fig 4 shows the distribution of the reported pleasantness scores for all 18 participants under each wind velocity condition. The scores of 1.0, 5.0, and 9.0 represent unpleasant, neutral, and pleasant states, respectively. The mean reported pleasantness scores for the 18 participants were 4.3, 5.5, 6.9, and 5.3 during exposure to wind with velocities of 0.44, 1.0, 2.0, and 4.0 m/s, respectively. These results indicate that the participants found the second-strongest wind (2.0 m/s) most pleasant, while the weakest wind (0.44 m/s) induced unpleasantness in most participants. Because pleasantness scores for the strongest wind condition (4.0 m/s) were lower than those for the second- and third-strongest wind conditions (1.0 and 2.0 m/s), the pleasantness scores had a nonlinear relationship with the wind velocity. Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 4. Distribution of reported pleasantness scores. Horizontal axis depicts the pleasantness scores, where values of 1.0, 5.0, and 9.0 denote unpleasant, neutral, and pleasant states, respectively. Vertical axis indicates each participant. The color of the dots represents the number of reports. Wind velocity of: (a) 0.44 m/s, (b) 1.0 m/s, (c) 2.0 m/s, and (d) 4.0 m/s. https://doi.org/10.1371/journal.pone.0299036.g004 Classification The mean classification accuracies of the binary classifiers are shown in Fig 5. The mean classification accuracies (±standard deviation) were 0.544 (±0.135), 0.571 (±0.125), 0.573 (±0.105), and 0.564 (±0.100) for logistic regression, SVM with the linear kernel, SVM with the RBF kernel, and ANN, respectively. SVM with the linear kernel, SVM with the RBF kernel, and ANN showed significantly higher classification accuracy than chance (p<0.05 in the one-sample t-test with Holm–Bonferroni correction [32]). Statistically significant differences in classification accuracy among the classifiers were not observed in the repeated-measures ANOVA (p = 0.39). Table 1 shows the feature importance scores for the four frequency bands. The SVM with the RBF kernel was used for the computation. None of the feature importance scores were significantly higher than 0 in any frequency band, and the scores did not exhibit differences among the frequency bands (p = 0.65 in repeated-measures ANOVA). Download: PPT PowerPoint slide PNG larger image TIFF original image Fig 5. Mean classification accuracy of the binary classifiers. Error bars represent standard deviations. *p<0.05 in t-test with Holm–Bonferroni correction. https://doi.org/10.1371/journal.pone.0299036.g005 Download: PPT PowerPoint slide PNG larger image TIFF original image Table 1. Feature importance of each frequency band. https://doi.org/10.1371/journal.pone.0299036.t001 Discussion In this study, the participants were exposed to four wind velocities with turbulent fluctuations in a hot and humid room. Within-participant binary classification using their EEG and reported pleasantness scores achieved higher classification accuracy than chance when SVM or ANN was utilized. In the analysis, we eliminated the possibility of decoding the wind velocity as much as possible by using EEG signals solely from the instantaneous pleasantness-related brain regions in the frontal cortex and by assigning different wind velocity conditions for training and testing the classifiers. Source-level analysis enabled the selection of brain regions used for classification. In an indoor experiment performed in an artificial climate chamber, we succeeded in changing the participants’ pleasantness by manipulating the airflow velocity within the room. Different wind velocity conditions successfully induced varying levels of pleasantness. Moreover, depending on their instantaneous physiological and psychological states, a wide range of pleasantness scores was reported by the same participant, even under the same wind velocity conditions. This result confirms that information about environmental conditions alone is insufficient to recognize the instantaneous pleasantness experienced by individuals, and physiological signals that reflect their emotions are helpful. Brain regions associated with emotions tend to differ according to the stimulus modality of the emotion elicitation method [33]. To date, some fMRI and positron emission tomography studies have investigated the brain regions associated with thermal comfort or pleasantness when perceiving non-painful thermal stimuli presented on the hand or body [18, 34–38]. Among these studies, Rolls et al. investigated the brain regions related to the instantaneous pleasantness of thermal stimuli using fMRI [18]. They reported that activities in the mid-orbitofrontal cortex, pregenual cingulate cortex, and ventral striatum were positively correlated with the subjective ratings of instantaneous pleasantness of thermal stimuli presented on the hand, while unpleasantness was correlated with the activity in the lateral orbitofrontal cortex. They also reported that the somatosensory cortex and ventral posterior insula showed activities correlated with the intensity of the thermal stimuli, and thus suggested that affective values and sensory properties of thermal stimuli were processed in different brain regions [18]. Accordingly, to make the classification of pleasantness of wind robust and less affected by other neural activities in the brain, we utilized source localization methods to estimate the source-level activity from scalp EEG and extracted features from the rostral anterior cingulate cortex, medial orbitofrontal cortex, and lateral orbitofrontal cortex for the classification analysis based on the Desikan–Killiany atlas [26]. Because it is difficult to measure the subcortical activity using EEG, the ventral striatum was not included in the analysis. As a result, we successfully decoded the instantaneous pleasantness of wind using the source-level EEG signals in these brain regions, although signals from their surrounding regions may also have affected the results due to signal leakage and spread of information in the source space [39, 40]. The result of the feature importance scores calculated using SVM with the RBF kernel showed that each frequency band alone did not have a large influence on the decoding performance in this study. This indicates that power in the four frequency bands were correlated with one another. Accordingly, we can expect that the pleasantness of wind can be classified using less features, leading to reduced computational cost during training the recognition models. In the analysis, we classified the pleasant and unpleasant states using the frequency-domain features of EEG, and the classification accuracies of both SVM and ANN were greater than the chance level with statistical significance. Because no statistical difference was observed among the classification accuracies of the machine learning methods utilized in this study, successful decoding of the pleasantness of wind from EEG may not be dependent on specific types of classifiers. As described earlier, we eliminated the influence of wind velocity on the classification accuracy. The test data consisted of trials from one wind velocity condition that was not included in the training data. Additionally, we excluded brain regions associated with somatosensory processing, such as the somatosensory and motor cortices, from the classification analysis. Thus, this study succeeded in decoding the pleasantness per se rather than decoding the tactile sensation (i.e., wind velocity). Although we achieved a classification accuracy higher than the chance level in this study, there are several limitations that should be acknowledged: The first limitation is the relatively long duration of the wind-blowing period in the experiment. Participants were exposed to the wind for 10 s in each trial. Future research should consider conducting EEG experiments with shorter wind-blowing periods to investigate whether pleasantness can be decoded in shorter time segments. In addition, continuous EEG experiments with varying wind strengths over time and frequent reports of pleasantness scores could help explore whether time-series changes in pleasantness can be tracked from EEG data. The second limitation relates to the number of classes used in the classification analysis. In this study, we divided the pleasantness scores into pleasant and unpleasant classes, performing binary classification. Due to the shortage of trials in some classes when attempting to divide pleasantness scores into more than two classes, we could not increase the number of classes in the analysis. Further experiments that include classification between more than two classes or using regression analysis will be necessary to enable the decoding of pleasantness levels at a finer scale from EEG data. The third limitation concerns the number of EEG electrodes. Although we used 64 electrodes in this study, it is preferable to use less electrodes for practical use. Accordingly, methods to decode pleasantness with a smaller number of EEG electrodes should be developed in future research. The last limitation concerns the relatively low classification accuracy. The classification accuracy in this study (57.3% in binary classification) is less than that of existing studies. Wu et al. reported group-level classification accuracies of up to 87.9% and 74.4% in their binary classification analyses between thermally comfortable and uncomfortable conditions, respectively [10, 11]. One potential reason for the difference in classification accuracy could be the different environmental factors manipulated in the experiments; wind velocity was manipulated in our study, while Wu et al. manipulated air temperature [10, 11]. Moreover, due to the inherent connection between environmental factors and thermal comfort, it is possible that the environmental factor (i.e., air temperature) itself affected the brain activity rather than thermal comfort, and directly contributed to the classification performance in the previous studies. In this study, we confirmed that pleasantness-related neural activity is reflected in the EEG data even after eliminating the possible effect of environmental factors. However, achieving a much higher classification accuracy is necessary for practical applications. A possible solution is to combine several physiological signals. It has been shown that combining EEG with other peripheral physiological signals, such as galvanic skin response, heart rate, and heart rate variability, increases the performance of emotion recognition [41]. Another solution is to integrate physiological signals with existing measures of thermal comfort, such as the PMV. The PMV provides a rough estimate of the thermal comfort, and physiological signals can provide complementary information. To summarize, the existing measures of thermal comfort combined with several types of physiological signals can realize high-performance recognition of the pleasantness of wind to provide a personalized comfortable environment that leads to increased human productivity and well-being in addition to energy efficiency.
TI - Classification of pleasantness of wind by electroencephalography
JF - PLoS ONE
DO - 10.1371/journal.pone.0299036
DA - 2024-02-27
UR - https://www.deepdyve.com/lp/public-library-of-science-plos-journal/classification-of-pleasantness-of-wind-by-electroencephalography-80noMaRQxv
SP - e0299036
VL - 19
IS - 2
DP - DeepDyve
ER -