Recognition of facial expression using spatial transformation network and convolutional neural networkKim, Jieun; Lee, Eung-Joo; Lee, Deokwoo
doi: 10.1117/12.2634030pmid: N/A
With the development of artificial intelligence, the field of sentiment analysis can be used in various industries such as computer-human interaction technology, personal status monitoring, criminal investigation, and entertainment. In the field of sentiment analysis, various methods such as facial expression, voice, EEG signal, and text are being studied. Among these methods, facial expression recognition is one of the approaches being actively studied because it has the advantage of being relatively easy to collect learning data and easy to apply to real life compared to other methods. Recently, research on facial expression recognition using deep learning has been actively conducted, and it shows relatively high performance. The method using deep learning has advantage of being easy to apply to a variety of data, but there is a limitation in large deviation in accuracy depending on the effect of occlusion, pose, and illumination in extracting feature points. In addition, in the case of expression recognition, similar objects such as face always exist in the data, and only some specific regions such as eyes, nose, and mouth have necessary information for learning, and remaining regions such as background and hair is considered as insignificant part of the data. Therefore learning all the features in the data not only takes a long time to learn, but also uses computing resources inefficiently. Therefore, we propose a convolutional neural network algorithm combined with a spatial transformation network which helps facial expression recognition by focusing on a specific part of the face.
Investigating the performance of an artificial neural network for solving the resource constrained project scheduling problem (RCPSP)Golab, Amir; Sedgh Gooya, Ehsan; Al Falou, Ayman; Cabon, Mikael
doi: 10.1117/12.2618499pmid: N/A
Project management plays a fundamental role in national development and economic improvement. Schedule management is also one of the knowledge areas of project management. This paper deals with the Resource-Constrained Project Scheduling Problem (RCPSP), which is a part of schedule management. The objective is to optimize and minimize the project duration while constraining the amount of resources during project scheduling. In this problem, resource constraints and precedence relationships of activities are known as important constraints for project scheduling. Many methods such as exact, heuristic, and meta-heuristic have been proposed by researchers to solve the problem, but there is a lack of investigation of the problem using new methods such as neural networks and machine learning. In this context, we investigate the function of a feed-forward neural network on the standard single-mode RCPSP. The artificial neural network learns based on the scheduling level characterized by parameters, namely network complexity, resource factor, resource strength, etc., calculated at each stage of project scheduling and identified priority rules. Therefore, after the learning process, the developed artificial neural network can automatically select an appropriate priority rule to filter out an unscheduled activity from the list of eligible activities and schedule all activities of the project in accordance with the specified project constraints.
Marker-less motion capture system using OpenPoseFeng, B.; Powell, D. W.; Doblas, A.
doi: 10.1117/12.2619059pmid: N/A
Motion capture systems are widely used to measure athletic performance and as a diagnostic tool in sports medicine. Standard motion capture systems record body movement using: (1) a set of cameras to localize body segments; or (2) specialized suits in which inertial measurement units are directly attached to body segments. The major drawbacks of these systems are limited portability, affordability, and accessibility. This contribution presents a markerless motion capture system using a commercially available sports camera and the OpenPose human pose estimation algorithm. We have validated the proposed markerless system by analyzing the human biometrics during running and jumping movements.
Re-defining radiology quality assurance (QA): artificial intelligence (AI)-based QA by restricted investigation of unequal scores (AQUARIUS)Wismüller, Axel; Stockmaster, Larry; Vosoughi, M. Ali
doi: 10.1117/12.2622234pmid: N/A
There is an urgent need for streamlining radiology Quality Assurance (QA) programs to make them better and faster. Here, we present a novel approach, Artificial Intelligence (AI)-Based Quality Assurance by Restricted Investigation of Unequal Scores (AQUARIUS), for re-defining radiology QA, which reduces human effort by up to several orders of magnitude over existing approaches. AQUARIUS typically includes automatic comparison of AI-based image analysis with natural language processing (NLP) on radiology reports. Only the usually small subset of cases with discordant reads is subsequently reviewed by human experts. To demonstrate the clinical applicability of AQUARIUS, we performed a clinical QA study on Intracranial Hemorrhage (ICH) detection in 1936 head CT scans from a large academic hospital. Immediately following image acquisition, scans were automatically analyzed for ICH using a commercially available software (Aidoc, Tel Aviv, Israel). Cases rated positive for ICH by AI (ICH-AI+) were automatically flagged in radiologists’ reading worklists, where flagging was randomly switched off with probability 50%. Using AQUARIUS with NLP on final radiology reports and targeted expert neuroradiology review of only 29 discordantly classified cases reduced the human QA effort by 98.5%, where we found a total of six non-reported true ICH+ cases, with radiologists’ missed ICH detection rates of 0.52% and 2.5% for flagged and non-flagged cases, respectively. We conclude that AQUARIUS, by combining AI-based image analysis with NLP-based pre-selection of cases for targeted human expert review, can efficiently identify missed findings in radiology studies and significantly expedite radiology QA programs in a hybrid human-machine interoperability approach.
RA-MISNet: multilevel information sharing network based on residual attention for crowd countingSang, Jun; Tian, Shaoli; Tan, Jinghan; Alam, Mohammad S.; Chen, Yan
doi: 10.1117/12.2618271pmid: N/A
Crowd counting has been a popular research topic in the field of computer vision due to the variation of human head scales and the interference of background noise. Some existing methods use multi-level feature fusion to solve scale variation, but the problem of background noise interference may be more serious due to the involvement of shallow features in the feature fusion process. In this paper, we propose a Multilevel Information Sharing Network based on Residual Attention(RA-MISNet) to solve this problem. The RA-MISNet consists of a feature extraction component, an information sharing module and a residual attention density map estimator. On the basis of solving the multi-scale problem, the residual attention mechanism is adopted by our proposed method to refine the population distribution information in sharing features at all levels, which can reduce the interference of complex texture background on density map regression. Furthermore, owing to the severe label noise interference problem in high-density crowd areas, we design a Regional Multi-level Segmentation Loss (RMS Loss) to divide the multi-level density regions with different label noise rates in a single crowd image and apply the corresponding granularity supervision constraints for each density level region. Extensive experiments on three crowd counting datasets (ShanghaiTech, UCF CC 50, UCF-QNRF) demonstrate the effectiveness and superiority of the proposed methods.
Can simpler be better? Review of methods for the detection of GAN-generated imageryAnkolekar, Adisree V.; Madappa, Raaga; Savakis, Andreas
doi: 10.1117/12.2632737pmid: N/A
As generative-adversarial-networks (GANs) continue to show progress in generating realistic imagery, there is a need to develop methods for distinguishing fake images from real images. This paper reviews state-ofthe- art methods for detecting real vs. GAN-generated images of faces. The methods used are NoiseScope, Resynthesis, Attribution Network, CNNDetector and DFT-based detection. Most methods are based on deeplearning architectures, except for the one using Discrete Fourier Transform (DFT) and a simple classifier based on azimuthal averaging of the image spectrum. While one might expect the deep-learning based methods to perform better, our initial experiments show that the DFT-based classifier performed the best and was the fastest and simplest to implement. These results illustrate that sometimes simpler methods can achieve better results, when comparing computation speed and performance, and point to the usefulness of considering a variety of approaches for the detection of fake imagery. The robustness of the methods were also assessed by adding different types of noise to the GAN generated images.
A virtual assistant for first responders using natural language understanding and optical character recognitionDo, Vickie; Huyen, Alexander; Joubert, Federick J.; Gabriel, Mina; Yun, Kyongsik; Lu, Thomas; Chow, Edward
doi: 10.1117/12.2620729pmid: N/A
Commercial deep learning capabilities are available for many applications such as computer vision processing and intelligent chat bots. The Google Cloud Platform product Google Dialogflow provides lifelike conversational artificial intelligence (AI) using machine learning (ML) to generate natural conversations between computers and humans. This ML utilizes natural language understanding (NLU) to recognize a user’s intent and extracts key information into a form of entities. We have developed a user-friendly application through understanding the hazardous material database, first aid safety guidelines and observing the process of first responders who access this information in the field. We created the Trusted and Explainable Artificial Intelligence for Saving Lives (TruePAL) virtual assistant using Dialogflow1 and TensorFlow2 paired with EasyOCR.3 The chatbot supports first responders by providing voice interaction which helps limit additional steps such as browsing through multiple categories when searching for information. Using feedback from our field interviews, the voice interface has been developed to enable the first responder to focus on the immediate emergency. With less distractions, the first responder is able to engage the incident more effectively. The partial hands-free TruePAL chatbot assistant improves the accessibility to the correct guidance by an average of 1.9 seconds compared to the widely used application, NIH WISER, which requires full attention to operate. We combined this intelligent chatbot with a separate visual processing capability to produce hazardous signage analysis and generate the proper guidance for first responders. With the evolving functionality of AI tools, the use of virtual assistants in first responder technology will be an advancement, benefiting the safety of both first responders and civilians.
Formation of a method and algorithm for automated analysis of the degree of attention and concentration of a personSemenishchev, Evgenii; Voronin, Viacheslav; Kurbakov, Mikhail; Lyakhov, Daniil; Egipko, Vladislav; Zelensky, Aleksandr
doi: 10.1117/12.2623051pmid: N/A
The article discusses the problem of constructing an algorithm for the automated detection of operator fatigue and monitoring its state by analyzing data obtained by machine vision systems in the visible range. As information parameters, a combined model is used, which includes an analysis of the speed of movement of the pupils and the degree of scattering of motion eyes. A multi-criteria smoothing method is used to identify the trend curve. The deviation of the scatter of displacements of the focus of view relative to the center of the object also indicates the degree of operator involvement in the technological or controlled process. The speed of movement of the pupils and the spread in the displacements of the focus of view relative to the center of the main large object were recorded. The work contains tables and graphs fixing the result of detecting deviations relative to the values obtained in the first minutes of the operator's work, from the time of tracking the test video.
Crowd detection and estimation for an earthquake early warning system using deep learningLamas, F.; Duguet, K.; Pezoa, J. E.; Montalva, G. A.; Torres, S. N.; Meng, W.
doi: 10.1117/12.2622392pmid: N/A
Earthquakes, and their cascading threats to economic and social sustainability, are a common problem between China and Chile. In such emergencies, automatic image recognition systems have become critical tools for preventing and reducing civilian casualties. Human crowd detection and estimation are fundamental for automatic recognition under life-threatening natural disasters. However, detecting and estimating crowds in scenes is nontrivial due to occlusion, complex behaviors, posture changes, and camera angles, among other issues. This paper presents the first steps in developing an intelligent Earthquake Early Warning System (EEWS) between China and Chile. The EEWS exploits the ability of deep learning architectures to properly model different spatial scales of people and the varying degrees of crowd densities. We propose an autoencoder architecture for crowd detection and estimation because it creates compressed representations for the original crowd input images in its latent space. The proposed architecture considers two cascaded autoencoders. The first performs reconstructive masking of the input images, while the second generates Focal Inverse Distance Transform (FIDT) maps. Thus, the cascaded autoencoders improve the ability of the network to locate people and crowds, thereby generating high-quality crowd maps and more reliable count estimates.
Improving face recognition by pose-aware quality assessment and judgementCao, Zhicheng; Shi, Liyi; Ju, Hua; Yang, Li; Pang, Liaojun
doi: 10.1117/12.2623001pmid: N/A
Face recognition has grown rapidly in the past several years due to advances in deep learning. More and more applications have emerged as this technology becomes more mature. However, face recognition under uncontrolled conditions is still quite challenging. For example, real-world applications usually encounter the issue of non-frontal standing pose which causes the face recognition system to degrade or even fail. Thus, this research work studies the issue of non-ideal facial pose in face recognition and propose to addresses this problem via pose-aware quality assessment and judgement. We first implement a standard face recognition system, consisting of an MTCNN face detection stage and a FaceNet face recognition stage. Then, we introduce a Quality Assessment and Judgement (QAJ) stage between the face detection stage and the face recognition stage. The QAJ stage conducts facial pose estimation which is realized through a DNN. Given a facial input, the QAJ stage assesses the facial pose and judges if the input is satisfactory in terms of quality. Inputs of poor quality will be screened and dropped out while inputs of high quality will be passed to the subsequent face recognition stage to output a final recognized identity. In the experiments, we compare the face recognition rates with and without the QAJ stage. Using a pose threshold of 15°, we find out that the recognition rate is improved by 2.83%, which is a significant improvement on the recognition performance and justifies our proposed technique of QAJ.