ACM Computing Surveys (CSUR)

Liu, Chang; Yu, Han

doi: 10.1145/3588764pmid: N/A

Promotional videos are rapidly becoming a popular medium for persuading people to change their behaviours in many settings (e.g., online shopping, social enterprise initiatives). Today, such videos are often produced by professionals, which is a time-, labour- and cost-intensive undertaking. In order to produce such contents to support large applications (e.g., e-commerce), the field of artificial intelligence (AI)-empowered persuasive video generation (AIPVG) has gained traction in recent years. This field is interdisciplinary in nature, which makes it challenging for new researchers to grasp. Currently, there is no comprehensive survey of AIPVG available. In this paper, we bridge this gap by reviewing key AI techniques that can be utilized to automatically generate persuasive videos. We offer a first-of-its-kind taxonomy which divides AIPVG into three major steps: (1) visual material understanding, which extracts information from the visual materials (VMs) relevant to the target of promotion; (2) visual storyline generation, which shortlists and arranges high-quality VMs into a sequence in order to compose a storyline with persuasive power; and (3) post-production, which involves background music generation and still image animation to enhance viewing experience. We also introduce the evaluation metrics and datasets commonly adopted in the field of AIPVG. We analyze the advantages and disadvantages of the existing works belonging to the above-mentioned steps, and discuss interesting potential future research directions.

Das, Ringki; Singh, Thoudam Doren

doi: 10.1145/3586075pmid: N/A

Sentiment analysis has come long way since it was introduced as a natural language processing task nearly 20 years ago. Sentiment analysis aims to extract the underlying attitudes and opinions toward an entity. It has become a powerful tool used by governments, businesses, medicine, marketing, and others. The traditional sentiment analysis model focuses mainly on text content. However, technological advances have allowed people to express their opinions and feelings through audio, image and video channels. As a result, sentiment analysis is shifting from unimodality to multimodality. Multimodal sentiment analysis brings new opportunities with the rapid increase of sentiment analysis as complementary data streams enable improved and deeper sentiment detection which goes beyond text-based analysis. Audio and video channels are included in multimodal sentiment analysis in terms of broadness. People have been working on different approaches to improve sentiment analysis system performance by employing complex deep neural architectures. Recently, sentiment analysis has achieved significant success using the transformer-based model. This paper presents a comprehensive study of different sentiment analysis approaches, applications, challenges, and resources then concludes that it holds tremendous potential. The primary motivation of this survey is to highlight changing trends in the unimodality to multimodality for solving sentiment analysis tasks.

Song, Yisheng; Wang, Ting; Cai, Puyu; Mondal, Subrota K.; Sahoo, Jyoti Prakash

doi: 10.1145/3582688pmid: N/A

Few-shot learning (FSL) has emerged as an effective learning method and shows great potential. Despite the recent creative works in tackling FSL tasks, learning valid information rapidly from just a few or even zero samples remains a serious challenge. In this context, we extensively investigated 200+ FSL papers published in top journals and conferences in the past three years, aiming to present a timely and comprehensive overview of the most recent advances in FSL with a fresh perspective and to provide an impartial comparison of the strengths and weaknesses of existing work. To avoid conceptual confusion, we first elaborate and contrast a set of relevant concepts including few-shot learning, transfer learning, and meta-learning. Then, we inventively extract prior knowledge related to few-shot learning in the form of a pyramid, which summarizes and classifies previous work in detail from the perspective of challenges. Furthermore, to enrich this survey, we present in-depth analysis and insightful discussions of recent advances in each subsection. What is more, taking computer vision as an example, we highlight the important application of FSL, covering various research hotspots. Finally, we conclude the survey with unique insights into technology trends and potential future research opportunities to guide FSL follow-up research.

Greca, Renan; Miranda, Breno; Bertolino, Antonia

doi: 10.1145/3579851pmid: N/A

Context: Software regression testing refers to rerunning test cases after the system under test is modified, ascertaining that the changes have not (re-)introduced failures. Not all researchers’ approaches consider applicability and scalability concerns, and not many have produced an impact in practice. Objective: One goal is to investigate industrial relevance and applicability of proposed approaches. Another is providing a live review, open to continuous updates by the community. Method: A systematic review of regression testing studies that are clearly motivated by or validated against industrial relevance and applicability is conducted. It is complemented by follow-up surveys with authors of the selected papers and 23 practitioners. Results: A set of 79 primary studies published between 2016–2022 is collected and classified according to approaches and metrics. Aspects relative to their relevance and impact are discussed, also based on their authors’ feedback. All the data are made available from the live repository that accompanies the study. Conclusions: While widely motivated by industrial relevance and applicability, not many approaches are evaluated in industrial or large-scale open-source systems, and even fewer approaches have been adopted in practice. Some challenges hindering the implementation of relevant approaches are synthesized, also based on the practitioners’ feedback.

Wang, Tian; Liang, Yuzhu; Shen, Xuewei; Zheng, Xi; Mahmood, Adnan; Sheng, Quan Z.

doi: 10.1145/3582270pmid: N/A

Sensor-cloud originates from extensive recent applications of wireless sensor networks and cloud computing. To draw a roadmap of the current research activities of the sensor-cloud community, we first investigate the state-of-the-art sensor-cloud literature reviews published since the late 2010s and discovered that these surveys have primarily studied the sensor-cloud in specific aspects, security-enabled solutions, efficient management mechanisms, and architectural challenges. While the existing surveys have reviewed the sensor-cloud from various perspectives, they are inadequate for the three key issues that require urgent attention in the sensor-cloud: reliability, energy, and heterogeneity. To fill this gap, we perform a thorough survey by examining the origins of the sensor-cloud and providing an in-depth and comprehensive discussion of these three key challenges. We summarize initial designs of the new edge-based schemes to address these challenges and identify several open issues and promising future research directions.

Dalzochio, Jovani; Kunst, Rafael; Barbosa, Jorge Luis Victória; Neto, Pedro Clarindo da Silva; Pignaton, Edison; ten Caten, Carla Schwengber; da Penha, Alex de Lima Teodoro

doi: 10.1145/3586100pmid: N/A

Military troops rely on maintenance management projects and operations to preserve the materials’ ordinary conditions or restore them to combat or military training. Maintenance management in the defense domain has its particularities, such as those related to the type of equipment operated, the environment and operating conditions, the need to maintain equipment readiness in cases of external aggression, and the security of the information. This study aims to understand the challenges, principles, scenarios, techniques, and open questions of predictive maintenance (PdM) in the military domain. We conducted a systematic literature review that resulted in the discussion of 43 articles, leading to the identification of 23 challenges and principles, 4 scenarios where predictive maintenance is crucial, besides discussing techniques used for PdM in the military domain. Our results contribute to understanding the perspective of PdM in the defense context.

Cinà, Antonio Emanuele; Grosse, Kathrin; Demontis, Ambra; Vascon, Sebastiano; Zellinger, Werner; Moser, Bernhard A.; Oprea, Alina; Biggio, Battista; Pelillo, Marcello; Roli, Fabio

doi: 10.1145/3585385pmid: N/A

The success of machine learning is fueled by the increasing availability of computing power and large training datasets. The training data is used to learn new models or update existing ones, assuming that it is sufficiently representative of the data that will be encountered at test time. This assumption is challenged by the threat of poisoning, an attack that manipulates the training data to compromise the model’s performance at test time. Although poisoning has been acknowledged as a relevant threat in industry applications, and a variety of different attacks and defenses have been proposed so far, a complete systematization and critical review of the field is still missing. In this survey, we provide a comprehensive systematization of poisoning attacks and defenses in machine learning, reviewing more than 100 papers published in the field in the past 15 years. We start by categorizing the current threat models and attacks and then organize existing defenses accordingly. While we focus mostly on computer-vision applications, we argue that our systematization also encompasses state-of-the-art attacks and defenses for other data modalities. Finally, we discuss existing resources for research in poisoning and shed light on the current limitations and open research questions in this research field.

André, Étienne; Liu, Shuang; Liu, Yang; Choppy, Christine; Sun, Jun; Dong, Jin Song

doi: 10.1145/3579821pmid: N/A

The Unified Modeling Language (UML) is a standard for modeling dynamic systems. UML behavioral state machines are used for modeling the dynamic behavior of object-oriented designs. The UML specification, maintained by the Object Management Group (OMG), is documented in natural language (in contrast to formal language). The inherent ambiguity of natural languages may introduce inconsistencies in the resulting state machine model. Formalizing UML state machine specification aims at solving the ambiguity problem and at providing a uniform view to software designers and developers. Such a formalization also aims at providing a foundation for automatic verification of UML state machine models, which can help to find software design vulnerabilities at an early stage and reduce the development cost. We provide here a comprehensive survey of existing work from 1997 to 2021 related to formalizing UML state machine semantics for the purpose of conducting model checking at the design stage.

Garg, Tanmay; Masud, Sarah; Suresh, Tharun; Chakraborty, Tanmoy

doi: 10.1145/3580494pmid: N/A

Detecting online toxicity has always been a challenge due to its inherent subjectivity. Factors such as the context, geography, socio-political climate, and background of the producers and consumers of the posts play a crucial role in determining if the content can be flagged as toxic. Adoption of automated toxicity detection models in production can thus lead to a sidelining of the various groups they aim to help in the first place. It has piqued researchers’ interest in examining unintended biases and their mitigation. Due to the nascent and multi-faceted nature of the work, complete literature is chaotic in its terminologies, techniques, and findings. In this article, we put together a systematic study of the limitations and challenges of existing methods for mitigating bias in toxicity detection.We look closely at proposed methods for evaluating and mitigating bias in toxic speech detection. To examine the limitations of existing methods, we also conduct a case study to introduce the concept of bias shift due to knowledge-based bias mitigation. The survey concludes with an overview of the critical challenges, research gaps, and future directions. While reducing toxicity on online platforms continues to be an active area of research, a systematic study of various biases and their mitigation strategies will help the research community produce robust and fair models.1

Rajapakse, Visal; Karunanayake, Ishan; Ahmed, Nadeem

doi: 10.1145/3583683pmid: N/A

Machine Learning (TinyML) is an upsurging research field that proposes to democratize the use of Machine Learning and Deep Learning on highly energy-efficient frugal Microcontroller Units (MCUs). Considering the general assumption that TinyML can only run inference, growing interest in the domain has led to work that makes them reformable, i.e., solutions that permit models to improve once deployed. This work presents a survey on reformable TinyML solutions with the proposal of a novel taxonomy. Here, the suitability of each hierarchical layer for reformability is discussed. Furthermore, we explore the workflow of TinyML and analyze the identified deployment schemes, available tools, and the scarcely available benchmarking tools. Finally, we discuss how reformable TinyML can impact a few selected industrial areas and discuss the challenges, and future directions, and its fusion with next-generation AI.