Empirical Software Engineering

Chen, Xi; Slowinska, Asia; Bos, Herbert

doi: 10.1007/s10664-015-9362-zpmid: N/A

Many reverse engineering techniques for data structures rely on the knowledge of memory allocation routines. Typically, they interpose on the system’s malloc and free functions, and track each chunk of memory thus allocated as a data structure. However, many performance-critical applications implement their own custom memory allocators. Examples include webservers, database management systems, and compilers like gcc and clang. As a result, current binary analysis techniques for tracking data structures fail on such binaries. We present MemBrush, a new tool to detect memory allocation and deallocation functions in stripped binaries with high accuracy. We evaluated the technique on a large number of real world applications that use custom memory allocators. We demonstrate that MemBrush can detect allocators/deallocators with a high accuracy which is 52 out of 59 for allocators, and 29 out of 31 for deallocators in SPECINT 2006. As we show, we can furnish existing reverse engineering tools with detailed information about the memory management API, and as a result perform an analysis of the actual application specific data structures designed by the programmer. Our system uses dynamic analysis and detects memory allocation and deallocation routines by searching for functions that comply with a set of generic characteristics of allocators and deallocators.

Haller, Istvan; Slowinska, Asia; Bos, Herbert

doi: 10.1007/s10664-015-9363-ypmid: N/A

Many existing techniques for reversing data structures in C/C ++ binaries are limited to low-level programming constructs, such as individual variables or structs. Unfortunately, without detailed information about a program's pointer structures, forensics and reverse engineering are exceedingly hard. To fill this gap, we propose MemPick, a tool that detects and classifies high-level data structures used in stripped binaries. By analyzing how links between memory objects evolve throughout the program execution, it distinguishes between many commonly used data structures, such as singly- or doubly-linked lists, many types of trees (e.g., AVL, red-black trees, B-trees), and graphs. We evaluate the technique on 10 real world applications, 4 file system implementations and 16 popular libraries. The results show that MemPick can identify the data structures with high accuracy.

Walkinshaw, Neil; Taylor, Ramsay; Derrick, John

doi: 10.1007/s10664-015-9367-7pmid: N/A

The ability to reverse-engineer models of software behaviour is valuable for a wide range of software maintenance, validation and verification tasks. Current reverse-engineering techniques focus either on control-specific behaviour (e.g., in the form of Finite State Machines), or data-specific behaviour (e.g., as pre / post-conditions or invariants). However, typical software behaviour is usually a product of the two; models must combine both aspects to fully represent the software’s operation. Extended Finite State Machines (EFSMs) provide such a model. Although attempts have been made to infer EFSMs, these have been problematic. The models inferred by these techniques can be non-deterministic, the inference algorithms can be inflexible, and only applicable to traces with specific characteristics. This paper presents a novel EFSM inference technique that addresses the problems of inflexibility and non-determinism. It also adapts an experimental technique from the field of Machine Learning to evaluate EFSM inference techniques, and applies it to three diverse software systems.

Maffort, Cristiano; Valente, Marco; Terra, Ricardo; Bigonha, Mariza; Anquetil, Nicolas; Hora, André

doi: 10.1007/s10664-014-9348-2pmid: N/A

Software architecture conformance is a key software quality control activity that aims to reveal the progressive gap normally observed between concrete and planned software architectures. However, formally specifying an architecture can be difficult, as it must be done by an expert of the system having a high level understanding of it. In this paper, we present a lightweighted approach for architecture conformance based on a combination of static and historical source code analysis. The proposed approach relies on four heuristics for detecting absences (something expected was not found) and divergences (something prohibited was found) in source code based architectures. We also present an architecture conformance process based on the proposed approach. We followed this process to evaluate the architecture of two industrial-strength information systems, achieving an overall precision of 62.7 % and 53.8 %. We also evaluated our approach in an open-source information retrieval library, achieving an overall precision of 59.2 %. We envision that an heuristic-based approach for architecture conformance can be used to rapidly raise architectural warnings, without deeply involving experts in the process.

Jaafar, Fehmi; Guéhéneuc, Yann-Gaël; Hamel, Sylvie; Khomh, Foutse; Zulkernine, Mohammad

doi: 10.1007/s10664-015-9361-0pmid: N/A

On the one hand, design patterns are solutions to recurring design problems, aimed at increasing reuse, flexibility, and maintainability. However, much prior work found that some patterns, such as the Observer and Singleton, are correlated with large code structures and argued that they are more likely to be fault prone. On the other hand, anti-patterns describe poor solutions to design and implementation problems that highlight weaknesses in the design of software systems and that may slow down maintenance and increase the risk of faults. They have been found to negatively impact change and fault-proneness. Classes participating in design patterns and anti-patterns have dependencies with other classes, e.g., static and co-change dependencies, that may propagate problems to other classes. We investigate the impact of such dependencies in object-oriented systems by studying the relations between the presence of static and co-change dependencies and (1) the fault-proneness, (2) the types of changes, and (3) the types of faults that these classes exhibit. We analyze six design patterns and 10 anti-patterns in 39 releases of ArgoUML, JFreeChart, and XercesJ, and investigate to what extent classes having dependencies with design patterns or anti-patterns have higher odds of faults than other classes. We show that in almost all releases of the three systems, classes having dependencies with anti-patterns are more fault-prone than others while this is not always true for classes with dependencies with design patterns. We also observe that structural changes are the most common changes impacting classes having dependencies with anti-patterns. Software developers could use this knowledge about the impact of design pattern and anti-pattern dependencies to better focus their testing and reviewing activities towards the most risky classes and to propagate changes adequately.

Baysal, Olga; Kononenko, Oleksii; Holmes, Reid; Godfrey, Michael

doi: 10.1007/s10664-015-9366-8pmid: N/A

When submitting patches for code review, individual developers are primarily interested in maximizing the chances of their patch being accepted in the least time possible. In principle, code review is a transparent process in which reviewers aim to assess the qualities of the patch on its technical merits in a timely manner; however, in practice the execution of this process can be affected by a variety of factors, some of which are external to the technical content of the patch itself. In this paper, we describe empirical studies of the code review processes for large, open source projects such as WebKit and Google Blink. We first consider factors that have been examined in previous studies — patch size, priority, and component — and then extend our enquiries to explore the effects of organization (which company is involved) and developer profile (review load and activity, patch writer experience) on code review response time and eventual outcome. Our approach uses a reverse engineered model of the patch submission process, and extracts key information from the issue-tracking and code review systems. Our findings suggest that these non-technical factors can significantly impact code review outcomes.

Adams, Bram; Kavanagh, Ryan; Hassan, Ahmed; German, Daniel

doi: 10.1007/s10664-015-9371-ypmid: N/A

Reuse of software components, either closed or open source, is considered to be one of the most important best practices in software engineering, since it reduces development cost and improves software quality. However, since reused components are (by definition) generic, they need to be customized and integrated into a specific system before they can be useful. Since this integration is system-specific, the integration effort is non-negligible and increases maintenance costs, especially if more than one component needs to be integrated. This paper performs an empirical study of multi-component integration in the context of three successful open source distributions (Debian, Ubuntu and FreeBSD). Such distributions integrate thousands of open source components with an operating system kernel to deliver a coherent software product to millions of users worldwide. We empirically identified seven major integration activities performed by the maintainers of these distributions, documented how these activities are being performed by the maintainers, then evaluated and refined the identified activities with input from six maintainers of the three studied distributions. The documented activities provide a common vocabulary for component integration in open source distributions and outline a roadmap for future research on software integration.

Calefato, Fabio; Lanubile, Filippo; Conte, Tayana; Prikladnicki, Rafael

doi: 10.1007/s10664-015-9372-xpmid: N/A

Communication in global software development is hindered by language differences in countries with a lack of English speaking professionals. Machine translation is a technology that uses software to translate from one natural language to another. The progress of machine translation systems has been steady in the last decade. As for now, machine translation technology is particularly appealing because it might be used, in the form of cross-language chat services, in countries that are entering into global software projects. However, despite the recent progress of the technology, we still lack a thorough understanding of how real-time machine translation affects communication. In this paper, we present a set of empirical studies with the goal of assessing to what extent real-time machine translation can be used in distributed, multilingual requirements meetings instead of English. Results suggest that, despite far from 100 % accurate, real-time machine translation is not disruptive of the conversation flow and, therefore, is accepted with favor by participants. However, stronger effects can be expected to emerge when language barriers are more critical. Our findings add to the evidence about the recent advances of machine translation technology and provide some guidance to global software engineering practitioners in regarding the losses and gains of using English as a lingua franca in multilingual group communication, as in the case of computer-mediated requirements meetings.

Tu, Yu-Cheng; Tempero, Ewan; Thomborson, Clark

doi: 10.1007/s10664-015-9374-8pmid: N/A

Effective communication is important to successful software development, but it is difficult to achieve. We believe transparency — the visibility of information to stakeholders — is an important factor in the effectiveness of communication in software projects. We theorise that more effective communication results from more transparent requirements documents. To test our theory, we conducted an experiment. We developed an operational definition of transparency with three attributes: accessibility, understandability, and relevance. We had students and software practitioners use requirements documents of differing levels of transparency based on these attributes to answer questions. We found that participants with the more transparent document spent less time, answered more questions correctly, and were more confident about their answers, than participants with the less transparent document. The results of our experiment provide evidence that our view of transparency may help evaluate the effectiveness of documents as a form of communication. Further work is needed to reproduce our results, and to determine whether they are generalizable to other types of stakeholders and forms of communication.

McIlroy, Stuart; Ali, Nasir; Khalid, Hammad; E. Hassan, Ahmed

doi: 10.1007/s10664-015-9375-7pmid: N/A

Mobile app reviews by users contain a wealth of information on the issues that users are experiencing. For example, a review might contain a feature request, a bug report, and/or a privacy complaint. Developers, users and app store owners (e.g. Apple, Blackberry, Google, Microsoft) can benefit from a better understanding of these issues – developers can better understand users’ concerns, app store owners can spot anomalous apps, and users can compare similar apps to decide which ones to download or purchase. However, user reviews are not labelled, e.g. we do not know which types of issues are raised in a review. Hence, one must sift through potentially thousands of reviews with slang and abbreviations to understand the various types of issues. Moreover, the unstructured and informal nature of reviews complicates the automated labelling of such reviews. In this paper, we study the multi-labelled nature of reviews from 20 mobile apps in the Google Play Store and Apple App Store. We find that up to 30 % of the reviews raise various types of issues in a single review (e.g. a review might contain a feature request and a bug report). We then propose an approach that can automatically assign multiple labels to reviews based on the raised issues with a precision of 66 % and recall of 65 %. Finally, we apply our approach to address three proof-of-concept analytics use case scenarios: (i) we compare competing apps to assist developers and users, (ii) we provide an overview of 601,221 reviews from 12,000 apps in the Google Play Store to assist app store owners and developers and (iii) we detect anomalous apps in the Google Play Store to assist app store owners and users.