A core goal of Continuous Integration (CI) is to make small incremental changes to software projects, which are integrated frequently into a mainline repository or branch. This paper presents an empirical study that investigates if developers adjust their commit activity towards the above-mentioned goal after projects start using CI. We analyzed the commit and merge activity in 93 GitHub projects that introduced the hosted CI system Travis CI, but have previously been developed for at least one year before introducing CI. In our analysis, we only found one non-negligible effect, an increased merge ratio, meaning that there were more merging commits in relation to all commits after the projects started using Travis CI. This effect has also been reported in related work. However, we observed the same effect in a random sample of 60 GitHub projects not using CI. Thus, it is unlikely that the effect is caused by the introduction of CI alone. We conclude that: (1) in our sample of projects, the introduction of CI did not lead to major changes in developers' commit activity, and (2) it is important to compare the commit activity to a baseline before attributing an effect to a treatment that may not be the cause for the observed effect.

Continue reading

During embedded system development, developers frequently change and reuse the existing C source code for the development of a new but behaviorally similar product. Such frequent changes generally decrease the understandability of C source code although the developers have to understand how it behaves and how to reuse it. So far, much research has been done on symbolic execution techniques that statically analyze the behavioral aspect of given source code. In this paper, we propose a symbolic execution-based approach to extracting a Micro State Transition Table (MSTT) that helps developers understanding the behavioral aspect of embedded C source code based on a fine-grained state transition model. As a case study, we applied the proposed approach to a collection of source files and then confirmed the correctness of the extracted MSTTs.

Continue reading

Cyber-physical systems (CPSs) play a critical role in automating public infrastructure and thus attract wide range of attacks. Assessing the effectiveness of defense mechanisms is challenging as realistic sets of attacks to test them against are not always available. In this short paper, we briefly describe smart fuzzing, an automated, machine learning guided technique for systematically producing test suites of CPS network attacks. Our approach uses predictive ma- chine learning models and meta-heuristic search algorithms to guide the fuzzing of actuators so as to drive the CPS into different unsafe physical states. The approach has been proven effective on two real-world CPS testbeds.

Continue reading

Software systems are often developed by many developers who have a varying range of skills and habits. These developers have a big impact on software quality. Understanding how different developers and developer characteristics impact the quality of a software is crucial to properly deploy human resources and help managers improve quality outcomes which is essential for software systems success. Addressing this concern, we conduct a study on how different developers and developer characteristics such as developer seniority in a system, frequency of commits, and interval between commits relate to Technical Debt (TD). We performed a large-scale analysis on 19,088 commits from 38 Apache Java systems and applied multiple statistical analysis tests to evaluate our hypotheses. Our empirical evaluation suggests that developers unequally increase and decrease TD, a developer seniority in a software system and frequency of commits are negatively correlated with the TD the developer induces, and a developer commit interval has a positive correlation with the TD the developer induces.

Continue reading

Repaying all technical debt (TD) present in a system may be unfeasible, as there is typically a shortage in the resources allocated for TD repayment. Therefore, TD prioritization is essential to best allocate such resources to determine which TD items are to be repaid first and which items are to be delayed until later releases. This study conducts a systematic literature review (SLR) to identify and analyze the currently researched TD prioritization approaches. The employed search strategy strove to achieve high completeness through the identification of a quasi-gold standard set, which was used to establish a search string to automatically retrieve papers from select research databases. The application of selection criteria, along with forward and backward snowballing, identified 24 TD prioritization approaches. The analysis of the identified approaches revealed a scarcity of approaches that account for cost, value, and resources constraint and a lack of industry evaluation. Furthermore, this SLR unveils potential gaps in the current TD prioritization research, which future research may explore.

Continue reading

Machine learning may enable the automated generation of test oracles. We have characterized emerging research in this area through a systematic literature review examining oracle types, researcher goals, the ML techniques applied, how the generation process was assessed, and the open research challenges in this emerging field.

Continue reading

This presentation will describe how we are using, and aiming to use, runtime verification, along with other varieties of formal verification and simulation-based testing, to together provide increased confidence in a range of autonomous systems.

Continue reading