Opentocs

Artificial Intelligence is becoming increasingly popular with organizations due to the success of Machine Learning and Deep Learning techniques. Using these techniques, data scientists learn from vast amounts of data to enhance behaviour in software-intensive systems. Despite the attractiveness of these techniques, however, there is a lack of systematic and structured design process for developing ML/DL models. The study uses a multiple-case study approach to explore the different activities and challenges data scientists face when developing ML/DL models in software-intensive embedded systems. In addition, we have identified seven different phases in the proposed design process leading to effective model development based on the case study. Iterations identified between phases and events which trigger these iterations optimize the design process for ML/DL models. Lessons learned from this study allow data scientists and engineers to develop high-performance ML/DL models and also bridge the gap between high demand and low supply of data scientists.

InnoSWDev 2014- Proceedings of the International Workshop on Innovative Software Development Methodologies and Practices

Full Citation in the ACM Digital Library

SESSION: New Paradigm

Software engineering for multi-tenancy computing challenges and implications

Jia Ru
John Grundy
Jacky Keung

Enhanced n-version programming and recovery block techniques for web service systems

Kuan-Li Peng
Chin-Yu Huang
Pin-Heng Wang
Chao-Jung Hsu

SESSION: Testing and Verification

Model checking partial software product line designs

Yufeng Shi
Ou Wei
Yu Zhou

Software reliability analysis considering the variation of testing-effort and change-point

Syuan-Zao Ke
Chin-Yu Huang
Kuan-Li Peng

Program structure aware fault localization

Heng Li
Yuzhen Liu
Zhenyu Zhang
Jian Liu

Toward a methodology to expose partially fixed concurrency bugs in modified multithreaded programs

To Tsui
Shangru Wu
W. K. Chan

SESSION: System Development and Management

Cloud-based support for global software engineering: potentials, risks, and gaps

Mohammed Al-qadhi
Jacky Keung

Scaling up analogy-based software effort estimation: a comparison of multiple hadoop implementation schemes

Passakorn Phannachitta
Jacky Keung
Akito Monden
Kenichi Matsumoto

A new business model of custom software development for agile software development

Yoshihito Kuranuki
Tsuyoshi Ushio
Tsutomu Yasui
Susumu Yamazaki

Developing enterprise mobile applications the easy way

Aharon Abadi
Yael Dubinsky
Andrei Kirshin
Yossi Mesika
Idan Ben-Harrush
Uzy Hadad

SESSION: Software Modeling, Analysis with Visualization Approach

Context-sensitive detection of information exposure bugs with symbolic execution

Paul Muntean
Claudia Eckert
Andreas Ibing

Supporting clone analysis with tag cloud visualization

Manamu Sano
Eunjong Choi
Norihiro Yoshida
Yuki Yamanaka
Katsuro Inoue

Sketch-based gradual model-driven development

Peiyuan Li
Naoyasu Ubayashi
Di Ai
Yu Ning Li
Shintaro Hosoai
Yasutaka Kamei

An empirical study of BM25 and BM25F based feature location techniques

Zhendong Shi
Jacky Keung
Qinbao Song

ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: Invited Presentation

Automated debugging: past, present, and future (ISSTA impact paper award)

Chris Parnin
Alessandro Orso

The paper titled “Are Automated Debugging Techniques Actually Helping Programmers?” was published in the proceedings of the International Symposium on Software Testing and Analysis (ISSTA) in 2011, and has been selected to receive the ISSTA 2021 Impact Paper Award. The paper investigated, through two user studies, how developers used and benefited from popular automated debugging techniques. The results of the studies provided (1) evidence that several assumptions made by automated debugging techniques did not hold in practice and (2) insights on limitations of existing approaches and how these limitations could be addressed. In this talk, we revisit the original paper and the work that led to it. We then assess the impact of that research by reviewing how the area of automated debugging has evolved since the paper was published. Finally, we conclude the talk by reflecting on the current state of the art in this area and discussing open issues and potential directions for future work.

ISSTA 2014- Proceedings of the 2014 International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: Concurrency and Verification

Runtime prevention of concurrency related type-state violations in multithreaded applications

Lu Zhang
Chao Wang

Performance regression testing of concurrent classes

Michael Pradel
Markus Huggler
Thomas R. Gross

Verifying atomicity via data independence

Ohad Shacham
Eran Yahav
Guy Golan Gueta
Alex Aiken
Nathan Bronson
Mooly Sagiv
Martin Vechev

Verification-aided regression testing

Fabrizio Pastore
Leonardo Mariani
Antti E. J. Hyvärinen
Grigory Fedyukovich
Natasha Sharygina
Stephan Sehestedt
Ali Muhammad

SESSION: Web Testing

Hybrid security analysis of web JavaScript code via dynamic partial evaluation

Omer Tripp
Pietro Ferrara
Marco Pistoia

Virtual DOM coverage for effective testing of dynamic web applications

Yunxiao Zou
Zhenyu Chen
Yunhui Zheng
Xiangyu Zhang
Zebao Gao

DOM-based test adequacy criteria for web applications

Mehdi Mirzaaghaei
Ali Mesbah

Cross-platform feature matching for web applications

Shauvik Roy Choudhary
Mukul R. Prasad
Alessandro Orso

SESSION: Artifact Studies

Covrig: a framework for the analysis of code, test, and coverage evolution in real software

Paul Marinescu
Petr Hosek
Cristian Cadar

CoREBench: studying complexity of regression errors

Marcel Böhme
Abhik Roychoudhury

SESSION: Static Analyses and Transformations

ARC++: effective typestate and lifetime dependency analysis

Xusheng Xiao
Gogul Balakrishnan
Franjo Ivančić
Naoto Maeda
Aarti Gupta
Deepak Chhetri

A type system for format strings

Konstantin Weitz
Gene Kim
Siwakorn Srisakaokul
Michael D. Ernst

Scalable detection of missed cross-function refactorings

Narcisa Andreea Milea
Lingxiao Jiang
Siau-Cheng Khoo

Tailored source code transformations to synthesize computationally diverse program variants

Benoit Baudry
Simon Allier
Martin Monperrus

SESSION: Test Selection and Reduction

Using test case reduction and prioritization to improve symbolic execution

Chaoqiang Zhang
Alex Groce
Mohammad Amin Alipour

FLOWER: optimal test suite reduction as a network maximum flow

Arnaud Gotlieb
Dusica Marijan

Coverage and fault detection of the output-uniqueness test selection criteria

Nadia Alshahwan
Mark Harman

Dodona: automated oracle data set selection

Pablo Loyola
Matt Staats
In-Young Ko
Gregg Rothermel

SESSION: Localization and Repair

CrashLocator: locating crashing faults based on crash stacks

Rongxin Wu
Hongyu Zhang
Shing-Chi Cheung
Sunghun Kim

Efficient predicated bug signature mining via hierarchical instrumentation

Zhiqiang Zuo
Siau-Cheng Khoo
Chengnian Sun

Semantic differential repair for input validation and sanitization

Muath Alkhalaf
Abdulbaki Aydin
Tevfik Bultan

Automatic repair for multi-threaded programs with Deadlock/Livelock using maximum satisfiability

Yiyan Lin
Sandeep S. Kulkarni

SESSION: Security

Make it work, make it right, make it fast: building a platform-neutral whole-system dynamic binary analysis platform

Andrew Henderson
Aravind Prakash
Lok Kwong Yan
Xunchao Hu
Xujiewen Wang
Rundong Zhou
Heng Yin

Automated testing for SQL injection vulnerabilities: an input mutation approach

Dennis Appelt
Cu Duy Nguyen
Lionel C. Briand
Nadia Alshahwan

SESSION: UI Testing

Reducing GUI test suites via program slicing

Stephan Arlt
Andreas Podelski
Martin Wehrle

SunCat: helping developers understand and predict performance problems in smartphone applications

Adrian Nistor
Lenin Ravindranath

A variability-based testing approach for synthesizing video sequences

José A. Galindo
Mauricio Alférez
Mathieu Acher
Benoit Baudry
David Benavides

Robust test automation using contextual clues

Rahulkrishna Yandrapally
Suresh Thummalapenta
Saurabh Sinha
Satish Chandra

SESSION: Efficiency and Optimizations

Efficient mutation analysis by propagating and partitioning infected execution states

René Just
Michael D. Ernst
Gordon Fraser

Lightweight automated detection of unsafe information leakage via exceptions

Benwen Zhang
James Clause

Integrated energy-directed test suite optimization

Ding Li
Yuchen Jin
Cagri Sahin
James Clause
William G. J. Halfond

Identifying optimal trade-offs between CPU time usage and temporal constraints using search

Shiva Nejati
Lionel C. Briand

SESSION: Generation and Propagation

Feedback-driven dynamic invariant discovery

Lingming Zhang
Guowei Yang
Neha Rungta
Suzette Person
Sarfraz Khurshid

Link: exploiting the web of data to generate test inputs

Leonardo Mariani
Mauro Pezzè
Oliviero Riganelli
Mauro Santoro

Empirically revisiting the test independence assumption

Sai Zhang
Darioush Jalali
Jochen Wuttke
Kıvanç Muşlu
Wing Lam
Michael D. Ernst
David Notkin

An empirical study of injected versus actual interface errors

Anna Lanzaro
Roberto Natella
Stefan Winter
Domenico Cotroneo
Neeraj Suri

SESSION: Tool Demonstrations

Legend: an agile DSL toolset for web acceptance testing

Tariq M. King
Gabriel Nunez
Dionny Santiago
Adam Cando
Cody Mack

ProCrawl: mining test models from multi-user web applications

Matthias Schur
Andreas Roth
Andreas Zeller

X-PERT: a web application testing tool for cross-browser inconsistency detection

Shauvik Roy Choudhary
Mukul R. Prasad
Alessandro Orso

Extending a search-based test generator with adaptive dynamic symbolic execution

Juan Pablo Galeotti
Gordon Fraser
Andrea Arcuri

Canalyze: a static bug-finding tool for C programs

Zhenbo Xu
Jian Zhang
Zhongxing Xu
Jiteng Wang

MuCheck: an extensible tool for mutation testing of haskell programs

Duc Le
Mohammad Amin Alipour
Rahul Gopinath
Alex Groce

The major mutation framework: efficient and scalable mutation analysis for Java

René Just

Defects4J: a database of existing faults to enable controlled testing studies for Java programs

René Just
Darioush Jalali
Michael D. Ernst

A format string checker for Java

Konstantin Weitz
Siwakorn Srisakaokul
Gene Kim
Michael D. Ernst

Constructing coding duels in Pex4Fun and code hunt

Nikolai Tillmann
Jonathan de Halleux
Tao Xie
Judith Bishop

SESSION: Doctoral Symposium

Reusing constraint proofs for scalable program analysis

Meixian Chen

Effective test generation and adequacy assessment for JavaScript-based web applications

Shabnam Mirshokraie

Efficient statistical debugging via hierarchical instrumentation

Zhiqiang Zuo

ISSTA 2015- Proceedings of the 2015 International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: Debugging

Evaluating the usefulness of IR-based fault localization techniques

Qianqian Wang
Chris Parnin
Alessandro Orso

Proactive detection of inadequate diagnostic messages for software configuration errors

Sai Zhang
Michael D. Ernst

An analysis of patch plausibility and correctness for generate-and-validate patch generation systems

Zichao Qi
Fan Long
Sara Achour
Martin Rinard

SESSION: Web Security

BrowserAudit: automated testing of browser security features

Charlie Hothersall-Thomas
Sergio Maffeis
Chris Novakovic

Detection and classification of malicious JavaScript via attack behavior modelling

Yinxing Xue
Junjie Wang
Yang Liu
Hao Xiao
Jun Sun
Mahinthan Chandramohan

Experience report: an empirical study of PHP security mechanism usage

Johannes Dahse
Thorsten Holz

SESSION: Mobile/Web Analysis

WuKong: a scalable and accurate two-phase approach to Android app clone detection

Haoyu Wang
Yao Guo
Ziang Ma
Xiangqun Chen

Systematic execution of Android test suites in adverse conditions

Christoffer Quist Adamsen
Gianluca Mezzetti
Anders Møller

DLint: dynamically checking bad coding practices in JavaScript

Liang Gong
Michael Pradel
Manu Sridharan
Koushik Sen

SESSION: Mobile Security

Scalable and precise taint analysis for Android

Wei Huang
Yao Dong
Ana Milanova
Julian Dolby

Dynamic detection of inter-application communication vulnerabilities in Android

Roee Hay
Omer Tripp
Marco Pistoia

Modelgen: mining explicit information flow specifications from concrete executions

Lazaro Clapp
Saswat Anand
Alex Aiken

SESSION: Concurrency Analysis

When truth is efficient: analysing concurrency

Ganesh Narayanaswamy

Pegasus: automatic barrier inference for stable multithreaded systems

Monika Dhok
Rashmi Mudduluru
Murali Krishna Ramanathan

ConcBugAssist: constraint solving for diagnosis and repair of concurrency bugs

Sepideh Khoshnood
Markus Kusano
Chao Wang

SESSION: Symbolic Execution

Enhancing reuse of constraint solutions to improve symbolic execution

Xiangyang Jia
Carlo Ghezzi
Shi Ying

S-looper: automatic summarization for multipath string loops

Xiaofei Xie
Yang Liu
Wei Le
Xiaohong Li
Hongxu Chen

Experience report: how is dynamic symbolic execution different from manual testing? a study on KLEE

Xiaoyin Wang
Lingming Zhang
Philip Tanofsky

SESSION: Regression Testing

Practical regression test selection with dynamic file dependencies

Milos Gligoric
Lamyaa Eloussi
Darko Marinov

Reliable testing: detecting state-polluting tests to prevent test dependency

Alex Gyori
August Shi
Farah Hariri
Darko Marinov

Empirical evaluation of pareto efficient multi-objective regression test case prioritisation

Michael G. Epitropakis
Shin Yoo
Mark Harman
Edmund K. Burke

SESSION: Search-Based Algorithms

Optimizing selection of competing features via feedback-directed evolutionary algorithms

Tian Huat Tan
Yinxing Xue
Manman Chen
Jun Sun
Yang Liu
Jin Song Dong

Automated software transplantation

Earl T. Barr
Mark Harman
Yue Jia
Alexandru Marginean
Justyna Petke

Automating performance bottleneck detection using search-based application profiling

Du Shen
Qi Luo
Denys Poshyvanyk
Mark Grechanik

SESSION: Verification

Test-case generation for runtime analysis and vice versa: verification of aircraft separation assurance

Marko Dimjašević
Dimitra Giannakopoulou

Reliability assessment for distributed systems via communication abstraction and refinement

Lin Gui
Jun Sun
Yang Liu
Jin Song Dong

Reusing constraint proofs in program analysis

Andrea Aquino
Francesco A. Bianchi
Meixian Chen
Giovanni Denaro
Mauro Pezzè

SESSION: Random Testing

Feedback-controlled random test generation

Kohsuke Yatoh
Kazunori Sakamoto
Fuyuki Ishikawa
Shinichi Honiden

Randomized stress-testing of link-time optimizers

Vu Le
Chengnian Sun
Zhendong Su

Automated unit test generation during software development: a controlled experiment and think-aloud observations

José Miguel Rojas
Gordon Fraser
Andrea Arcuri

SESSION: Domain-Specific Testing

Calculation coverage testing in scientific applications

Yoshiki Sato
Shumpei Hozumi
Shigeru Chiba

Automatic fault injection for driver robustness testing

Kai Cong
Li Lei
Zhenkun Yang
Fei Xie

Preventing data errors with continuous testing

Kıvanç Muşlu
Yuriy Brun
Alexandra Meliou

SESSION: Model-Based Testing

Automatic generation of system test cases from use case specifications

Chunhui Wang
Fabrizio Pastore
Arda Goknil
Lionel Briand
Zohaib Iqbal

RTCM: a natural language based, automated, and practical test case generation framework

Tao Yue
Shaukat Ali
Man Zhang

SESSION: Tool Demonstrations

Dynamic taint tracking for Java with phosphor (demo)

Jonathan Bell
Gail Kaiser

TSTL: a language and tool for testing (demo)

Alex Groce
Jervis Pinto
Pooria Azimi
Pranjal Mittal

CanaryAdvisor: a statistical-based tool for canary testing (demo)

Alexander Tarvo
Peter F. Sweeney
Nick Mitchell
V.T. Rajan
Matthew Arnold
Ioana Baldini

SAMC: a fast model checker for finding heisenbugs in distributed systems (demo)

Tanakorn Leesatapornwongsa
Haryadi S. Gunawi

SESSION: Doctoral Symposium

Making your crashes work for you (doctoral symposium)

Peter Ohmann

Scalable program analysis through proof caching (doctoral symposium)

Andrea Aquino

Collaborative testing across shared software components (doctoral symposium)

Teng Long

Cost-aware combinatorial interaction testing (doctoral symposium)

Gulsen Demiroz

Mining change history for test-plan generation (doctoral symposium)

Thomas Rolfsnes

ISSTA 2017: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: Improving Testing

One test to rule them all

Alex Groce
Josie Holmes
Kevin Kellar

Test reduction has long been seen as critical for automated testing. However, traditional test reduction simply reduces the length of a test, but does not attempt to reduce semantic complexity. This paper extends previous efforts with algorithms for normalizing and generalizing tests. Rewriting tests into a normal form can reduce semantic complexity and even remove steps from an already delta-debugged test. Moreover, normalization dramatically reduces the number of tests that a reader must examine, partially addressing the ``fuzzer taming'' problem of discovering distinct faults in a set of failing tests. Generalization, in contrast, takes a test and reports what aspects of the test could have been changed while preserving the property that the test fails. Normalization plus generalization aids understanding of tests, including tests for complex and widely used APIs such as the NumPy numeric computation library and the ArcPy GIS scripting package. Normalization frequently reduces the number of tests to be examined by well over an order of magnitude, and often to just one test per fault. Together, ideally, normalization and generalization allow a user to replace reading a large set of tests that vary in unimportant ways with reading one annotated summary test.

ISSTA 2018- Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: ISSTA 2018 Retrospective Impact Paper Award

Test input generation with Java PathFinder: then and now (invited talk abstract)

Sarfraz Khurshid
Corina S. Păsăreanu
Willem Visser

The paper Test Input Generation With Java PathFinder was published in the International Symposium on Software Testing and Analysis (ISSTA) 2004 Proceedings, and has now been selected to receive the ISSTA 2018 Retrospective Impact Paper Award. The paper described black-box and white-box techniques for the automated testing of software systems. These techniques were based on model checking and symbolic execution and incorporated in the Java PathFinder analysis tool. The main contribution of the paper was to describe how to perform efficient test input generation for code manipulating complex data that takes into account complex method preconditions and evaluate the techniques for generating high coverage tests.

ISSTA 2019- Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: Keynote

Some challenges for software testing research (invited talk paper)

Nadia Alshahwan
Andrea Ciancone
Mark Harman
Yue Jia
Ke Mao
Alexandru Marginean
Alexander Mols
Hila Peleg
Federica Sarro
Ilya Zorin

This paper outlines 4 open challenges for Software Testing in general and Search Based Software Testing in particular, arising from our experience with the Sapienz System Deployment at Facebook. The challenges may also apply more generally, thereby representing opportunities for the research community to further benefit from the growing interest in automated test design in industry.

ISSTA 2020: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis

Full Citation in the ACM Digital Library

SESSION: Fuzzing

WEIZZ: automatic grey-box fuzzing for structured binary formats

Andrea Fioraldi
Daniele Cono D'Elia
Emilio Coppa

Fuzzing technologies have evolved at a fast pace in recent years, revealing bugs in programs with ever increasing depth and speed. Applications working with complex formats are however more difficult to take on, as inputs need to meet certain format-specific characteristics to get through the initial parsing stage and reach deeper behaviors of the program.

SIGSOFT Awards

Conference Awards

Opentocs

ICSSP '16- Proceedings of the International Workshop on Software and Systems Process

SESSION: Process adaptation and tailoring

SESSION: Process tooling and simulation

SESSION: Effort, cost, and measurement

SESSION: Process design and improvement

SESSION: System evolution and regulated domains

ICSSP '20: Proceedings of the International Conference on Software and System Processes

SESSION: Main Track

InnoSWDev 2014- Proceedings of the International Workshop on Innovative Software Development Methodologies and Practices

SESSION: New Paradigm

SESSION: Testing and Verification

SESSION: System Development and Management

SESSION: Software Modeling, Analysis with Visualization Approach

ISSTA 2021: Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis

SESSION: Invited Presentation

ISSTA 2014- Proceedings of the 2014 International Symposium on Software Testing and Analysis

SESSION: Concurrency and Verification

SESSION: Web Testing

SESSION: Artifact Studies

SESSION: Static Analyses and Transformations

SESSION: Test Selection and Reduction

SESSION: Localization and Repair

SESSION: Security

SESSION: UI Testing

SESSION: Efficiency and Optimizations

SESSION: Generation and Propagation

SESSION: Tool Demonstrations

SESSION: Doctoral Symposium

ISSTA 2015- Proceedings of the 2015 International Symposium on Software Testing and Analysis

SESSION: Debugging

SESSION: Web Security

SESSION: Mobile/Web Analysis

SESSION: Mobile Security

SESSION: Concurrency Analysis

SESSION: Symbolic Execution

SESSION: Regression Testing

SESSION: Search-Based Algorithms