Empirical Standards

Select all that apply:

General

Engineering research Research that invents and evaluates technological artifacts

Multimethodology or mixed methods Studies that use two or more approaches to data collection or analysis to corroborate, complement and expand research findings (multi-methodology) or combine and integrate inductive research with deductive research (mixed methods), often but not necessarily relying on qualitative and/or quantitative data.

Qualitative

Action researchEmpirical research that investigates how an intervention, like the introduction of a method or tool, affects a real-life context

Case study"An empirical inquiry that investigates a contemporary phenomenon (the "case") in depth and within its real-world context, especially when the boundaries between phenomenon and context are unclear" (Yin 2017)

Grounded theoryA study of a specific area of interest or phenomenon that involves iterative and interleaved rounds of qualitative data collection and analysis, leading to key patterns (e.g. concepts, categories)

Qualitative survey (i.e. interviews)Research comprising semi-structured or open-ended interviews

Quantitative

BenchmarkingA study in which a software system is assessed using a standard tool (i.e. a benchmark) for competitively evaluating and comparing methods, techniques or systems "according to specific characteristics such as performance, dependability, or security” (Kistowski et al. 2015).

Data science Studies that analyze software engineering phenomena or artifacts using data-centric analysis methods such as machine learning or other computational intelligence approaches as well as search-based approaches

Experiment (with human participants)A study in which an intervention is deliberately introduced to observe its effects on some aspects of reality under controlled conditions

Optimization study (including search-based software engineering)Research studies that focus on the formulation of software engineering problems as search problems, and apply optimization techniques to solve such problems

Quantitative longitudinal studyA study focusing on the changes in and evolution of a phenomenon over time

Quantitative simulationA study that involves developing and using a mathematical model that imitates a real-world system's behavior, which often entails problem understanding, data collection, model development, verification, validation, design of experiments, data analysis, and implementation of results.

Questionnaire surveyA study in which a sample of respondents answer a series of (mostly structured) questions, typically through a computerized or paper form

Repository MiningA study that quantitatively analyzes a dataset extracted from a platform hosting of structured or semi-structured text (e.g a source code repository)

Literature review

Case surveyA study that aims to generalize results about a complex phenomenon by systematically converting qualitative descriptions available in published case studies into quantitative data and analyzing the converted data

Systematic literature reviewA study that appraises, analyses, and synthesizes primary or secondary literature to provide a complete, exhaustive summary of current evidence regarding one or more specific topics or research questions

Other

Meta-scienceA paper that analyses an issue of research methodology or makes recommendations for conducting research

ReplicationA study that deliberately repeats a previous study (the "original study") to determine whether its results can be reproduced

An empirical method not listed above