In May 2019, ACM SIGSOFT launched an initiative to improve the quality of research papers and peer reviews at software engineering venues. The initiative is co-chaired by Paul Ralph (Dalhousie University) and Romain Robbes (Free University of Bozen-Bolzano).
The Initiative seeks to improve the quality of papers and peer reviews by generating and evolving empirical standards: brief, public documents that describe expectations for a certain kind of study. These standards are not vague criteria like "soundness" and "presentation." They list specific practices like random assignment (for experiments) and reference snowballing (for systematic reviews).
Preliminary standards were announced in October 2020. All active researchers are encouraged to review the standards relevant to their research programs and report any potential problems or oversights by submitting an issue to the GitHub repo. More standards are in development. To volunteer to contribute, contact Paul Ralph. You do not have to be a member of ACM or SIGSOFT to report an issue or contribute to the standards.
A detailed report, explaining how the standards were generated and how they should be used is also available from the repo.
Any scholarly venue can adopt these standards. Representatives of journals, conferences or other venues who would like to explore adopting the standards should contact us directly.