2nd International Workshop on Practical Reproducible Evaluation of Computer Systems

Topics | Program | Organization | Contact

The P-RECS’19 workshop will be held as a full-day meeting at ACM HPDC 2019 in Phoenix, Arizona, USA on June 24th, 2019. This year, HPDC runs under the ACM Federated Computing Research Conference. This large federated conference will assemble 11 affiliated conferences and will provide excellent opportunities for interdisciplinary networking and learning.

The P-RECS workshop focuses heavily on practical, actionable aspects of reproducibility in broad areas of computational science and data exploration, with special emphasis on issues in which community collaboration can be essential for adopting novel methodologies, techniques and frameworks aimed at addressing some of the challenges we face today. The workshop brings together researchers and experts to share experiences and advance the state of the art in the reproducible evaluation of computer systems, featuring contributed papers and invited talks.


We expect submissions from topics such as, but not limited to:


09:00-09:15 Welcome
09:15-10:15 Keynote (Dr. Carl Kesselman)
10:15-10:45 Coffee break
10:45-12:15 Paper Presentations 1
12:15-13:30 Lunch (hosted by HPDC/FCRC)
13:30-15:00 Paper Presentations 2
15:00-15:30 Coffee Break
15:30-16:30 Open discussion
16:30-17:00 Closing remarks

Keynote Address

Carl Kesselman (University of Southern California)

Title: Making Lightening Strike Twice: Achieving reproducibility and impact in a data-driven scientific environment.

Abstract: A cornerstone of the scientific method is the ability for one scientist to reproduce the results of another scientist. This requires that investigators take explicit steps such as ensuring that protocols are well defined, reagents and cell lines characterized and validated, etc. A critical aspect of this process is describing what data has been collected and how it is analyzed. While science has always been driven by the collection, analysis and sharing of data, technology advances have shifted data processing from the role of a final analysis step to a core and integral part of the scientific method. However, with the increased complexity of computational methods and shear volume of data, achieving reproducibility of a data-driven scientific investigation becomes correspondingly more difficult. In my talk, I will describe the properties that data in a scientific investigation should have to promote reproducibility. Specifically, reproducibility requires that data should be Findable, Interoperable, Accessible, or Reusable, or FAIR. I will describe methods and tools that can help promote reproducibility in data-driven scientific research and will illustrate with examples from FaceBase, an NIH funded consortium that is generating data associated with caniofacial development and malformation.

Bio: Dr. Carl Kesselman specializes in grid computing technologies. This term was developed by him and professor Ian Foster in the book The Grid: Blueprint for a New Computing Infrastructure. He and Foster are winners of the British Computer Society’s Lovelace Medal for their grid work. He is institute fellow at the University of Southern California’s Information Sciences Institute and a professor in the Epstein Department of Industrial and Systems Engineering, at the University of Southern California.

Papers Session 1

Papers Session 2


Submit (single-blind) via EasyChair. We look for two categories of submissions:


Authors are invited to submit manuscripts in English not exceeding 5 pages of content. The 5-page limit includes figures, tables and appendices, but does not include references, for which there is no page limit. Submissions must use the ACM Master Template (please use the sigconf format with default options).


The proceedings will be archived in both the ACM Digital Library and IEEE Xplore through SIGHPC. In addition, pre-print versions of the accepted articles will be published in this website (as allowed by ACM’s publishing policy).


These tools can be used used to automate your experiments (not an exhaustive list): CK, CWL, Popper, ReproZip, Sciunit, Sumatra.

Important Dates


Program Committee


Please address workshop questions to ivo@cs.ucsc.edu.