The Institute for Replication (I4R) and the University of Sheffield will be organising the AI Replication Game @ Sheffield on June 17th, 2024. For more information and to register for the AI Replication Game, please see the information below. Virtual participation is possible and coauthorship to a meta paper is granted.

The AI Replication Game @ Sheffield University is a one-day event on June 17th, 2024 that brings researchers together to collaborate on reproducing quantitative results published in high-ranking social science journals. Replication is a crucial aspect of scientific research, ensuring that results are reliable and reproducible. By participating in the Replication Game, you will not only contribute to the integrity of research in your field, but also have the opportunity to network with fellow researchers and develop your coding and AI skills. The Replication Game is open to all researchers: academics, post-docs, and graduate students. Knowledge of Python,R or Stata is essential.

Participation: Researchers participating in the AI Replication Game @ University of Sheffield will be randomly assigned to one of three teams: Machine, Cyborg or Human. Machine and Cyborg teams will have access to (commercially available) LLM models to conduct their work; Human teams of course rely only on unaugmented human skills. Each team consists of 3 members with similar research interests and varying skill levels. Teams will be asked to check for coding errors and conduct a robustness reproduction, which is the ability to duplicate the results of a prior study using the same data but different procedures as were used by the original investigator.

During the event, participants are expected to read the paper and familiarise themselves with the code and data shared alongside the paper. Teams then will work together in groups of ~3 to check for coding errors and conduct sensitivity analysis. No work is conducted before/after the event other than answering a short survey.

Coauthorship: all participants will get coauthorship on a metaresearch journal paper which combines the work of all teams.

Schedule: The AI Replication Game @ University of Sheffield will start at 9:15 AM local time, and end at 4PM local time. Virtual participation is possible.

Please register here: before June 1st, 2024.