GUIDELINES for TRANSPARENCY and OPENNESS PROMOTION (TOP) in SCIENCE
Published in Science in 2015, the TOP guidelines outline best practices in scholarly research to promote transparency and openness. Mind & Life is committed to supporting these goals for the projects we fund. The guidelines below represent the policies of the Mind & Life Institute for the various topics listed. All Mind & Life Institute grantees are expected to follow these guidelines in their funded research projects.
All data, program code and other methods should be appropriately cited. Such materials should be recognized as original intellectual contributions (separate from any related publications) and afforded recognition through specific citation.
- All data sets and program code used in a report or publication generated by this funding should be cited in the text and listed in the reference section.
- References for data sets and program code should include a persistent identifier, such as a Digital Object Identifier (DOI). Persistent identifiers ensure future access to unique published digital objects, such as a text or data set. Persistent identifiers are assigned to data sets by digital archives, such as institutional repositories and partners in the Data Preservation Alliance for the Social Sciences (Data-PASS).
- Data set citation example: Campbell, Angus, and Robert L. Kahn. American National Election Study, 1948. ICPSR07218-v3. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 1999. http://doi.org/10.3886/ICPSR07218.v3
All participants’ demographics should be described thoroughly, including but not limited to age, gender, race, ethnicity, socioeconomic status, and education level. The Mind & Life Institute encourages research on populations outside of those commonly studied (e.g., white, wealthy, undergraduate, etc.). When interpreting findings, researchers should clearly state the limits of generalization to other populations, and frame any conclusions within the specific context of the study. This can be done by adding a Constraints on Generality (COG) statement to the discussion section of a manuscript.
Data, Analytic Methods (Code), and Research Materials Transparency
The policy of the Mind & Life Institute is that grantees indicate explicitly in all reports, publications, and other research communications whether the data, methods used in the analysis, and materials used to conduct the research will be made available (after publication) to any researcher for purposes of reproducing the results or replicating the procedure.
- Prior to receiving funds, grantees must indicate if they will or will not make their data, analytic methods, and study materials available to other researchers.
- If a grantee agrees to make materials available, the grantee must specify where that material will be available.
Design and Analysis Transparency
The policy of the Mind & Life Institute is to fund proposals where grantees plan to follow standards for disclosing key aspects of the research design and data analysis. Grantees are encouraged to review the standards available for many research areas from http://www.equator-network.org/ and use those that are relevant for the reported research.
Preregistration of Studies and Analysis Plans
Preregistration of studies involves registering the study design, variables, and treatment conditions prior to conducting the research. Including an analysis plan involves specification of sequence of analyses or the statistical model that will be reported. Preregistration is strongly encouraged, as it leads to improved rigor around theory and statistics, and helps reduce publication of false positive results. (See here for helpful information and FAQ about preregistration in psychology.)
The policy of the Mind & Life Institute is to fund proposals where grantees indicate whether the proposed research will be preregistered in an independent, institutional registry (e.g., http://clinicaltrials.gov/, http://socialscienceregistry.org/, http://openscienceframework.org/, http://egap.org/design-registration/, http://ridie.3ieimpact.org/).
Prior to receiving funds, grantees must indicate if they will or will not preregister the research in an independent, institutional registry, and whether or not this will include an analysis plan.
- If a grantee will preregister the research, the grantee must confirm that the study will be registered prior to conducting the research and that the preregistration will adhere to the disclosure requirements of the institutional registry or those required for the preregistered badge maintained by the Center for Open Science.
- If a grantee will preregister the research with an analysis plan, the grantee must also:
- confirm all preregistered analyses will be reported, or, if there are changes in the analysis plan following preregistration, those changes will be disclosed with explanation for the changes in any reports or publications.
- clearly distinguish analyses that were preregistered from those that were not, such as having separate sections in the results for confirmatory and exploratory analyses in any reports or publications.
The policy of the Mind & Life Institute is to encourage submission of replication studies in response to specific solicitations for replication proposals or through a general funding mechanism.
On initial submission, authors of proposed replication studies should:
- denote in the cover letter that the study is a replication study submission.
- include a full description of the introduction justifying the importance of the replication and methods with a complete analysis plan.