Any complex project flows more smoothly with forethought and careful planning. Researcher experiences indicate that the following are worth serious consideration before beginning an evidence synthesis project:
1. Study team
a. Number:
- Minimum: 2 (for objectivity) to screen abstracts and full-text.
- Ideal: 3, with 3rd member providing input for consensus on disputed items.
- Without 3rd member to settle disagreements, plan in advance the method team will use to resolve split decisions on item inclusion. Apply method consistently and report it in any presentation/publication on the research.
b. Expertise:
- Searching (bibliographic databases, grey literature) – librarian or experienced searcher
- Review process – someone who has performed the type of review you propose to do at least once before
- Evaluation of articles for scientific rigor/methodologic quality and risk of bias
- Biostatistics – For a meta-analysis, someone with expertise in missing and/or inconsistent data could significantly impact the quality of your review, its reception by the scientific community, and its generalizability.
2. Time
a. Overall: A full systematic review will probably take about 18 months from conception to submission for publication.
b. Individual time commitments:
- Depend upon the number of studies being reviewed and the tasks.
- 1st round of reviews (titles and abstracts) UF libraries experience estimates 2-15 hours
- 2nd round of reviews (full-text) - possibly twice the above estimate
- Depend on the experience of the reviewers: Horton et al found that median data extraction time varied from 171 minutes per selected article for experienced data extractors to 246.
- Most of the projects with which UF HSCL librarians have been involved have stalled when study team reviewers faced the time-consuming processes of reviewing or extracting data.
3. Protocol
A written protocol lays out the methods, facilitating writing up that section when the team begins to draft for publication and clarifying to the entire team what the project will entail. In the event new team members are recruited, the written protocol can bring them up to speed quickly. The protocol should address the following issues:
- Specific research question being addressed (emphasis on specific-this is the best point in the project to prevent "mission creep")
- Purpose for the project or for addressing the research question
- Names of team members and role(s) each will play in the project
- Deadlines for various phases of the project
- Frequency and means of communication between study team members (recommend regular intervals rather than after completion of phases to avoid the worst stalling of the project)
- Inclusion/exclusion criteria for studies
- Method for resolving disagreements (e.g. on studies to be included)
- Process for addressing shifts in study team membership, including time commitment shifts
- Dissemination strategy: target conferences/journals, timeline, responsibilities of team members
