RTIPs Submission and Review Process:
This guide is intended for program developers who are interested in submitting cancer control and cancer survivorship interventions for review and potential inclusion on the National Cancer Institute (NCI) Research-tested Intervention Programs (RTIPs) website.
This guide provides a brief introduction to RTIPs, describes how to submit an intervention to RTIPs, and explains the steps involved in an RTIPs review. This document will help you, as a program developer, to understand:
RTIPs is a searchable database of cancer control interventions. It provides program planners and public health practitioners easy and immediate access to information about interventions and their research base. In addition, it provides access to research-tested program materials that can be used to implement these interventions in community or clinical settings. RTIPs is a voluntary system in which program developers may elect to participate. RTIPs is prominently featured in the Programs section on the Cancer Control P.L.A.N.E.T. (Plan, Link, Act, Network with Evidence-based Tools) Home page at http://ccplanet.cancer.gov. It can also be accessed directly at http://rtips.cancer.gov/rtips/.
Benefits of Participating in an RTIPs Review
RTIPs reviews do require a commitment of time on the part of the program developer to prepare for submission and participate in the review process. However, the effort invested can return a number of benefits:
Determining Your Program's Submission Readiness
Before you decide to submit an intervention to RTIPs, you will need to ensure that it addresses one of the RTIPs topic areas and meets the criteria for inclusion. These two important components of submission readiness are described in the next section.
RTIPs Topic Areas
RTIPs accepts interventions in 12 topic areas:
Eligibility Criteria for Inclusion
Interventions must meet the following criteria to be eligible for an RTIPs review:
If your intervention meets the above criteria, the RTIPs Development Team welcomes you to submit your program for consideration, following the steps described in the next section. If you have questions about submission or are uncertain about whether your intervention meets the above criteria, please go to the Contact Us page of the RTIPs website and submit your question or concern.
Submitting Your Intervention to RTIPs
There are three steps involved in submitting an intervention to RTIPs:
Step 1: Program Registration
If you are interested in pursuing inclusion in RTIPs, your first step is to complete a Program Registration Form on the RTIPs website at http://rtips.cancer.gov/rtips/register/index.do. This form asks you to confirm whether your intervention meets each of the five eligibility criteria noted above; you will also provide your contact information.
Step 2: Initial Submission
If your program appears to meets the eligibility criteria based on the registration form, the RTIPs Development Team will send you an email inviting you to submit materials so that an official determination can be made about your program's eligibility. The following materials are requested at this stage:
Step 3: Kick-Off Call
The third and final step of the submission process is to participate in a kick-off call with the RTIPs Development Team. The kick-off call provides an opportunity for both parties to ask questions, discuss the RTIPs review process, and identify additional materials, if necessary, that will be needed before the intervention is officially accepted for RTIPs review.
Once it has been determined that both parties wish to proceed with the review, you will be sent an Initial Approval Form. This form is used to finalize important details pertaining to your review (e.g., confirmation of the studies and outcomes that will be reviewed, confirmation of the products to be disseminated, and authorization to initiate program review).
The RTIPs Review Process
After your intervention has been formally accepted for review, it is assigned to one of the Review Coordinators (RCs) on the RTIPs Development Team. The RCs are Ph.D.-level evaluation and behavioral/social science specialists who serve as a technical liaison between developers and the pair of expert reviewers assigned to each review. The expert reviewers are Ph.D.-level professionals who have expertise in one or more of the RTIPs topic areas. The identity of the reviewers assigned to your review will be kept confidential to ensure the integrity of the review process.
Both the RC and the reviewers have a role in rating the intervention. The reviewers rate the intervention's Research Integrity, while the RC rates Intervention Impact, Dissemination Capability, and RE-AIM criteria. More information about the RTIPs Rating System is provided in the next section. To facilitate the reviewers' work, the RC provides the reviewers with a review packet that includes all the study articles and other materials to be reviewed; detailed notes, including any supplemental information you may provide to explain how the study or studies were conducted; and information on the rating anchors. The RC is also responsible for drafting the RTIPs summary for your program, which will be posted on the RTIPs website at the conclusion of the review.
The RTIPs Rating System
Intervention evaluation and program materials are evaluated in four areas for the RTIPs review. These rating areas are described below.
Research Integrity reflects the overall confidence reviewers can place in the findings of a program's evaluation based on its scientific rigor. The Research Integrity rating system comprises 16 criteria scored by independent experts. Scores on each criterion are given on a 5-point scale ranging from low quality to high quality. The overall integrity score is an average of the 16 criteria reflecting the merits of the science that went into the program evaluation.
Intervention Impact describes whether, and to what degree, a program is usable and appropriate for widespread application and dissemination. This rating is determined by the RC. Population Reach and Effect Sizes are separately rated on a 5-point scale; these ratings are then combined using the RTIPs Intervention Impact rating table to determine the impact score.
Dissemination Capability refers to the readiness of program materials for use by others as well as a program's capability to offer services and resources to facilitate dissemination. The rating is given on a 5-point scale ranging from low quality (1.0) to high quality (5.0). Dissemination capability is measured through the assessment of three areas:
RE-AIM is a five-step framework designed to enhance the quality, speed, and public health impact of efforts to translate research into practice:
A RE-AIM scoring instrument has been developed to assess the first four steps above. The instrument consists of 22 items within 4 dimensions: Reach (5 items), Effectiveness (3 items), Adoption (6 items), and Implementation (8 items). The Maintenance step is not currently assessed by RTIPs. Additional information about RE-AIM can be found at http://re-aim.org.
As noted in the eligibility criteria, RTIPs programs are required to have messages, materials, and/or other components, in English, available for dissemination to potential implementers in U.S. community or clinical settings. Before your review begins, the RTIPs Development Team will work with you to determine how these materials will be provided to RTIPs users. There are two dissemination options:
RTIPs programs are also required to have written implementation guidance explaining the steps required to implement the program. If you do not have an existing implementation guide, one will be created for your program using an established NCI template. The Implementation Guide will be developed by the RTIPs Development Team from information you provide, or you may draft one using the NCI template.
Summary Approval and Publication
At the conclusion of your review, you will receive a draft intervention summary containing descriptive information about the program; information about the study reviewed, including key findings; and the final RTIPs ratings. An example of a completed RTIPs summary can be found at http://rtips.cancer.gov/rtips/programDetails.do?programId=892493
You will have the opportunity to request clarification on the ratings and suggest edits to the summary before providing your final consent to post the summary via the Final Approval Forms. NCI considers all edits by developers and may confer with the RTIPs Development Team to resolve any technical questions or concerns.
Research to Reality (R2R) Promotion
For each completed RTIPs intervention summary, an associated R2R Featured Partners segment will be produced for the R2R website. Initiated in 2012, the Featured R2R Partners page provides a profile about the intervention's developer(s). This initiative has been created by NCI to help foster an ongoing dialogue around successes, issues, and challenges associated with the implementation, adaptation, and evaluation of the intervention. You will be notified by the RTIPs Development Team and R2R Team when your intervention summary and R2R Featured Partners profile are published on the RTIPs and R2R websites, respectively.
For an example of a dialogue with an R2R featured partner, please visit visit the Research to Reality Web page of the RTIPs site.
RTIPs is a way for you to share advances in implementation science and improve practices in cancer control and survivorship. Although participating in a review does require some time and effort, the benefits of having your program included in RTIPs should prove to be worth the investment. Below is a flowchart to help illustrate the process.