Introduction:

The integrity of trial data, protection of the rights, and well-being of study participants is measured by monitoring. Monitoring is an ongoing process conducted before, during and after the trial and is classified in four distinct types of visits (pre-initiation, initiation, routine and close out). The designated site monitor conducts periodic visits by visiting the site, this is known as on-site monitoring (often the site monitors are sponsor employees or contracted independent monitors depending on the type of trial, or the trial manager for non-commercial trials, or on-boarding local monitors may be beneficial, as they have experience in the local language, culture, and practices.) ^{[1] [2]}

In Remote monitoring the monitor or clinical research associate (CRA) reviews the data without visiting the investigational site via secure online workspaces or other platforms. The monitor executes the protocol in regular manner and data is recorded in the electronic case report forms (eCRFs). The investigational site staff if expected to upload all the trial related documents such as informed consent forms, source documents, lab reports to online workspace so that, the data can be immediately reviewed by the site monitor. The data entered in eCRF is verified against the source documents available online hence this monitoring visit is termed as remote monitoring visit. ^{[2], [3]}

Centralized monitoring involves analytical evaluation carried out by central monitors at a central location other than the site at which the clinical investigation is being conducted. Risk based monitoring is the process of assuring that the quality of clinical trials is maintained by identifying, assessing, monitoring, and reducing the risks that could affect the safety of a study ^{[2], [4]}.

The FDA defines centralized monitoring as “Consists of a remote evaluation carried out by sponsor personnel or representatives (e.g., clinical monitors, data management personnel, or statisticians) at a location other than the sites at which the clinical investigation is being conducted.” It also enables sponsor/CROs to identify trends such as range, consistency, data completeness or unusual distribution of data within and/or between clinical research sites involved in the study. ^[5]

Until August 2013 when US FDA issued a guidance on RBM, pharma-sponsored studies followed the standard monitoring technique of regular on-site monitoring with 100% Source Data Verification (SDV). The US FDA guidance on RBM was published in August 2013. Thus, the real drive to adopt and implement RBM at industry level started post- August 2013. However, there are different approached towards implementing RBM across organisations.^[6],[7]

Risk-based monitoring is the primary feature of centralized monitoring techniques. Automated reviews facilitate the need for manual intervention which helps uncover errors. The potential risk alerts are generated by the RBM technologies using machine learning. The prospective analysis of data collected during the trial conduct helps these technologies identify trends and patterns. Any issues identified at the patient, site, or study levels are notified by the RBM technology to the user. The continuous tracking and analysis of trend information and any time available actionable data enables sponsors to take corrective action in a timely manner, improving the patient safety and data quality^{[8], [9]}. 

A risk-based monitoring plan is different than the traditional monitoring plan, as it uses various monitoring approaches like centralized monitoring, on-site monitoring, and off-site monitoring, and shares the responsibilities with various cross-functional resources. It should clearly describe the monitoring approaches to be used regarding what should be achieved through centralized monitoring and what should be done during on-site visits and off-site monitoring for controlling and mitigating various types of risk/issue(s). An RBM plan is considered a “dynamic” plan, as the frequency and extent of on-site and off-site monitoring activities will vary based on the critical risk/ issue(s) findings, performance of the sites, the quality of data coming in from different sites, and the identification of new risks. For instance, the sites that are not performing well or are having more quality issues will receive more attention or on-site visits than the sites which are performing as, or better than, expected. An RBM plan must also suggest managing unresolved key issues or significant instances of site non-compliance. 

The successful implementation of RBM is solely not dependent on the technology solution but also on the people and process. The impact technology has on the clinical operations must be performed but it may be influenced by interdependent variables. 

The below mentioned measures are considered to aid in evaluating the impact RBM technology has on organization:

•   --> Portfolio conversion time
•   --> Number of supporting systems retired
•   --> Costs associated with maintaining technology
•   --> Time needs to implement technology solution in existing process and platforms
•   --> Costs involved in external sourcing of trials using technologies
•   --> Cycle time for all monitoring activities
•   --> Database locks irrespective of trial phase ^[10]

Correctly implemented RBM can help shorten the time taken to clean and evaluate the study data. The RBM technology evaluates the data an ongoing basis which further helps reduce the time from end of study to reporting of study results. Regulatory agencies are focusing on more robust patient safety and data quality credibility. Sponsors who are implementing the RBM technology with traceability and audit capabilities are better equipped to meet current and emerging regulatory requirements. ^{[8], [9]}

Post COVID-19 pandemic, the companies naturally experimented with transitioning to remote-site monitoring to avoid trial disruptions. The data collected during these times clearly indicate that the effectiveness of remote-site monitoring is equivalent to that of on-site monitoring.  Distinctly, the rapid shift from on-site to remote-site monitoring for most clinical trials during the pandemic indicates that implementing RBM model without compromising the monitoring effectiveness is feasible ^{[11], [12], [13], [14], [15], [16]}

Methodology:

This study used a survey method to collect data about the opinions and experiences of clinical research professionals who have worked on risk-based management systems at some point of time in their clinical research career. The survey was available on the electronic portal and the link to the survey was shared with participants using various modes of communication like mail, phone, social media, etc.  The survey is conducted among clinical research professionals having more than five years of work experience in the field of clinical research. The forecasted data collection timeline was of 15 months. The statistical tool was used to calculate the sample size for this project by the biostatistician involved in the study. The statistician confirmed that the projected sample size for this study should be 500 however, a sample size of 300 is considered fair and 1000 is excellent.  

Results:

The questionnaire was sent to multiple clinical research professionals worldwide. However, the complete response to the questionnaire was received from 511 participants.

Descriptive analysis of general information collected from the participants:

Figure 1

59.3% (303 participants) of the respondent’s current organization is CRO, while 34.1% (174 participants) work for sponsor companies, and 6.1% (34 participants) work for other type of organization i.e., Site Management Organizations.

Figure 2

39.7% (203 participants) have experience of working only in the global trial, 11.9% (61 participants) have experience of working only in the local trial and 48.3% (247 participants) have work experiences of both local and global trials.

76.1% (390 respondents) had experience in working with risk-based monitoring while 23.9% (122 respondents) had no previous working experience in RBM. 30.50% (156 participants) have experience of 1-5 years, 33.30% (170 respondents) have 6-10 years of experience, and 12.30% (63 respondents) have 11-15 years of experience in the field. 23.9% (122 respondents) chose to not disclose their experience. Hence for statistical analysis, the response of 122 participants was not considered leading to an adjustment in the percentage. 87.5% (447 respondents) had been working in their current role from 1-5 years, 11% (56 respondents) had been in their current role of from 6 -10 years, and 1.6% (8 respondents) had less than 1 year of experience in their current role.

Figure 3

The above figure represents the current designation of the respondent in different organizations at different roles. The majority of the respondents are clinical research associates, Project managers, clinical trial managers, and central monitors including trip report reviewers.

The next question was ‘Do you feel RBM implementation reduces cost in overall trial management?’. 51.7% (264 participants) agree that RBM implementation reduces the cost in overall trial management, 26% (133 participants) strongly agree to this, 14.9% (76 participants) have neutral opinion while a small percentage of 5.9% (30 participants) and 1.6% (8 participants) disagree and strongly disagree respectively that RBM implementation reduces the cost of the trial.

Figure 4

The participants were then asked, ‘Reduction in 100% SDV means lesser frequency of onsite monitoring visit thus influencing the cost involved in the trial’. 49.9% (255 participants) agree that reduction in 100% SDV reduces the monitoring visit thus influences the cost involved in the trial. After performing a Cross table analysis of participants having CRO experience versus experience working in sponsor in reduction in 100% SDV means lesser frequency of onsite monitoring visit thus influencing the cost involved in the trial, 156 participants working in CRO and 75 participants working for sponsor agree that reduction in 100% SDV means less monitoring visit thus influencing the cost involved in the trial. Chi-square test results of Cross table analysis of participants having CRO experience versus experience working in sponsor in reduction in 100% SDV means lesser frequency of onsite monitoring visit thus influencing the cost involved in the trial showed that the p -value (0.00) appears in the same row in the Asymptotic significance(2-sided) column. The result is non-significant as the value is less than 0.05.

The participants were asked, ‘Research says potential financial impacts comes from longer RBM trials -often phase 3’. They were given the options of Agree, disagree, strongly agree, strongly disagree, and neither agree nor disagree. 56.07% (286) of the participants agree with the statement and 25.49% (130) of the participants strongly agree.

Figure 5

The participants were also asked if they agree with the statement, ‘Incorporating quality by design at the concept stage improves the protocol design and monitoring plan’. They were given the options of Agree, disagree, strongly agree, strongly disagree, and neither agree nor disagree. 27.00% of the participants strongly agree with the statement. 60.46% and 12.52% of the participants agree and disagree with the statement respectively.

Figure 6

The participants were asked, ‘What, according to them are the key elements of real-time quality management?’ They were given the options of Fixed performance indices, Key risk indicators, monitoring adjustments, error predictions, Quality metrics, and site performance metrics. They could select more than one option. 70.84% (362) participants said Site performance metrics, 62.42% (311) participants said quality metrics, 56.75% (290) of participants said Key risk indicator, 40.11% (205) of participants said fixed performance indices, 20.35% (104) participants said error prediction and 14.67% (75) said monitoring adjustments.

The next question was, how quality is managed in RBM trials. They were given the options of Developing Risk based monitoring plan/ Risk mitigation plan, developing systems ensuring audit trails, Training the personnel working in RBM trials, setting tolerance limits/ acceptable errors, Identification and analysis of risk, and Communication of potential observation and corrective action taken. 77.88% (398) of the participants said Developing Risk-based monitoring plan/ Risk mitigation plan, 50.09% (256) of the participants chose developing systems ensuring audit trails and 38.35% (196) of the participants said Training the personnel working in RBM trials.

The next question to the participants was ‘Data management reviews and a modest, targeted level of on-site monitoring still play meaningful, complementary roles to Clinical Study Manager in the delivery of a high-quality study.’ They were asked to Strongly agree, strongly disagree, agree, disagree or neither agree nor disagree. 46.77% (239) of the participants agree and 38.16% (195) of the participants strongly agree. 10.95% (56) of the participants neither agree nor disagree.

Figure 7

Discussion:

The maximum number of participants participating in the survey had experience working in the CRO and managing both the global and local trials having either or all the components of RBM implemented in the study.

The systems developed by the clinical research industry to support the successful implementation and adoption of RBM model faces a challenge of different operating model used across the industry hence the new system must be versatile and adaptable to these operating models. The general capabilities of the RBM system would include a source system that supports the risk indicators, mapping engine, which allows the source data to be mapped to an internal data warehouse that supports RBM,  system to load unstructured data in combination with structured data, system to segregate areas for exploration and standard reporting,  framework allowing both incremental and cumulative data, a system allowing creation of data snapshots to support the traceability of decisions made during the course of the trial, system sending alerts to appropriate users, defined and stored accessible metadata, data transfers must be validated against a data standards catalogue, and includes data integrity checks and sends error message and rejects the transfer if it fails integrity checks while the survey data indicates that system that supports the risk indicators, has a mapping engine, which allows the source data to be mapped to an internal data warehouse that supports RBM, allows creation of data snapshots to support the traceability of decisions made during the course, System sending alerts to appropriate users and , and includes data integrity checks and sends error message and rejects the transfer if it fails integrity checks would be the ideal capabilities looked for in the RBM system. ^[17]

Sponsor should select the technologies that are able to examine the baseline risk, supports both the on-site and centralized monitoring techniques, is cost efficient and provides a process for systematic review of the trials risk profile. Subsequently, developing study monitoring plan, defining critical data and processes, developing quality and risk management plan, and implementing risk assessment will help the sponsors to adapt to RBM faster. ^{[18], [19]}

The reviewed literature suggests that even though implementation of RBM system will incur large financial cost however, this cost can be nullified in long run phase 2 and phase 3 trials as RBM will reduce the number of onsite monitoring visits in turn reducing the cost associated to the monitoring visits. The survey data also complies and agrees that implementation of RBM can increase the budget initially however this cost is saved in the long run trials due to reduced onsite monitoring activities^{. [20], [21]}

The available literature emphasized that quality in RBM can be achieved by developing risk-based monitoring plan/ Risk mitigation plan, developing systems ensuring audit trails, training the personnel’s working in RBM trials, setting tolerance limits/ acceptable errors, identification and analysis of risk, communication of potential observation and corrective action taken. The survey data agreed to that quality in RBM can be achieved by developing risk-based monitoring plan/ Risk mitigation plan, developing systems ensuring audit trails, training the personnel’s working in RBM trials, setting tolerance limits/ acceptable errors, identification and analysis of risk, communication of potential observation and corrective action taken. Since we are talking about the risk-based monitoring the development of risk-based monitoring plan is vital.^[22]

RBM also suggest development of quality management plan as a standalone document that defines the quality parameters and quality tolerance limits. The literature and survey suggested that integrated quality management plan revolves around the following principles that quality is built in from the protocol development and managed through process of continuous improvement, quality goals and metrics are prospectively identified throughout the study duration, and risk to quality are prospectively identified and mitigated.^[23]

Conclusion:

Though traditional monitoring model is very well developed and adopted by the industry personnel a swift acceptance of RBM model was seen during and post the COVID-19 pandemic. The concept of alternative monitoring methods and RBM proved to be boon to the industry in maintaining compliance with respect to IP, safety reporting and remote monitoring. Quality and Risk-based monitoring model runs parallel thus real time quality management must be implemented. The elements of real time quality management can be attributed to defining of quality, site performance metrices, defining key risk indicators, error predictions and allow monitoring adjustments. We can infer that any organizations implementing RBM technologies faces lot of challenges at the initial stage with respect to selection of systems, the management of time, cost, people. However, in the long term the RBM systems prove to be beneficial in reducing the cost, reducing the time involved and manual efforts

References

1. Gruppo Italiano per lo Studio della Sopravvivenza nell'Infarto Miocardico-Italian group for the study of the survival of myocardial infarction.

2.Temple, R. Policy Developments in Regulatory Approval. Statistics in Medicine. 21: 2939-2948 (2002).

3.Guidance for industry, E6 Good Clinical Practice: Consolidated Guidance, 1996, section 5.18.3.

4.Usher, R. PhRMA BioResearch Monitoring Committee Perspective on Acceptable Approaches for Clinical Trial Monitoring. Drug Inf J. 44: 477-483 (2010). 18 FDA, Concept Paper: Quality in FDA-Regulated Clinical Research; Background to HSP/BIMO Workshop 5/10- 5/11/07, (4/26/07).

5.Ashok G., RBM Implementation: Developing An Effective Risk Based Monitoring Plan; Cited from https://www.clinicalleader.com/doc/rbm-implementation-developing-an-effective-risk-based-monitoring-plan-0001; cited on 03Oct2019

6.Heema Desai. “A Pilot Study to Understand the Challenges and Benefits of Implementing Risk Based Monitoring in Clinical Research” MAR Pharmaceutical Sciences 1.1 (2020); cited on 12Dec2021

7.The Foundation of Risk-Based Monitoring Success available on www.perkinelmer.com; cited on 02Oct2020

8.http://www.transceleratebiopharmainc.com/wpcontent/uploads/2013/10/TransCelerate-RBM-Position-Paper-FINAL-30MAY2013.pdf; cited on 21Jan 2021

9.A.Ghone, The Risk Based Monitoring Plan, 30 Oct 2017, applied clinical trials, available on https://www.appliedclinicaltrialsonline.com/view/risk-based-monitoring-plan, cited on 15 Sep 2020.

10.Addressing Ever-Rising Cost in Conducting Clinical Trials. Feb. 6, 2013. Accessible at: http://blog.covance.com/2013/02/cost-in-conducting-clinical-trials/

11.Morrison et al. Monitoring the Quality of Conduct of Clinical Trials: A Survey of Current Practices. Clin Trials.

12.Lessons Learned from An RBM Implementation; available on https://www.clinicalleader.com/doc/lessons-learned-from-an-rbm-implementation-0001 3/3 cited on 10 Oct 2020

13.7 Reasons You’re Not Ready For Risk Based Monitoring; available on https://www.clinicalleader.com/doc/reasons-you-re-not-ready-for-risk-based-monitoring-0001 2/3; cited on 10 Nov 2020

14.Barnes S, Katta N, Sanford N, Staigers T, Verish T. Technology Considerations to Enable the Risk-Based Monitoring Methodology. Ther Innov Regul Sci. 2014;48(5):536-545. doi:10.1177/2168479014546336 accessed on 24Jan2022

15.Oleson, Kimberly & Fay, Margaret. (2013). Fay MF and Oleson K. A Rationale for a Risk-Based Approach to Clinical Trial Conduct and Monitoring Practices. The Monitor. 2013;12: 9-14. The Monitor. 12. 9-14. Cited on 17 Oct 2021

16.Koreith K. 2011. The high cost and questionable impact of 100% SDV. CenterWatch Monthly 18:1, 15–7

17.Barnes S, Katta N, Sanford N, Staigers T, Verish T. Technology considerations to enable the risk-based monitoring methodology. Therapeutic Innovation & Regulatory Science. 2014;48: 536-545.

18.Peter M.J.M. Jongen, Cornelis A. van den Bogert, Catharina W.E. van de Laar, Kim Notenboom, Elysée T.M. Hille & Ingrid Hegger (2016) Risk indicator taxonomy for supervision of clinical trials on medicinal products, Current Medical Research and Opinion, 32:7, 1269-1276, DOI: 10.1185/03007995.2016.1170671

19.RBM Interactive guide; available on

https://www.transceleratebiopharmainc.com/rbminteractiveguide/what-is-risk-based-monitoring-rbm/introduction/ ;accessed on 13 Jan 2020

20.How Risk-Based Monitoring Can Improve Clinical Trial Operations; available on https://www.anjusoftware.com/about/all-news/insights/how-risk-based-monitoring-can-improve-clinical-trial-operations 3/9; cited on 10 Oct           2020

21.Sugarman J, Califf RM. Ethics and regulatory complexities for pragmatic clinical trials. JAMA. 2014 Jun 18;311(23):2381-2. doi: 10.1001/jama.2014.4164. PMID: 24810723.

22.Reflection paper on risk-based quality management in clinical trials EMA/269011/2013; cited on 17 Dec 2020

23.Jules Mitchel, MBA, PhD | Dean Gittleman, MS | Judith M. Schloss Markowitz, MS, Laura Suciu, MS | Michael R. Hamrell, PhD, RAC, FRAPS, RQAP-GCP, CCRA “A 21st Century Approach to QA Oversight of Clinical Trial Performance and Clinical Data Integrity” December 2013; cited on 21 Dec 2021.