A Systematic Review to Assess Accuracy and Reliability of the Ultrasound (US) as Pre-Operative Non-Invasive Imaging Modality in Diagnosing Rotator Cuff Tears (RCTs) in Comparison to Magnetic Resonance Imaging (MRI).

Dr Neeraj Sharma *¹, Dr. Priya Sharma ², Dr Pankaj Sharma ³, Dr. Nizar Mohamed KV ⁴

1. M.S Orthopaedics, LLH Hospital Musaffah, Abu Dhabi.
2. MD Anaesthesia, NMC Royal Hospital Abu Dhabi.
3. Department of Radiodiagnosis, AIIMS Rishikesh.
4. Mch Orthopaedics, Zayed Military Hospital, Abu Dhabi.

                                                                                                                                                                               
*Correspondence to: Dr Pankaj Sharma, Department of Radiodiagnosis, AIIMS Rishikesh.

Copyright

© 2024 Dr Pankaj Sharma. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Received: 17 January 2024

Published: 01 February 2024

Abstract

Background: With the background of availability of high frequency transducers and with the improvement in training guidelines for shoulder US over past 10 years, this systematic literature review has been done to assess whether the accuracy and reliability of the US in detecting RCTs has become comparable to MRI.

Aims: To improve the quality of care of the patient by providing them with a more convenient and readily available yet accurate alternative.

Method: The Cochrane Library and PubMed databases were systematically searched for full-text journal articles published between January 1, 2013, and December 31, 2022. Studies comparing the accuracy of US with MRI for detecting RCTs with arthroscopy/ open surgery as reference were included for the review. QUADAS-2 with QUADAS-C was utilized for assessment of the risk of bias. Data analysis was done and presented graphically in the form of Forest Plot and Summary Receiver Operating Characteristic curve (SROC).

Results: 9 studies including 938 patients published between January 2013 and December 2022 were included for the systemic review. On analysis of pooled result, US showed a sensitivity of 0.901 and specificity of 0.69, whereas MRI had sensitivity of 0.912 and specificity of 0.689.

Conclusion: The overall diagnostic accuracy and reliability of shoulder US is comparable to MRI in detection of RCTs. This study supports the utilization of US as first-line diagnostic modality in patients with suspicion of RCTs in facilities where experienced operator is available along with high resolution equipment.

Key Words: Shoulder ultrasound, Rotator cuff tears, MRI.

Introduction

Rotator Cuff (RC) muscles function as the primary dynamic stabilizer of humeral head to the glenoid cavity preventing humeral head from migrating upwards [1,2].

RC pathologies are the most common causes of the shoulder pain particularly in the middle and old age [3]. They include a broad spectrum of diseases ranging from impingement syndrome, tendinopathy, calcific tendinopathy to partial and full thickness tendon tears [4]. Chronic neglected RCTs may lead to wear of glenohumeral joint and superior migration of humeral head which is also known as rotator cuff arthropathy [5].

In patients with RCTs, early detection and repair of tears significantly shortens the repair time and reduces the occurrence of post operative retears leading to better outcome [6]. Comprehensive anatomical details of the RCTs are a prerequisite in preoperative surgical planning, counselling patients and prognosticating patients regarding likely outcome of the surgery [7,8].

Three radiological tests: magnetic resonance imaging (MRI), magnetic resonance arthrography (MRA) and ultrasound (US) have been utilized to detect RCTs and for preoperative planning [1]. None of these currently available radiological investigations are 100% accurate in assessment of RCTs [1].

Although MRA is slightly better, it is not preferred by both clinicians as well as patients as it requires a dye to be injected inside the shoulder. MRI has been widely utilized for diagnosing the etiology of the shoulder pain [9]. MRI has the advantages of being non-invasive, non- radioactive, with excellent tissue contrast and anatomical repeatability. It can perform scanning in multiple angles, in multiple planes and can provide images of different tissue structures at the same time [3]. It can provide detailed information about RC tears such as its size, location, muscle retraction and atrophy and fatty degeneration. It also provides information about adjacent intraarticular structures such as labrum and gleno-humeral ligaments [10].

These characteristics have made MRI the preferred imaging modality in diagnosing RCTs. However, it is inconvenient to patients as it is costly, time consuming and may be associated with claustrophobia. The machine itself is costly and requires lot of space, hence it is available in limited centers. There is long waiting list for these scans. It also lacks capability of dynamic and functional assessment of the joint [11,12].

MRI also have some absolute contraindications as it cannot be used in the presence of cardiac pacemakers, automatic defibrillators,  cochlear implants and metallic orbital foreign bodies [9].

In view of these limitations of MRI, there is requirement of an alternative cheaper, readily available and more convenient imaging modality for diagnosing RCTs [13]. US was introduced for diagnosing RCTs by Seltzer in 1979. Not only it is cheaper, but it is also portable and readily available [5]. Unlike MRI, it has no absolute contraindications. It can dynamically examine the patient in multiple scanning planes as compared to static MRI [2, 13]. Overall, it is better tolerated by the patient. As the operator can interact with the patient, diagnostic yield may be increased by asking patient to point at the symptomatic area. It can be performed by orthopedic surgeon at the first point of contact [14]. Moreover, it can be used for therapeutic management in procedures such as intraarticular steroid injections, and for barbotage therapy for painful calcific tendinitis [15].

However, its utilization has been limited due to several reasons. It is operator dependent, and interpretation depends on the skill and experience of the operator. Optimal images may be difficult to obtain in the obese or muscular patients [1,16,17]. Long learning curve and high interobserver variability principally in partial thickness tears are other limitations [1]. Its effectiveness when diagnosing RCTs has been questioned due to its limited ability in reporting on characteristics such as tear size, muscle retraction, muscle atrophy and fatty degeneration of the muscle [18].

There has been increase in demand for the imaging modalities for the diagnosis of RCTs [19]. Multiple studies have been published comparing the accuracy and reliability of shoulder US with the MRI [2,20]. The outcomes of these studies have been quite variable. This variability in the results of shoulder US have been attributed to inadequate training and experience of the ultrasonologist, multiple observers, lack of definition of diagnostic criteria and unavailability of high frequency linear transducers. This inconsistency in the accuracy of shoulder US in the published work has discouraged many from recommending it as the first line imaging modality for RCTs [19].

Over the last two decades these limitations have been addressed to large extent by introduction of high frequency linear transducers and introduction of proper training guidelines. With background of improvement in the operator training and availability of high frequency transducers, this systematic literature review has been done to assess whether the accuracy and reliability of the US in detecting RCTs has become comparable to MRI over the past ten years. The aim of the study is to improve the quality of care of the patient by providing them with a more convenient and readily available yet accurate alternative. As the definitive diagnosis is made earlier, the shoulder surgeons may propose the required surgery at an early stage, especially in full thickness tears, thereby improving the outcomes of the surgery [21]. However, if this turns out to be a negative trial, then further improvement in the US technique, instruments and operator training may be recommended.

Methods

This systematic review was conducted as per research proposed earlier submitted in December 2022 in the University of Essex (Online).

This systematic review was conducted based on the checklists of the  PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). The 27 item PRISMA checklist was used as it provides guidance for transparent reporting of systematic reviews thereby assisting in enhancing the validity, applicability and replicability of the review making it more useful [22].

Ethical approval was taken from the University of Essex. Patients’ consents were not necessary as the analysis was performed based on data available in published articles [23].

The PubMed and Cochrane Library databases were systematically searched for full-text journal articles evaluating the performance of MRI and US in the detection of RCTs published between January 1, 2013, and December 31, 2022. Only studies published in the last ten years were included to analyze whether US has become comparable to MRI with introduction of improved instruments, and training.

Published articles were retrieved utilizing 2 methods [12]

1. A search was performed using the following keywords: ((Ultrasonograph) OR (Ultrasound) OR (Sonography) OR (Sonograph) OR (US)) AND ((MR) OR (Magnetic Resonance Imaging)) AND ((sensitivity) OR (specificity) OR (diagnostic accuracy) OR (accuracy) OR (reliability)) AND ((shoulder) OR (biceps tendon) OR (rotator cuff) OR (supraspinatus) OR (infraspinatus) OR (Subscapularis) OR (teres minor) OR (tendinopathy) OR (shoulder pain) OR (bursitis) OR (bursitis))) (Farooqi et al, 2021). This search strategy was adapted to both the database – PubMed and Cochrane Library. After excluding duplicated articles, initial screening was done by reading title and abstract of the retrieved articles.
2. Subsequently, manual screening of the reference lists of relevant articles including previous systematic reviews was done to retrieve additional studies [24].

Selection Criteria

The inclusion criteria were studies that [23]

1. Involved human patients;
2. Compare the sensitivity, specificity and diagnostic accuracy of US and MRI in detection of RCTs;
3. Included arthroscopy or open surgery as reference standard to prove/disprove the imaging findings.
4. Studies in English language.

Only studies where diagnostic outcome data of all three: US, MRI and arthroscopy were available were included in the review to avoid bias and to avoid discrepancy in the results.

The exclusion criteria were [23,24,25]:

1. Review articles, meta-analyses, systematic reviews, commentaries, letters, case reports, or congress proceedings.
2. Experimental studies involving animal or cadavers.
3. Studies in language other than English.
4. Studies lacking sufficient diagnostic outcome data.
5. Studies done on other specified shoulder disorders such as osteoarthritis or rheumatoid arthritis as this review was specific for RCTs.
6. Studies with more than a year gap between US/ MRI and arthroscopy/ open surgery as a longer gap may cause further progression of the tear.

Quality assessment

Shortcomings in the design, conduct, and analysis may introduce bias and may compromise the results of systematic reviews studying the accuracy of diagnostic tests. QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2) is one of the recommended tool to assess the risk of bias and concerns regarding applicability of the single index test diagnostic test accuracy (DTA) studies [26].

The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies– Comparative) tool is an extension of QUADAS-2 tool and is designed specifically to identify and assess the risk of bias in comparative DTA studies. In addition to 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing), QUADAS-C has additional questions to each QUADAS-2 domain (Yang et al, 2021). This systematic review utilized QUADAS-C together with QUADAS-2 for assessing separately the risk of bias judgments for single test accuracy estimates and for comparative accuracy estimates for each included study [26]. As per recommendation of Yang et al. (2021), each part was evaluated for the risk of bias and assigned a score of low, high or unclear. The results of all the included studies were tabulated.

Data Extraction

Following data was retrieved and tabulated from each study [23,27]:

1. General information: the first author’s surname, year of publication, period of study, primary study design (i.e. prospective versus retrospective).
2. Sample size and participant characteristics: total number, number of patients with US or MRI as index test, average age and number of male vs female patients.
3. Instruments used for Index tests: details of instruments used for US and MRI.
4. Sensitivity, specificity and accuracy of each of the index test with arthroscopy or open surgery as reference followed by number of true positives (TP), true negatives (TN), false positives (FP) and false negatives (FN) was tabulated separately for partial thickness tears and full thickness tears (whenever available) [24].

Data analysis

Data Analysis was done utilizing MetaDTA a free interactive online application which allows to conduct meta-analysis of diagnostic test accuracy, with or without gold standard [28]. The data analysis was graphically presented in two forms:

1. Forest plots were constructed to display the pooled sensitivity and specificity results of all included studies for US and MRI separately and for visual examination of variation in test accuracy [1,29].
2. For both US as well as MRI, the Summary Receiver Operating Characteristic Curve (SROC) were constructed with sensitivity and False Positive Rate (1-specificity) on the y- and x- axes respectively to provide a global summary of the test performance [28].

Results

The detailed article search strategy identified 1667 articles. First 323 studies were excluded as these were duplicated. Further 1162 were excluded at initial screening of titles and abstracts as they were either not relevant or did not meet inclusion criteria.  Remaining 179 full text articles of potentially eligible studies were assessed. 3 articles could not be assessed as they were not available with the library. Further hand search of the references identified 4 articles.  173 studies did not meet our inclusion criteria and were excluded. 1 study was excluded as it did not have sufficient diagnostic outcome data. Finally, 9 studies which compared the diagnostic accuracy of US with MRI with arthroscopy/ open surgery as reference for diagnosing RCT published between January 2013 and December 2022 were identified and were included for the systemic review (Figure 1).

Figure 1: Selection flow chart for study screening process used to determine study eligibility and inclusion into the systematic review

Risk of Bias

All the studies were reviewed for risk of bias utilizing QUADAS-2 tool in combination of QUADAS-C tool (Table 1). None of the studies had “high risk” of bias in any of the domain. However, none of the included studies had “low risk” of bias in all the domains. In almost all the studies, arthroscopy (reference test) was conducted with prior knowledge of results of the index test. Hence “unclear risk” of bias was marked in reference standard in both QUADAS-2 and QUADAS-C tool.

? indicates low risk; ? indicates high risk; ? indicates unclear risk.

Table 1: Methodological quality summary: review author’s judgements about quality and bias assessment of each included diagnostic accuracy study using QUADAS–2 and QUADAS- C tool.

In none of the studies, index test had “high” or “unclear” risk of bias on QUADAS-2 assessment as both US as well as MRI were conducted in all the studies preoperatively prior to arthroscopic examination. Hence, the findings of the arthroscopic examination (reference standard) could not have been known at time of US and MRI. Index tests had “unclear” risk of bias in two studies on QUADAS-C assessment where information if either US or MRI were conducted without knowing the result of other was unavailable. Five out of nine studies did not clearly mention the interval between the US/ MRI and the arthroscopic examination. Hence, they were “unclear” risk of bias in five studies in QUADAS-2 and QUADAS-C in the flow and timing category.

The systemic review included 9 studies comparing diagnostic accuracy and reliability of US in detecting RCT as compared to MRI with arthroscopy as reference in 938 patients. The review included 5 prospective studies and 4 retrospective analyses with sample sizes ranging from 19 to 255 patients. Basic demographic data with basic characteristics of all the included studies was tabulated and presented in Table 2.

Table 2: Basic demographic data of the included studies

In four out of 9 studies, the results of diagnostic accuracy and reliability of US and MRI in partial thickness tears and full thickness tears were not presented separately. Rather sensitivity, specificity, accuracy, positive predictive value (PPV) and negative predictive value (NPV) of US and MRI in all type of tears whether partial or full thickness tears were presented together. In retrospective analysis by Ward et al, 2018 [30], the results of diagnostic accuracy of US and MRI of two different rotator cuff muscles: subscapularis and supraspinatus were presented separately (Table 3).

Table 3: Diagnostic performance of included studies where all type of tears assessed together.

In rest of the 5 studies, outcome of comparative analysis of MRI and US were tabulated separately for partial-thickness and full thickness tears (Table 4 & 5).

Table 5: Diagnostic performance data (PPV and NPV) of studies where partial thickness tears and full thickness tears assessed separately

For data analysis, the True positive (TP), False positive (FP), False Negative (FN) and True negative (TN) of all types of tears were combined and tabulated for US (Table 6) and MRI (Table 7) separately.

FN=false negative, FP=false positive, TN=true negative, TP=true positive.

Table 6: Diagnostic accuracy data of US for all studies combined

FN=false negative, FP=false positive, TN=true negative, TP=true positive.

Table 7: Diagnostic accuracy data of MRI for all studies combined

The results of each study were then pooled and weighted by the number of shoulders evaluated to determine overall effect. Results assessing the diagnostic performance of US for detecting RCTs in patients as generated from the 9 studies, involving 517 patients showed a sensitivity of 0.901 (95% CI 0.701 -0.973) and specificity of 0.690 (95% CI 0.482- 0.842). The diagnostic performance of MRI for detecting RCTs in patients as generated from the 9 studies, involving 540 patients showed a sensitivity of 0.912 (95% CI 0.798- 0.965) and specificity of 0.689 (95% CI 0.585 – 0.778).

Forest plots were constructed to display the pooled sensitivity and specificity results of all included studies for US and MRI separately utilizing Meta DTA application (Figure 2, 3, 4 & 5). Similary, Summary Receiver Operating Characteristic Curve (SROC) were constructed of all included studies with summary estimates, 95% confidence region and 95% prediction region of shoulder US (Figure 6) and shoulder MRI (Figure 7). On visual examination, both sensitivity as well as specificity of US seems to be comparable for detecting RCTs.

Figure 2: Forest Plot for pooled sensitivity of US for detecting RCTs.

Figure 3: Forest Plot for pooled sensitivity of MRI for detecting RCTs.

Figure 4: Forest Plot for pooled specificity of US in detecting RCTs

Figure 5: Forest Plot for pooled specificity of MRI for detecting RCTs.

Figure 6: Summary Receiver Operating Characteristic Curve (SROC) of shoulder US of all included studies.

Figure 7: Summary Receiver Operating Characteristic curve (SROC) of shoulder MRI of all included studies

Discussion

The demand for the non-invasive imaging modalities in the evaluation of RCTs is increasing [38]. Whether to proceed with conservative or operative management is largely dependent on both clinical findings as well as imaging results [39]. The reliability and accuracy of the imaging modality used whether MRI or US is a major concern. Therefore, it is essential to compare the diagnostic capabilities of US and MRI for diagnosing rotator cuff pathologies [23].

Summary of the main results

9 studies including 938 patients published between January 2013 and December 2022 were identified and were included for the systemic review. There was significant heterogenicity in the study designs of the included studies. On assessment on QUADAS-2 and QUADAS-C tool for risk of bias, none of the included studies had neither “high risk” of bias in any of the domain, nor had “low risk” of bias in all the domains. 

This systemic review involving 9 studies has shown that the overall diagnostic accuracy and reliability of shoulder US (sensitivity 90%, specificity 69%) is comparable to shoulder MRI (sensitivity 91%, specificity 69%). Even on visual examination on Forest Plot, both sensitivity as well as specificity of US seemed to be comparable for detecting RCTs.

However, considerable heterogeneity was observed in the results of US and MRI in diagnosing RCTs. This may be due to the variation in the criteria of defining a diagnostic test as positive or negative, different instruments, different techniques and variations in the study groups. All the three included tests: US, MRI and arthroscopy are known to be operator and reader dependent. Variations in operator or reader experience may be another reason for the observed variations between the studies, especially for studies of US which are especially operator dependent [1].

Strength of the study

The results of this systematic review were based on usage of wide search term with comprehensive search aimed to identify all comparative studies published during past ten years.

The systematic review was conducted based on guidelines and checklist of PRISMA to improve the transparency, reliability and robustness of the systematic review.

As theis systematic review utilized QUADAS-C together with QUADAS-2 for assessing the risk of bias and quality assessment, separate judgments for single test accuracy estimates for US and MRI as well as comparative accuracy estimates were established. None of the included study had “high risk” of bias.

The data analysis was done utilizing software MetaDTA which specifically allows for conducting meta-analysis of the diagnostic tests.

This systemic review shows that with improvement in US technology and operator training, US has become comparable to MRI in diagnosing RCTs over the past ten years. Considering cost, availability and patients’ convenience and comfort, US is a better option for diagnosis of RCTs in most settings compared to MRI [40].

Both prospective and retrospective studies have their own limitations. The strength of evidence obtainable from retrospective studies is weakened by a dependence on reported information. Moreover, there is no possibility of applying controls against bias and confounding factors. Although prospective blinded studies provide more robust evidence, still they have been criticized for lacking clinical relevance because information is not normally withheld in clinical decision-making. In our review, most prospective studies have shown better results of US for diagnosing RCTs than retrospective analysis. This may be due to the reason that more emphasis may be given on the procedures by the operator if he is aware that the result of the procedure is going to be included in the study [18].  By including both prospective as well as retrospective studies, the limitations of either were countered.

Limitations of study

Although the inclusion of studies was done using a rigorous process, still there are many deficiencies [18]. Since arthroscopy or open surgery was taken as the reference test, the included studies had only those patients who had high clinical suspicion of RCTs. Various subgroup analysis was avoided due to lack of sufficient data. There was significant heterogenicity in the designs of the included studies. While some papers included in our study individually analyzed partial and full thickness tears, others combined the two for analysis. The frequency of probes used was different in each study. In few studies, for preoperative analysis, the patients had either undergone US or MRI only [16]. Even the experience level of the operating physician was different in each study. Few of the included studies had poor reporting of participant characteristics and study design. On quality assessment, none of the studies had “low risk” score in all the characteristics. These factors are likely to introduce bias and represent limitations to this study [18].

Comparison with similar systemic reviews

The findings of this systematic review are comparable to those of systematic review done by de Jesus et al, 2009 [41]. In their review, they found US and MRI to be comparable in sensitivity and specificity in the diagnosis of both full and partial thickness tears. They suggested US to be used as investigation of choice for screening for RCTs where adequately trained examiner was available.

On qualitative assessment of included studies, this review has similar opinion as Lenza et al., 2014 [1]. In their review, the strength of evidence for included studies was limited as they were heterogenous and methodologically flawed. As per them, both US and MRI may have poor sensitivity in detecting partial thickness tears. Hence Lenza et al., 2014 [1] recommended the need for well designed studies that directly compare MRI, MRA and US for detection of RCTs.

In a similar meta-analysis involving 23 studies with 2054 shoulders, Farooqi et al., 2021[42], found no statistically significant difference in diagnostic sensitivity, specificity or accuracy of US or MRI for diagnosing any type of supraspinatus tear.

Implications for Practice

To decide which imaging modality should be utilized, various considerations should be considered. In presence of claustrophobia, metallic implants and stimulators and in scenario of limited availability of MRI, US should obviously be preferred first. In clinical suspicion of RCTs and in presence of high frequency linear probe and skilled operator, US should again be preferred. Otherwise, MRI should be preferred as not only it is not dependent on the skills of the operator, but also it is more versatile with ability to diagnose wider range of shoulder pathologies. US may be recommended after MRI wherever there is clinical concern for subscapularis tear missed on MRI scans [43]. An optimum application of the technique and knowledge of imaging pitfalls is necessary to increase diagnostic accuracy [44]. 

Finally, we agree with Ward et al., 2018 [34] where he has recommended institutions to organize regular combined radiology/ shoulder surgery multidisciplinary team meetings for feedback from intraoperative findings and for discussion of complex cases [45]. Audits should be done by all institutions to validate the accuracy of their pre-arthroscopy US scans [46].

Implications for research

Further studies with larger cohort of patients should be performed with blinded design where the duration of time between the index tests and reference test is limited. Separate studies designed to assess the reliability and accuracy US and MRI specifically for the assessment of the partial thickness tears and subscapularis tendons are required [44].

Conclusion

Shoulder ultrasound, in the hands of an experienced radiologist with modern high-resolution equipment can produced comparable results with MRI. The present study supports the utilization of US as first-line diagnostic modality in patients with suspicion of RCTs in facilities where experienced operator is available along with high resolution equipment [37].

Finally, since none of the available modalities are 100% accurate, incorporation of newer technologies such as 3D US, 3D MRI, 3T MRI and SE should be considered whenever clinically indicated as per their availability.

Please click here to view  complete article with figures and tables