Musculoskeletal ultrasound imaging of the plantar forefoot in patients with rheumatoid arthritis: inter-observer agreement between a podiatrist and a radiologist

As technology has improved, clinical expertise in performing musculoskeletal ultrasound (MSUS) has also advanced dramatically [1, 2]. The advantages of ultrasound imaging over conventional radiography, computed tomography, radioisotope scan and MRI (Magnetic Resonance Imaging) are reported as being that it is: painless, does not use ionising radiation, less expensive, can be performed real time, needs no special environment and is clinically readily accessible [1].

There have been a number of studies that have utilised MSUS to diagnose specific soft tissue foot pain in healthy volunteers [3–5] and soft tissue problems in Rheumatoid Arthritis (RA) [6, 7]. More specifically for the forefoot ultrasonographic findings of metatarsophalangeal joint (MTPJ) synovitis [8, 9], plantar flexor tenosynovitis [10] and plantar bursitis have been described [11].

MSUS is, however, affirmed as being highly operator dependent [12]. The procedure itself has no known specific side effects, although harm that may result from incorrect acquisition and interpretation of images has been highlighted [2]. The reported challenges in training for this skill are primarily due to the quality and interpretation of the ultrasound images that are acknowledged as being greatly dependent on the expertise and experience of the operator [13]. Knowledge about the basic principles relevant to sound waves and a detailed anatomical knowledge of the structures under investigation are therefore mandatory [14].

Models for the use of MSUS by clinicians other than radiologists, have been demonstrated [15, 16] and specific competencies for MSUS performed by non radiologists have been rigorously developed [17–19]. Within the most recent proposed framework for development of competencies in MSUS scanning techniques, due to the training challenges, tailored learning to areas directly relevant to a clinician's discrete field of practice is proposed [19].

This study provided an opportunity to evaluate the inter-observer agreement between a radiologist and podiatrist who had followed recommended training guidelines for MSUS, in the MSUS assessment of the forefoot of patients with RA. This work forms part of an ultrasound based research programme investigating the clinical relevance of soft tissue inflammation within the forefoot of patients with RA.

Overall agreement was 83.3% for presence or absence of bursitis, 81.8% for presence or absence of erosion and 68.3% presence or absence of synovitis. Kappa scores for from the primary data revealed moderate agreement for bursitis (kappa 0.522; p < 0.01) and erosions (kappa 0.636; p < 0.01) and fair agreement for synovitis (kappa 0.216; p < 0.05).

The sensitivity of the podiatrist using MSUS was 82.4% for detection of bursitis, 83.0% for detection of erosion and 84.0% for detection of synovitis. Specificity of the podiatrist using MSUS was 88.9% for detection of bursitis, 80.7% for detection of erosion and 35.9% for detection of synovitis.

Following the consensus meeting and test on a randomised selection of thirty six images from the DIASUS machine, good levels of agreement were achieved between the podiatrist and radiologist for 8/9 bursitis, 4/10 synovitis, 6/9 erosion, 8/8 normal images (kappa 0.702; p < 0.01).

The use of MSUS assessment of the foot and ankle in clinical practice would be beneficial to patients with RA in facilitating more effective timely referral and management of foot problems. This is the first study to investigate tailored learning of MSUS to the discrete field of foot and ankle practice in evaluating the inter-observer agreement in the use of MSUS between an allied health professional (podiatrist) and an expert radiologist on imaging of the foot. We have demonstrated good agreement for bursitis and erosions, but only fair agreement for the presence of synovitis.

Competency assessment in MSUS is an important issue [17]. Usually, reliability in technique is reported by rheumatologists who have trained or are being trained in MSUS that are tested against experienced MSUS sonographers or radiologists [25–28]. The podiatrist in this study had followed recommended BSR training in MSUS techniques followed by further training and mentorship from expert radiologists. Podiatrists are regularly involved in the assessment and management of musculoskeletal foot and ankle pathology. With extended scope practice in the use of MSUS by podiatrists there are the potentially valuable benefits to patients mentioned above, as well as lower costs in service provision.

In our study overall exact agreement between the radiologist and podiatrist was recorded as 83.3% for bursitis, 68.3% for synovitis and 81.8% for erosions. Acceptable sonographic images were obtained by the podiatrist with moderate agreements for bursitis (kappa 0.522, p <0.01) and erosions (kappa 0.636, p <0.01). Low agreements for synovitis (kappa 0.216, p <0.05) were obtained initially, however following further training, levels of agreement for all three variables increased to a good standard (kappa 0.702, p <0.01).

Within the MSUS literature the foot is under-investigated and those who have reported on assessments of the foot joints have observed similar low agreement scores for synovitis [25–28]. Inter-observer reliability among 14 experts in MSUS produced overall good agreements for all examined joints (kappa 0.76) although low agreement for ankle/toe joints (0.28 kappa) was reported [27]. In evaluating scanning technique and diagnostic criteria, a group of 23 experts in MSUS who scanned shoulder, wrist/hand, ankle/foot and knee joints, reported exact overall agreement of 91% for synovitis, 87% for cortical abnormalities and 83.5% for bursitis but only 'fair' agreements for the ankle and foot region (kappa 0.54) [28]. In MSUS examination of synovitis of the metacarpophalangeal joints and MTPJs the learning curve of three rheumatologists in MSUS techniques was investigated [26]. The agreements at the final evaluations were good for two of the fellows, (kappa 0.63 and 0.62), consistent with our findings, however for the third fellow was poor (kappa 0.18). This highlights the variability of learning requirements for the technique [26].

The use of MSUS by the podiatrist within this study did reveal good sensitivity and specificity for detecting forefoot bursitis (82.4% and 88.9%) and erosion (83.0% and 80.7%), although in detection of MTP synovitis sensitivity was good, specificity was low (84.0% and 35.9%) with over-reporting of false positives. As reported by others [27] we did encounter difficulties in using MSUS to detect synovitis in the MTPJs from the plantar aspect of the foot, especially if deformities and/or subluxation of those joints were present. Interestingly others have scanned each joint in the dorsal aspect of the hands and feet, reporting that this technique was preferred to a palmer or plantar scan because of its reliability for detecting synovitis [26] although they did not mention technique where deformity and subluxation existed in the MTPJs. From our experiences within this study we recommend that further work on establishing reliability of protocols for MSUS assessment of the foot and ankle be conducted.

A number of potential limitations within this current study should be acknowledged. The prevalence of synovitis, erosion and bursitis within the forefoot was not validated by any other 'gold standard' imaging technique such as MRI. Other authors have not attempted to address this issue either although results from the OMERACT MSUS special interest group highlighted the need to consider comparison of MSUS data with histology and MRI [29]. MRI however remains more costly and less readily accessible than MSUS which made it less feasible to be included in this study. Limitations relating to the sample should also be highlighted. The sample was relatively small and comprised patients with severe disease therefore generalizibility to the whole population of patients with RA needs to be confirmed. Finally, the images selected for analysis after the consensus meeting were taken from part of the cohort that the podiatrist and radiologist used for the initial reliability analyses. Whilst the consensus meeting did take place once data collection was complete, an element of recall bias may have been introduced and thus should be acknowledged.

This study was the first to attempt to investigate inter-observer agreement in the use of diagnostic ultrasound between an allied health professional (podiatrist) and an expert radiologist. Performance of MSUS in image acquisition and interpretation by the podiatrist was of an acceptable standard during the primary investigation and following further training levels of agreement increased to a good standard. The foot is a complex structure with many small joints and podiatrists are well placed to be involved in its MSUS assessment, however we have highlighted the importance of appropriate training and further mentorship from experts in MSUS imaging techniques. Finally, we have recommended that further work is undertaken in establishing reliable protocols for MSUS assessment of the foot.