Skip to main content

Quantitative ultrasound imaging of Achilles tendon integrity in symptomatic and asymptomatic individuals: reliability and minimal detectable change



Quantifying the integrity of the Achilles tendon (AT) is a rehabilitation challenge. Adopting quantitative ultrasound measurements (QUS measurements) of the AT could fill this gap by 1) evaluating the test-retest reliability and accuracy of QUS measurements of the AT; 2) determining the best protocol for collecting QUS measurements in clinical practice.


A total of 23 ATs with symptoms of Achilles tendinopathy and 63 asymptomatic ATs were evaluated. Eight images were recorded for each AT (2 visits × 2 evaluators × 2 images). Multiple sets of QUS measurements were taken: geometric (thickness, width, area), first-order statistics (computed from a grayscale histogram distribution: echogenicity, variance, skewness, kurtosis, entropy) and texture features (computed from co-occurrence matrices: contrast, energy, homogeneity). A generalizability study quantified the reliability and standard error of measurement (accuracy) of each QUS measurement, and a decision study identified the best measurement taking protocols.


Geometric QUS measurements demonstrated excellent accuracy and reliability. QUS measurements computed from the grayscale histogram distribution revealed poor accuracy and reliability. QUS measurements derived from co-occurrence matrices showed variable accuracy and moderate to excellent reliability. In clinical practice, using an average of the results of three images collected by a single evaluator during a single visit is recommended.


The use of geometric QUS measurements enables quantification of AT integrity in clinical practice and research settings. More studies on QUS measurements derived from co-occurrence matrices are warranted.

Peer Review reports


The Achilles tendon (AT) is the largest and strongest tendon in the human body. The great tensile loads, occurring predominantly during its elongation or contraction, make it vulnerable to overuse injuries. Although the prevalence and incidence of midsubstance Achilles tendinopathy (i.e., in the middle third of the tendon) are high in athletes, cases are also frequently reported in sedentary individuals [16]. The aetiology and pathogenesis of AT tendinopathy have been the subject of much research, but with inconsistent findings [2, 3, 7]. Hence, treating people suffering from this pathology remains challenging for rehabilitation professionals and the success rate of conservative treatments is variable [810].

Ultrasound imaging allows in vivo visualization of the biological integrity of the tendon. It is a safe, rapid, non-invasive, relatively inexpensive and popular method used in the assessment of AT tendinopathy [11, 12]. When looking at ultrasound images (UIs) of healthy ATs, well-organized and parallel alignment of the collagen fibres (i.e., fibrillary striation) are highlighted by alternating parallel bright bands (hyperechoic) of collagen and dark bands (hypoechoic) of extracellular matrix [12, 13]. The paratenon of a healthy AT appears as an uninterrupted, well-defined bright line surrounding the tendon [12, 13] (Fig. 1). Conversely, in people with midsubstance AT tendinopathy, the fibrillar striation pattern is often altered as a result of a disorganization of the collagen fibres and a thickened and hypoechoic portion of the AT reflects an increase in the quantity of extracellular matrix and tenocytes [8, 14, 15]. This will typically translate to focal thickening along the AT, presence of dark (hypoechoic) intratendinous regions and sometimes irregular contours of the tendon on UIs [13] (Fig. 1).

Fig. 1
figure 1

a ROI of a healthy AT in longitudinal view; b ROI of a healthy AT in transverse view; c grayscale histogram derived from the ROI of image (b); d Pathologic AT in longitudinal view; e Pathologic AT in transverse view with arrows indicating the AT’s thickness at different locations in the sagittal plane; f grayscale histogram derived from the ROI of image (e)

Interpretation of an UI of the AT is generally semi-objective. The general appearance of the image is annotated based on the different contrasts observed (e.g., heterogeneous, homogenous, focal or diffuse abnormalities) and the maximum thickness of the AT is often measured using a two-point digital caliper function on the US machine. This interpretation is largely influenced by the evaluator’s experience with the recording technique and ability to interpret an UI [16, 17]. Recent technological advances have helped to promote the development of new quantitative ultrasound (QUS) outcome measures extracted from an UI, specifically from a particular region of interest (ROI). Digital UIs can now be broken down into a multitude of micro pixels, and numerical values (e.g., average thickness, tendon width and area) can be measured. The echogenicity of a ROI within an image can also be quantified by allocating a numerical grayscale value to each of those micro pixels [18, 19].

The usefulness of new UI analysis techniques has been demonstrated in various studies on animals and humans [20]. For example, these techniques have helped to quantify changes in the composition of an exercised muscle compared to an unexercised muscle in an elderly population [2123]. These techniques have also revealed differences in the histological composition of the supraspinatus muscle and the quadriceps muscle in adults [24] and have been successfully used to detect structural changes in four key muscles in youths with neuromuscular disorders [25]. Moreover, new UI analysis techniques have enabled the differentiation of persons with Achilles tendinopathy from healthy individuals [26, 27] and have been effective in detecting focal and diffuse abnormalities in the AT [28].

Very few studies have been conducted to evaluate the reliability of QUS measurements of the AT. This is worrisome considering that the reliability of the QUS measurement of AT thickness, a key diagnostic criterion for Achilles tendinopathy, is rarely reported. To our knowledge, studies that have investigated test-retest reliability of QUS measurements of the AT have shown a moderate to good level of reliability [2933]. In addition, it was shown that ultrasound image recording is greatly influenced by the evaluator, even among highly experienced ultrasonographers (weak inter-evaluator reliability [34]). Various factors such as the pressure applied on the probe and its alignment can influence recorded image properties and thus alter the quantitative values extracted [35, 36]. Information about the reliability and minimal detectable change is essential in order to develop evidence-based measurement taking protocols, empowering clinicians and researchers to quantify the tendinous changes observed in Achilles tendinopathy and incorporate these findings into clinical practice.

The primary objective of this study was to evaluate the reliability and minimal detectable change (MDC) of AT QUS measurements in people with symptoms consistent with midsubstance Achilles tendinopathy affecting at least one lower limb, as well as in completely asymptomatic individuals. The secondary objective was to recommend the best QUS measurement collection protocol possible, which could be subsequently used to characterize AT integrity in clinical practice or in research projects. It is anticipated that all QUS measurements, when collected by the same evaluator, will be reliable (Φ ≥0.75) and accurate (MDCNORMALIZED ≤ 15 %) and that a QUS measurement taking protocol in which a single evaluator averages the results of at least three images obtained during a single visit will be recommended in clinical practice.



A group of 20 individuals with clinical signs or symptoms of unilateral or bilateral midsubstance Achilles tendinopathy and a group of 23 asymptomatic individuals agreed to take part in this study. Individuals with symptoms consistent with Achilles tendinopathy had to have experienced pain over four weeks, evoked pain on palpation in the middle third of the AT and a VISA-A score below 100. The VISA-A questionnaire [37], completed by all the participants, is a reliable and validated measurement tool with an interest in AT pain, ability to function in daily life and during athletic activities. Eight questions are summed to produce an overall score, which is used as an indicator of the pathology’s severity. Scores range from 0 to 100, with a low score indicating greater severity. Asymptomatic participants were to have no pain or previous history of pain in the AT, no observable sign of Achilles tendinopathy or pain in the ankle, and a VISA-A score equal to 100 [38]. The criteria for the inclusion and exclusion of all participants, as well as each group’s specific characteristics, are summarized in Fig. 2. Finally, before conducting any formal testing, ultrasound visualization of the two ATs was performed for each participant to verify its integrity (i.e., normal tendon structure) at and around its insertion and also to rule out complete rupture of the AT. This experiment was approved by the Centre for Interdisciplinary Research in Rehabilitation of Greater Montreal (CRIR) Research Ethics Committee (Certificate: CRIR-557–1110). Participants were fully informed of the nature of the study and asked to sign a consent form before participating.

Fig. 2
figure 2

General criteria for inclusion and exclusion of participants and participants’ characteristics

Clinical examination

Initially, all the participants underwent a clinical examination conducted by an experienced physiotherapist specialized in musculoskeletal disorders with over 10 years of experience. This examination aimed at detecting signs and symptoms typically present with midsubstance Achilles tendinopathy. Special attention was given to the visualization of the AT (without ultrasonography), with emphasis on finding the characteristic thickening sometimes present in its middle third, as well as evoked pain on palpation of the AT’s middle third. A series of manoeuvres was carried out to apply passive and active tensions to selected structures with the intention of reproducing the participant’s symptoms at the AT: manually resisted contraction of the sural triceps, passive stretch of the sural triceps muscle, repeated unilateral heel rise test and repeated unilateral jump.

Ultrasound image recording

Device and settings

All of the ultrasound examinations were conducted using a Philips HD11 1.0.6 ultrasound machine with a 5–12 MHz 50 mm linear array transducer (Philips Medical Systems, Bothell, WA). Image field depth was set to 2 cm, gain was set to 85 dB, probe frequency to 12 MHz and a unique focal zone (set at a depth of 0,5 cm) was positioned at the level of the AT. These main machine settings, as well as all the other options (e.g., compress, map, smooth, X-resolution) were maintained across all examinations performed for all participants in order to standardize the recorded images across all participants.


A physiotherapist (M-J Nadeau) and a resident in physiatry (A. Desrochers) conducted all ultrasound examinations and recorded all of the AT images using a precise protocol (see next section). Both had previously received 10 h of practical training in AT ultrasound examination from an experienced physiatrist recognized by his peers in musculoskeletal ultrasonography (M. Lamontagne).

Image recording protocol

A summary of the image recording protocol is illustrated in Fig. 3. Initial visit (test): Each participant was placed in a prone position, with both feet dangling over the end of the table, and ankles positioned at about 5° of plantar flexion using a splint to immobilize the foot. Once placed in this position, the AT’s enthesis on the calcaneus was located by ultrasonography, and the skin marked at this location. The enthesis was defined as the most distal point of the insertion of the AT on the calcaneus. Another mark, made at a distance of 6 cm proximal to the enthesis, served as a standardized location for the center of the probe when performing the recording of all the ultrasound images. The images were recorded in this precise location since studies indicate that the incidence of Achilles tendinopathy is higher at this level (i.e., middle third) [2, 7]. The first evaluator recorded two images of the AT in the longitudinal view, as well as two additional images in the transverse view. During the recording of each of these images, the probe was removed and then repositioned on the skin with the center of the probe continually aligned with the mark on the skin. Once these four images were recorded, the first evaluator erased all the marks drawn on the skin before the second evaluator repeated the same image collection protocol. Particular attention was paid to the probe’s positioning on the tendon, taking care to apply minimal pressure on the probe and to align the transducer according to fiber orientation, with respect to the local referential (i.e., x, y, and z axis) defined by the tendon itself. Hence, the transducer was not necessarily perfectly aligned with the traditional anatomical planes and may have deviated slightly from them (i.e., yaw, roll and pitch movements of the transducer).

Fig. 3
figure 3

Summary of image recording protocol

Second visit (retest)

After a minimal 10-min rest period, the evaluators repeated the image collection sequence described above. No significant change in QUS measurements was anticipated as each participant had to remain at rest between the two sessions.

Image analysis

To calculate the different QUS measurements and facilitate characterization of the integrity of the AT, ultrasound images initially recorded in DICOM format were converted to JPEG format. An interactive 2D viewing and image analysis software, developed by the research team using MATLAB Image Processing Toolbox (The Mathworks, Natick, MA), was used to extract the QUS measurements. The development of this program was inspired by work previously realized by a research team based at the University of Pittsburgh that used QUS measurements to characterize shoulder tendons (i.e., supraspinatus, biceps) [3941]. Each image selected for analysis appeared on screen and the evaluators (i.e., the physiotherapist and a trained research associate) traced a standardized ROI directly on the image, using markers. For blinding purposes, all images recorded during visits 1 and 2 by a unique evaluator were uploaded as a block of images prior to starting the image analysis. Thereafter, each image was presented in a random order to the evaluator to conduct the image analysis. While conducting the image analysis, only the image appeared on the computer screen and all other information was blinded with a black frame generated by the program. As described in detail below, the anatomical landmarks defining the ROIs in the longitudinal and transverse images of the AT differed between both views (Fig. 1a and b).

In the longitudinal view images, the ROI included a 1-cm length area, centered in the middle of the image and captured 6 cm proximal to the tendon enthesis (Fig. 1a). For transverse view images, the ROI was defined by the tendon’s contour (Fig. 1b). The ROI outline in both images was established to include the AT’s fibres and exclude the paratenon.

These two ROIs were used to extract the following QUS measurements selected for this study: thickness, width (only for transverse images), area (only for transverse images), echogenicity, variance, skewness, kurtosis, entropy, contrast, energy and homogeneity.


The average thickness of the tendon’s ROI is calculated in the longitudinal view. One hundred equidistant points are plotted respectively on the upper and lower edges of the AT and the distance between each pair of points is calculated. The 100 distance measurements are then averaged and represent the thickness. In the transverse view, the thickness of the AT is determined by encompassing the tendon with a rectangle (Fig. 1b). The height of the rectangle reflects the tendon’s maximum thickness (Fig. 1e).


The width is determined from the rectangle that encompasses the tendon, as defined above. The rectangle’s width reflects the tendon’s width (Fig. 1b).


The tendon’s area corresponds to the area of the region delimited by the tendon’s outline.

To calculate the other QUS measurements, the ROI is fragmented into multiple micro pixels (micro pixel = 0,0057 mm2) by the software and a numerical grayscale colour value is allocated to each micro pixel. The grayscale is a scale of colors used in imagery that ranges from 0 = black to white = 255 for a total of 256 possible shades.

The micro pixels’ grayscale values included in the ROI are initially represented by a grey level frequency distribution curve found in the ROI (grayscale histogram) (Fig. 1c and f). The following first-order statistics can be calculated from this distribution curve: echogenicity, variance, skewness, kurtosis and entropy. Additional information on QUS measurements are provided in Table 1.

Table 1 Definitions of QUS measurements and their mathematical formulas

Next, a co-occurrence matrix is calculated. Texture analysis using a co-occurrence matrix is based on the repeated occurrence of a typical pixel configuration in the image’s ROI. It considers how many pairs of pixels with specific grayscale values and a specific predefined spatial relationship (distance and relative orientation angle) are present in an ROI. In this study, pairs of pixels were calculated in four directions (angles = 0°, 45°, 90° 135°) and a distance of 10 pixels was determined. The following texture indicator measurements are derived from this matrix: contrast, energy and homogeneity (Table 1).

All of the QUS measurements can be classified into three categories: geometric measurements (thickness, width, area), measurements computed from a grayscale histogram (echogenicity, variance, skewness, kurtosis, entropy) and measurements computed from a co-occurrence matrix (contrast, energy, homogeneity).

It is expected that a healthy tendon would have a more heterogeneous appearance because of the alternation of its black and white stripes, with a larger range of values on the grayscale. A pathological area in a tendon would, in contrast, have a darker, more homogeneous appearance, with grayscale values closer to zero (black). Hence, the following QUS measurement values are expected to be found in a pathological tendon: increased thickness, width, area, skewness, kurtosis, homogeneity and energy, as well as reduced echogenicity, variance, entropy and contrast [40].

Statistical analysis

Outcome measures

The overall averages, standard deviations and confidence intervals of the results of the 8 images obtained for each QUS measurement in longitudinal and transverse views were calculated for all of the images for all tendons (n = 86) and separately for all of the images of symptomatic tendons (n = 23) and for all of the images of asymptomatic tendons (n = 63). The percentage difference between the averaged results of these two sub-groups was also calculated.


The generalizability theory was used to determine the reliability of the different QUS measurements taken for the symptomatic and asymptomatic tendons. On the basis of the analysis of variance, this theory is mainly divided by 2 studies: the generalizability study (G-Study) and the dependability study (D-Study) [42]. Unlike the traditional theory of reliability that provides a unique random error term, the G-Study divides the error into different facets (sources of variance) relevant to our study and allows for the magnitude of the variance attributed to each facet to be determined. Therefore, in this study, the G-study determined the magnitude of the variance attributed to the subject (S), evaluator (E), visit (V), image (I), and random errors resulting from the interactions between these different sources of variance (SE, SV, SI, EV, EI, VI, SEV, SEI, SVI, EVI), thus leaving much less unexplained variance. In the G-Study, the variance component assigned to the subject (S) represents the difference between the subjects. This proportion of variance is error-free. The unexplained residual error is solely from the interaction between all sources of error and corresponds to the combination of the variances of subjects, evaluators, visits, and images (SEVI). Unlike the traditional theory of reliability that assumes that reliability exists independently of the measurement protocol design, the D-study relies upon information generated from the G-study to determine the reliability of specific simulated protocol designs and provides information to optimize reliability according to, for example, the context in which the measurement is being used (e.g., clinical practice versus research). In this study, the impact of different experimental protocols on the reliability coefficients (Φ), standard error of measurement (SEM) and normalized minimal detectable change (MDCNORM) for each QUS measurement was determined. Since it is documented in studies that inter-rater reliability of QUS measurements is clearly inferior to intra-rater reliability, a single evaluator was used in the D-study. Improvements which may be obtained by averaging 1–3 images during a single visit or by averaging the images obtained during two visits by a single evaluator were compared. The G- and D-studies allow for the calculation of dependability coefficients (Φ), ranging from 0 (no reliability) to 1 (perfect reliability). In general, the dependability coefficient (Φ) can be interpreted as follows: poor reliability (Φ < 0.50), moderate (0.50 ≤ Φ < 0.75), good (0.75 ≤ Φ < 0.90) and excellent (Φ ≥ 0.90). However, there is some consensus that reliability indicators must exceed 0.90 for clinical measurements on an individual basis in order to minimize error and ensure reasonable validity [43]. More liberal reliability scores are allowed on a group basis, particularly for research purposes. The generalizability analysis was conducted using PC GENOVA statistical software, Version 2.2.

Standard error of measurement

Because the dependability coefficient (Φ) can be high despite substantial variability in the measurements, the standard error of measurement (SEM) has also been reported. The absolute SEM is estimated using the same units as the primary outcome measure. The SEM, which is the square root of the error variance, reflects the accuracy of a measurement.

Minimal detectable change

The absolute minimal detectable change (MDCABS) was calculated to determine the extent of the absolute change required to detect a difference that could be interpreted as a real difference exceeding the measurement error. For a 90 % confidence level (z = 1.65), which is considered sufficient for clinical decision-making, the MDCABS was calculated using the following equation:

$$ \mathrm{M}\mathrm{D}{\mathrm{C}}_{\mathrm{ABS}} = 1.65\kern0.5em \times \kern0.5em \surd 2\kern0.5em \times \kern0.5em \mathrm{S}\mathrm{E}\mathrm{M} $$

In order for the MDC to be independent from the unit of measurement and to facilitate its interpretation, the MDCABS has been subsequently normalized relative to the average obtained (MDCNORM) and calculated using the following equation:

$$ \mathrm{M}\mathrm{D}{\mathrm{C}}_{\mathrm{NORM}} = \left(\mathrm{M}\mathrm{D}{\mathrm{C}}_{\mathrm{ABS}}\ /\ \mathrm{overall}\ \mathrm{group}\ \mathrm{average}\right) \times 100. $$

MDCNORM ≤15 % reflects excellent measurement accuracy.


The overall averages, standard deviations and confidence intervals of the results of the 8 images obtained for each QUS measurement in longitudinal and transverse views for the complete set of tendons (n = 86) as well as for the symptomatic tendons (n = 23) and asymptomatic tendons (n = 63) are summarized in Table 2.

Table 2 Averages, standard deviations and confidence intervals of the different QUS measurements

Sources of variance

The magnitude of each variance component (source of error), expressed as a percentage of the total variance for each QUS measurement for symptomatic and asymptomatic tendons, is presented in Tables 3 and 4 for images recorded in longitudinal and transverse views, respectively. Aside from the main source of variance associated with the subject (S) in most cases, the evaluator (E) is the systematic error with the highest variance percentage, up to 13.7 % of the total variance. The other systematic errors (visit and image) are negligible and vary from 0 to 1.9 % of the total variance. A significant proportion of random error is attributable to sources of variance that involve an interaction between the subject and the evaluator (SE, SEV, SEI) with proportions of up to 32.9 %, 40.0 %, and 16.3 % of the total variance, respectively. The contribution of the other errors (SV, SI, EV, EI VI, SVI, EVI) is lower, with percentages ranging from 0 to 10.8 %, where 10.8 % represents SVI interaction. The unexplained residual error (SEVI) is variable (1.5 to 27.3 %) for all of the measurements, with the exception of kurtosis (21.1 to 43.9 %) and skewness (22.7 to 39.2 %), which remains slightly higher.

Table 3 Magnitude of variance components for QUS measurements computed for the images recorded in longitudinal view
Table 4 Magnitude of variance components for QUS measurements computed for the images recorded in transverse view

Reliability and minimal detectable change

The reliability and MDC of different hypothetical QUS measurement acquisition protocol designs are described in Tables 5 and 6 for the images in longitudinal and transverse views, respectively. Different trends in reliability and MDC of the QUS measurements are observed for the three main measurement categories.

Table 5 Impact of different hypothetical protocols (D-Study) for QUS measurements recorded in longitudinal view
Table 6 Impact of different hypothetical protocols (D-Study) for QUS measurements recorded in transverse view

The reliability and MDC of the results for the protocol design in which the QUS measurement results of three images taken by a single evaluator in a single visit (E = 1, V = 1, I = 3) are averaged were compared for the three main measurement categories. This measurement scenario is compatible with clinical practice.

Geometric measurements

In general, these QUS measurements have shown good to excellent reliability, with dependability coefficients ranging from 0.88 to 0.98 and good accuracy with a MDC 90%NORM <15 % obtained in most cases. Only the thickness of symptomatic ATs in longitudinal view had a MDC 90%NORM value greater than 15 % (MDC 90 % NORM = 23.66 %) which still remains acceptable.

Measurements computed from a grayscale histogram

Echogenicity stands out in this category by its excellent results with dependability coefficients ranging from 0.88 to 0.92 and a MDC90%NORM ranging from 8.56 to 15.51 %. The entropy also seems promising with a tendency for slightly better reliability than other QUS measurements in this category (all Φ =0.77 except for one Φ = 0.34), and an excellent MDC 90%NORM ranging from 2.18 to 4.95 %. However, the other QUS measurements in this category have shown only weak to moderate reliability, with most dependability coefficients below the threshold of 0.75 (Φ range = 0.49–0.79). These measurements also showed a large MDC90%NORM (with the exception of entropy) ranging from 26.09 to 76.02 %.

Measurements computed from a co-occurrence matrix

In general, these QUS measurements have shown moderate to excellent reliability with Φ ranging from Φ = 0.69 to Φ = 0.92 and a variable MDC90%NORM ranging from 2.36 to 33.67 %. Of these measurements, homogeneity stands out with good to excellent reliability (Φ = 0.75 to Φ = 0.92) and excellent MDC 90%NORM ranging from 2.36 to 3.52 %.

Impact of averaging results from multiple images or visits

D-study reliability and error measurement estimates were computed for 6 experimental designs (Tables 5 and 6). Improved reliability and decreased MDC were obtained in all cases by increasing the number of recorded images. Slightly larger improvements in reliability and MDC were observed by recording images from two visits, in all cases.


Evaluator as a significant source of variability

The evaluator represents a significant source of variability in the recording of UIs. The high level of technical skills and manual dexterity during UI recording requires extensive clinical experience and may contribute to variability associated with the evaluator [44]. In the present study, the limited experience of both ultrasonographers (i.e., evaluators) might have increased the variability of the evaluator (E) facet. However, Gellhorn et al. revealed excellent inter-rater reliability for the cross sectional area (CSA) measurement of the patellar tendon (comparable to the AT in terms of shape, content and superficial location) between a novice and an experienced sonographer [45]. The important interplay between the evaluator and the participants during an UI recording is highlighted by the high proportion of variance in all QUS measurements attributed to sources of error involving interaction between the subject and the evaluator (SE, SEV, SEI). Every AT is different and those anatomical and physiological differences (oblique orientation of the tendon, the subject’s weak natural echogenicity, blurred outline of the tendon, etc.) are expressed by in the subject facet (S). These dissimilarities make capturing an image of certain tendons more challenging than others and may explain why an evaluator may have more difficulty in assessing some subjects than others (SE). Consequently, it is recommended that a single evaluator record US images and extract QUS measurements, particularly when the goal is to monitor treatment effects over time.

Superiority of geometric measurements and echogenicity

The excellent results obtained in this study, in terms of reliability and accuracy, of geometric QUS measurements of area, thickness and width (Φ obtained mostly at the top of the clinical acceptable threshold of 0.90 and a MDC90%NORM <15 %) are similar or better than those obtained in comparable studies targeting the AT [3133, 46]. Although echogenicity is a measurement computed from a grayscale histogram, it behaves as a geometric measurement and has also shown excellent reliability and accuracy (mostly all Φ > 0.90 and MDCNORM < 15 %). Echogenicity has been previously studied, mainly on muscles, and has shown good reliability for repeated measurements (variation coefficients ranging from 5 to 11 %) [4749]. Strict compliance of a standardized measurement taking protocol and the use of software to extract the geometric QUS measurements may explain, among other reasons, the favourable results obtained in this study. Continued use of the geometric QUS measurements, as well as echogenicity, is therefore encouraged in quantifying AT integrity.

The poor results, in terms of reliability and MDC, of the QUS measurements computed from a grayscale histogram (variance, skewness, kurtosis) obtained in this study confirm the need for refinement and further study before advocating the use of these QUS measurements in the assessment of AT integrity. For a hypothetical protocol in which the evaluator averages the results of three images recorded during a single visit (E = 1, V = 1, I = 3), the dependability coefficients obtained for variance, skewness and kurtosis range from 0.49 to 0.79, with the majority of measurements falling under the threshold established to ensure good reliability of 0.75. The accuracy results are also disappointing (MDC90%NORM ranging from 26.09 to 76.02 %). Only entropy stands out for its general good reliability (all Φ =0.77 except for one Φ = 0.34) and high accuracy (MDC 90%NORM ranging from 2.18 to 4.95 %) in this study when using the above-described protocol.

The reliability of QUS measurements computed from a grayscale histogram was also studied previously by two research teams that obtained similar results to those of the present study. Collinger et al. assessed the reliability and accuracy of various QUS measurements extracted from longitudinal images of the long head of the biceps and the supraspinatus tendons based on the generalizability theory [39]. The reliability and accuracy scores, determined with a D-study for an E = 1, V = 1, I = 2 protocol, are similar to the present study. Good reliability and accuracy of thickness (Φ ranging from 0.92 to 0.94; MDC90%NORM ranging from 9.42 to 14.49 %) and echogenicity (Φ ranging from 0.79 to 0.85; MDC90%NORM ranging from 16.03 to 18.72 %), as well as low reliability and weak accuracy of the variance, skewness and kurtosis have been found (Φ ranging from 0.57 to 0.69; MDC90% greater than 15 % and up to 297.35 %). Entropy demonstrated moderate reliability and good accuracy (Φ ranging from 0.64 to 0.68; MDC90% ranging from 5.56 to 5.58 %). Slightly better results in terms of reliability of the QUS measurements were obtained in the present study compared to the study by Collinger. The superior reliability of the AT QUS measurements is possibly explained by the fact that this tendon is easier to assess than shoulder tendons due to its superficial position, alignment and surrounding structures. Nielsen et al. assessed the test-retest reliability for repeated measurements of the QUS measurements of variance, skewness and kurtosis of the supraspinatus muscle. A single evaluator repeated 30 images during a single visit (E = 1, V = 1, I = 30) [49]. This team also obtained results consistent with those in this study, that is, low reliability in these QUS measurements, with variation coefficients (analogous to SEM expressed as a percentage of the grand mean) ranging from 14 to 35 %.

Knowledge of the theoretical foundations and the underlying calculation of the different QUS measurements computed from a grayscale histogram is essential in understanding the disappointing results, in terms of reliability and accuracy, of these measurements. The skewness and kurtosis QUS measurements are calculated directly from the shape of the grey level frequency distribution curve (grayscale histogram), while the shape of this curve also influences variance and entropy. For its part, the QUS measurement of echogenicity is an average of the grey scale values for all of the pixels in the ROI and does not take into account the shape of the distribution curve. The appearance of an anatomical structure on an ultrasound image can vary significantly depending upon the angle and the pressure applied to the tissues with the probe, both in terms of shape and echogenicity [50]. The edges of the ROI are sensitive to these variations in echogenicity, which can change the shape of the grey level frequency distribution curve without having a significant influence on the echogenicity’s average value for the ROI.

Differences between transverse and longitudinal AT images

In this study, reliability and MDC of QUS measurements are generally similar between transverse and longitudinal images. The QUS measurement of thickness in a longitudinal view of symptomatic ATs is the exception to the rule as it tends to be less reliable (−8.58 %) and less accurate (−48.74 %) than in a transverse view. This difference might be explained by the fact that when the probe is positioned longitudinally to the AT fibres, it can be repositioned at different locations or angles on the tendon’s sagittal plane for each image (Fig. 1e). In a longitudinal view, the thickness measurement is captured only for a slice located directly under the probe and it is difficult to ensure that it is located on the thickest portion of the tendon. Therefore, it appears preferable that AT QUS thickness measurements are also taken in the transverse view, at a location considered relevant and determined following a full excursion of the transducer along the AT in both planes. Other thickness measures previously reported in the literature (e.g., true thickness measure) may also deserve to be explored in the future, especially with regard to the thickness measured in the transverse view [32]. In addition, since our QUS thickness measurement in the longitudinal view reflects the average thickness of a targeted area, it is likely that its value is reduced in comparison to the maximum thickness found in this region captured in the transverse view of the AT.

Co-occurrence matrix shows promise in quantitative ultrasound imaging

The co-occurrence matrix is an image analysis method that considers the spatial organization of the pixels, as opposed to the grayscale histogram that only considers the grey scale values of the pixels, without taking into account their position on the image or their interaction with the surrounding pixels [18, 19, 51]. In our study, better reliability was achieved for QUS measurements drawn from a co-occurrence matrix in comparison to the reliability of the QUS measurements computed from a grayscale histogram. Collinger et al. obtained similar results [39]. Superior reliability may be explained by the fact that the co-occurrence matrix studies pairs of pixels, and not the isolated value of each pixel’s grey level.

Proposing a measurement collection protocol for clinical practice

In clinical practice, it is difficult to consider having more than one assessment visit in which additional images would be recorded (V = 2). Even though one or more additional visits positively influences the reliability and accuracy of QUS measurements, productivity constraints should be considered. A protocol in which the averaged result obtained from three images collected by a unique evaluator during a single visit seems to represent a good compromise. The clinical applicability of AT QUS measurements becomes highly relevant since the time required for recording them is, in the latter protocol, at most 10 min, which can help in the clinical decision making process and in practice.

Study limitations

This study has several limitations. Within the context of this reliability study, all measurements were taken in an identical location across all participants according to a standardized protocol. The location selected was set at 6 cm proximal to the enthesis of the AT considering that AT tendinopathy typically occurs between 2–6 cm proximal to its enthesis on the calcaneus [52, 53]. Hence, measurements of the symptomatic tendons were not necessarily done exactly at the pathology’s precise location for all participants which, in turn, may have minimized the variance between the pathological tendons and underestimated the reliability and accuracy of the measurements. Two separate tasks, which are both dependent upon the evaluator, must be performed when obtaining QUS measurements. The first task involves obtaining the ultrasound image (image acquiring) and the second consists of processing the acquired image in order to extract the desired quantitative values (image analyzing). Both “image acquiring” and “image analyzing” can affect reliability separately. For example, when the evaluator plots the delineation of the ROI while analyzing the UIs, the sometimes-blurred outline of the AT increases this task’s difficulty. Syha et al. found that reliability of thickness measurements of the AT was more reliable when the ROI was traced automatically compared to manual tracing [54, 55]. In the present study, it is impossible to differentiate the error related to the recording of the image from that of the analysis of the image. Further studies are required to isolate these potential sources of variability that are currently encompassed within the evaluator facet. Another source of variability and potential limitation of the study is that we are quantifying the integrity of a three dimensional tendon using two dimensional UIs.


This study focused on the reliability of three types of QUS measurements: geometric QUS measurements, QUS measurements computed from a grayscale histogram, and QUS measurements computed from a co-occurrence matrix. Even though additional validity and responsiveness studies are necessary, the favourable results of geometric QUS measurements and of the echogenicity further support their use in clinical practice and research protocols. These measurements could be used in longitudinal follow-up to capture the progress of the severity of AT tendinopathy and the impact of follow-up treatment in clinical practice or in rehabilitation research protocols. These measurements may also be useful in a transversal context in order to compare individuals between themselves or against standards established in clinical practice or in future studies. Furthermore, it is imperative that particular emphasis be given to adhering to a rigorous, standardized measurement-taking protocol when acquiring UIs to reach an acceptable level of reliability and accuracy. With respect to QUS measurements computed from grayscale histograms, in light of the results obtained in this study, the use of these QUS measurements in evaluating AT integrity should be reconsidered. Lastly, QUS measurements computed from a co-occurrence matrix are promising and additional studies on this emerging method of image analysis are necessary.





Achilles tendon


Centre for Interdisciplinary Research in Rehabilitation of Greater Montreal


Cross sectional area






Minimal detectable change


Absolute minimal detectable change


Normalized minimal detectable change


Quantitative ultrasound


Region of interest




Standard error of measurement




Ultrasound image




Victorian Institute of Sport Assessment- Achilles


  1. De Jonge S, Van den Berg C, De Vos R, Van der Heide H, Weir A, Verhaar J, Bierma-Zeinstra S, Tol J. Incidence of midportion Achilles tendinopathy in the general population. Br J Sports Med. 2011;45(13):1026–8.

    Article  PubMed  Google Scholar 

  2. Alfredson H, Lorentzon R. Chronic achilles tendinosis. Sports Med. 2000;29(2):135–46.

    Article  CAS  PubMed  Google Scholar 

  3. Rolf C, Movin T. Etiology, histopathology, and outcome of surgery in achillodynia. Foot Ankle Int. 1997;18(9):565–9.

    Article  CAS  PubMed  Google Scholar 

  4. Kvist M. Achilles tendon injuries in athletes. Sports Med. 1994;18(3):173–201.

    Article  CAS  PubMed  Google Scholar 

  5. Krolo I, Višković K, Ikić Đ, Klarić-Ćustović R, Marotti M, Cicvara T. The risk of sports activities–the injuries of the Achilles tendon in sportsmen. Coll Antropol. 2007;31(1):275–8.

    PubMed  Google Scholar 

  6. Lopes AD, Junior MLCH, Yeung SS, Costa LOP. What are the Main Running-Related Musculoskeletal Injuries? Sports Med. 2012;42(10):891–905.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Maffulli N, Sharma P, Luscombe KL. Achilles tendinopathy: aetiology and management. J R Soc Med. 2004;97(10):472–6.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Alfredson H, Cook J. A treatment algorithm for managing Achilles tendinopathy: new treatment options. Br J Sports Med. 2007;41(4):211–6.

    Article  PubMed  PubMed Central  Google Scholar 

  9. McShane JM, Ostick B, McCabe F. Noninsertional Achilles tendinopathy: pathology and management. Curr Sports Med Rep. 2007;6(5):288–92.

    PubMed  Google Scholar 

  10. Magnussen RA, Dunn WR, Thomson AB. Nonoperative treatment of midportion Achilles tendinopathy: a systematic review. Clin J Sport Med. 2009;19(1):54–64.

    Article  PubMed  Google Scholar 

  11. Brasseur J, Zeitoun-Eiss D, Renoux J, Bach G. Apport de l’échographie dans l’exploration du pied et de la cheville. EM-Consulte Podologie. 2010. doi:10.1016/S0292-062X(10)53770-X.

    Google Scholar 

  12. Bianchi S, Martinoli C. Ultrasound of the Musculoskeletal System. 1st ed. Berlin Heidelberg: Springer; 2007.

    Google Scholar 

  13. Jacobson JA. Fundamentals of musculoskeletal ultrasound. 1st ed. Philadelphia: Elsevier Health Sciences; 2007.

  14. Sharma P, Maffulli N. Biology of tendon injury: healing, modeling and remodeling. J Musculoskelet Neuronal Interact. 2006;6(2):181.

    CAS  PubMed  Google Scholar 

  15. Khan KM, Cook JL, Bonar F, Harcourt P, Åstrom M. Histopathology of common tendinopathies. Sports Med. 1999;27(6):393–408.

    Article  CAS  PubMed  Google Scholar 

  16. Hertzberg BS, Kliewer MA, Bowie JD, Carroll BA, DeLong DH, Gray L, Nelson RC. Physician Training Requirements in Sonography: How Many Cases Are Needed for Competence? AJR Am J Roentgenol. 2000;174(5):1221–7.

    Article  CAS  PubMed  Google Scholar 

  17. Krupinski EA. Computer-aided Detection in Clinical Environment: Benefits and Challenges for Radiologists1. Radiology. 2004;231(1):7–9.

    Article  PubMed  Google Scholar 

  18. Castellano G, Bonilha L, Li L, Cendes F. Texture analysis of medical images. Clin Radiol. 2004;59(12):1061–9.

    Article  CAS  PubMed  Google Scholar 

  19. Bharati MH, Liu JJ, MacGregor JF. Image texture analysis: methods and comparisons. Chemometr Intell Lab Syst. 2004;72(1):57–71.

    Article  CAS  Google Scholar 

  20. Schie HT, Bakker EM, Jonker AM, Weeren P. Ultrasonographic tissue characterization of equine superficial digital flexor tendons by means of gray level statistics. Am J Vet Res. 2000;61(2):210–9.

    Article  PubMed  Google Scholar 

  21. Sipila S, Suominen H. Ultrasound imaging of the quadriceps muscle in elderly athletes and untrained men. Muscle Nerve. 1991;14(6):527–33.

    Article  CAS  PubMed  Google Scholar 

  22. Sipilä S, Suominen H. Muscle ultrasonography and computed tomography in elderly trained and untrained women. Muscle Nerve. 1993;16(3):294–300.

    Article  PubMed  Google Scholar 

  23. Sipilä S, Suominen H. Quantitative ultrasonography of muscle: detection of adaptations to training in elderly women. Arch Phys Med Rehabil. 1996;77(11):1173–8.

    Article  PubMed  Google Scholar 

  24. Nielsen PK, Jensen BR, Darvann T, Jørgensen K, Bakke M. Quantitative ultrasound tissue characterization in shoulder and thigh muscles–a new approach. BMC Musculoskelet Disord. 2006;7(1):2.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Pillen S, Verrips A, Van Alfen N, Arts I, Sie L, Zwarts M. Quantitative skeletal muscle ultrasound: diagnostic value in childhood neuromuscular disease. Neuromuscul Disord. 2007;17(7):509–16.

    Article  CAS  PubMed  Google Scholar 

  26. Bashford GR, Tomsen N, Arya S, Burnfield JM, Kulig K. Tendinopathy discrimination by use of spatial frequency parameters in ultrasound B-mode images. IEEE Trans Med Imaging. 2008;27(5):608–15.

    Article  CAS  PubMed  Google Scholar 

  27. van Schie H, De Vos R, De Jonge S, Bakker E, Heijboer M, Verhaar J, Tol J, Weinans H. Ultrasonographic tissue characterisation of human Achilles tendons: quantification of tendon structure through a novel non-invasive approach. Br J Sports Med. 2010;44(16):1153–9.

    Article  PubMed  Google Scholar 

  28. Tuthill TA, Rubin JM, Fowlkes JB, Jamadar DA, Bude RO. Frequency analysis of echo texture in tendon. Ultrasound Med Biol. 1999;25(6):959–68.

    Article  CAS  PubMed  Google Scholar 

  29. O’Connor PJ, Grainger AJ, Morgan S, Smith K, Waterton J, Nash A. Ultrasound assessment of tendons in asymptomatic volunteers: a study of reproducibility. Eur Radiol. 2004;14(11):1968–73.

    Article  PubMed  Google Scholar 

  30. Wearing SC, Grigg NL, Hooper SL, Smeathers JE. Conditioning of the Achilles tendon via ankle exercise improves correlations between sonographic measures of tendon thickness and body anthropometry. J Appl Physiol. 2011;110(5):1384–9.

    Article  PubMed  Google Scholar 

  31. Schmidt W, Schmidt H, Schicke B, Gromnica-Ihle E. Standard reference values for musculoskeletal ultrasonography. Ann Rheum Dis. 2004;63(8):988–94.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Fredberg U, Bolvig L, Andersen NT, Stengaard-Pedersen K. Ultrasonography in evaluation of Achilles and patella tendon thickness. Ultraschall Med. 2008;29(01):60–5.

    Article  CAS  PubMed  Google Scholar 

  33. Brushøj C, Henriksen B, Albrecht-Beste E, Hölmich P, Larsen K, Nielsen MB. Reproducibility of ultrasound and magnetic resonance imaging measurements of tendon size. Acta Radiol. 2006;47(9):954–9.

    Article  PubMed  Google Scholar 

  34. Ying M, Yeung E, Li B, Li W, Lui M, Tsoi C-W. Sonographic evaluation of the size of Achilles tendon: the effect of exercise and dominance of the ankle. Ultrasound Med Biol. 2003;29(5):637–42.

    Article  PubMed  Google Scholar 

  35. Larivière C, Gagnon D, De Oliveira JE, Henry SM, Mecheri H, Dumas J-P. Reliability of Ultrasound Measures of the Transversus Abdominis: Effect of Task and Transducer Position. PM R. 2013;5(2):104–13.

    Article  PubMed  Google Scholar 

  36. Lariviere C, Gagnon D, De Oliveira JE, Henry SM, Mecheri H, Dumas J-P. Ultrasound Measures of the Lumbar Multifidus: Effect of Task and Transducer Position on Reliability. PM R. 2013;5(8):678–87.

    Article  PubMed  Google Scholar 

  37. Robinson J, Cook JL, Purdam C, Visentini P, Ross J, Maffulli N, Taunton J, Khan K. The VISA-A questionnaire: a valid and reliable index of the clinical severity of Achilles tendinopathy. Br J Sports Med. 2001;35(5):335–41.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Vestergrd Iversen J, Bartels EM, Langberg H. The Victorian Institute of Sports Assessment-Achilles Questionnaire (VISA-A)-reliable tool for measuring Achilles tendinopathy. Int J Sports Phys Ther. 2012;7(1):76–84.

    PubMed  Google Scholar 

  39. Collinger JL, Gagnon D, Jacobson J, Impink BG, Boninger ML. Reliability of quantitative ultrasound measures of the biceps and supraspinatus tendons. Acad Radiol. 2009;16(11):1424–32.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Collinger JL, Fullerton B, Impink BG, Koontz AM, Boninger ML. Validation of greyscale-based quantitative ultrasound in manual wheelchair users: relationship to established clinical measures of shoulder pathology. Am J Phys Med Rehabil. 2010;89(5):390.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Collinger JL, Impink BG, Ozawa H, Boninger ML. Effect of an intense wheelchair propulsion task on quantitative ultrasound of shoulder tendons. PM R. 2010;2(10):920–5.

    Article  PubMed  Google Scholar 

  42. Roebroeck ME, Harlaar J, Lankhorst GJ. The application of generalizability theory to reliability assessment: an illustration using isometric force measurements. Phys Ther. 1993;73(6):386–95.

    CAS  PubMed  Google Scholar 

  43. Portney LG, Watkins MP. Foundations of clinical research: Applications to practice. 3rd ed. Prentice Hall; 2007

  44. McCreesh KM, Crotty JM, Lewis JS. Acromiohumeral distance measurement in rotator cuff tendinopathy: is there a reliable, clinically applicable method? A systematic review. Br J Sports Med. 2015;49(5):298–305.

    Article  PubMed  Google Scholar 

  45. Gellhorn AC, Carlson MJ. Inter-rater, intra-rater, and inter-machine reliability of quantitative ultrasound measurements of the patellar tendon. Ultrasound Med Biol. 2013;39(5):791–6.

    Article  PubMed  Google Scholar 

  46. Tweedell AJ, Ryan ED, Scharville MJ, Rosenberg JG, Sobolewski EJ, Kleinberg CR. The influence of ultrasound measurement techniques on the age-related differences in Achilles tendon size. Exp Gerontol. 2016;76:68–71.

    Article  PubMed  Google Scholar 

  47. Scholten R, Pillen S, Verrips A, Zwarts M. Quantitative ultrasonography of skeletal muscles in children: normal values. Muscle Nerve. 2003;27(6):693–8.

    Article  CAS  PubMed  Google Scholar 

  48. Reimers CD, Schlotter B, Eicke BM, Witt TN. Calf enlargement in neuromuscular diseases: a quantitative ultrasound study in 350 patients and review of the literature. J Neurol Sci. 1996;143(1):46–56.

    Article  CAS  PubMed  Google Scholar 

  49. Nielsen PK, Jensen BR, Darvann T, Jørgensen K, Bakke M. Quantitative ultrasound image analysis of the supraspinatus muscle. Clin Biomech (Bristol, Avon). 2000;15:S13–6.

    Article  Google Scholar 

  50. Arend CF. Top ten pitfalls to avoid when performing musculoskeletal sonography: what you should know before entering the examination room. Eur J Radiol. 2013;82(11):1933–9.

    Article  PubMed  Google Scholar 

  51. Haralick RM, Shanmugam K, Dinstein IH. Textural features for image classification. IEEE Trans Syst Man Cybern Syst. 1973;3(6):610–21.

    Article  Google Scholar 

  52. Lagergren C, Lindholm Å. Vascular distribution in the Achilles tendon; an angiographic and microangiographic study. Acta Chir Scand. 1959;116(5–6):491.

    CAS  PubMed  Google Scholar 

  53. Clain MR, Baxter DE. Achilles tendinitis. Foot Ankle Int. 1992;13(8):482–7.

    Article  CAS  Google Scholar 

  54. Syha R, Peters M, Birnesser H, Niess A, Hirschmueller A, Dickhuth H, Sandrock M. Computer-based quantification of the mean Achilles tendon thickness in ultrasound images–effect of tendinosis. Br J Sports Med. 2007;41(12):897–902.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Syha R, Grau S, Nieß AM, Hein T, Werner M, Ketelsen D, Schick F, Best R. Computer-based quantification of the Achilles tendon thickness in sequential B-mode ultrasound images: a study of feasibility and reliability. Arch Orthop Trauma Surg. 2014;134(10):1443–9.

    Article  CAS  PubMed  Google Scholar 

Download references


The authors would like to thank Dominic Véronneau from OrthoAction for providing the modified ankle-foot orthosis used.


Marie-Josée Nadeau received a M.Sc. scholarship of the Ordre professionnel de la physiothérapie du Québec (The Quebec Professional Order of Physiotherapy) (OPPQ). Amélie Desrochers and Martin Lamontagne received clinical research grants from the Association québécoise des médecins du sport (AQMS) and the Canadian Academy of Sport and Exercise Medicine (CASEM). Dany H. Gagnon was a scholarly researcher supported by the Fonds de la recherche du Québec-santé (Quebec Health Research Fund) (FRQS) at the time of the study. Dany H. Gagnon now co-chairs the Initiative for the Development of New Technologies and Practices in Rehabilitation (INSPIRE) affiliated with the Lindsay Rehabilitation Award funded by the LRH Foundation. The equipment and material required for the research completed at the Pathokinesiology Laboratory was financed by the Canada Foundation for Innovation (CFI) and the Occupational Health and Safety Research Institute Robert-Sauvé (IRSST).

Availability of data and material

All data and materials can be made available from the corresponding author upon request.

Authors’ contributions

MJN, AD, ML, CL, DHG participated in the conception and design of the study. MJN and AD undertook data acquisition. MJN and DHG processed data, carried out the statistical analysis and the interpretation of the results, and drafted the initial manuscript. AD, ML and CL assisted in the drafting of the manuscript. All authors were involved in revising the initial manuscript, and read and approved the final version to be published.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

This study was approved by the Research Ethics Committee of the Centre for Interdisciplinary Research in Rehabilitation of Greater Montreal (CRIR) (Certificate: CRIR-557–1110). All participants were fully informed of the nature of the study and signed a consent form before entering the study.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Dany H. Gagnon.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nadeau, MJ., Desrochers, A., Lamontagne, M. et al. Quantitative ultrasound imaging of Achilles tendon integrity in symptomatic and asymptomatic individuals: reliability and minimal detectable change. J Foot Ankle Res 9, 30 (2016).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: