- Research
- Open Access
- Open Peer Review
This article has Open Peer Review reports available.
The Queensland high risk foot form (QHRFF) – is it a reliable and valid clinical research tool for foot disease?
https://doi.org/10.1186/1757-1146-7-7
© Lazzarini et al.; licensee BioMed Central Ltd. 2014
Received: 20 May 2013
Accepted: 22 January 2014
Published: 28 January 2014
Abstract
Background
Foot disease complications, such as foot ulcers and infection, contribute to considerable morbidity and mortality. These complications are typically precipitated by “high-risk factors”, such as peripheral neuropathy and peripheral arterial disease. High-risk factors are more prevalent in specific “at risk” populations such as diabetes, kidney disease and cardiovascular disease. To the best of the authors’ knowledge a tool capturing multiple high-risk factors and foot disease complications in multiple at risk populations has yet to be tested. This study aimed to develop and test the validity and reliability of a Queensland High Risk Foot Form (QHRFF) tool.
Methods
The study was conducted in two phases. Phase one developed a QHRFF using an existing diabetes foot disease tool, literature searches, stakeholder groups and expert panel. Phase two tested the QHRFF for validity and reliability. Four clinicians, representing different levels of expertise, were recruited to test validity and reliability. Three cohorts of patients were recruited; one tested criterion measure reliability (n = 32), another tested criterion validity and inter-rater reliability (n = 43), and another tested intra-rater reliability (n = 19). Validity was determined using sensitivity, specificity and positive predictive values (PPV). Reliability was determined using Kappa, weighted Kappa and intra-class correlation (ICC) statistics.
Results
A QHRFF tool containing 46 items across seven domains was developed. Criterion measure reliability of at least moderate categories of agreement (Kappa > 0.4; ICC > 0.75) was seen in 91% (29 of 32) tested items. Criterion validity of at least moderate categories (PPV > 0.7) was seen in 83% (60 of 72) tested items. Inter- and intra-rater reliability of at least moderate categories (Kappa > 0.4; ICC > 0.75) was seen in 88% (84 of 96) and 87% (20 of 23) tested items respectively.
Conclusions
The QHRFF had acceptable validity and reliability across the majority of items; particularly items identifying relevant co-morbidities, high-risk factors and foot disease complications. Recommendations have been made to improve or remove identified weaker items for future QHRFF versions. Overall, the QHRFF possesses suitable practicality, validity and reliability to assess and capture relevant foot disease items across multiple at risk populations.
Keywords
Background
Foot disease contributes to considerable hospitalisation [1–4], amputation [5–8], institutionalisation [9, 10], and death [2, 11]; yet foot disease complications and these outcomes are largely preventable [5, 6, 12, 13]. “High risk factors” (such as peripheral neuropathy, peripheral arterial disease or foot deformity) significantly increase the risk of developing “foot disease complications” (such as foot ulcers, infection or ischaemia) [1, 14–16]. Diabetes populations are frequently acknowledged as the leading “at risk population” for foot disease due to the increased risk of developing high risk factors, and in turn foot disease complications, from diabetes [5–8, 17]. However, other chronic disease populations (such as chronic kidney disease [15, 16], cardiovascular disease [5, 6, 18, 19] and some cancers [5, 6, 8]) have now been shown to cause comparable rates of high risk factors and foot disease complications to diabetes, and thus, are also becoming more readily identified as “at risk populations” for foot disease.
Best practice foot disease management has been shown to significantly reduce hospitalisation, amputation, mortality and overall costs within different at risk populations [5, 12, 13, 20]. These multi-faceted best practice interventions commonly include screening for high-risk factors, multi-disciplinary management of foot disease complications, clinical training, evidence-based clinical pathway utilisation and regular capture and analysis of foot disease clinical data [5, 6, 12, 13, 20]. In consideration of the growing problem of foot disease, and the potential future improvements observed with best practice clinical management and research, it is imperative that any clinical tools to assess, capture, measure or analyse patient outcomes in at risk populations are valid and reliable.
There are a multitude of studies that have investigated a specific high risk foot factor or foot disease complication within multiple at risk populations [3, 5, 6, 21]. Furthermore, numerous studies have investigated multiple high risk factors and foot disease complications in specific at risk populations such as diabetes [2, 15, 17]. However, very few studies have investigated multiple high risk foot factors and foot disease complications in multiple at risk populations. This situation appears to be mirrored by the clinical tools available to measure and report on foot disease. For example many validated single-item tools are available which measure specific high-risk factors or foot disease complications within multiple at risk populations [22–24]; such as the ankle brachial index to capture and measure peripheral arterial disease within various at risk populations [22]. Furthermore, many validated multi-item tools measuring multiple high-risk factors and foot disease complications in specific at risk populations have been developed [15, 25–28]; such as the University of Texas Diabetic Foot Classification System to capture and measure multiple high risk factors and foot disease complications in people with diabetes only [25]. However, to the best of the authors’ knowledge a multi-item tool designed to measure multiple high-risk factors and foot disease complications in multiple at risk populations is yet to be developed and tested for validity and reliability.
Recently a Diabetic Foot Form (DFF) tool was developed to measure multiple high-risk factors and foot disease complications in people with diabetes only in diverse Queensland (Australia) settings [29]. The DFF was a multi-item tool developed from a number of existing clinical tools recommended in the literature to reliably capture various high-risk factors and foot disease complications in the diabetes population [29]. The implementation of the DFF in diverse Queensland clinical sites, in conjunction with other multi-faceted strategies, resulted in improved capture, measurement and management of high risk factors and foot disease complications and a corresponding reduction in outcomes such as hospitalisation and amputation [29, 30]. However, the tool was not tested for validity or reliability and was designed only to measure multiple high-risk factors and foot disease complications in the specific at risk population of people with diabetes.
In this study, we sought to modify the DFF tool to enable the measurement of multiple high risk factors and foot disease complications in multiple at risk populations. It was intended that the new multi-item tool would align with best practice principles for clinical tools including being easily interpreted, practical to use and possessing high face, content and criterion validity, and inter- and intra-rater reliability [31–33]. Thus, the aims of this study were firstly to develop a multi-item Queensland High Risk Foot Form (QHRFF) tool to capture multiple high risk factors and foot disease complications in multiple applicable at risk populations, and secondly, to investigate the validity and reliability of the tool’s individual items when used by clinicians representing different levels of foot disease expertise.
Methods
The study was conducted in two phases. Phase one involved development of the Queensland High Risk Foot Form (QHRFF) tool and phase two tested the validity and reliability of the QHRFF tool. Approval was granted from Institutional Ethics Committees and informed consent was obtained from all individual participants (patients and clinicians) for this study.
Phase one – development of the tool
Phase one primarily aimed to select items for the development of a QHRFF tool that were practical to collect in an Australian clinical setting, applicable to multiple 'at risk’ populations, and provided high face and content validity. The overarching procedures used for item selection included using the original DFF [29] as the starting tool to build the new QHRFF, searching the electronic literature for other recommended foot disease tools, establishing an expert panel to guide development and using several rounds of stakeholder consultation to refine the tool.
The original DFF contained 64 items pertaining to the construct of foot disease in diabetes populations [29] and was used as the starting tool to modify into the QHRFF. The DFF was originally developed using similar procedures to those used in this study and appeared to possess high practicality, face and content validity [29]. Furthermore, the DFF had been routinely used in over 25 High Risk Foot Service sites throughout Queensland for three years to collect standard clinical data on diabetes-related foot disease [30].
An electronic literature search was undertaken of relevant electronic databases including MEDLINE (all years to June 2011), CINAHL (all years to June 2011) and relevant professional websites to identify existing foot disease-related tools. The basic terms searched included tools to identify peripheral neuropathy, peripheral arterial disease, foot deformity, foot ulceration, foot infection and amputation or synonyms.
An expert panel was established from members of the Queensland Statewide Diabetes Foot Working Group; a working group of the Queensland Statewide Diabetes Clinical Network. The panel comprised 14 expert clinicians, with between five and 25 years of experience in the area of foot disease management, from the fields of endocrinology, vascular surgery, podiatry, nursing, public health, quality improvement and research. The panel had the responsibility to decide upon the final items for the QHRFF tool after each round of consultation.
Refinements of the tool were achieved through numerous rounds of consultation and trialling of draft QHRFF versions with external stakeholders in relation to practicality, face and content validity. Stakeholders included up to 200 multidisciplinary professionals from sites registered to use the existing DFF, members of the Queensland Health Statewide Diabetes Clinical Network, Statewide Renal Clinical Network and Statewide Podiatry Network.
At the conclusion of phase one the expert panel and aforementioned network’s management committees determined, via consensus endorsement, that each item of the QHRFF possessed high practicality, face and content validity, and was applicable across multiple 'at risk’ populations (particularly diabetes, cardiovascular disease and chronic kidney disease) in Australian clinical settings. See Phase One Results for further details regarding the final QHRFF.
Phase two – validity and reliability
The final endorsed QHRFF tool developed in phase one was then tested for validity and reliability. The general procedure for testing involved using podiatrists with different levels of foot disease expertise to assess patients with different levels and severity of high risk factors and foot disease. The authors decided to test only QHRFF items that directly related to the foot disease construct, thus, excluding items such as patients’ name, and facility. To test the validity of each item a general criterion measure (the agreement between two 'experts’) was initially assessed for reliability. Each item was then tested, using the podiatrists with different experience levels, against the criterion measure to evaluate magnitudes of concurrent criterion validity. The inter- and intra-rater reliability was also tested for the level of agreement on each item by podiatrists with different levels of experience.
Setting and participants
The testing was conducted within the High Risk Foot Services (HRFS), Metro North Hospital and Health Service, Brisbane, Australia. Eligible clinician participants were recruited from podiatrists practicing a minimum of one session per week in a HRFS. Four podiatrists were chosen as representative of the majority of clinicians managing foot disease within Queensland Health, and thus, potentially using the QHRFF in future. For the purposes of the study, levels of expertise were categorised using the Queensland Health 'Health Practitioner award’ [34]. Thus, expert clinicians were defined as either a 'consultant clinician’ (level 6) or 'specialist clinician’ (level 5) [34] and working in an acute hospital setting, plus, a member of the expert panel to ensure they understood the original intended construct for the QHRFF tool. General clinicians were defined as a 'senior clinician’ (level 4) or 'clinician’ (level 3) [34], working in a community setting, and thought to be representative of the general podiatry clinical workforce managing foot disease in Queensland. One of each level was recruited; one level 6, one level 5, one level 4 and one level 3 (however, the level 3 had been acting in a level 4 position at the time of the study). Written informed consent was obtained from all participants (patients and clinicians) prior to commencement of their study participation.
Eligible patient participants were consecutively recruited from patients already attending Community HRFS clinics for the care of high risk factors or foot disease complications; defined as a previous or current foot ulcer. Exclusion criteria included patients with a cognitive deficit, signs or symptoms of a systemic infection, younger than 18 years of age, or unwilling to provide written consent to participate. The authors considered that patients with previous or current foot ulcers would ensure that the majority of high risk factors and foot disease items had the realistic possibility of being present or absent, and thus, could be suitably tested on each participant. Furthermore, it was thought this population should possess the moderate prevalence rates, yet unpredictable mixes and severities of each item, of different high risk factors and foot disease complications, that are suggested in the literature to improve statistical robustness for validity and reliability studies [31].
Three different patient cohorts were used; one cohort to test the reliability of the criterion measures (agreement between 'experts’) (n = 32), another cohort to simultaneously test the criterion validity (an 'expert’ diagnosis against general clinicians) and inter-rater reliability (n = 43), and the last cohort to test the intra-rater reliability of a general clinician (n = 19). The recruitment of consecutive community patients did mean that patients may have been familiar to the level 3 or level 4 clinicians; however, not to the expert 'gold standard’ clinicians working in the hospital setting. To minimise the risk of patients being familiar to the level 3 or level 4 clinicians’ seven different Community HRFS clinics were used to recruit patient participants. The level 3 and level 4 clinicians had only worked at two of the seven clinics recruiting patients.
Procedures
A designated research assistant coordinated all procedures. Training of each clinician consisted of being provided with a QHRFF manual that gave a definition of each item [35], a 1–2 hour training session on instructions and tips to use the QHRFF tool, and each was encouraged to trial the tool on their existing patients and clarify any queries with the research assistant prior to testing.
The general assessment procedure for each validity or reliability test consisted of patients having their feet examined by at least two different clinicians, blinded from each other’s assessment, within the one clinical visit (validity and inter-rater reliability). However, to ensure all clinicians had an equal opportunity to determine the patient’s debridement and wound management needs, all were permitted to visually inspect the patient’s feet together for up to five minutes prior to any clinician ratings. In this initial inspection period, the clinicians were instructed to only visually inspect the need for debridement and previous wound dressings whilst not conversing with or touching the patient or each other. The order of clinician assessment after this inspection period was then at the discretion of the research assistant based on clinician availability. The first clinician would have the additional task of debriding the wound or callus if they deemed necessary and the last clinician the additional task of completing any clinical management.
The research assistant ensured all clinicians were blinded from each other’s assessments in separate rooms and that all examination records were de-identified. Each clinician conducted the assessment of patient’s feet using the QHRFF to record their assessment and management recommendations. Demographic, medical history and medication information were available from the patients’ medical records or by direct communication with the patients themselves. To minimise assessment bias all historical foot-related records (including progress notes, DFF tools or QHRFFs tools) were removed from the medical record prior to each clinician’s assessment.
Criterion measure
The criterion measure was tested on the first cohort of patients. The criterion measure (i.e. 'gold standard’ diagnosis) for each QHRFF item in this study was defined by the authors as the diagnosis made by an expert clinician. Expert clinicians were pragmatically chosen as a general criterion measure for all items, instead of using multiple resource and time intensive individual gold standard-recognised criterion measures (such as nerve conduction studies for neuropathy), due to the resource and time constraints of the study. Best practice dictates that any criterion measure should be reliable, free from bias and measure the same item as the new tool [31]. Thus, the study’s criterion measure, of an expert clinician, required testing to determine its reliability, independence (free from bias) and applicability, prior to its use as a 'gold standard’ criterion measure in the criterion-related validity tests. The authors tested the criterion measure by testing the reliability of the agreement between two independent (blinded from one another) experts (one level 6 and one level 5) to determine the magnitude of reliability of agreement on their 'gold standard’ diagnoses for each item on the same patient cohort. The “general assessment procedure” as described above was utilised for each patient. Once the criterion measure for each item was determined to be reliable, the criterion validity of each QHRFF item was tested.
Criterion validity
The criterion validity and inter-rated reliability was then tested simultaneously on a second cohort of patients. Criterion validity was tested using the concurrent criterion validity method by comparing the reliable criterion measure (one of the expert clinicians) against each of the two representative 'general’ clinicians (one level 3 and one level 4). Thus, two separate criterion assessments were carried out for each item; one using the level 3 clinician against the criterion measure, and another the level 4 clinician against the criterion measure. Again the “general assessment procedure” as described above was utilised for each patient when testing for criterion validity.
Inter-rater reliability
Inter-rater reliability tests were performed simultaneously with the criterion validity tests on the same second cohort of patients and were tested using the expert clinician (level 5), senior clinician (level 4) and clinician (level 3). Thus, three inter-rater reliability measures of agreements were carried out for each item; one testing the agreement between the level 5 and level 4 clinician, a second between the level 5 and level 3 clinician, and a third between the level 4 and level 3 clinician. “General assessment procedures” as described above were again utilised for each patient when testing for inter-rater reliability.
Intra-rater reliability
Intra-rater reliability was tested on a third cohort of patients. The clinician with the least expertise (level 3) was used to test intra-rater reliability as it was hypothesised that the clinician with the least expertise would have the most variability of the tested clinicians. The two different time points to test intra-rater reliability were between one and four weeks apart. This time period was chosen as it was considered that a minimum of one week (of full clinical load) would be necessary to adequately reduce recall bias in a clinician, plus, a maximum of four weeks would not be sufficient time for the majority of items to markedly change and thus items would remain stable between ratings. Any items that did not fit this criteria were excluded from intra-rater testing. Furthermore, this time period aligned with any necessary follow up treatment time period for patients, and thus, was convenient to patient participants. To control for any potential changes in foot ulcer characteristics over time (for example ulcer combined surface area and clinical signs of infection) digital images were used. Digital photographic images taken of foot ulcers at the first rating were used at the second rating to standardise the foot ulcer characteristics across two time points [36]. Images were taken perpendicular and 30-60 cm away from ulcers at the first rating, and incorporated two calibrated measures across the length and width of the ulcer in the image. These images were reviewed by the clinician at the second rating to determine the foot ulcer characteristics only.
Statistical analysis
All data was analysed using SPSS 19.0 for Windows (SPSS Inc., Chicago, IL, USA) or GraphPad Software. Descriptive statistics were used to display the single demographic, medical co-morbidity, high-risk factors and foot disease variables for each cohort of patient participants; using means and standard deviations (SD) for continuous variables (which were normally distributed) or proportions for categorical variables. Each above descriptive cohort variable was collected from the clinician with most expertise or on the first rating of the intra-rater reliability cohort. Chi-squared test of independence and ANOVA were used to test for differences between the three cohorts’ characteristics. A significance level of p < 0.05 was used throughout.
All QHRFF items were tested for validity and reliability; except those stated items considered not to impact on the foot disease construct. Criterion validity was tested by calculating sensitivity, specificity and positive predictive values. Ordinal variables were collapsed into dichotomous data to enable calculations of sensitivity, specificity and positive predictive values. All measures of agreements between clinicians were tested using Kappa (K) for dichotomous variables, weighted Kappa (wK) for ordinal variables and intra-class correlation (ICCs) (model 2, 1) for continuous variables [31, 37, 38]. Kappa and wK value (Standard Errors (SE)) strengths were categorised as: no agreement < 0; slight agreement = 0 – 0.20; fair agreement = 0.21 - 0.40; moderate agreement = 0.41 – 0.6; substantial agreement = 0.61 – 0.8; and near perfect agreement = 0.81 – 1.0 [31, 37, 38]. ICC (SD) strengths were categorised as: weak-moderate agreement < 0.75 and strong agreement > 0.75 [31].
Results
Phase one – development of the tool
The literature search identified 174 papers that reported on tools to identify or measure foot disease complications and/or high-risk factors, with the majority of papers specifically relating to diabetes populations (90 (52%)). Tools reported in other populations were in conditions commonly identified in the literature to be associated with lower limb amputation [5–8]. These included cardiovascular disease (peripheral arterial disease), chronic kidney disease, malignancy, infection and other neurological conditions [5–8]. No tool was identified that was specifically designed to identify multiple high-risk factors and foot disease complications in multiple at risk populations.
Fifty-nine individual tools were identified from the literature search; 23 were considered to have adequate practicality, face validity and applicability to an Australian clinical context to be considered for the QHRFF. These tools and the original DFF tool [29] were considered by the expert panel for QHRFF item selection. At this point, the expert panel determined that the tool should be divided into a clinical assessment record section that informed a separate data collection section, and thus, only the data collection section would require testing.
Queensland high risk foot data collection form (QHRFF).
Supporting tools or literature for QHRFF items
QHRFF item | Supporting tool or literature |
|---|---|
General demographics* | [35] |
Indigenous status | [14] |
Health professionals attending* | |
Medical co-morbidity history* | |
Medical (diabetes) history | |
Diabetes year diagnosed (duration) | |
Recent BGLs > 15 mmol/L | |
HbA1c result | |
CKD | |
ESRF | |
High-risk factor history* | |
Clinical diagnoses of high-risk factors | |
Neuropathy | |
PAD | Foot pulses, ankle brachial index and/or toe systolic pressure [1, 14, 21, 22, 39, 46–49] |
Acute Charcot | |
Foot deformity | |
Risk classification | |
Clinical diagnoses of foot disease | |
(Ulcer) type | |
Combined surface area mm2 | |
Combined surface area (change since last visit) | |
Clinical signs of infection | |
UTWCS grade | |
Ulcer depth# | |
Clinical management principles performed* | |
Phase two – validity and reliability
Demographic, co-morbidity and high-risk foot complication data for each patient cohort (n (%) unless otherwise stated)
Criterion measure | Criterion validity* & inter-rater reliability* | Intra-rater reliability | p Value | |
|---|---|---|---|---|
General demographics | ||||
Numbers | 32 | 43 | 19 | |
Male | 25 (78%) | 37 (86%) | 17 (90%) | 0.501 |
Age, mean (SD) (years)# | 69 (13) | 68 (13) | 70 (15) | 0.826 |
Age range (years) | 48 – 90 | 36 – 89 | 42 – 90 | |
Indigenous | 1 (3%) | 0 (0%) | 1 (5%) | NA |
Health professionals attending (in the past week) | ||||
Podiatrist | 32 (100%) | 43 (100%) | 19 (100%) | 1.000 |
GP | 2 (6%) | 10 (23%) | 2 (11%) | 0.103 |
Nurse | 3 (9%) | 12 (28%) | 4 (21%) | 0.141 |
Orthotist | 0 | 0 | 0 | NA |
Physician | 2 (6%) | 1 (2%) | 1 (5%) | NA |
Surgeon | 1 (3%) | 3 (7%) | 0 | NA |
Other | 0 | 0 | 0 | NA |
Medical co-morbidity history | ||||
Diabetes (type 1 or 2) | 28 (88%) | 35 (81%) | 16 (84%) | 0.384 |
Type 2 diabetes | 28 (88%) | 31 (72%) | 14 (74%) | 0.256 |
Diabetes duration (years)# | 12 (7) | 23 (11) | 20 (14) | 0.001** |
Recent BGLs > 15 mmol/L | 5 (16%) | 4 (9%) | 5 (26%) | 0.220 |
HbA1c (SD)# | 9.2 (3.5) | 8.1 (1.6) | 8.4 (2.7) | 0.450 |
Hypertension | 28 (88%) | 30 (70%) | 11 (58%) | 0.053 |
Dyslipidaemia | 23 (72%) | 25 (58%) | 9 (47%) | 0.201 |
Smoker | 5 (16%) | 11 (26%) | 3 (16%) | 0.492 |
CVD | 21 (66%) | 21 (49%) | 9 (47%) | 0.281 |
CKD | 5 (16%) | 13 (30%) | 5 (26%) | 0.339 |
ESRF | 0 | 1 (2%) | 0 | NA |
Other (non-listed) | 29 (91%) | 36 (84%) | 11 (58%) | 0.013** |
High-risk history | ||||
Neuropathy | 31 (97%) | 37 (86%) | 18 (95%) | 0.214 |
PAD | 14 (44%) | 21 (49%) | 8 (42%) | 0.853 |
Previous foot ulcer | 27 (84%) | 37 (86%) | 17 (90%) | 0.878 |
Current foot ulcer | 32 (100%) | 42 (98%) | 19 (100%) | 0.549 |
New foot ulcer | 21 (66%) | 42 (98%) | 19 (100%) | ≤ 0.001** |
Previous amputation | 16 (50%) | 17 (40%) | 7 (37%) | 0.566 |
Clinical diagnoses of high-risk factors | ||||
Neuropathy | 31 (97%) | 38 (88%) | 19 (100%) | 0.146 |
Any PAD | 14 (44%) | 22 (51%) | 8 (42%) | 0.735 |
Acute Charcot | 1 (3%) | 1 (2%) | 0 | NA |
Foot deformity | 26 (81%) | 33 (77%) | 17 (90%) | 0.501 |
Acute risk classification | 32 (100%) | 42 (98%) | 19 (100%) | 0.549 |
Clinical diagnoses of foot disease | ||||
Ulcer type: neuropathic | 14 (44%) | 14 (33%) | 8 (42%) | 0.572 |
Ulcer type: neuroischaemic | 9 (28%) | 16 (37%) | 8 (42%) | 0.555 |
Ulcer type: ischaemic | 0 | 3 (7%) | 1 (5%) | NA |
Ulcer type: other | 9 (28%) | 9 (21%) | 2 (11%) | 0.339 |
Combined surface area (mm2)# | 122 (191) | 352 (899) | 379 (1436) | 0.469 |
Any clinical infection | 6 (19%) | 14 (33%) | 2 (11%) | 0.125 |
UTWCS (Grade 1A) | 13 (41%) | 14 (33%) | 7 (37%) | 0.770 |
Deep ulcer (Grade 2 or 3)^ | 8 (25%) | 6 (14%) | 2 (11%) | 0.394 |
Criterion measure
Criterion measure reliability – measure of agreement between gold standard experts
QHRFF item | % | K(SE) | Strength of agreement [[37]] |
|---|---|---|---|
General demographics | |||
Indigenous status* | 77.4 | 0.45 (0.17) | Moderate |
Health professionals attending (in the past week) | |||
Podiatrist | 96.9 | -- | -- |
GP | 81.2 | 0.78 (0.21) | Substantial |
Nurse | 84.4 | 0.47 (0.18) | Moderate |
Orthotist | 100 | -- | -- |
Physician | 96.9 | 0.78 (0.21) | Substantial |
Surgeon | 100 | 1.00 (0.01) | Near perfect |
Other | 100 | -- | -- |
Medical co-morbidity history | |||
Medical (diabetes) history | 100 | 1.00 (0.01) | Near perfect |
Diabetes duration# | 0.94 (0.88–0.98) | Strong | |
Recent BGLs > 15 mmol/L* | 90.6 | 0.81 (0.11) | Near perfect |
HbA1c result# | 0.93 (0.79–0.98) | Strong | |
Hypertension | 87.5 | 0.60 (0.17) | Moderate |
Dyslipidaemia | 75.0 | 0.46 (0.16) | Moderate |
Smoker | 96.9 | 0.89 (0.11) | Near perfect |
CVD | 93.8 | 0.86 (0.10) | Near perfect |
CKD | 100 | 1.00 (0.01) | Near perfect |
ESRF | 96.9 | -- | -- |
Other (non-listed) | 46.9 | 0.04 (0.09) | Slight |
High-risk history | |||
Neuropathy | 93.8 | 0.48 (0.31) | Moderate |
PAD | 84.4 | 0.68 (0.13) | Substantial |
Previous foot ulcer | 84.4 | 0.45 (0.21) | Moderate |
Current foot ulcer | 96.9 | -- | -- |
New foot ulcer | 83.9 | 0.58 (0.15) | Moderate |
Previous amputation | 90.6 | 0.81 (0.10) | Near perfect |
Clinical diagnoses of high-risk factors | |||
Neuropathy | 96.9 | 0.65 (0.32) | Substantial |
PAD* | 77.4 | 0.61 (0.12) | Substantial |
Acute Charcot | 100 | 1.00 (0.01) | Near perfect |
Foot deformity | 81.3 | 0.46 (0.19) | Moderate |
Risk classification* | 96.8 | -- | -- |
Clinical diagnoses of foot disease | |||
Ulcer type* | 87.5 | 0.85 (0.08) | Near perfect |
Combined surface area mm2# | 0.73 (0.52-0.86) | Moderate | |
Clinical signs of infection* | 84.3 | 0.68 (0.15) | Substantial |
UTWCS grade# | 0.56 (0.28–0.76) | Weak-moderate | |
Ulcer depth^ | 78.1 | 0.67 (0.13) | Substantial |
Clinical management principles performed | |||
Debrided ulcer/callus* | 93.8 | -- | -- |
Dressing optimum* | 96.9 | -- | -- |
Antibiotics required* | 84.4 | 0.47 (0.19) | Moderate |
Off-loading optimum* | 68.8 | 0.41 (0.14) | Moderate |
Footwear optimum* | 65.6 | 0.28 (0.16) | Fair |
Educated patient* | 100 | -- | -- |
Criterion validity
Criterion validity summary statistics for senior clinician (Sensitivity, specificity and positive predictive value) & inter-rater reliability between expert and senior clinicians (Kappa and strength of agreement)
QHRFF field | % | K(SE) | Strength of agreement [[37]] | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) |
|---|---|---|---|---|---|---|
General demographics | ||||||
Any indigenous status | 100 | -- | -- | -- | 1.00 (0.90-1.00) | -- |
Health professionals attending (in the past week) | ||||||
Podiatrist | 100 | -- | -- | 1.00 (0.90-1.00) | -- | 1.00 (0.90-1.00) |
GP | 86.0 | 0.64 (0.14) | Substantial | 0.80 (0.44-0.96) | 0.88 (0.71-0.96) | 0.67 (0.35-0.88) |
Nurse | 93.0 | 0.83 (0.09) | Near perfect | 0.92 (0.60-1.00) | 0.94 (0.77-0.99) | 0.85 (0.54-0.97) |
Orthotist | 100 | -- | -- | -- | 1.00 (0.90-1.00) | -- |
Physician | 93.0 | -- | -- | 0 (0 – 0.95) | 0.95 (0.83-0.99) | 0 (0 – 0.80) |
Surgeon | 95.3 | 0.64 (0.24) | Substantial | 0.67 (0.13-0.98) | 0.98 (0.85-1.00) | 0.67 (0.13-0.98) |
Other | 100 | -- | -- | -- | 1.00 (0.90-1.00) | -- |
Medical co-morbidity history | ||||||
Any medical (diabetes) history | 100 | 1.00 (0.01) | Near perfect | 1.00 (0.88-1.00) | 1.00 (0.60-1.00) | 1.00 (0.88-1.00) |
Diabetes duration# | 0.90 (0.80– 0.95) | Strong | ||||
Recent BGLs > 15 mmol/L | 93.0 | 0.69 (0.16) | Substantial | 0.57 (0.20-0.88) | 1.00 (0.88-1.00) | 1.00 (0.40-1.00) |
HbA1c result# | 1.00 (0.99–1.00) | Strong | ||||
Hypertension | 93.0 | 0.83 (0.09) | Near perfect | 0.97 (0.81-1.00) | 0.85 (0.54-0.97) | 0.94 (0.77-0.99) |
Dyslipidaemia | 95.3 | 0.90 (0.07) | Near perfect | 0.96 (0.78-1.00) | 0.94 (0.71-1.00) | 0.96 (0.78-1.00) |
Smoker | 90.7 | 0.72 (0.13) | Substantial | 0.64 (0.32-0.88) | 1.00 (0.87-1.00) | 1.00 (0.56-1.00) |
CVD | 90.7 | 0.81 (0.09) | Near perfect | 0.86 (0.63-0.96) | 0.95 (0.75-1.00) | 0.95 (0.72-1.00) |
CKD | 81.4 | 0.49 (0.15) | Moderate | 0.46 (0.20-0.74) | 0.97 (0.81-1.00) | 0.86 (0.42-0.99) |
ESRF | 1.00 | 1.00 (0.01) | Near perfect | 1.00 (0.05-1.00) | 1.00 (0.90-1.00) | 1.00 (0.05-1.00) |
Other (non-listed) | 57.1 | 0.09 (0.12) | Slight | 0.57 (0.40-0.73) | 0.57 (0.20-0.88) | 0.87 (0.65-0.97) |
High-risk history | ||||||
Neuropathy | 93.0 | 0.73 (0.15) | Substantial | 0.95 (0.80-0.99) | 0.83 (0.36-0.99) | 0.97 (0.84-1.00) |
PAD | 88.4 | 0.77 (0.10) | Substantial | 0.86 (0.63-0.96) | 0.91 (0.69-0.98) | 0.90 (0.67-0.98) |
Previous foot ulcer | 79.1 | 0.28 (0.18) | Fair | 0.84 (0.67-0.93) | 0.50 (0.14-0.86) | 0.91 (0.75-0.98) |
Current foot ulcer | 95.3 | -- | -- | 0.98 (0.86-1.00) | 0 (0 – 0.95) | 0.98 (0.86-1.00) |
New foot ulcer | 93.0 | -- | -- | 0.95 (0.82-0.99) | 0 (0 – 0.95) | 0.98 (0.86-1.00) |
Previous amputation | 97.7 | 0.95 (0.05) | Near perfect | 0.94 (0.69-1.00) | 1.00 (0.84-1.00) | 1.00 (0.76-1.00) |
Clinical diagnoses of high-risk factors | ||||||
Neuropathy | 97.7 | 0.90 (0.10) | Near perfect | 0.97 (0.85-1.00) | 1.00 (0.46-1.00) | 1.00 (0.88-1.00) |
Any PAD | 93.0 | 0.86 (0.8) | Near perfect | 0.91 (0.69-0.98) | 0.95 (0.74-1.00) | 0.95 (0.74-1.00) |
Acute Charcot | 100 | 1.00 (0.01) | Near perfect | 1.00 (0.05-1.00) | 1.00 (0.90-1.00) | 1.00 (0.05-1.00) |
Foot deformity | 86.0 | 0.51 (0.16) | Moderate | 1.00 (0.87-1.00) | 0.4 (0.14-0.73) | 0.85 (0.69-0.94) |
Acute risk classification | 97.7 | -- | -- | 1.00 (0.90-1.00) | 0 (0 – 0.95) | 0.98 (0.86-1.00) |
Clinical diagnoses of foot disease | ||||||
Ulcer type: neuropathic (neuropathic or neuroisch.) | 90.7 | 0.76 (0.11) | Substantial | 0.88 (0.72-0.96) | 1.0 (0.63-1.00) | 1.00 (0.86-1.00) |
Ulcer type: ischaemic | 79.1 | 0.58 (0.13) | Moderate | 0.75 (0.51-0.90) | 0.83 (0.60-0.94) | 0.79 (0.54-0.93) |
(Neuroisch. or ischaemic) | ||||||
Combined surface area mm2# | 0.93 (0.87–0.96) | Strong | ||||
Any clinical infection | 88.4 | 0.72 (0.12) | Substantial | 0.71 (0.42-0.90) | 0.97 (0.80-1.00) | 0.91 (0.57-1.00) |
UTWCS (Grade 1A) | 83.7 | 0.65 (12) | Substantial | 0.85 (0.56-0.97) | 0.83 (0.64-0.93) | 0.71 (0.44-0.89) |
Deep ulcer | 95.3 | 0.81 (0.13) | Near perfect | 0.71 (0.30-0.95) | 0.97 (0.84-1.00) | 0.83 (0.36-0.99) |
(Grade 2 or 3)^ | ||||||
Clinical management principles performed | ||||||
Ulcer debridement is required | 95.3 | 0.66 (0.01) | Substantial | 1.00 (0.89-1.00) | 1.00 (0.20-1.00) | 1.00 (0.89-1.00) |
Dressing is optimum | 100 | -- | -- | 1.00 (0.90-1.00) | -- | 1.00 (0.90-1.00) |
Antibiotics are required | 93.0 | 0.82 (0.10) | Near perfect | 0.83 (0.51-0.97) | 0.94 (0.77-0.99) | 0.83 (0.51-0.97) |
Off-loading is optimum | 93.0 | 0.80 (0.11) | Substantial | 0.80 (0.44-0.96) | 0.97 (0.82-1.00) | 0.89 (0.51-0.99) |
Footwear is optimum | 72.1 | 0.31 (0.16) | Fair | 0.50 (0.20-0.80) | 0.82 (0.64-0.92) | 0.45 (0.18-0.75) |
Educated patient | 97.7 | -- | -- | 0.98 (0.86-1.00) | -- | 1.00 (0.90-1.00) |
Criterion validity summary statistics for clinician (Sensitivity, specificity and positive predictive value) & inter-rater reliability between expert and clinician (Kappa and strength of agreement)
QHRFF field | % | K(SE) | Strength of agreement [[37]] | Sensitivity (95% CI) | Specificity (95% CI) | PPV (95% CI) |
|---|---|---|---|---|---|---|
General demographics | ||||||
Any indigenous status | 100 | -- | -- | -- | 1.00 (0.90-1.00) | -- |
Health professionals attending (in the past week) | ||||||
Podiatrist | 88.4 | -- | -- | 0.88 (0.74-0.96) | -- | 0.88 (0.74-0.96) |
GP | 81.4 | 0.39 (0.17) | Fair | 0.40 (0.14-0.73) | 0.94 (0.78-0.99) | 0.67 (0.24-0.94) |
Nurse | 79.1 | 0.37 (0.16) | Fair | 0.33 (0.11-0.65) | 0.97 (0.81-1.00) | 0.80 (0.30-0.99) |
Orthotist | 100 | -- | -- | -- | 1.00 (0.90–1.00) | -- |
Physician | 97.6 | 0.66 (0.32) | Substantial | 1.00 (0.05-1.00) | 0.98 (0.86-1.00) | 0.50 (0.03-0.97) |
Surgeon | 90.7 | 0.45 (0.23) | Moderate | 0.67 (0.13-0.98) | 0.93 (0.79-0.98) | 0.40 (0.07-0.83) |
Other | 100 | -- | -- | -- | 0.98 (0.86-1.00) | -- |
Medical co-morbidity history | ||||||
Any medical (diabetes) history | 100 | 100 (0.01) | Near perfect | 1.00 (0.88-1.00) | 1.00 (0.60-1.00) | 1.00 (0.88-1.00) |
Diabetes duration# | 0.95 (0.90–0.97) | Strong | ||||
Recent BGLs > 15 mmol/L | 90.7 | 0.62 (0.17) | Substantial | 1.00 (0.40-1.00) | 0.90 (0.75-0.97) | 0.50 (0.17-0.83) |
HbA1c result# | 1.00 (0.99–1.00) | Strong | ||||
Hypertension | 86.0 | 0.68 (0.12) | Substantial | 0.87 (0.68-0.96) | 0.85 (0.54-0.97) | 0.93 (0.75-0.99) |
Dyslipidaemia | 79.1 | 0.58 (0.12) | Moderate | 0.76 (0.54-0.90) | 0.83 (0.58-0.96) | 0.86 (0.64-0.96) |
Smoker | 88.4 | 0.69 (0.13) | Substantial | 0.73 (0.39-0.93) | 0.94 (0.78-0.99) | 0.80 (0.44-0.96) |
CVD | 79.1 | 0.58 (0.12) | Moderate | 0.71 (0.48-0.88) | 0.86 (0.64-0.96) | 0.83 (0.58-0.96) |
CKD | 86.0 | 0.65 (0.13) | Substantial | 0.69 (0.39-0.90) | 0.93 (0.76-0.99) | 0.82 (0.48-0.97) |
ESRF | 97.7 | 0.66 (0.32) | Substantial | 1.00 (0.05-1.00) | 0.98 (0.86-1.00) | 0.50 (0.03-0.97) |
Other (non-listed) | 79.1 | 0.19 (0.19) | Slight | 0.89 (0.73-0.96) | 0.29 (0.05-0.7) | 0.86 (0.70-0.95) |
High-risk history | ||||||
Neuropathy | 93.0 | 0.73 (0.15) | Substantial | 0.95 (0.80-0.99) | 0.83 (0.36-0.99) | 0.97 (0.84-1.00) |
PAD | 86.0 | 0.72 (0.10) | Substantial | 0.76 (0.52-0.91) | 0.95 (0.75-1.00) | 0.94 (0.69-1.00) |
Previous foot ulcer | 79.1 | 0.28 (0.18) | Fair | 0.84 (0.67-0.93) | 0.50 (0.14-0.86) | 0.91 (0.75-0.98) |
Current foot ulcer | 97.7 | -- | -- | 1.00 (0.90-1.00) | 0 (0 – 0.95) | 0.98 (0.86-1.00) |
New foot ulcer | 97.4 | -- | -- | 0.90 (0.76-0.97) | 0 (0 – 0.95) | 0.97 (0.85-0.99) |
Previous amputation | 95.3 | 0.90 (0.07) | Near perfect | 0.88 (0.62-0.98) | 1.00 (0.84-1.00) | 1.00 (0.75-1.00) |
Clinical diagnoses of high-risk factors | ||||||
Neuropathy | 100 | 1.00 (0.01) | Near perfect | 1.00 (0.89-1.00) | 1.00 (0.46-1.00) | 1.00 (0.89-1.00) |
Any PAD | 95.3 | 0.91 (0.06) | Near perfect | 0.91 (0.69-0.98) | 1.00 (0.81-1.00) | 1.00 (0.80-1.00) |
Acute Charcot | 100 | 1.00 (0.01) | Near perfect | 1.00 (0.05-1.00) | 1.00 (0.90-1.00) | 1.00 (0.05-1.00) |
Foot deformity | 93.0 | 0.80 (0.11) | Substantial | 0.97 (0.82-1.00) | 0.80 (0.44-0.96) | 0.94 (0.79-0.99) |
Acute risk classify | 97.7 | -- | -- | 1.00 (0.90-1.00) | 0 (0 – 0.95) | 0.98 (0.86-1.00) |
Clinical diagnoses of foot disease | ||||||
Ulcer type: neuropathic | 83.7 | 0.55 (0.14) | Moderate | 1.00 (0.86-1.00) | 0.46 (0.20-0.74) | 0.81 (0.64-0.91) |
(Neuropathic or neuroisch.) | ||||||
Ulcer type is ischaemic | 79.1 | 0.57 (0.13) | Moderate | 0.74 (0.48-0.90) | 0.83 (0.62-0.95) | 0.78 (0.52-0.93) |
(Neuroisch. or ischaemic) | ||||||
Combined surface area mm2# | 0.99 (0.99–1.00) | Strong | ||||
Any clinical infection | 81.4 | 0.58 (0.13) | Moderate | 0.71 (0.42-0.90) | 0.86 (0.67-0.95) | 0.71 (0.42-0.90) |
UTWCS (Grade 1A) | 81.4 | 0.61 (0.12) | Substantial | 0.86 (0.56-0.97) | 0.79 (0.60-0.91) | 0.69 (0.41-0.86) |
Deep ulcer | 93.0 | 0.73 (0.15) | Substantial | 0.71 (0.30-0.95) | 0.97 (0.84-1.00) | 0.83 (0.36-0.99) |
(Grade 2 or 3)^ | ||||||
Clinical management principles performed | ||||||
Ulcer debridement is required | 95.3 | 0.39 (0.22) | Fair | 1.00 (0.89-1.00) | 0.50 (0.03-0.97) | 0.98 (0.86-1.00) |
Dressing is optimum | 97.7 | -- | -- | 0.98 (0.86-1.00) | -- | 1.00 (0.90-1.00) |
Antibiotics are required | 90.7 | 0.76 (0.11) | Substantial | 0.75 (0.43-0.93) | 0.97 (0.81-1.00) | 0.90 (0.54-0.99) |
Off-loading is optimum | 79.1 | 0.50 (0.14) | Moderate | 0.80 (0.44-0.96) | 0.79 (0.60-0.90) | 0.53 (0.27-0.78) |
Footwear is optimum | 79.1 | 0.47 (0.15) | Moderate | 0.70 (0.35-0.92) | 0.82 (0.64-0.92) | 0.54 (0.26-0.80) |
Educated patient | 100 | -- | -- | 1.00 (0.90-1.00) | -- | 1.00 (0.90-1.00) |
Inter-rater reliability
Reliability measure of agreement summary statistics: Inter-rater (Senior clinician & clinician) & intra-rater (Clinician)
Inter-rater reliability | Intra-rater reliability | |||||
|---|---|---|---|---|---|---|
QHRFF field | % | K(SE) | Strength of agreement [[37]] | % | K(SE) | Strength of agreement [[37]] |
General demographics | ||||||
Indigenous status* | 83.7 | -- | -- | 100 | 1.00 (0.01) | Near Perfect |
Health professionals attending (in the past week) | ||||||
Podiatrist | 88.4 | -- | -- | N/A | N/A | N/A |
GP | 81.4 | 0.45 (0.15) | Moderate | N/A | N/A | N/A |
Nurse | 81.4 | 0.47 (0.14) | Moderate | N/A | N/A | N/A |
Orthotist | 100 | -- | -- | N/A | N/A | N/A |
Physician | 90.7 | -- | -- | N/A | N/A | N/A |
Surgeon | 90.7 | 0.45 (0.23) | Moderate | N/A | N/A | N/A |
Other | 97.7 | -- | -- | N/A | N/A | N/A |
Medical co-morbidity history | ||||||
Medical (diabetes) history | 95.3 | 0.94 (0.05) | Near perfect | 100 | 1.00 (0.01) | Near Perfect |
Diabetes duration# | 0.94 (0.87–0.97) | Strong | 1.00 (0.99–1.00) | Strong | ||
Recent BGLs >15 mmol/L* | 83.7 | 0.67 (0.12) | Substantial | N/A | N/A | N/A |
HbA1c result# | 1.00 (1.00–1.00) | Strong | 1.00 (1.00–1.00) | |||
Hypertension | 88.4 | 0.73 (0.11) | Substantial | 89.5 | 0.78 (0.15) | Substantial |
Dyslipidaemia | 83.7 | 0.67 (0.11) | Substantial | 94.7 | 0.90 (0.10) | Near Perfect |
Smoker | 88.4 | 0.64 (0.15) | Substantial | 94.7 | 0.83 (0.17) | Near Perfect |
CVD | 88.4 | 0.76 (0.10) | Substantial | 84.2 | 0.69 (0.17) | Substantial |
CKD | 86.1 | 0.58 (0.15) | Moderate | 94.7 | 0.86 (0.14) | Near Perfect |
ESRF | 97.7 | 0.66 (0.32) | Substantial | 100 | -- | -- |
Other (non-listed) | 69.1 | 0.34 (0.11) | Fair | 78.9 | 0.57 (0.19) | Moderate |
High-risk history | ||||||
Neuropathy | 90.7 | 0.66 (0.16) | Substantial | 94.7 | -- | -- |
PAD | 83.7 | 0.67 (0.11) | Substantial | 78.9 | 0.60 (0.16) | Moderate |
Previous foot ulcer | 90.7 | 0.72 (0.13) | Substantial | 94.7 | 0.64 (0.33) | Substantial |
Current foot ulcer | 97.7 | -- | -- | 100 | -- | -- |
New foot ulcer | 97.4 | -- | -- | 100 | -- | -- |
Previous amputation | 97.7 | 0.95 (0.05) | Near perfect | 100 | 1.00 (0.01) | Near Perfect |
Clinical diagnoses of high-risk factors | ||||||
Neuropathy | 97.7 | 0.90 (0.10) | Near perfect | 94.7 | -- | -- |
PAD* | 97.9 | 0.97 (0.03) | Near perfect | 84.2 | 0.69 (0.16) | Substantial |
Acute charcot | 100 | 1.00 (0.01) | Near perfect | 100 | -- | -- |
Foot deformity | 83.7 | 0.38 (0.18) | Fair | 84.2 | 0.31 (0.30) | Fair |
Risk classification | 100 | -- | -- | 100 | -- | -- |
Clinical diagnoses of foot disease | ||||||
Ulcer type* | 88.4 | 0.81 (0.08) | Near perfect | 84.2 | 0.77 (0.14) | Substantial |
Combined surface area mm2# | 0.94 (0.89–0.97) | Strong | 1.00 (1.00–1.00) | Strong | ||
Clinical signs infection* | 76.7 | 0.60 (0.12) | Moderate | 89.5 | 0.46 (0.32) | Moderate |
UTWCS grade# | 0.90 (0.83–0.95) | Strong | 0.98 (0.94–0.99) | Strong | ||
Ulcer depth^ | 95.3 | 0.90 (0.10) | Near perfect | 94.7 | 0.79 (0.23) | Substantial |
Clinical management principles performed | ||||||
Debrided ulcer/callus* | 97.7 | 0.66 (0.32) | Substantial | 94.7 | -- | -- |
Dressing optimum* | 97.7 | -- | -- | 100 | -- | -- |
Antibiotics required* | 79.1 | 0.54 (0.12) | Moderate | 89.5 | 0.44 (0.33) | Moderate |
Off-loading optimum* | 72.1 | 0.36 (0.13) | Fair | 68.4 | 0.32 (0.23) | Fair |
Footwear optimum* | 67.4 | 0.33 (0.14) | Fair | 68.4 | 0.37 (0.21) | Fair |
Educated patient* | 97.7 | -- | -- | 89.5 | -- | -- |
Intra-rater reliability
Table 6 also displays the intra-rater reliability results for the level 3 clinician. The median (interquartile range) period between the first and second ratings for the intra-rater reliability testing was 2(1–2) weeks. Twenty-three items were able to be statistically tested. Six items (26%) recorded near perfect categories of agreement, ten (43%) substantial/strong categories, four (17%) moderate categories and three (13%) weak/fair categories. Thus, overall 20 (87%) of these items recorded at least a moderate category of reliability (K > 0.4; ICC > 0.75). Those items scoring weak categories of agreement were foot deformity, optimal offloading and optimal footwear.
Discussion
The QHHRF appears to be the first multi-item tool developed and tested to identify multiple high-risk factors and foot disease complications in multiple at risk populations. Our findings indicate that the majority of the tool’s items demonstrates at least moderate categories of validity (face, content and criterion validity) and reliability (inter-rater and intra-rater); particularly those in the domains of identifying relevant medical co-morbidity history, and clinical diagnoses of high-risk factors and foot disease complications. However, some items in the domains of identifying different health professionals previously attending the patient and general clinical management principles performed appear to have weaker categories of validity and reliability that need addressing in future versions of the tool.
The QHRFF went through a number of developmental, validity and reliability tests to determine its status as a valid and reliable tool. The magnitude of validity (or accuracy) in this study was evaluated via the methods of face, content and importantly criterion validity. Face and content validity are considered the least robust of the validity tests due to their inherent subjectivity [31, 33]. However, they are considered important factors in the development phase to ensure the tool can actually measure the general identified construct [31, 33]; in this case foot disease. At the completion of phase one, the expert panel and network stakeholders endorsements implied high practicality, face and content validity.
Criterion validity is considered to be the most objective validity test [31, 33]. To test concurrent criterion validity, as used by this study, a satisfactory criterion measure must be used. In this study the criterion measure to test criterion validity was an expert’s clinical diagnosis. Overall, the 'expert’ criterion measure used for this study were rated as having at least moderate categories of reliability for all but three items; any other (non-listed) co-morbidity, optimum footwear and UTWCS grade. The other (non-listed) co-morbidity and optimum footwear items rated in the weak categories in most other inter- or intra-rater reliability agreement tested and should be reviewed or removed in future versions of the QHRFF. The weak-moderate agreement for the UTWCS grade was unexpected as this tool has been validated extensively in the past [62], however, all other inter- or intra-rater agreements on the UTWCS were rated as substantial/strong categories of reliability. Thus, it is recommended that the UTWCS grade be retained in future with more emphasis applied on the UTWCS grading system in the training sessions provided. It is certainly possible that the criterion measure for each item may have been more robust if the individual gold standard test for each individual item were used; for example an angiogram to diagnose peripheral arterial disease [22] or nerve conduction studies to diagnose neuropathy [44]. However, such an approach would have been particularly resource and time intensive and was thus not practical within the resources available to this study.
The QHRFF items were tested for concurrent criterion validity by using two different blinded representative general clinicians’ ratings compared to the criterion measure (a reliable expert’s diagnosis) on the same patients. Positive predictive values from this testing indicated the majority of QHRFF items had at least moderate validity when used by clinicians with different levels of expertise. The high positive predictive values suggest that the proportion of people with a positive test result for individual QHRFF items actually had the relevant medical co-morbidity, high-risk factor or foot disease complication of interest. Although there were a few notable exceptions such as identifying different health professionals attending previously, recent blood glucose levels (BGLs) > 15 mmol/L, ESRF, optimal offloading and optimum footwear. However, overall the QHRFF tool appears to demonstrate acceptable validity in the majority of its items to be considered a valid tool to test the foot disease construct in multiple at risk populations.
Reliability (or consistency) is a major prerequisite of any useful items to be measured [31, 33]. In this study inter-rater and intra-rater measures of agreement were used for reliability. Inter-rater reliability was primarily assessed using three clinicians with different representative levels of expertise in managing foot disease. Overall, all items consistently displayed at least moderate categories of reliability, except for any other (non-listed) co-morbidity, optimum footwear and previous foot ulcer. Optimum footwear was also identified to have weak categories of intra-rater reliability; along with identifying foot deformity and optimum offloading. Thus, the QHRFF tool appears to demonstrate satisfactory reliability to collect the majority of items in the foot disease construct. However, the authors recognise that it would have been preferable to test reliability with more clinicians across a larger sample of patients, yet this was beyond the resourcing available for this study.
Notwithstanding, the aforementioned methodological limitations, the study does incorporate many best practice research design methods for testing validity and reliability. These methods included testing the reliability of the criterion measure to diagnose and measure each item, testing the subsequent concurrent criterion validity of representative clinicians, and testing the reliability of the tool when used by clinicians with different levels of expertise. Furthermore, the study employed patient samples that were highly representative of the demographics and co-morbidity prevalence rates found in multiple 'at risk’ populations for high risk factors and foot disease complications. These high prevalence rates not only suggest the tool was tested in the construct it was designed to measure, but literature suggests higher prevalence rates improve the statistical robustness of validity and reliability results [31]. Lastly, many existing validated single-item tools [23, 46, 60, 62] were incorporated within the QHRFF tool and this potentially adds weight to the validity reported in this study. Thus, overall the QHRFF tool appears to be a valid and reliable tool to collect the vast majority of items contained within the foot disease construct and can be used as a general tool to complement existing specific tools in the field of foot disease.
Limitations
Several limitations have already been outlined in this study including the use of a general criterion measure of an experts’ clinical diagnoses for each item, only using a limited number of representative clinicians and testing intra-rater reliability on one clinician in a small sample. Other limitations include: not performing a systematic literature review (and thus some evidence based items and tools may have been overlooked); using only podiatrists as the clinical raters; using historically defined strength categories for validity and reliability; not testing the tool for construct validity, predictive validity, or factor analysis; and the settings used were existing HRFS only. It is recommended that any future research into this tool should address these methodological limitations by investigating the factor analysis, construct and predictive criterion validity of the tool’s items in a much larger and broader sample of patients with more multi-disciplinary clinician raters. Furthermore, if resources permit, the use of individual gold standard criterion measures and/or existing concurrent validated single-item tools should be considered.
However, there are a number of recommendations from this study’s findings that should be considered for implementation in future versions of the QHRFF tool. Firstly, those items reporting weaker categories of validity or reliability and that potentially collect duplicate information to other more reliable items should be removed; including any other (non-listed) co-morbidity, combined surface area (change), recent BGLs greater than 15 mmol/L and new ulcer. Secondly, some items’ definitions should be reviewed in an attempt to improve the item’s future reliability and validity. For example, previous foot ulcer definition could be modified to identify a previously “healed” foot ulcer, 'health professionals attending’ previously requires an exact retrospective time period that patients were in attendance in alignment with similar literature (such as “in the previous two weeks”) [64, 65] and optimum offloading should have a more explicit definition referring directly to non-removable offloading definitions in the existing literature [14]. Thirdly, the validation of criteria for identifying a foot deformity and optimum footwear are urgently required for these 'at risk’ populations. To the best of the author’s knowledge such tools for these items have been developed [14, 50] yet are to be validated and this study was no exception. However, in the meantime the authors recommend adding the numerical foot deformity score to the QHRFF to improve accuracy and test for validity. Lastly, a systematic literature review for other reported independent associates for foot disease (such as cancer, arthritis, depression, trauma, vision impairment, mobility impairment and social determinant factors) and other outcome measures for foot disease (such as quality of life measures) should be performed to ensure all 'at risk’ populations and outcome measures for foot disease are identified, considered and potentially included in the next QHRFF version.
Conclusions
The QHRFF tool appears to be the first multi-item tool developed and tested to identify multiple high-risk factors and foot disease complications in multiple at risk populations. Acceptable validity and reliability were demonstrated for the majority of items; particularly in the domains of identifying relevant medical co-morbidity history and clinically diagnosing high-risk factors and foot disease complications. However, recommendations to remove, add or redefine items registering weak validity or reliability scores should be implemented to improve future versions of the QHRFF tool; particularly in the domains of health professionals attending and clinical management principles. Overall, the QHRFF tool appears to demonstrate the practicality, validity and reliability required to facilitate robust clinical assessment and data capture to measure the large burden of foot disease facing our multiple at risk populations.
Abbreviations
- ANOVA:
-
Analysis of variance
- BGL:
-
Blood glucose levels
- CINAHL:
-
Cumulative index to nursing and allied health literature
- CKD:
-
Chronic kidney disease
- DFF:
-
Diabetic foot form
- ESRF:
-
End stage renal failure
- GP:
-
General practitioner
- HRFS:
-
High risk foot service
- ICC:
-
Intra-class correlation
- K:
-
Kappa
- wK:
-
Weighted Kappa
- MEDLINE:
-
Medical literature analysis and retrieval system online
- PPV:
-
Positive predictive value
- QHRFF:
-
Queensland high risk foot form
- SD:
-
Standard deviation
- SE:
-
Standard error
- UTWCS:
-
University of Texas wound classification system.
Declarations
Acknowledgements
The authors wish to acknowledge the significant contribution made to the project by Ms Helen Martin, Ms Kimberley Canning, Ms Helen Sheahan and the entire Metro North Hospital and Health Service Podiatry Department. Furthermore, the authors wish to acknowledge the significant support of the Queensland Health Statewide Diabetes Clinical Network, Diabetes Foot Working Group, Renal Clinical Network and Podiatry Network. The authors are also grateful to the grant funding for the project provided by the Metro North Hospital & Health Service Clinical Practice Incentive Payment Grants. None of these contributors have a relevant conflict of interest. Lastly, the authors wish to acknowledge Dr Sheree Nix for reviewing final drafts of the manuscript.
Authors’ Affiliations
References
- Bakker K, Apelqvist J, Schaper NC: Practical guidelines on the management and prevention of the diabetic foot 2011. Diabetes/Metab Res Rev. 2012, 28 (Suppl 1): 225-231.View ArticleGoogle Scholar
- Australian Institute of Health & Welfare (AIHW): Diabetes: Australian facts 2008. 2008, Canberra: Australian Government, http://www.aihw.gov.au/publications/cvd/daf08/daf08.pdf Accessed 25th June 2013Google Scholar
- Barrois B, Labalette C, Rousseau P, Corbin A, Colin D, Allaert F, Saumet JL: A national prevalence study of pressure ulcers in French hospital inpatients. J Wound Care. 2008, 17: 373-View ArticlePubMedGoogle Scholar
- Lazzarini PA, Gurr JM, Rogers JR, Schox A, Bergin SM: Diabetes foot disease: the Cinderella of Australian diabetes management?. J Foot Ankle Res. 2012, 5: 24-10.1186/1757-1146-5-24.View ArticlePubMedPubMed CentralGoogle Scholar
- Moxey PW, Gogalniceanu P, Hinchliffe RJ, Loftus IM, Jones KJ, Thompson MM, Holt PJ: Lower extremity amputations–a review of global variability in incidence. Diabetic Med: J Br Diabetic Assoc. 2011, 28: 1144-1153. 10.1111/j.1464-5491.2011.03279.x.View ArticleGoogle Scholar
- Ephraim PL, Dillingham TR, Sector M, Pezzin LE, MacKenzie EJ: Epidemiology of limb loss and congenital limb deficiency: a review of the literature. Arch Phys Med Rehabil. 2003, 84: 747-761.View ArticlePubMedGoogle Scholar
- Global Lower Extremity Amputation Study Group: Epidemiology of lower extremity amputation in centres in Europe, North America and East Asia. Br J Surg. 2000, 87: 328-337. 10.1046/j.1365-2168.2000.01344.x.View ArticleGoogle Scholar
- Lazzarini PA, O’Rourke SR, Russell AW, Clark D, Kuys SS: What are the key conditions associated with lower limb amputations in a major Australian teaching hospital?. J Foot Ankle Res. 2012, 5: 12-10.1186/1757-1146-5-12.View ArticlePubMedPubMed CentralGoogle Scholar
- Lavery LA, Houtum WHV, Armstrong DG: Institutionalization following diabetes-related lower extremity amputation. Am J Med. 1997, 103: 383-388. 10.1016/S0002-9343(97)00163-0.View ArticlePubMedGoogle Scholar
- Dillingham TR, Yacub JN, Pezzin LE: Determinants of postacute care discharge destination after dysvascular lower limb amputation. PM&R. 2011, 3: 336-344.View ArticleGoogle Scholar
- Armstrong DG, Wrobel J, Robbins JM: Guest editorial: are diabetes-related wounds and amputations worse than cancer?. Int Wound J. 2007, 4: 286-287. 10.1111/j.1742-481X.2007.00392.x.View ArticlePubMedGoogle Scholar
- Canavan RJ, Unwin NC, Kelly WF, Connolly VM: Diabetes- and nondiabetes-related lower extremity amputation incidence before and after the introduction of better organized diabetes foot care: continuous longitudinal monitoring using a standard method. Diabetes Care. 2008, 31: 459-463.View ArticlePubMedGoogle Scholar
- Krishnan S, Nash F, Baker N, Fowler D, Rayman G: Reduction in diabetic amputations over 11 years in a defined U.K. population: benefits of multidisciplinary team work and continuous prospective audit. Diabetes Care. 2008, 31: 99-101.View ArticlePubMedGoogle Scholar
- National Health & Medical Research Council (NHMRC) Guidelines: National evidence-based guideline on prevention, identification and management of foot complications in diabetes (Part of the guidelines on management of type 2 diabetes). 2011, Melbourne: Baker IDI Heart & Diabetes InstituteGoogle Scholar
- Kaminski M, Frescos N, Tucker S: Prevalence of risk factors for foot ulceration in patients with end-stage renal disease on haemodialysis. Intern Med J. 2012, 42: e120-e128. 10.1111/j.1445-5994.2011.02605.x.View ArticlePubMedGoogle Scholar
- Freeman A, May K, Frescos N, Wraight PR: Frequency of risk factors for foot ulceration in individuals with chronic kidney disease. Intern Med J. 2008, 38: 314-320. 10.1111/j.1445-5994.2007.01528.x.View ArticlePubMedGoogle Scholar
- Tapp RJ, Shaw JE, Courten MPD, Dunstan DW, Welborn TA, Zimmet PZ: Foot complications in type 2 diabetes: an Australian population‒based study. Diabet Med. 2003, 20: 105-113. 10.1046/j.1464-5491.2003.00881.x.View ArticlePubMedGoogle Scholar
- Reed T, Veith FJ, Gargiulo NJ, Timaran CH, Ohki T, Lipsitz EC, Malas MB, Wain RA, Suggs WD: System to decrease length of stay for vascular surgery. J Vasc Surg. 2004, 39: 395-399. 10.1016/j.jvs.2003.09.015.View ArticlePubMedGoogle Scholar
- Lynch EA, Hillier SL, Stiller K, Campanella RR, Fisher PH: Sensory retraining of the lower limb after acute stroke: a randomized controlled pilot trial. Arch Phys Med Rehabil. 2007, 88: 1101-1107. 10.1016/j.apmr.2007.06.010.View ArticlePubMedGoogle Scholar
- Jeffcoate WJ, van Houtum WH: Amputation as a marker of the quality of foot care in diabetes. Diabetologia. 2004, 47: 2051-2058. 10.1007/s00125-004-1584-3.View ArticlePubMedGoogle Scholar
- American College of Cardiology Foundation/American Heart Association (ACCF/AHA): ACCF/AHA focused update of the guideline for the management of patients with peripheral artery disease (Updating the 2005 guideline): a report of the American College of Cardiology Foundation/American Heart Association Task Force on practice guidelines. Circulation. 2011, 124: 2020-2045. 10.1161/CIR.0b013e31822e80c3.View ArticleGoogle Scholar
- Caruana MF, Bradbury AW, Adam DJ: The validity, reliability, reproducibility and extended utility of ankle to brachial pressure index in current vascular surgical practice. Eur J Vasc Endovasc Surg: The Official J Eur Soc Vasc Surg. 2005, 29: 443-451. 10.1016/j.ejvs.2005.01.015.View ArticleGoogle Scholar
- Young D, Schuerman S, Flynn K, Hartig K, Moss D, Altenburger B: Reliability and responsiveness of an 18 site, 10-g monofilament examination for assessment of protective foot sensation. J Geriatr Phys Ther (2001). 2011, 34: 95-98. 10.1519/JPT.0b013e31820aabe5.View ArticleGoogle Scholar
- Stausberg J, Lehmann N, Kröger K, Maier I, Niebel W: Reliability and validity of pressure ulcer diagnosis and grading: an image-based survey. Int J Nurs Stud. 2007, 44: 1316-1323. 10.1016/j.ijnurstu.2006.06.006.View ArticlePubMedGoogle Scholar
- Armstrong DG, Lavery LA, Harkless LB: Treatment-based classification system for assessment and care of diabetic feet. Ostomy Wound Manage. 1996, 42: 50-Google Scholar
- Ince P, Abbas ZG, Lutale JK, Basit A, Ali SM, Chohan F, Morbach S, Mollenberg J, Game FL, Jeffcoate WJ: Use of the SINBAD classification system and score in comparing outcome of foot ulcer management on three continents. Diabetes Care. 2008, 31: 964-967. 10.2337/dc07-2367.View ArticlePubMedGoogle Scholar
- Lipsky BA, Weigelt JA, Sun X, Johannes RS, Derby KG, Tabak YP: Developing and validating a risk score for lower-extremity amputation in patients hospitalized for a diabetic foot infection. Diabetes Care. 2011, 34: 1695-1700. 10.2337/dc11-0331.View ArticlePubMedPubMed CentralGoogle Scholar
- Bergin SM, Brand CA, Colman PG, Campbell DA: A questionnaire for determining prevalence of diabetes related foot disease (Q-DFD): construction and validation. J Foot Ankle Res. 2009, 2: 34-10.1186/1757-1146-2-34.View ArticlePubMedPubMed CentralGoogle Scholar
- Lazzarini PA, O¿Rourke SR, Russell AW, Derhy PH, Kamp MC: Standardising practices improves clinical diabetic foot management: the Queensland Diabetic Foot Innovation Project, 2006–09. Aust Health Rev. 2012, 36: 8-15. 10.1071/AH10978.View ArticlePubMedGoogle Scholar
- Lazzarini PA, O’Rourke SR, Russell AW, Derhy PH, Kamp MC: Standardising practices improves ambulatory diabetic foot management and reduces amputations: the Queensland Diabetic Foot Innovation Project, 2006–2009. J Foot Ankle Res. 2011, 4: O25-10.1186/1757-1146-4-S1-O25.View ArticlePubMed CentralGoogle Scholar
- Portney LG, Watkins MP: Foundations of clinical research applications to practice. 2009, New Jersey, USA: Pearson Prentice, 3Google Scholar
- Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW: Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res: An Int J Qual Life Aspects Of Treatment, Care And Rehabil. 2012, 21: 651-657. 10.1007/s11136-011-9960-1.View ArticleGoogle Scholar
- Trevethan R: Self-assessment of foot health: requirements, issues, practicalities, and challenges. J Am Podiatr Med Assoc. 2009, 99: 460-471.View ArticlePubMedGoogle Scholar
- Queensland Industrial Relations Commission: Health Practitioners (Queensland health) Certified Agreement (No. 2). 2011, Brisbane: Queensland Government, http://www.health.qld.gov.au/eb/hp.asp Accessed 25th June 2013Google Scholar
- Queensland Health Statewide Diabetes Clinical Network: High risk foot form (manual): Statewide Diabetic Foot Working Group. 2011, Brisbane: Queensland Health, http://www.health.qld.gov.au/caru/networks/diabetes.asp Accessed 25th June 2013Google Scholar
- Bowling FL, King L, Paterson JA, Hu J, Lipsky BA, Matthews DR, Boulton AJM: Remote assessment of diabetic foot ulcers using a novel wound imaging system. Wound Repair And Regeneration: Official Publication Of The Wound Healing Society [And] The European Tissue Repair Society. 2011, 19: 25-30. 10.1111/j.1524-475X.2010.00645.x.View ArticleGoogle Scholar
- Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
- Landis JR, Koch GG: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977, 33: 363-374. 10.2307/2529786.View ArticlePubMedGoogle Scholar
- Scottish Intercollegiate Guidelines Network (SIGN): Management of diabetes: a national clinical guideline. Edited by: Scotland NQI. 2010, Edinburgh: SIGNGoogle Scholar
- Scottish Intercollegiate Guidelines Network (SIGN): Management of diabetes: a national clinical guideline. 2010, Edinburgh: SIGNGoogle Scholar
- National Vascular Disease Prevention Alliance (NVDPA): Guidelines for the management of absolute cardiovascular disease risk. 2012, : National Stroke FoundationGoogle Scholar
- UK Prospective Diabetes Study (UKPDS) Group: Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998, 352: 837-853.View ArticleGoogle Scholar
- National Health & Medical Research Council (NHMRC) Guideline: National evidence based guideline for diagnosis, prevention and management of chronic kidney disease in type 2 diabetes. 2009, Canberra: Diabetes AustraliaGoogle Scholar
- Perkins BA, Olaleye D, Zinman B, Bril V: Simple screening tests for peripheral neuropathy in the diabetes clinic. Diabetes Care. 2001, 24: 250-256. 10.2337/diacare.24.2.250.View ArticlePubMedGoogle Scholar
- Smieja M, Hunt DL, Edelman D, Etchells E, Cornuz J, Simel DL: Clinical examination for the detection of protective sensation in the feet of diabetic patients. International Cooperative Group for Clinical Examination Research. Journal Of General Internal Medicine. 1999, 14: 418-424.View ArticleGoogle Scholar
- Romanos MT, Raspovic A, Perrin BM: The reliability of toe systolic pressure and the toe brachial index in patients with diabetes. J Foot Ankle Res. 2010, 3: 31-10.1186/1757-1146-3-31.View ArticlePubMedPubMed CentralGoogle Scholar
- Pérez-Martin A, Meyer G, Demattei C, Böge G, Laroche JP, Quéré I, Dauzat M: Validation of a fully automatic photoplethysmographic device for toe blood pressure measurement. Eur J Vasc Endovasc Surg: The Official J Eur Soc Vasc Surg. 2010, 40: 515-520. 10.1016/j.ejvs.2010.06.008.View ArticleGoogle Scholar
- Scanlon C, Park K, Mapletoft D, Begg L, Burns J: Interrater and intrarater reliability of photoplethysmography for measuring toe blood pressure and toe-brachial index in people with diabetes mellitus. J Foot Ankle Res. 2012, 5: 13-10.1186/1757-1146-5-13.View ArticlePubMedPubMed CentralGoogle Scholar
- Holland-Letz T, Endres HG, Biedermann S, Mahn M, Kunert J, Groh S, Pittrow D, von Bilderling P, Sternitzky R, Diehm C: Reproducibility and reliability of the ankle-brachial index as assessed by vascular experts, family physicians and nurses. Vasc Med (London, England). 2007, 12: 105-112. 10.1177/1358863X07077281.View ArticleGoogle Scholar
- Abbott CA, Carrington AL, Ashe H, Bath S, Every LC, Griffiths J, Hann AW, Hussein A, Jackson N, Johnson KE, et al: The North-West Diabetes Foot Care Study: incidence of, and risk factors for, new diabetic foot ulceration in a community-based patient cohort. Diabetic Med: J Br Diabetic Assoc. 2002, 19: 377-384. 10.1046/j.1464-5491.2002.00698.x.View ArticleGoogle Scholar
- Monteiro-Soares M, Boyko E, Ribeiro J, Ribeiro I, Dinis-Ribeiro M: Risk stratification systems for diabetic foot ulcers: a systematic review. Diabetologia. 2011, 54: 1190-1199. 10.1007/s00125-010-2030-3.View ArticlePubMedGoogle Scholar
- Peters EJG, Lavery LA: Effectiveness of the diabetic foot risk classification system of the International Working Group on the Diabetic Foot. Diabetes Care. 2001, 24: 1442-1447. 10.2337/diacare.24.8.1442.View ArticlePubMedGoogle Scholar
- Widatalla AH, Mahadi SEI, Shawer MA, Elsayem HA, Ahmed ME: Implementation of diabetic foot ulcer classification system for research purposes to predict lower extremity amputation. Int J Diabetes In Developing Countries. 2009, 29: 1-5. 10.4103/0973-3930.50707.View ArticleGoogle Scholar
- Schaper NC: Diabetic foot ulcer classification system for research purposes: a progress report on criteria for including patients in research studies. Diabetes/Metab Res Rev. 2004, 20 (Suppl 1): S90-S95.View ArticleGoogle Scholar
- Oyibo SO, Jude EB, Tarawneh I, Nguyen HC, Armstrong DG, Harkless LB, Boulton AJ: The effects of ulcer size and site, patient’s age, sex and type and duration of diabetes on the outcome of diabetic foot ulcers. Diabetic Med: J Br Diabetic Assoc. 2001, 18: 133-138. 10.1046/j.1464-5491.2001.00422.x.View ArticleGoogle Scholar
- Sheehan P, Jones P, Caselli A, Giurini JM, Veves A: Percent change in wound area of diabetic foot ulcers over a 4-week period is a robust predictor of complete healing in a 12-week prospective trial. Diabetes Care. 2003, 26: 1879-1882. 10.2337/diacare.26.6.1879.View ArticlePubMedGoogle Scholar
- Zimny S, Pfohl M: Healing times and prediction of wound healing in neuropathic diabetic foot ulcers: a prospective study. Experimental And Clinical Endocrinology & Diabetes: Official Journal, German Society Of Endocrinology [And] German Diabetes Association. 2005, 113: 90-93. 10.1055/s-2004-830537.View ArticleGoogle Scholar
- Cardinal M, Eisenbud DE, Phillips T, Harding K: Early healing rates and wound area measurements are reliable predictors of later complete wound closure. Wound Repair And Regeneration: Official Publication Of The Wound Healing Society [And] The European Tissue Repair Society. 2008, 16: 19-22. 10.1111/j.1524-475X.2007.00328.x.View ArticleGoogle Scholar
- Warriner RA, Snyder RJ, Cardinal MH: Differentiating diabetic foot ulcers that are unlikely to heal by 12 weeks following achieving 50% percent area reduction at 4 weeks. Int Wound J. 2011, 8: 632-637. 10.1111/j.1742-481X.2011.00860.x.View ArticlePubMedGoogle Scholar
- Lavery LA, Armstrong DG, Murdoch DP, Peters EJG, Lipsky BA: Validation of the Infectious Diseases Society of America’s diabetic foot infection classification system. Clinical Infectious Diseases: An Official Publication Of The Infectious Diseases Society Of America. 2007, 44: 562-565. 10.1086/511036.View ArticleGoogle Scholar
- Lipsky BA, Berendt AR, Cornia PB, Pile JC, Peters EJG, Armstrong DG, Deery HG, Embil JM, Joseph WS, Karchmer AW, et al: Executive summary: 2012 Infectious Diseases Society of America clinical practice guideline for the diagnosis and treatment of diabetic foot infections. Clinical Infectious Diseases: An Official Publication Of The Infectious Diseases Society Of America. 2012, 54: 1679-1684. 10.1093/cid/cis460.View ArticleGoogle Scholar
- Armstrong DG, Lavery LA, Harkless LB: Validation of a diabetic wound classification system. The contribution of depth, infection, and ischemia to risk of amputation. Diabetes Care. 1998, 21: 855-859. 10.2337/diacare.21.5.855.View ArticlePubMedGoogle Scholar
- Oyibo SO, Jude EB, Tarawneh I, Nguyen HC, Harkless LB, Boulton AJM: A comparison of two diabetic foot ulcer classification systems: the Wagner and the University of Texas wound classification systems. Diabetes Care. 2001, 24: 84-88. 10.2337/diacare.24.1.84.View ArticlePubMedGoogle Scholar
- Menz HB, Gill TK, Taylor AW, Hill CL: Predictors of podiatry utilisation in Australia: the North West Adelaide Health Study. J Foot Ankle Res. 2008, 1: 8-10.1186/1757-1146-1-8.View ArticlePubMedPubMed CentralGoogle Scholar
- Fletcher AK, Dolben J: A hospital survey of the care of elderly patients with diabetes mellitus. Age & Ageing. 1996, 25: 349-352. 10.1093/ageing/25.5.349.View ArticleGoogle Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
