Skip to main content

Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

The Queensland high risk foot form (QHRFF) – is it a reliable and valid clinical research tool for foot disease?

Journal of Foot and Ankle Research volume 7, Article number: 7 (2014) Cite this article

Abstract

Background

Foot disease complications, such as foot ulcers and infection, contribute to considerable morbidity and mortality. These complications are typically precipitated by “high-risk factors”, such as peripheral neuropathy and peripheral arterial disease. High-risk factors are more prevalent in specific “at risk” populations such as diabetes, kidney disease and cardiovascular disease. To the best of the authors’ knowledge a tool capturing multiple high-risk factors and foot disease complications in multiple at risk populations has yet to be tested. This study aimed to develop and test the validity and reliability of a Queensland High Risk Foot Form (QHRFF) tool.

Methods

The study was conducted in two phases. Phase one developed a QHRFF using an existing diabetes foot disease tool, literature searches, stakeholder groups and expert panel. Phase two tested the QHRFF for validity and reliability. Four clinicians, representing different levels of expertise, were recruited to test validity and reliability. Three cohorts of patients were recruited; one tested criterion measure reliability (n = 32), another tested criterion validity and inter-rater reliability (n = 43), and another tested intra-rater reliability (n = 19). Validity was determined using sensitivity, specificity and positive predictive values (PPV). Reliability was determined using Kappa, weighted Kappa and intra-class correlation (ICC) statistics.

Results

A QHRFF tool containing 46 items across seven domains was developed. Criterion measure reliability of at least moderate categories of agreement (Kappa > 0.4; ICC > 0.75) was seen in 91% (29 of 32) tested items. Criterion validity of at least moderate categories (PPV > 0.7) was seen in 83% (60 of 72) tested items. Inter- and intra-rater reliability of at least moderate categories (Kappa > 0.4; ICC > 0.75) was seen in 88% (84 of 96) and 87% (20 of 23) tested items respectively.

Conclusions

The QHRFF had acceptable validity and reliability across the majority of items; particularly items identifying relevant co-morbidities, high-risk factors and foot disease complications. Recommendations have been made to improve or remove identified weaker items for future QHRFF versions. Overall, the QHRFF possesses suitable practicality, validity and reliability to assess and capture relevant foot disease items across multiple at risk populations.

Peer Review reports

Background

Foot disease contributes to considerable hospitalisation [1–4], amputation [5–8], institutionalisation [9, 10], and death [2, 11]; yet foot disease complications and these outcomes are largely preventable [5, 6, 12, 13]. “High risk factors” (such as peripheral neuropathy, peripheral arterial disease or foot deformity) significantly increase the risk of developing “foot disease complications” (such as foot ulcers, infection or ischaemia) [1, 14–16]. Diabetes populations are frequently acknowledged as the leading “at risk population” for foot disease due to the increased risk of developing high risk factors, and in turn foot disease complications, from diabetes [5–8, 17]. However, other chronic disease populations (such as chronic kidney disease [15, 16], cardiovascular disease [5, 6, 18, 19] and some cancers [5, 6, 8]) have now been shown to cause comparable rates of high risk factors and foot disease complications to diabetes, and thus, are also becoming more readily identified as “at risk populations” for foot disease.

Best practice foot disease management has been shown to significantly reduce hospitalisation, amputation, mortality and overall costs within different at risk populations [5, 12, 13, 20]. These multi-faceted best practice interventions commonly include screening for high-risk factors, multi-disciplinary management of foot disease complications, clinical training, evidence-based clinical pathway utilisation and regular capture and analysis of foot disease clinical data [5, 6, 12, 13, 20]. In consideration of the growing problem of foot disease, and the potential future improvements observed with best practice clinical management and research, it is imperative that any clinical tools to assess, capture, measure or analyse patient outcomes in at risk populations are valid and reliable.

There are a multitude of studies that have investigated a specific high risk foot factor or foot disease complication within multiple at risk populations [3, 5, 6, 21]. Furthermore, numerous studies have investigated multiple high risk factors and foot disease complications in specific at risk populations such as diabetes [2, 15, 17]. However, very few studies have investigated multiple high risk foot factors and foot disease complications in multiple at risk populations. This situation appears to be mirrored by the clinical tools available to measure and report on foot disease. For example many validated single-item tools are available which measure specific high-risk factors or foot disease complications within multiple at risk populations [22–24]; such as the ankle brachial index to capture and measure peripheral arterial disease within various at risk populations [22]. Furthermore, many validated multi-item tools measuring multiple high-risk factors and foot disease complications in specific at risk populations have been developed [15, 25–28]; such as the University of Texas Diabetic Foot Classification System to capture and measure multiple high risk factors and foot disease complications in people with diabetes only [25]. However, to the best of the authors’ knowledge a multi-item tool designed to measure multiple high-risk factors and foot disease complications in multiple at risk populations is yet to be developed and tested for validity and reliability.

Recently a Diabetic Foot Form (DFF) tool was developed to measure multiple high-risk factors and foot disease complications in people with diabetes only in diverse Queensland (Australia) settings [29]. The DFF was a multi-item tool developed from a number of existing clinical tools recommended in the literature to reliably capture various high-risk factors and foot disease complications in the diabetes population [29]. The implementation of the DFF in diverse Queensland clinical sites, in conjunction with other multi-faceted strategies, resulted in improved capture, measurement and management of high risk factors and foot disease complications and a corresponding reduction in outcomes such as hospitalisation and amputation [29, 30]. However, the tool was not tested for validity or reliability and was designed only to measure multiple high-risk factors and foot disease complications in the specific at risk population of people with diabetes.

In this study, we sought to modify the DFF tool to enable the measurement of multiple high risk factors and foot disease complications in multiple at risk populations. It was intended that the new multi-item tool would align with best practice principles for clinical tools including being easily interpreted, practical to use and possessing high face, content and criterion validity, and inter- and intra-rater reliability [31–33]. Thus, the aims of this study were firstly to develop a multi-item Queensland High Risk Foot Form (QHRFF) tool to capture multiple high risk factors and foot disease complications in multiple applicable at risk populations, and secondly, to investigate the validity and reliability of the tool’s individual items when used by clinicians representing different levels of foot disease expertise.

Methods

The study was conducted in two phases. Phase one involved development of the Queensland High Risk Foot Form (QHRFF) tool and phase two tested the validity and reliability of the QHRFF tool. Approval was granted from Institutional Ethics Committees and informed consent was obtained from all individual participants (patients and clinicians) for this study.

Phase one – development of the tool

Phase one primarily aimed to select items for the development of a QHRFF tool that were practical to collect in an Australian clinical setting, applicable to multiple 'at risk’ populations, and provided high face and content validity. The overarching procedures used for item selection included using the original DFF [29] as the starting tool to build the new QHRFF, searching the electronic literature for other recommended foot disease tools, establishing an expert panel to guide development and using several rounds of stakeholder consultation to refine the tool.

The original DFF contained 64 items pertaining to the construct of foot disease in diabetes populations [29] and was used as the starting tool to modify into the QHRFF. The DFF was originally developed using similar procedures to those used in this study and appeared to possess high practicality, face and content validity [29]. Furthermore, the DFF had been routinely used in over 25 High Risk Foot Service sites throughout Queensland for three years to collect standard clinical data on diabetes-related foot disease [30].

An electronic literature search was undertaken of relevant electronic databases including MEDLINE (all years to June 2011), CINAHL (all years to June 2011) and relevant professional websites to identify existing foot disease-related tools. The basic terms searched included tools to identify peripheral neuropathy, peripheral arterial disease, foot deformity, foot ulceration, foot infection and amputation or synonyms.

An expert panel was established from members of the Queensland Statewide Diabetes Foot Working Group; a working group of the Queensland Statewide Diabetes Clinical Network. The panel comprised 14 expert clinicians, with between five and 25 years of experience in the area of foot disease management, from the fields of endocrinology, vascular surgery, podiatry, nursing, public health, quality improvement and research. The panel had the responsibility to decide upon the final items for the QHRFF tool after each round of consultation.

Refinements of the tool were achieved through numerous rounds of consultation and trialling of draft QHRFF versions with external stakeholders in relation to practicality, face and content validity. Stakeholders included up to 200 multidisciplinary professionals from sites registered to use the existing DFF, members of the Queensland Health Statewide Diabetes Clinical Network, Statewide Renal Clinical Network and Statewide Podiatry Network.

At the conclusion of phase one the expert panel and aforementioned network’s management committees determined, via consensus endorsement, that each item of the QHRFF possessed high practicality, face and content validity, and was applicable across multiple 'at risk’ populations (particularly diabetes, cardiovascular disease and chronic kidney disease) in Australian clinical settings. See Phase One Results for further details regarding the final QHRFF.

Phase two – validity and reliability

The final endorsed QHRFF tool developed in phase one was then tested for validity and reliability. The general procedure for testing involved using podiatrists with different levels of foot disease expertise to assess patients with different levels and severity of high risk factors and foot disease. The authors decided to test only QHRFF items that directly related to the foot disease construct, thus, excluding items such as patients’ name, and facility. To test the validity of each item a general criterion measure (the agreement between two 'experts’) was initially assessed for reliability. Each item was then tested, using the podiatrists with different experience levels, against the criterion measure to evaluate magnitudes of concurrent criterion validity. The inter- and intra-rater reliability was also tested for the level of agreement on each item by podiatrists with different levels of experience.

Setting and participants

The testing was conducted within the High Risk Foot Services (HRFS), Metro North Hospital and Health Service, Brisbane, Australia. Eligible clinician participants were recruited from podiatrists practicing a minimum of one session per week in a HRFS. Four podiatrists were chosen as representative of the majority of clinicians managing foot disease within Queensland Health, and thus, potentially using the QHRFF in future. For the purposes of the study, levels of expertise were categorised using the Queensland Health 'Health Practitioner award’ [34]. Thus, expert clinicians were defined as either a 'consultant clinician’ (level 6) or 'specialist clinician’ (level 5) [34] and working in an acute hospital setting, plus, a member of the expert panel to ensure they understood the original intended construct for the QHRFF tool. General clinicians were defined as a 'senior clinician’ (level 4) or 'clinician’ (level 3) [34], working in a community setting, and thought to be representative of the general podiatry clinical workforce managing foot disease in Queensland. One of each level was recruited; one level 6, one level 5, one level 4 and one level 3 (however, the level 3 had been acting in a level 4 position at the time of the study). Written informed consent was obtained from all participants (patients and clinicians) prior to commencement of their study participation.

Eligible patient participants were consecutively recruited from patients already attending Community HRFS clinics for the care of high risk factors or foot disease complications; defined as a previous or current foot ulcer. Exclusion criteria included patients with a cognitive deficit, signs or symptoms of a systemic infection, younger than 18 years of age, or unwilling to provide written consent to participate. The authors considered that patients with previous or current foot ulcers would ensure that the majority of high risk factors and foot disease items had the realistic possibility of being present or absent, and thus, could be suitably tested on each participant. Furthermore, it was thought this population should possess the moderate prevalence rates, yet unpredictable mixes and severities of each item, of different high risk factors and foot disease complications, that are suggested in the literature to improve statistical robustness for validity and reliability studies [31].

Three different patient cohorts were used; one cohort to test the reliability of the criterion measures (agreement between 'experts’) (n = 32), another cohort to simultaneously test the criterion validity (an 'expert’ diagnosis against general clinicians) and inter-rater reliability (n = 43), and the last cohort to test the intra-rater reliability of a general clinician (n = 19). The recruitment of consecutive community patients did mean that patients may have been familiar to the level 3 or level 4 clinicians; however, not to the expert 'gold standard’ clinicians working in the hospital setting. To minimise the risk of patients being familiar to the level 3 or level 4 clinicians’ seven different Community HRFS clinics were used to recruit patient participants. The level 3 and level 4 clinicians had only worked at two of the seven clinics recruiting patients.

Procedures

A designated research assistant coordinated all procedures. Training of each clinician consisted of being provided with a QHRFF manual that gave a definition of each item [35], a 1–2 hour training session on instructions and tips to use the QHRFF tool, and each was encouraged to trial the tool on their existing patients and clarify any queries with the research assistant prior to testing.

The general assessment procedure for each validity or reliability test consisted of patients having their feet examined by at least two different clinicians, blinded from each other’s assessment, within the one clinical visit (validity and inter-rater reliability). However, to ensure all clinicians had an equal opportunity to determine the patient’s debridement and wound management needs, all were permitted to visually inspect the patient’s feet together for up to five minutes prior to any clinician ratings. In this initial inspection period, the clinicians were instructed to only visually inspect the need for debridement and previous wound dressings whilst not conversing with or touching the patient or each other. The order of clinician assessment after this inspection period was then at the discretion of the research assistant based on clinician availability. The first clinician would have the additional task of debriding the wound or callus if they deemed necessary and the last clinician the additional task of completing any clinical management.

The research assistant ensured all clinicians were blinded from each other’s assessments in separate rooms and that all examination records were de-identified. Each clinician conducted the assessment of patient’s feet using the QHRFF to record their assessment and management recommendations. Demographic, medical history and medication information were available from the patients’ medical records or by direct communication with the patients themselves. To minimise assessment bias all historical foot-related records (including progress notes, DFF tools or QHRFFs tools) were removed from the medical record prior to each clinician’s assessment.

Criterion measure

The criterion measure was tested on the first cohort of patients. The criterion measure (i.e. 'gold standard’ diagnosis) for each QHRFF item in this study was defined by the authors as the diagnosis made by an expert clinician. Expert clinicians were pragmatically chosen as a general criterion measure for all items, instead of using multiple resource and time intensive individual gold standard-recognised criterion measures (such as nerve conduction studies for neuropathy), due to the resource and time constraints of the study. Best practice dictates that any criterion measure should be reliable, free from bias and measure the same item as the new tool [31]. Thus, the study’s criterion measure, of an expert clinician, required testing to determine its reliability, independence (free from bias) and applicability, prior to its use as a 'gold standard’ criterion measure in the criterion-related validity tests. The authors tested the criterion measure by testing the reliability of the agreement between two independent (blinded from one another) experts (one level 6 and one level 5) to determine the magnitude of reliability of agreement on their 'gold standard’ diagnoses for each item on the same patient cohort. The “general assessment procedure” as described above was utilised for each patient. Once the criterion measure for each item was determined to be reliable, the criterion validity of each QHRFF item was tested.

Criterion validity

The criterion validity and inter-rated reliability was then tested simultaneously on a second cohort of patients. Criterion validity was tested using the concurrent criterion validity method by comparing the reliable criterion measure (one of the expert clinicians) against each of the two representative 'general’ clinicians (one level 3 and one level 4). Thus, two separate criterion assessments were carried out for each item; one using the level 3 clinician against the criterion measure, and another the level 4 clinician against the criterion measure. Again the “general assessment procedure” as described above was utilised for each patient when testing for criterion validity.

Inter-rater reliability

Inter-rater reliability tests were performed simultaneously with the criterion validity tests on the same second cohort of patients and were tested using the expert clinician (level 5), senior clinician (level 4) and clinician (level 3). Thus, three inter-rater reliability measures of agreements were carried out for each item; one testing the agreement between the level 5 and level 4 clinician, a second between the level 5 and level 3 clinician, and a third between the level 4 and level 3 clinician. “General assessment procedures” as described above were again utilised for each patient when testing for inter-rater reliability.

Intra-rater reliability

Intra-rater reliability was tested on a third cohort of patients. The clinician with the least expertise (level 3) was used to test intra-rater reliability as it was hypothesised that the clinician with the least expertise would have the most variability of the tested clinicians. The two different time points to test intra-rater reliability were between one and four weeks apart. This time period was chosen as it was considered that a minimum of one week (of full clinical load) would be necessary to adequately reduce recall bias in a clinician, plus, a maximum of four weeks would not be sufficient time for the majority of items to markedly change and thus items would remain stable between ratings. Any items that did not fit this criteria were excluded from intra-rater testing. Furthermore, this time period aligned with any necessary follow up treatment time period for patients, and thus, was convenient to patient participants. To control for any potential changes in foot ulcer characteristics over time (for example ulcer combined surface area and clinical signs of infection) digital images were used. Digital photographic images taken of foot ulcers at the first rating were used at the second rating to standardise the foot ulcer characteristics across two time points [36]. Images were taken perpendicular and 30-60 cm away from ulcers at the first rating, and incorporated two calibrated measures across the length and width of the ulcer in the image. These images were reviewed by the clinician at the second rating to determine the foot ulcer characteristics only.

Statistical analysis

All data was analysed using SPSS 19.0 for Windows (SPSS Inc., Chicago, IL, USA) or GraphPad Software. Descriptive statistics were used to display the single demographic, medical co-morbidity, high-risk factors and foot disease variables for each cohort of patient participants; using means and standard deviations (SD) for continuous variables (which were normally distributed) or proportions for categorical variables. Each above descriptive cohort variable was collected from the clinician with most expertise or on the first rating of the intra-rater reliability cohort. Chi-squared test of independence and ANOVA were used to test for differences between the three cohorts’ characteristics. A significance level of p < 0.05 was used throughout.

All QHRFF items were tested for validity and reliability; except those stated items considered not to impact on the foot disease construct. Criterion validity was tested by calculating sensitivity, specificity and positive predictive values. Ordinal variables were collapsed into dichotomous data to enable calculations of sensitivity, specificity and positive predictive values. All measures of agreements between clinicians were tested using Kappa (K) for dichotomous variables, weighted Kappa (wK) for ordinal variables and intra-class correlation (ICCs) (model 2, 1) for continuous variables [31, 37, 38]. Kappa and wK value (Standard Errors (SE)) strengths were categorised as: no agreement < 0; slight agreement = 0 – 0.20; fair agreement = 0.21 - 0.40; moderate agreement = 0.41 – 0.6; substantial agreement = 0.61 – 0.8; and near perfect agreement = 0.81 – 1.0 [31, 37, 38]. ICC (SD) strengths were categorised as: weak-moderate agreement < 0.75 and strong agreement > 0.75 [31].

Results

Phase one – development of the tool

The literature search identified 174 papers that reported on tools to identify or measure foot disease complications and/or high-risk factors, with the majority of papers specifically relating to diabetes populations (90 (52%)). Tools reported in other populations were in conditions commonly identified in the literature to be associated with lower limb amputation [5–8]. These included cardiovascular disease (peripheral arterial disease), chronic kidney disease, malignancy, infection and other neurological conditions [5–8]. No tool was identified that was specifically designed to identify multiple high-risk factors and foot disease complications in multiple at risk populations.

Fifty-nine individual tools were identified from the literature search; 23 were considered to have adequate practicality, face validity and applicability to an Australian clinical context to be considered for the QHRFF. These tools and the original DFF tool [29] were considered by the expert panel for QHRFF item selection. At this point, the expert panel determined that the tool should be divided into a clinical assessment record section that informed a separate data collection section, and thus, only the data collection section would require testing.

Overall 87 items were initially identified from the considered tools. At the conclusion of phase one procedures the final endorsed QHRFF data collection tool was made up of 46-items (excluding general identification items) covering seven domains (Figure 1). The seven content domains included identifying general demographics, different health professionals attending, medical co-morbidity history, high-risk factor history, clinical diagnosis of high-risk factors, clinical diagnoses of foot disease complications, and clinical management principles performed. A QHRFF manual was developed to provide definitions and instructions on each item contained in the tool [35]. Table 1 outlines the tools or literature used to support each QHRFF item. At the conclusion of phase one, the expert panel concluded that the QHRFF tool’s items had the required high clinical practicality, applicability to multiple at risk populations, and good face and content validity. Subsequently, the tool was endorsed for use by the Queensland Health Statewide Diabetes, Renal and Podiatry Clinical Networks. Thus, the tool was ready for validity and reliability testing.

Figure 1
figure1

Queensland high risk foot data collection form (QHRFF).

Full size image
Table 1 Supporting tools or literature for QHRFF items
Full size table

Phase two – validity and reliability

Forty items were tested for validity and reliability unless otherwise stated. The items not tested were those considered not to impact on the foot disease construct; i.e. 'facility’, 'visit type’, 'todays visit to HRFS’, 'separation status’, and 'date of referral’. Table 2 displays the general demographic, medical co-morbidity history, high-risk factor history, clinical diagnoses of high-risk factors and foot disease complications variable prevalence for the three patient cohorts used. No significant differences were noted for these variables, except for diabetes duration and any other co-morbidity (p < 0.05). All cohorts contained variables with moderate prevalence rates (> 15%) [31], except for different health professionals attending previously, co-morbidity of end stage renal failure (ESRF), acute Charcot and ischaemic ulcers.

Table 2 Demographic, co-morbidity and high-risk foot complication data for each patient cohort (n (%) unless otherwise stated)
Full size table

Criterion measure

Table 3 displays the reliability results for the criterion measure of expert clinicians. Thirty-two items were able to be statistically tested. Nine items (28%) recorded near perfect categories of agreement, nine (28%) substantial/strong categories, eleven (34%) moderate categories and three (9%) weak/fair categories. Thus, overall 29 (91%) of these items recorded at least a moderate category of reliability (K > 0.4; ICC > 0.75). The items recording weaker categories of reliability included other (non-listed) condition, University of Texas Wound Classification System (UTWCS) grade and optimum footwear.

Table 3 Criterion measure reliability – measure of agreement between gold standard experts
Full size table

Criterion validity

Tables 4 and 5 display the criterion validity results for both the senior (level 4) clinician and clinician (level 3) respectively tested against the criterion measure for each item. Thirty-six items were able to be statistically tested for sensitivity, specificity or positive predictive values (PPV) on both clinicians. Thus, 72 different tests were each performed for sensitivity, specificity and PPV. Sixty-one (85%), 59 (82%) and 60 (83%) items recorded at least moderate categories (> 0.7) for sensitivity, specificity and PPV respectively. Conversely, three (8%), five (14%) and four (11%) out of the 36 items recorded weak categories (< 0.7) for sensitivity, specificity and PPV respectively on both clinicians tested. The items registering weak categories of validity (sensitivity, specificity or PPV on both clinicians) included identifying a patient that had attended a GP, physician or surgeon; chronic kidney disease (CKD) or other (non-listed) condition; previous, current and new foot ulcer all had poor specificity in particular; and optimum footwear had poor PPVs.

Table 4 Criterion validity summary statistics for senior clinician (Sensitivity, specificity and positive predictive value) & inter-rater reliability between expert and senior clinicians (Kappa and strength of agreement)
Full size table
Table 5 Criterion validity summary statistics for clinician (Sensitivity, specificity and positive predictive value) & inter-rater reliability between expert and clinician (Kappa and strength of agreement)
Full size table

Inter-rater reliability

Tables 4, 5 and 6 display the inter-rater reliability results for all three tests. Ninety-six different tests were able to be statistically tested for inter-rater reliability across the three different pairs of clinician agreement. Twenty four items (25%) recorded near perfect categories of agreement, 43 (45%) substantial/strong categories, 17 (18%) moderate categories and 12 (12%) weak/fair categories. Thus, overall 84 (88%) of these items recorded at least moderate categories of reliability (K > 0.4; ICC > 0.75). The items recording weak categories of reliability across two or more agreements tested included other (non-listed) condition, previous foot ulcer and optimum footwear.

Table 6 Reliability measure of agreement summary statistics: Inter-rater (Senior clinician & clinician) & intra-rater (Clinician)
Full size table

Intra-rater reliability

Table 6 also displays the intra-rater reliability results for the level 3 clinician. The median (interquartile range) period between the first and second ratings for the intra-rater reliability testing was 2(1–2) weeks. Twenty-three items were able to be statistically tested. Six items (26%) recorded near perfect categories of agreement, ten (43%) substantial/strong categories, four (17%) moderate categories and three (13%) weak/fair categories. Thus, overall 20 (87%) of these items recorded at least a moderate category of reliability (K > 0.4; ICC > 0.75). Those items scoring weak categories of agreement were foot deformity, optimal offloading and optimal footwear.

Discussion

The QHHRF appears to be the first multi-item tool developed and tested to identify multiple high-risk factors and foot disease complications in multiple at risk populations. Our findings indicate that the majority of the tool’s items demonstrates at least moderate categories of validity (face, content and criterion validity) and reliability (inter-rater and intra-rater); particularly those in the domains of identifying relevant medical co-morbidity history, and clinical diagnoses of high-risk factors and foot disease complications. However, some items in the domains of identifying different health professionals previously attending the patient and general clinical management principles performed appear to have weaker categories of validity and reliability that need addressing in future versions of the tool.

The QHRFF went through a number of developmental, validity and reliability tests to determine its status as a valid and reliable tool. The magnitude of validity (or accuracy) in this study was evaluated via the methods of face, content and importantly criterion validity. Face and content validity are considered the least robust of the validity tests due to their inherent subjectivity [31, 33]. However, they are considered important factors in the development phase to ensure the tool can actually measure the general identified construct [31, 33]; in this case foot disease. At the completion of phase one, the expert panel and network stakeholders endorsements implied high practicality, face and content validity.

Criterion validity is considered to be the most objective validity test [31, 33]. To test concurrent criterion validity, as used by this study, a satisfactory criterion measure must be used. In this study the criterion measure to test criterion validity was an expert’s clinical diagnosis. Overall, the 'expert’ criterion measure used for this study were rated as having at least moderate categories of reliability for all but three items; any other (non-listed) co-morbidity, optimum footwear and UTWCS grade. The other (non-listed) co-morbidity and optimum footwear items rated in the weak categories in most other inter- or intra-rater reliability agreement tested and should be reviewed or removed in future versions of the QHRFF. The weak-moderate agreement for the UTWCS grade was unexpected as this tool has been validated extensively in the past [62], however, all other inter- or intra-rater agreements on the UTWCS were rated as substantial/strong categories of reliability. Thus, it is recommended that the UTWCS grade be retained in future with more emphasis applied on the UTWCS grading system in the training sessions provided. It is certainly possible that the criterion measure for each item may have been more robust if the individual gold standard test for each individual item were used; for example an angiogram to diagnose peripheral arterial disease [22] or nerve conduction studies to diagnose neuropathy [44]. However, such an approach would have been particularly resource and time intensive and was thus not practical within the resources available to this study.

The QHRFF items were tested for concurrent criterion validity by using two different blinded representative general clinicians’ ratings compared to the criterion measure (a reliable expert’s diagnosis) on the same patients. Positive predictive values from this testing indicated the majority of QHRFF items had at least moderate validity when used by clinicians with different levels of expertise. The high positive predictive values suggest that the proportion of people with a positive test result for individual QHRFF items actually had the relevant medical co-morbidity, high-risk factor or foot disease complication of interest. Although there were a few notable exceptions such as identifying different health professionals attending previously, recent blood glucose levels (BGLs) > 15 mmol/L, ESRF, optimal offloading and optimum footwear. However, overall the QHRFF tool appears to demonstrate acceptable validity in the majority of its items to be considered a valid tool to test the foot disease construct in multiple at risk populations.

Reliability (or consistency) is a major prerequisite of any useful items to be measured [31, 33]. In this study inter-rater and intra-rater measures of agreement were used for reliability. Inter-rater reliability was primarily assessed using three clinicians with different representative levels of expertise in managing foot disease. Overall, all items consistently displayed at least moderate categories of reliability, except for any other (non-listed) co-morbidity, optimum footwear and previous foot ulcer. Optimum footwear was also identified to have weak categories of intra-rater reliability; along with identifying foot deformity and optimum offloading. Thus, the QHRFF tool appears to demonstrate satisfactory reliability to collect the majority of items in the foot disease construct. However, the authors recognise that it would have been preferable to test reliability with more clinicians across a larger sample of patients, yet this was beyond the resourcing available for this study.

Notwithstanding, the aforementioned methodological limitations, the study does incorporate many best practice research design methods for testing validity and reliability. These methods included testing the reliability of the criterion measure to diagnose and measure each item, testing the subsequent concurrent criterion validity of representative clinicians, and testing the reliability of the tool when used by clinicians with different levels of expertise. Furthermore, the study employed patient samples that were highly representative of the demographics and co-morbidity prevalence rates found in multiple 'at risk’ populations for high risk factors and foot disease complications. These high prevalence rates not only suggest the tool was tested in the construct it was designed to measure, but literature suggests higher prevalence rates improve the statistical robustness of validity and reliability results [31]. Lastly, many existing validated single-item tools [23, 46, 60, 62] were incorporated within the QHRFF tool and this potentially adds weight to the validity reported in this study. Thus, overall the QHRFF tool appears to be a valid and reliable tool to collect the vast majority of items contained within the foot disease construct and can be used as a general tool to complement existing specific tools in the field of foot disease.

Limitations

Several limitations have already been outlined in this study including the use of a general criterion measure of an experts’ clinical diagnoses for each item, only using a limited number of representative clinicians and testing intra-rater reliability on one clinician in a small sample. Other limitations include: not performing a systematic literature review (and thus some evidence based items and tools may have been overlooked); using only podiatrists as the clinical raters; using historically defined strength categories for validity and reliability; not testing the tool for construct validity, predictive validity, or factor analysis; and the settings used were existing HRFS only. It is recommended that any future research into this tool should address these methodological limitations by investigating the factor analysis, construct and predictive criterion validity of the tool’s items in a much larger and broader sample of patients with more multi-disciplinary clinician raters. Furthermore, if resources permit, the use of individual gold standard criterion measures and/or existing concurrent validated single-item tools should be considered.

However, there are a number of recommendations from this study’s findings that should be considered for implementation in future versions of the QHRFF tool. Firstly, those items reporting weaker categories of validity or reliability and that potentially collect duplicate information to other more reliable items should be removed; including any other (non-listed) co-morbidity, combined surface area (change), recent BGLs greater than 15 mmol/L and new ulcer. Secondly, some items’ definitions should be reviewed in an attempt to improve the item’s future reliability and validity. For example, previous foot ulcer definition could be modified to identify a previously “healed” foot ulcer, 'health professionals attending’ previously requires an exact retrospective time period that patients were in attendance in alignment with similar literature (such as “in the previous two weeks”) [64, 65] and optimum offloading should have a more explicit definition referring directly to non-removable offloading definitions in the existing literature [14]. Thirdly, the validation of criteria for identifying a foot deformity and optimum footwear are urgently required for these 'at risk’ populations. To the best of the author’s knowledge such tools for these items have been developed [14, 50] yet are to be validated and this study was no exception. However, in the meantime the authors recommend adding the numerical foot deformity score to the QHRFF to improve accuracy and test for validity. Lastly, a systematic literature review for other reported independent associates for foot disease (such as cancer, arthritis, depression, trauma, vision impairment, mobility impairment and social determinant factors) and other outcome measures for foot disease (such as quality of life measures) should be performed to ensure all 'at risk’ populations and outcome measures for foot disease are identified, considered and potentially included in the next QHRFF version.

Conclusions

The QHRFF tool appears to be the first multi-item tool developed and tested to identify multiple high-risk factors and foot disease complications in multiple at risk populations. Acceptable validity and reliability were demonstrated for the majority of items; particularly in the domains of identifying relevant medical co-morbidity history and clinically diagnosing high-risk factors and foot disease complications. However, recommendations to remove, add or redefine items registering weak validity or reliability scores should be implemented to improve future versions of the QHRFF tool; particularly in the domains of health professionals attending and clinical management principles. Overall, the QHRFF tool appears to demonstrate the practicality, validity and reliability required to facilitate robust clinical assessment and data capture to measure the large burden of foot disease facing our multiple at risk populations.

Abbreviations

ANOVA:

Analysis of variance

BGL:

Blood glucose levels

CINAHL:

Cumulative index to nursing and allied health literature

CKD:

Chronic kidney disease

DFF:

Diabetic foot form

ESRF:

End stage renal failure

GP:

General practitioner

HRFS:

High risk foot service

ICC:

Intra-class correlation

K:

Kappa

wK:

Weighted Kappa

MEDLINE:

Medical literature analysis and retrieval system online

PPV:

Positive predictive value

QHRFF:

Queensland high risk foot form

SD:

Standard deviation

SE:

Standard error

UTWCS:

University of Texas wound classification system.

References

  1. 1.

    Bakker K, Apelqvist J, Schaper NC: Practical guidelines on the management and prevention of the diabetic foot 2011. Diabetes/Metab Res Rev. 2012, 28 (Suppl 1): 225-231.

  2. 2.

    Australian Institute of Health & Welfare (AIHW): Diabetes: Australian facts 2008. 2008, Canberra: Australian Government, http://www.aihw.gov.au/publications/cvd/daf08/daf08.pdf Accessed 25th June 2013

  3. 3.

    Barrois B, Labalette C, Rousseau P, Corbin A, Colin D, Allaert F, Saumet JL: A national prevalence study of pressure ulcers in French hospital inpatients. J Wound Care. 2008, 17: 373-

  4. 4.

    Lazzarini PA, Gurr JM, Rogers JR, Schox A, Bergin SM: Diabetes foot disease: the Cinderella of Australian diabetes management?. J Foot Ankle Res. 2012, 5: 24-10.1186/1757-1146-5-24.

  5. 5.

    Moxey PW, Gogalniceanu P, Hinchliffe RJ, Loftus IM, Jones KJ, Thompson MM, Holt PJ: Lower extremity amputations–a review of global variability in incidence. Diabetic Med: J Br Diabetic Assoc. 2011, 28: 1144-1153. 10.1111/j.1464-5491.2011.03279.x.

  6. 6.

    Ephraim PL, Dillingham TR, Sector M, Pezzin LE, MacKenzie EJ: Epidemiology of limb loss and congenital limb deficiency: a review of the literature. Arch Phys Med Rehabil. 2003, 84: 747-761.

  7. 7.

    Global Lower Extremity Amputation Study Group: Epidemiology of lower extremity amputation in centres in Europe, North America and East Asia. Br J Surg. 2000, 87: 328-337. 10.1046/j.1365-2168.2000.01344.x.

  8. 8.

    Lazzarini PA, O’Rourke SR, Russell AW, Clark D, Kuys SS: What are the key conditions associated with lower limb amputations in a major Australian teaching hospital?. J Foot Ankle Res. 2012, 5: 12-10.1186/1757-1146-5-12.

  9. 9.

    Lavery LA, Houtum WHV, Armstrong DG: Institutionalization following diabetes-related lower extremity amputation. Am J Med. 1997, 103: 383-388. 10.1016/S0002-9343(97)00163-0.

  10. 10.

    Dillingham TR, Yacub JN, Pezzin LE: Determinants of postacute care discharge destination after dysvascular lower limb amputation. PM&R. 2011, 3: 336-344.

  11. 11.

    Armstrong DG, Wrobel J, Robbins JM: Guest editorial: are diabetes-related wounds and amputations worse than cancer?. Int Wound J. 2007, 4: 286-287. 10.1111/j.1742-481X.2007.00392.x.

  12. 12.

    Canavan RJ, Unwin NC, Kelly WF, Connolly VM: Diabetes- and nondiabetes-related lower extremity amputation incidence before and after the introduction of better organized diabetes foot care: continuous longitudinal monitoring using a standard method. Diabetes Care. 2008, 31: 459-463.

  13. 13.

    Krishnan S, Nash F, Baker N, Fowler D, Rayman G: Reduction in diabetic amputations over 11 years in a defined U.K. population: benefits of multidisciplinary team work and continuous prospective audit. Diabetes Care. 2008, 31: 99-101.

  14. 14.

    National Health & Medical Research Council (NHMRC) Guidelines: National evidence-based guideline on prevention, identification and management of foot complications in diabetes (Part of the guidelines on management of type 2 diabetes). 2011, Melbourne: Baker IDI Heart & Diabetes Institute

  15. 15.

    Kaminski M, Frescos N, Tucker S: Prevalence of risk factors for foot ulceration in patients with end-stage renal disease on haemodialysis. Intern Med J. 2012, 42: e120-e128. 10.1111/j.1445-5994.2011.02605.x.

  16. 16.

    Freeman A, May K, Frescos N, Wraight PR: Frequency of risk factors for foot ulceration in individuals with chronic kidney disease. Intern Med J. 2008, 38: 314-320. 10.1111/j.1445-5994.2007.01528.x.

  17. 17.

    Tapp RJ, Shaw JE, Courten MPD, Dunstan DW, Welborn TA, Zimmet PZ: Foot complications in type 2 diabetes: an Australian population‒based study. Diabet Med. 2003, 20: 105-113. 10.1046/j.1464-5491.2003.00881.x.

  18. 18.

    Reed T, Veith FJ, Gargiulo NJ, Timaran CH, Ohki T, Lipsitz EC, Malas MB, Wain RA, Suggs WD: System to decrease length of stay for vascular surgery. J Vasc Surg. 2004, 39: 395-399. 10.1016/j.jvs.2003.09.015.

  19. 19.

    Lynch EA, Hillier SL, Stiller K, Campanella RR, Fisher PH: Sensory retraining of the lower limb after acute stroke: a randomized controlled pilot trial. Arch Phys Med Rehabil. 2007, 88: 1101-1107. 10.1016/j.apmr.2007.06.010.

  20. 20.

    Jeffcoate WJ, van Houtum WH: Amputation as a marker of the quality of foot care in diabetes. Diabetologia. 2004, 47: 2051-2058. 10.1007/s00125-004-1584-3.

  21. 21.

    American College of Cardiology Foundation/American Heart Association (ACCF/AHA): ACCF/AHA focused update of the guideline for the management of patients with peripheral artery disease (Updating the 2005 guideline): a report of the American College of Cardiology Foundation/American Heart Association Task Force on practice guidelines. Circulation. 2011, 124: 2020-2045. 10.1161/CIR.0b013e31822e80c3.

  22. 22.

    Caruana MF, Bradbury AW, Adam DJ: The validity, reliability, reproducibility and extended utility of ankle to brachial pressure index in current vascular surgical practice. Eur J Vasc Endovasc Surg: The Official J Eur Soc Vasc Surg. 2005, 29: 443-451. 10.1016/j.ejvs.2005.01.015.

  23. 23.

    Young D, Schuerman S, Flynn K, Hartig K, Moss D, Altenburger B: Reliability and responsiveness of an 18 site, 10-g monofilament examination for assessment of protective foot sensation. J Geriatr Phys Ther (2001). 2011, 34: 95-98. 10.1519/JPT.0b013e31820aabe5.

  24. 24.

    Stausberg J, Lehmann N, Kröger K, Maier I, Niebel W: Reliability and validity of pressure ulcer diagnosis and grading: an image-based survey. Int J Nurs Stud. 2007, 44: 1316-1323. 10.1016/j.ijnurstu.2006.06.006.

  25. 25.

    Armstrong DG, Lavery LA, Harkless LB: Treatment-based classification system for assessment and care of diabetic feet. Ostomy Wound Manage. 1996, 42: 50-

  26. 26.

    Ince P, Abbas ZG, Lutale JK, Basit A, Ali SM, Chohan F, Morbach S, Mollenberg J, Game FL, Jeffcoate WJ: Use of the SINBAD classification system and score in comparing outcome of foot ulcer management on three continents. Diabetes Care. 2008, 31: 964-967. 10.2337/dc07-2367.

  27. 27.

    Lipsky BA, Weigelt JA, Sun X, Johannes RS, Derby KG, Tabak YP: Developing and validating a risk score for lower-extremity amputation in patients hospitalized for a diabetic foot infection. Diabetes Care. 2011, 34: 1695-1700. 10.2337/dc11-0331.

  28. 28.

    Bergin SM, Brand CA, Colman PG, Campbell DA: A questionnaire for determining prevalence of diabetes related foot disease (Q-DFD): construction and validation. J Foot Ankle Res. 2009, 2: 34-10.1186/1757-1146-2-34.

  29. 29.

    Lazzarini PA, O¿Rourke SR, Russell AW, Derhy PH, Kamp MC: Standardising practices improves clinical diabetic foot management: the Queensland Diabetic Foot Innovation Project, 2006–09. Aust Health Rev. 2012, 36: 8-15. 10.1071/AH10978.

  30. 30.

    Lazzarini PA, O’Rourke SR, Russell AW, Derhy PH, Kamp MC: Standardising practices improves ambulatory diabetic foot management and reduces amputations: the Queensland Diabetic Foot Innovation Project, 2006–2009. J Foot Ankle Res. 2011, 4: O25-10.1186/1757-1146-4-S1-O25.

  31. 31.

    Portney LG, Watkins MP: Foundations of clinical research applications to practice. 2009, New Jersey, USA: Pearson Prentice, 3

  32. 32.

    Terwee CB, Mokkink LB, Knol DL, Ostelo RWJG, Bouter LM, de Vet HCW: Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res: An Int J Qual Life Aspects Of Treatment, Care And Rehabil. 2012, 21: 651-657. 10.1007/s11136-011-9960-1.

  33. 33.

    Trevethan R: Self-assessment of foot health: requirements, issues, practicalities, and challenges. J Am Podiatr Med Assoc. 2009, 99: 460-471.

  34. 34.

    Queensland Industrial Relations Commission: Health Practitioners (Queensland health) Certified Agreement (No. 2). 2011, Brisbane: Queensland Government, http://www.health.qld.gov.au/eb/hp.asp Accessed 25th June 2013

  35. 35.

    Queensland Health Statewide Diabetes Clinical Network: High risk foot form (manual): Statewide Diabetic Foot Working Group. 2011, Brisbane: Queensland Health, http://www.health.qld.gov.au/caru/networks/diabetes.asp Accessed 25th June 2013

  36. 36.

    Bowling FL, King L, Paterson JA, Hu J, Lipsky BA, Matthews DR, Boulton AJM: Remote assessment of diabetic foot ulcers using a novel wound imaging system. Wound Repair And Regeneration: Official Publication Of The Wound Healing Society [And] The European Tissue Repair Society. 2011, 19: 25-30. 10.1111/j.1524-475X.2010.00645.x.

  37. 37.

    Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.

  38. 38.

    Landis JR, Koch GG: An application of hierarchical kappa-type statistics in the assessment of majority agreement among multiple observers. Biometrics. 1977, 33: 363-374. 10.2307/2529786.

  39. 39.

    Scottish Intercollegiate Guidelines Network (SIGN): Management of diabetes: a national clinical guideline. Edited by: Scotland NQI. 2010, Edinburgh: SIGN

  40. 40.

    Scottish Intercollegiate Guidelines Network (SIGN): Management of diabetes: a national clinical guideline. 2010, Edinburgh: SIGN

  41. 41.

    National Vascular Disease Prevention Alliance (NVDPA): Guidelines for the management of absolute cardiovascular disease risk. 2012, : National Stroke Foundation

  42. 42.

    UK Prospective Diabetes Study (UKPDS) Group: Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998, 352: 837-853.

  43. 43.

    National Health & Medical Research Council (NHMRC) Guideline: National evidence based guideline for diagnosis, prevention and management of chronic kidney disease in type 2 diabetes. 2009, Canberra: Diabetes Australia

  44. 44.

    Perkins BA, Olaleye D, Zinman B, Bril V: Simple screening tests for peripheral neuropathy in the diabetes clinic. Diabetes Care. 2001, 24: 250-256. 10.2337/diacare.24.2.250.

  45. 45.

    Smieja M, Hunt DL, Edelman D, Etchells E, Cornuz J, Simel DL: Clinical examination for the detection of protective sensation in the feet of diabetic patients. International Cooperative Group for Clinical Examination Research. Journal Of General Internal Medicine. 1999, 14: 418-424.

  46. 46.

    Romanos MT, Raspovic A, Perrin BM: The reliability of toe systolic pressure and the toe brachial index in patients with diabetes. J Foot Ankle Res. 2010, 3: 31-10.1186/1757-1146-3-31.

  47. 47.

    Pérez-Martin A, Meyer G, Demattei C, Böge G, Laroche JP, Quéré I, Dauzat M: Validation of a fully automatic photoplethysmographic device for toe blood pressure measurement. Eur J Vasc Endovasc Surg: The Official J Eur Soc Vasc Surg. 2010, 40: 515-520. 10.1016/j.ejvs.2010.06.008.

  48. 48.

    Scanlon C, Park K, Mapletoft D, Begg L, Burns J: Interrater and intrarater reliability of photoplethysmography for measuring toe blood pressure and toe-brachial index in people with diabetes mellitus. J Foot Ankle Res. 2012, 5: 13-10.1186/1757-1146-5-13.

  49. 49.

    Holland-Letz T, Endres HG, Biedermann S, Mahn M, Kunert J, Groh S, Pittrow D, von Bilderling P, Sternitzky R, Diehm C: Reproducibility and reliability of the ankle-brachial index as assessed by vascular experts, family physicians and nurses. Vasc Med (London, England). 2007, 12: 105-112. 10.1177/1358863X07077281.

  50. 50.

    Abbott CA, Carrington AL, Ashe H, Bath S, Every LC, Griffiths J, Hann AW, Hussein A, Jackson N, Johnson KE, et al: The North-West Diabetes Foot Care Study: incidence of, and risk factors for, new diabetic foot ulceration in a community-based patient cohort. Diabetic Med: J Br Diabetic Assoc. 2002, 19: 377-384. 10.1046/j.1464-5491.2002.00698.x.

  51. 51.

    Monteiro-Soares M, Boyko E, Ribeiro J, Ribeiro I, Dinis-Ribeiro M: Risk stratification systems for diabetic foot ulcers: a systematic review. Diabetologia. 2011, 54: 1190-1199. 10.1007/s00125-010-2030-3.

  52. 52.

    Peters EJG, Lavery LA: Effectiveness of the diabetic foot risk classification system of the International Working Group on the Diabetic Foot. Diabetes Care. 2001, 24: 1442-1447. 10.2337/diacare.24.8.1442.

  53. 53.

    Widatalla AH, Mahadi SEI, Shawer MA, Elsayem HA, Ahmed ME: Implementation of diabetic foot ulcer classification system for research purposes to predict lower extremity amputation. Int J Diabetes In Developing Countries. 2009, 29: 1-5. 10.4103/0973-3930.50707.

  54. 54.

    Schaper NC: Diabetic foot ulcer classification system for research purposes: a progress report on criteria for including patients in research studies. Diabetes/Metab Res Rev. 2004, 20 (Suppl 1): S90-S95.

  55. 55.

    Oyibo SO, Jude EB, Tarawneh I, Nguyen HC, Armstrong DG, Harkless LB, Boulton AJ: The effects of ulcer size and site, patient’s age, sex and type and duration of diabetes on the outcome of diabetic foot ulcers. Diabetic Med: J Br Diabetic Assoc. 2001, 18: 133-138. 10.1046/j.1464-5491.2001.00422.x.

  56. 56.

    Sheehan P, Jones P, Caselli A, Giurini JM, Veves A: Percent change in wound area of diabetic foot ulcers over a 4-week period is a robust predictor of complete healing in a 12-week prospective trial. Diabetes Care. 2003, 26: 1879-1882. 10.2337/diacare.26.6.1879.

  57. 57.

    Zimny S, Pfohl M: Healing times and prediction of wound healing in neuropathic diabetic foot ulcers: a prospective study. Experimental And Clinical Endocrinology & Diabetes: Official Journal, German Society Of Endocrinology [And] German Diabetes Association. 2005, 113: 90-93. 10.1055/s-2004-830537.

  58. 58.

    Cardinal M, Eisenbud DE, Phillips T, Harding K: Early healing rates and wound area measurements are reliable predictors of later complete wound closure. Wound Repair And Regeneration: Official Publication Of The Wound Healing Society [And] The European Tissue Repair Society. 2008, 16: 19-22. 10.1111/j.1524-475X.2007.00328.x.

  59. 59.

    Warriner RA, Snyder RJ, Cardinal MH: Differentiating diabetic foot ulcers that are unlikely to heal by 12 weeks following achieving 50% percent area reduction at 4 weeks. Int Wound J. 2011, 8: 632-637. 10.1111/j.1742-481X.2011.00860.x.

  60. 60.

    Lavery LA, Armstrong DG, Murdoch DP, Peters EJG, Lipsky BA: Validation of the Infectious Diseases Society of America’s diabetic foot infection classification system. Clinical Infectious Diseases: An Official Publication Of The Infectious Diseases Society Of America. 2007, 44: 562-565. 10.1086/511036.

  61. 61.

    Lipsky BA, Berendt AR, Cornia PB, Pile JC, Peters EJG, Armstrong DG, Deery HG, Embil JM, Joseph WS, Karchmer AW, et al: Executive summary: 2012 Infectious Diseases Society of America clinical practice guideline for the diagnosis and treatment of diabetic foot infections. Clinical Infectious Diseases: An Official Publication Of The Infectious Diseases Society Of America. 2012, 54: 1679-1684. 10.1093/cid/cis460.

  62. 62.

    Armstrong DG, Lavery LA, Harkless LB: Validation of a diabetic wound classification system. The contribution of depth, infection, and ischemia to risk of amputation. Diabetes Care. 1998, 21: 855-859. 10.2337/diacare.21.5.855.

  63. 63.

    Oyibo SO, Jude EB, Tarawneh I, Nguyen HC, Harkless LB, Boulton AJM: A comparison of two diabetic foot ulcer classification systems: the Wagner and the University of Texas wound classification systems. Diabetes Care. 2001, 24: 84-88. 10.2337/diacare.24.1.84.

  64. 64.

    Menz HB, Gill TK, Taylor AW, Hill CL: Predictors of podiatry utilisation in Australia: the North West Adelaide Health Study. J Foot Ankle Res. 2008, 1: 8-10.1186/1757-1146-1-8.

  65. 65.

    Fletcher AK, Dolben J: A hospital survey of the care of elderly patients with diabetes mellitus. Age & Ageing. 1996, 25: 349-352. 10.1093/ageing/25.5.349.

Download references

Acknowledgements

The authors wish to acknowledge the significant contribution made to the project by Ms Helen Martin, Ms Kimberley Canning, Ms Helen Sheahan and the entire Metro North Hospital and Health Service Podiatry Department. Furthermore, the authors wish to acknowledge the significant support of the Queensland Health Statewide Diabetes Clinical Network, Diabetes Foot Working Group, Renal Clinical Network and Podiatry Network. The authors are also grateful to the grant funding for the project provided by the Metro North Hospital & Health Service Clinical Practice Incentive Payment Grants. None of these contributors have a relevant conflict of interest. Lastly, the authors wish to acknowledge Dr Sheree Nix for reviewing final drafts of the manuscript.

Author information

Affiliations

  1. Allied Health Research Collaborative, Metro North Hospital & Health Service, Queensland Health, Rode Road, Chermside, Brisbane, QLD, 4032, Australia
    • Peter A Lazzarini
    • , Vanessa Ng
    •  & Suzanne S Kuys
  2. Department of Podiatry, Metro North Hospital & Health Service, Queensland Health, Brisbane, Australia
    • Peter A Lazzarini
    • , Vanessa Ng
    •  & Ewan M Kinnear
  3. School of Clinical Sciences, Queensland University of Technology, Brisbane, Australia
    • Peter A Lazzarini
    •  & Lloyd F Reed
  4. School of Medicine, The University of Queensland, Brisbane, Australia
    • Maarten C Kamp
  5. Department of Endocrinology, Metro North Hospital & Health Service, Queensland Health, Brisbane, Australia
    • Maarten C Kamp
  6. Centre for Musculoskeletal Research, Griffith Health Institute, Griffith University, Gold Coast, Australia
    • Suzanne S Kuys
  7. Clinical Epidemiology Unit, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand
    • Cameron Hurst
  8. Institute of Health and Biomedical Innovation, Queensland University of Technology, Brisbane, Australia
    • Peter A Lazzarini
    •  & Lloyd F Reed
Authors
  1. Search for Peter A Lazzarini in:
  2. Search for Vanessa Ng in:
  3. Search for Ewan M Kinnear in:
  4. Search for Maarten C Kamp in:
  5. Search for Suzanne S Kuys in:
  6. Search for Cameron Hurst in:
  7. Search for Lloyd F Reed in:

Corresponding author

Correspondence to Peter A Lazzarini.

Additional information

Competing interests

The authors have no relevant conflict of interest to disclose.

Authors’ contributions

PAL conceived, designed, researched data, contributed to discussion, wrote and reviewed/edited the manuscript. VN provided research assistance, researched data, contributed to discussion and reviewed/edited the manuscript. EMK conceived, contributed to discussion and reviewed/edited the manuscript. MSK, SSK and LR designed, contributed to discussion, and reviewed/edited the manuscript. CH designed, researched data and reviewed/edited the manuscript. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Lazzarini, P.A., Ng, V., Kinnear, E.M. et al. The Queensland high risk foot form (QHRFF) – is it a reliable and valid clinical research tool for foot disease?. J Foot Ankle Res 7, 7 (2014). https://doi.org/10.1186/1757-1146-7-7

Download citation

Keywords

  • Foot
  • Disease
  • Tool
  • Valid
  • Reliable

Journal of Foot and Ankle Research

ISSN: 1757-1146