Assess
6.4 Defining audit criteria/standards
The terms ‘criteria’ and ‘standards’ often lead to confusion as these terms have been used differently by various professional groups and writers across healthcare, and are frequently used interchangeably.
Audit criteria are clearly defined, measurable, explicit statements, which are used to assess the quality of care.
For criteria to be valid and lead to improvements in care, they need to be:
- evidence-based
- related to important aspects of care
- measurable (NICE, 2002)
The National Institute of Health and Care Excellence recommends using high quality clinical practice guidelines to develop audit criteria (NICE, 2002). There are several different appraisal tools available to help you evaluate the quality of clinical guidelines (for example, the AGREE II checklist developed by the AGREE Collaboration; see also Siering et al., 2013). These appraisal tools can be utilised to determine whether or not guidelines represent a suitable source for deriving your audit criteria. However, published clinical guidelines are scarce in veterinary medicine, with professional consensus statements offering the closest available option in many cases.
If clinical guidelines or up-to-date systematic reviews are not available, a literature review may be carried out to identify (Acquire) the best (Appraise) and most up-to-date evidence from which audit criteria may be generated.
You may already have evidence-based clinical protocols or guidelines for your practice, which you can use to define ‘local’ audit criteria. Where there are no known or available criteria, one other option is to compare your audit data to historical clinical records over time.
Each audit criterion should have a performance level or target assigned to it (usually expressed as a percentage). Again, some overlap and confusion exists between different publications and guidance about clinical audit – some sources use the term ‘standard’ to define the performance level or target for expected compliance.
A structure audit evaluating practice waiting times:
|
Criterion |
Target/performance level |
|
All patients should be seen within 15 minutes of appointment time |
90% |
A process audit of equine laminitis:
|
Criterion |
Target/performance level |
|
All equids diagnosed with acute laminitis, with no clinical signs of systemic illness (such as systemic inflammatory response syndrome (SIRS)), should receive a follow-up appointment for endocrine laboratory testing |
100% |
An outcome audit of perioperative hypothermia management:
|
Criterion |
Target/performance level |
|
All dogs with postoperative hypothermia should have rectal temperature measured hourly during recovery until a temperature of ≥37·5°C is reached (adapted from Rose et al., 2016b) |
100% |
In many cases, we would aim to achieve 100% compliance with our evidence-based ‘best practice’, as set out by our audit criteria. However in practice, performance levels are a compromise between clinical importance, practicability and acceptability and for various reasons, it may not always be possible to meet 100% compliance.
Where you have set your target performance level based on published literature, you should note that levels of performance achieved in trials or research studies are helpful, but they often include very well defined study populations and should not be regarded as uniformly achievable in unselected patient populations in a practice setting. Clinical practice benchmarking can also be used to set and maintain target levels of performance.
There may be justifiable reasons why some cases might not comply with a specific audit criterion, and these cases should not be included in your audit data analysis. These exceptions should be defined along with your audit criteria, prior to data collection.
Example Scenario
Small animal dental imaging
Although published practice guidelines emphasise the importance of dental health in canine and feline quality of life (Bellows et al., 2019), Tom was unable to identify evidence to help him set the criteria and target performance level for his audit. The practice team agree on a criterion of owner-reported improvement following dental treatment. Tom found a systematic review of oral health-related quality of life following dental treatment under general anaesthesia in children (Knapp et al., 2017). Considering this to be a useful proxy for owner assessment of quality of life in his patients, Tom uses data from studies included in this review for his audit. Tom sets a target performance level of 65% of dental cases being reported to have improved quality of life following treatment, based on 63% of children having met or exceeded a minimal clinically important difference in quality of life following dental treatment (Knapp et al., 2017).
Key point:
In the absence of species-specific evidence to set his target performance level, Tom utilised comparative information from a systematic review of a similar area in human medicine.
Example Scenario
Completion of financial consent forms
The criteria for Matthew’s audit were defined by local consensus as:
1) the percentage of financial consent forms with the estimate filled in
2) the percentage of financial consent forms with the owner or owner representative’s signature
Key point:
Matthew’s audit team agree that the target performance level for each process criterion should be set at 100% based on guidance provided by the RCVS that informed consent and documentation of consent is essential. The team agreed that emergency cases brought in by a transporter only, with no owner or representative present, would be excluded from the clinical audit (exceptions to the audit criteria).