Appraise
4. Step 1: Determine the level of evidence
4.1. What is the study type (or design)?
4.1 What is the study type (or design)?
When reading a paper, it is important to determine what type of study was conducted so that you can establish whether the study type is appropriate to help answer your question.
This is important because different study types are more (or less) appropriate to answer different question types. This will be covered in more detail later on, in Is the study design appropriate to answer your question?
In order to decide on the study type, you will need to look at the methods section of the paper. The author may state which study type is used, but sometimes careful reading may contradict this.
A brief description of the common study types is outlined under 'Study types and descriptions' below (adapted from Dean, 2013). There is also information on identifying study types in the RCVS Knowledge EBVM Toolkit 4 – What type of study is it? .
Study types and descriptions
Adapted from Dean (2013)
Evidence syntheses: Studies that summarise evidence
A systematic review is a defined and rigorous method of appraising, collating and summarising the information from published papers addressing a specific question. The methods used to search the literature, assess the quality, and make conclusions are explicitly stated in the methods section.
A meta-analysis is a quantitative statistical analysis (generally) conducted as part of a systematic review. The results of different clinical trials relating to a specific question are statistically analysed and summarised. By combining the data, a meta-analysis provides more robust evidence than each individual study is able to on its own.
Also referred to as Knowledge Summary, critically appraised topic (CAT), research synthesis or BestBET. An evidence summary is a standardised summary of research evidence based on a clinical question generated from a specific patient situation or problem, producing a clinical conclusion, or summary.
Intervention (experimental) studies: The researcher designs an intervention (e.g. treatment, drug therapy, surgical method, etc.)
A randomised controlled trial is an intervention study used to assess treatments or other interventions. Study subjects are randomly allocated to either the intervention group or a control group (which receives either no treatment, a placebo, the current best treatment or a comparator). As allocation of subjects is performed randomly, all other characteristics of the population should be equally distributed across the groups, thus decreasing bias. Therefore, evidence of a cause–effect relationship is more credible in these types of studies. Ideally, the study should be ‘blinded’, so that anyone involved with assessing study outcomes does not know which treatment each animal received, in order to limit conscious or unconscious bias.
Observational studies: The researcher has no influence on which animals get the intervention; they only 'observe'
A cohort study is an observational study where exposed and unexposed groups (cohorts) are followed over a defined period of time and occurrence of the outcome of interest (e.g. disease) is measured. Cohort studies can identify risk factors associated with the outcome and estimate incidence.
A case-control study is a retrospective study (occasionally prospective) comparing animals with the disease (cases) and without the disease (controls) of interest. The animals’ histories are examined to identify risk factors for the disease.
A cross-sectional study looks at a sample of the population at a single point in time, most commonly to determine the prevalence of a certain disease.
A study comparing a group of animals before and after an event, or intervention. The effect of the event, or intervention, can then be identified by comparing the data sets.
A diagnostic test validation study is used to establish the usefulness of new diagnostic tests. Animals are tested using the new diagnostic test and the current gold standard to establish the sensitivity, specificity and likelihood ratios for the new diagnostic test.
Descriptive (non-comparative) studies: Description of what is happening – case findings, report a rare occurrence, etc.
Descriptive studies cannot be used to measure risk, causation, treatment effects or prevalence of a disease.
A case series is a description of the presentation, diagnosis, treatment and/or outcome of a group of animals with the same disease. There are no disease-free animals for comparison, and any differences in management are not randomly allocated (for example, they may be due to the owners’ preferences or different protocols between centres).
A case report is a description of a single case (or small number of cases).
Expert opinion can be one individual’s opinion or part of an elicitation process based on a panel of experts used to answer a question of interest. Expert opinion may provide some evidence where no information is available (e.g. new treatment efficacy or application to a new population). However these will almost always include some form of bias, for example, the selection of the evidence included.