Psychology dissertation quantitative bias analysis translates assumptions about systematic error into corrected estimates or ranges rather than leaving bias as a vague limitation. It can examine how exposure misclassification, outcome misclassification, selection, and uncontrolled confounding might change an observational result.
This guide explains deterministic and probabilistic approaches, bias parameters, plausible ranges, simulation, psychology-specific examples, software, interpretation, and transparent reporting. Quantitative bias analysis does not reveal the true answer. It shows what follows if clearly stated bias assumptions are approximately correct.
What psychology dissertation quantitative bias analysis entails
Conventional confidence intervals usually quantify random sampling variation under a fitted model. They do not automatically incorporate systematic errors caused by invalid measurement, non-participation, missing selection mechanisms, or omitted common causes. Quantitative bias analysis, often shortened to QBA, explicitly models one or more of those errors.
The method starts with an observed association and a bias model. Researchers specify parameters describing the bias, calculate the estimate that would be expected under those assumptions, and repeat the process across scenarios or probability distributions. The output reveals the direction and potential magnitude of change.
Phillips argued for quantifying uncertainty from systematic errors rather than discussing every limitation qualitatively. QBA supports that aim, but its output is only as credible as the design, data, assumptions, and evidence used for bias parameters.
| Approach | Bias inputs | Main output | Best use |
|---|---|---|---|
| Deterministic analysis | One value or a grid of values | Corrected estimate for each scenario | Transparent tipping points and scenario tables |
| Probabilistic analysis | Distributions for bias parameters | Distribution of corrected estimates | Propagating uncertainty across many plausible scenarios |
| Bounding analysis | Defensible parameter limits | Range compatible with the bounds | When exact distributions are difficult to justify |
| Multiple-bias analysis | Parameters for several linked errors | Jointly corrected estimates | Studies threatened by more than one important bias |
Table of Contents
Decide whether QBA matches the research question
QBA is most useful when a dissertation has a clearly defined estimand, a conventional estimate, a specific material bias, and enough external or internal evidence to specify plausible bias parameters. It is not a substitute for defining the exposure, outcome, target population, time zero, contrast, and effect measure.
The approach is especially relevant to observational psychology studies using self-report, administrative records, volunteer samples, retrospective exposure histories, proxy measures, or incomplete adjustment. It can also support experimental work affected by non-adherence, attrition, outcome misclassification, or differential missingness, provided the selected bias model fits the problem.
Do not perform QBA merely because software is available. First identify the bias pathway with a causal diagram and explain why it matters. The psychology dissertation limitations guide can help distinguish a material threat from a generic limitation.
Define the target estimate before correcting bias
State whether the target is a risk ratio, odds ratio, risk difference, mean difference, regression coefficient, prevalence ratio, or another measure. A correction method designed for a two-by-two risk ratio cannot be transferred automatically to a standardised mean difference or multilevel coefficient.
Define the population and analysis sample. A selection-bias analysis may target all eligible students, all invited students, respondents, or complete cases. These populations can yield different effects. The correction must correspond to the population named in the research question.
Record the unadjusted and adjusted conventional estimates with confidence intervals. The conventional model should already use a defensible measured confounder set, functional form, clustering structure, and missing-data strategy. QBA investigates remaining error; it should not conceal an avoidable modelling problem.
Choose the bias to model
Exposure or outcome misclassification
Misclassification QBA asks how imperfect classification changes an association. For a binary measure, common bias parameters are sensitivity and specificity. These may be non-differential or vary according to the other variable, recruitment group, assessor, language, or study wave.
For example, a dissertation may classify high social-anxiety symptoms using a brief screening threshold. The screen is not a clinical diagnosis. Its sensitivity and specificity might differ across language versions or exposure groups, creating outcome misclassification.
Selection bias
Selection QBA models how inclusion, participation, retention, or complete-case analysis depends jointly on exposure and outcome. Parameters may be stratum-specific probabilities of entering or remaining in the analysed sample. The model must reflect the actual selection mechanism rather than a generic attrition percentage.
A university wellbeing survey may have lower response among distressed students and among students with heavy paid-work commitments. If both variables influence participation, conditioning on response can distort their association. Use the selection bias guide to map the pathway before assigning probabilities.
Uncontrolled confounding
Confounding QBA represents an omitted or imperfectly measured common cause. Typical inputs include its prevalence across exposure groups and its association with the outcome after considering measured covariates. Parameters must correspond to a coherent scale and target effect.
A study of peer victimisation and later depressive symptoms might be concerned about family adversity that was not measured. Plausible prevalence differences and outcome associations could be drawn from a comparable cohort, a validation subsample, or carefully justified ranges.
Build defensible bias parameters
Bias parameters should come from the strongest relevant evidence available. Prioritise internal validation data collected in the same population and under the same procedures. Next consider external validation studies with comparable instruments, thresholds, languages, age groups, settings, and time periods.
If empirical evidence is limited, use structured expert elicitation or broad scenario ranges and label the source honestly. Do not present an arbitrary value as literature-based. Record every search, inclusion decision, conversion, and judgement used to create the range.
Account for correlation among parameters. Sensitivity and specificity may trade off because of a threshold. Selection probabilities across strata may share a common recruitment process. Drawing each value independently can generate combinations that no expert considers plausible.
| Bias | Example parameters | Potential evidence | Key caution |
|---|---|---|---|
| Binary exposure misclassification | Sensitivity and specificity by outcome status | Validation interview or device record | Differential error can change direction unpredictably |
| Binary outcome misclassification | Sensitivity and specificity by exposure group | Clinical assessment or adjudication sample | Screening accuracy may vary by setting or language |
| Selection bias | Selection probability in exposure-outcome strata | Recruitment log or follow-up records | Observed response rates do not identify every joint probability |
| Uncontrolled confounding | Confounder prevalence and outcome association | Comparable cohort or measured proxy | The omitted construct may be multidimensional |
Run a deterministic bias analysis
A deterministic analysis fixes the bias parameters at selected values. Begin with a base scenario, then vary one or more parameters across a prespecified grid. Calculate a corrected estimate for every combination and display the results in a table, contour plot, or line plot.
Include scenarios representing minimal, moderate, and severe bias, but tie those labels to evidence. A sensitivity of 0.80 is not universally moderate. Its plausibility depends on the instrument, respondent, threshold, population, and reference standard.
Deterministic analysis is easy to audit and useful for identifying tipping points. It does not provide a probability that a scenario is true. A wide grid also does not mean that every combination within it is equally plausible.
Run a probabilistic bias analysis
A probabilistic bias analysis assigns distributions to uncertain bias parameters. Each simulation draws a parameter set, applies the bias correction, and stores the corrected estimate. Repeating the process produces a distribution reflecting the modelled systematic uncertainty.
The method described by Fox and colleagues for probabilistic analysis of misclassified binary variables reconstructs data that might have been observed under specified sensitivity and specificity distributions. Simulation intervals can incorporate systematic and random error when the implementation samples both components appropriately.
Choose distributions based on plausible support and shape. Beta distributions fit probabilities between zero and one. Triangular or trapezoidal distributions can express a minimum, maximum, and most plausible region. Truncation should prevent impossible values without hiding substantial prior uncertainty.
Set and report the number of simulations, random seed, convergence or stability checks, failed or impossible draws, and correlation structure. Repeat the analysis with alternative distributions to determine whether conclusions depend on a particular parameterisation.
Do not confuse simulation intervals with confidence intervals
A conventional confidence interval describes sampling uncertainty under model assumptions. A simulation interval from probabilistic QBA describes the combined output of sampling variation and the chosen bias distributions only if both are incorporated. Some implementations report systematic-error distributions separately.
Name the interval precisely. Report its percentile limits, what sources of uncertainty it includes, and which it omits. Avoid stating that it contains the true effect with a fixed probability unless a coherent Bayesian interpretation and prior model justify that claim.
Gustafson’s review of probabilistic approaches to systematic error emphasises that standard analyses typically omit selection, measurement, and unobserved-confounding uncertainty. Adding a simulation does not eliminate uncertainty about the bias model itself.
A psychology dissertation misclassification example
Consider a hypothetical longitudinal study of frequent cyberbullying exposure and clinically elevated anxiety six months later. Exposure is self-reported with a short checklist. Anxiety is determined by a validated screening scale rather than a diagnostic interview. The adjusted conventional risk ratio is 1.50.
The researcher identifies validation evidence suggesting exposure sensitivity from 0.75 to 0.90 and specificity from 0.90 to 0.98. Outcome-screen sensitivity may range from 0.78 to 0.92 and specificity from 0.85 to 0.95. The evidence does not justify assuming identical accuracy across exposure groups.
A deterministic analysis first examines non-differential error, then plausible differential scenarios. A probabilistic analysis assigns supported distributions, preserves defensible correlations, adds random error, and repeats the correction. The dissertation reports the conventional estimate beside the corrected distribution rather than choosing only the most favourable result.
If many plausible differential-error scenarios move the association close to one, the original causal claim is fragile. If estimates remain elevated across credible inputs, confidence increases only with respect to the modelled misclassification. Confounding, selection, reverse causation, and model misspecification remain separate concerns.
A psychology dissertation selection example
Suppose a diary study examines irregular work schedules and daily emotional exhaustion. Participation at the final assessment is lower among students working the longest hours and among those reporting greater exhaustion. Complete-case analysis may therefore condition on a common effect of exposure and outcome.
The researcher uses recruitment and follow-up logs to estimate ranges for retention within exposure-outcome strata. A deterministic grid shows how the effect changes across plausible selection probabilities. A probabilistic model then represents uncertainty around those probabilities.
This analysis requires care because outcome status for non-respondents may be unknown. Auxiliary measures, earlier outcomes, or administrative follow-up can inform assumptions but may not identify them. Report which probabilities were observed, derived, or assumed.
Model uncontrolled confounding carefully
Specify one omitted construct or a well-defined composite. State its prevalence across exposure groups and its independent relationship with the outcome. Do not describe the parameter merely as “amount of confounding.” Readers need to understand the hypothetical variable and causal pathways.
Benchmark ranges against measured covariates on comparable scales. The confounding variables guide explains why colliders and mediators should not be treated as ordinary controls. A bias analysis based on the wrong causal role can be precisely wrong.
Compare QBA with the E-value analysis guide. An E-value gives a minimum joint strength needed to move a relative estimate to a target. QBA instead calculates corrected estimates under specified prevalence and association scenarios. Use the tool that answers the dissertation’s actual sensitivity question.
Combine multiple biases without hiding assumptions
Real studies may contain exposure misclassification, selection, and uncontrolled confounding simultaneously. Correcting one bias can reveal or amplify another. A multiple-bias model applies corrections in an order justified by the data-generating process and propagates uncertainty across them.
Complexity grows quickly. Start with separate one-bias analyses to understand direction and sensitivity. Add a joint model only when the parameters, dependencies, correction order, and software are defensible. Report both separate and combined results so readers can see which assumptions drive change.
Do not interpret a broad multiple-bias interval as failure. It may be the honest result of limited validation information. Conversely, a narrow interval can be misleading if the parameter distributions were unjustifiably restrictive.
Use software reproducibly
The current CRAN episensr package implements basic analyses for misclassification, selection bias, unmeasured confounding, probabilistic sensitivity, and multiple biases. Use the function matching the data structure and estimand, and read its documentation before interpreting output.
Record the package and software versions, code, seed, data table or model input, bias parameter sources, distributions, correlations, simulation count, and output processing. Validate a simple deterministic scenario by hand or with an independent implementation.
A systematic review of applied QBA found substantial variation in methods and reporting. Reproducible code is necessary but not sufficient. The dissertation must explain why its inputs and model are substantively plausible.
| Report element | Minimum information | Reason |
|---|---|---|
| Conventional result | Estimate, interval, population, and model | Provides the comparison point |
| Bias model | Target bias, equations or named method, and assumptions | Shows what was corrected |
| Parameters | Values or distributions, sources, and dependence | Allows plausibility assessment |
| Computation | Software, version, code, seed, draws, and diagnostics | Supports reproducibility |
| Results | Scenario table or distribution with clearly named intervals | Prevents selective emphasis |
| Interpretation | Biases addressed, omitted threats, and decision implications | Keeps conclusions proportionate |
Quality checks before interpretation
- Confirm that the bias model matches the causal diagram and target estimate.
- Verify that sensitivity, specificity, prevalence, and selection probabilities remain within valid ranges.
- Check that parameter definitions match the software documentation.
- Inspect impossible corrected cell counts or failed simulation draws.
- Assess whether distributions are too narrow, truncated, or unsupported.
- Repeat the analysis under alternative credible parameter sources.
- Separate deterministic tipping points from probabilistic summaries.
- Report results that strengthen and weaken the conventional conclusion.
- Retain other design, measurement, and modelling limitations.
Frequently asked questions
What is quantitative bias analysis?
It is a family of methods that calculates how an estimate changes under explicit assumptions about systematic errors such as misclassification, selection bias, or uncontrolled confounding.
Is QBA the same as an E-value?
No. An E-value is a threshold for unmeasured confounding on a relative scale. QBA can model detailed bias parameters and produce corrected estimates across specified scenarios.
What is the difference between deterministic and probabilistic QBA?
Deterministic QBA uses fixed parameter values or grids. Probabilistic QBA samples from distributions, producing a distribution of corrected estimates under the stated model.
Where should bias parameters come from?
Prefer internal validation data, followed by comparable external validation studies. Structured expert elicitation and broad sensitivity ranges are alternatives when evidence is limited.
Can QBA prove the corrected estimate is true?
No. The corrected estimate is conditional on the bias model and parameters. QBA clarifies consequences of assumptions rather than recovering truth automatically.
Can I model several biases together?
Yes, but correction order, dependence, and parameter uncertainty must follow a defensible data-generating process. Report separate analyses before or alongside the joint model.
Should every psychology dissertation use QBA?
No. Use it when a material bias can be parameterised credibly and the analysis changes how readers evaluate the result. A poorly justified simulation can obscure rather than clarify uncertainty.
Conclusion
Psychology dissertation quantitative bias analysis makes systematic-error assumptions visible and testable. Strong work defines the estimand, targets a specific bias, justifies every parameter, examines alternative scenarios, verifies computation, and interprets corrected estimates conditionally.
If you need methodological support, choose ethical guidance that helps you map the bias, locate validation evidence, verify code, and report uncertainty. Consultation should complement supervision, protect confidential data, and strengthen your independent reasoning rather than replace it.
