Psychology Dissertation Selection Bias
psychology dissertation selection bias occurs when entering, remaining in, or being analysed in a study depends on factors that distort the relationship under investigation. It can arise before recruitment, during consent, through attrition, or when researchers restrict analysis to complete cases, service users, survivors, or another selected group.
This guide explains how to recognise selection mechanisms, distinguish selection bias from confounding and ordinary sampling limitations, reduce avoidable bias through design, use causal diagrams to evaluate conditioning, analyse missing participation cautiously, and report what the final sample can support. The examples apply to survey, cohort, experimental, clinical, educational, workplace, and secondary-data psychology dissertations.
What Psychology Dissertation Selection Bias means in psychology research
Selection bias is not simply the fact that a sample differs from a national population. The key question is whether the process determining who contributes data creates a misleading exposure-outcome association or a misleading estimate for the target population. A volunteer sample can limit generalisability without necessarily biasing every association. Conversely, a large sample can still produce a biased estimate when participation depends jointly on variables related to the exposure and outcome.
A useful modern distinction separates bias in estimating an effect within the source population from bias in transporting that effect to a wider target population. The methodological resource on types of bias in causal diagrams shows why selection, confounding, measurement, and reporting problems should not be collapsed into one vague statement that a sample is “unrepresentative.”
Define the target before diagnosing bias
State the exposure or comparison, outcome, source population, eligible population, analysed population, and estimand. For example, “the association between weekly employment hours and end-of-term burnout among full-time postgraduate students enrolled at three universities” is more diagnosable than “work and student wellbeing.”
Then ask what population the conclusion is meant to describe. If the dissertation targets only enrolled full-time students, excluding part-time students may be a scope decision rather than bias. If the conclusion is later extended to all postgraduate students, that exclusion becomes relevant to external validity.
Table of Contents
Selection bias, sampling bias, confounding, and missing data
| Problem | Central question | Typical psychology example | Primary response |
|---|---|---|---|
| Selection bias | Did inclusion or analysis condition on a process that distorts the focal relationship? | Only counselling users are analysed when distress and service access both affect attendance | Map selection causes, redesign where possible, and use justified adjustment or sensitivity analysis |
| Sampling limitation | Does the achieved sample cover the population to which results are applied? | A university volunteer sample is used to discuss all working adults | Narrow the target, improve recruitment, or qualify transportability |
| Confounding | Does a common cause influence both exposure and outcome? | Prior depression affects social media use and later symptoms | Measure and adjust for a defensible confounder set |
| Missing data | Why are particular values absent? | Participants skip trauma items while completing the rest of the survey | Investigate the missingness process and use an appropriate method |
| Measurement bias | Are variables measured differently across groups or conditions? | Interviewers probe one treatment group more intensively | Standardise measurement and evaluate validity |
These problems can coexist. Attrition produces missing outcome data and may also create selection bias if remaining under observation depends on causes of both the exposure and outcome. Review the site’s guides to psychology dissertation sampling, missing data, and confounding variables as connected but distinct decisions.
How conditioning creates selection bias
Selection can be represented by a variable indicating whether a person enters the study, responds, remains at follow-up, has a complete record, or is retained for analysis. Conditioning occurs when the study design or analysis restricts attention to one level of that variable. Regression adjustment, stratification, matching, and complete-case analysis can also condition on variables.
The collider mechanism
A collider is a common effect of two variables. Suppose distress increases the likelihood of seeking counselling, while access to insurance also increases attendance. Among counselling attendees, distress and insurance can become associated even if they are unrelated in the source population. Restricting the study to attendees conditions on counselling use, the collider.
Elwert and Winship’s review of endogenous selection bias explains how conditioning on common outcomes can generate misleading associations even when no observations are visibly deleted during analysis. Selection may be produced by design restrictions, data availability, adjustment, or stratification.
Not every conditioned variable is a collider, and conditioning on a collider does not guarantee severe bias in every dataset. The size and direction depend on the causal structure and strength of relevant relationships. An open-access study of collider scope shows why the magnitude can vary substantially. Avoid claiming that any selected sample is automatically unusable.
Use a selection diagram
Draw a directed acyclic graph with the exposure, outcome, their plausible causes, and a selection node. Add arrows into selection from factors that influence recruitment, consent, retention, record completeness, or inclusion in the analytic dataset. DAGitty can help draw and inspect the graph, although substantive knowledge must determine the arrows.
For a longitudinal study of baseline loneliness and later depression, both worsening symptoms and relocation may affect follow-up. Relocation may also relate to employment instability, which affects loneliness and depression. The diagram should represent these mechanisms rather than using a generic arrow labelled “dropout.”
Where selection bias enters a dissertation
Eligibility and exclusion criteria
Eligibility rules define the study population and can improve safety, measurement consistency, and feasibility. They become problematic when justified after observing results, when they remove participants based on post-exposure variables, or when conclusions extend beyond the eligible group. The inclusion and exclusion criteria guide explains how to define rules before recruitment and record their consequences.
For example, excluding participants who start psychological treatment during a stress study may condition on a response to worsening symptoms. Treatment initiation can be affected by earlier stress and predict later outcomes. Removing these participants can change the meaning of the comparison.
Recruitment and self-selection
People decide whether to notice an invitation, trust the research team, meet the practical demands, and consent. Topic salience, available time, internet access, literacy, incentives, perceived stigma, and current symptoms can all influence participation. Recruitment through a single online community may select people who are engaged with the topic in ways relevant to the outcome.
Do not describe self-selection only as “participants may have been more interested.” Specify why interest may relate to the variables studied. In a body-image survey advertised as a study of appearance concerns, people with unusually high concern may be especially likely to participate. Neutral but honest recruitment wording and multiple recruitment routes can reduce avoidable topic-driven selection.
Nonresponse after contact
A sampling frame does not remove selection bias if response depends on relevant characteristics. Record how many people were approached, opened the invitation, consented, began the study, and provided analysable data when those figures are available. Compare respondents and nonrespondents on ethical, legitimately available frame variables without seeking protected information merely for diagnostics.
Attrition and loss to follow-up
Attrition is central to a longitudinal dissertation. Dropout that differs only by a variable unrelated to the focal association may mainly reduce precision. Dropout related to exposure, outcome, their causes, or their consequences can change the composition of the retained group and distort estimates.
Imagine an intervention study in which participants with little early improvement are more likely to stop responding, especially when assigned to a demanding programme. An analysis limited to completers may overstate benefit. Retention procedures, reasons for withdrawal, intention-to-treat principles where relevant, missing-data methods, and sensitivity analyses should be planned before results are known.
Complete records and administrative datasets
Secondary datasets include only people who interact with the systems that generate records. A clinical database reflects help-seeking, referral, access, documentation, and service retention. An employee wellbeing dataset may contain only current staff and miss those who left because of adverse conditions. Restricting analysis to records with every field completed can add another selection stage.
Describe the data-generating process in the secondary data analysis methods. “A large real-world dataset” is not evidence that selection is negligible.
Post-randomisation selection
Random assignment protects the initial treatment comparison in expectation, but selecting participants after allocation can undermine that protection. Per-protocol samples, attendance thresholds, adverse-event exclusions, and available-case analyses may compare groups whose post-randomisation behaviour reflects treatment and prognosis.
State whether the target is the effect of assignment, treatment received, adherence, or another estimand. These questions require different assumptions. Do not rename a completer analysis “intention to treat.”
Psychology-specific examples
| Research question | Selection mechanism | Possible distortion | Design response |
|---|---|---|---|
| Does social anxiety predict help-seeking? | Recruitment occurs only through counselling services | Service access and symptom severity both affect inclusion | Recruit across service and community settings, or narrow the target |
| Does remote work affect burnout? | Survey includes current employees only | Workers who left because of burnout are absent | Include former employees where feasible and discuss survivor selection |
| Does a mindfulness programme reduce distress? | Only session completers are analysed | Persistence may reflect early benefit, motivation, and baseline distress | Preserve randomised groups and analyse attrition explicitly |
| Is screen time associated with sleep? | Parents opt children into a device-tracking study | Privacy attitudes and concern about use affect participation | Use broad recruitment, compare participation stages, and qualify scope |
| Does trauma exposure predict memory performance? | Participants with incomplete trauma items are excluded | Disclosure willingness may relate to trauma severity and cognition | Protect privacy, assess item missingness, and run sensitivity analyses |
Prevent selection bias through design
Map the participant pathway
Create a flow map from the target population to the analysed dataset. Include identification, invitation, eligibility screening, consent, baseline completion, allocation if applicable, follow-up, exclusions, and final analysis. At each transition, list plausible causes of moving forward or leaving.
Use recruitment that matches the target
Choose channels that reach relevant subgroups, provide accessible formats, translate materials appropriately, minimise unnecessary burden, and make incentives proportionate. Monitor recruitment across prespecified characteristics when ethically permitted. Quotas can improve coverage but do not automatically eliminate bias within quota groups.
Separate safety exclusions from convenience exclusions
Safety, consent capacity, and task validity can justify exclusions. Convenience alone needs scrutiny. If the study requires a laptop only because the survey was designed poorly for mobile devices, the device restriction may exclude participants by income or location. Pilot the participant journey and remove barriers that do not serve the research question.
Plan retention ethically
Use realistic follow-up schedules, concise measures, clear reminders, flexible appointment options, accessible contact methods, and transparent compensation. Collect only necessary contact data, store identifiers separately, and honour withdrawal. Retention goals do not override voluntary participation.
Analyse selection without promising a complete repair
No single statistical procedure makes an unrepresentative or selectively observed sample unbiased. Analysis should follow a stated causal model and acknowledge unsupported assumptions.
Describe every selection stage
Report counts and reasons where possible. Compare characteristics across invited, enrolled, retained, and analysed groups using variables collected legitimately. These comparisons can reveal patterns but cannot prove the absence of bias, especially for unmeasured factors.
Use weighting when assumptions are plausible
Inverse probability of selection or censoring weights can create a pseudo-population in which observed selection predictors are balanced. The method requires adequate measurement of selection causes, correct models, positivity, and stable weights. Examine extreme weights and report diagnostics. A small dissertation sample may not support a complex selection model.
Use multiple imputation for missing values, not as a slogan
Multiple imputation can reduce bias and preserve uncertainty when its model includes variables related to missingness and the analysis. It does not recover people who were never approached or variables never measured, and it does not guarantee protection under nonignorable missingness.
Run targeted sensitivity analyses
Compare primary estimates under alternative defensible inclusion rules, complete-case and imputed analyses, different weight specifications, or plausible assumptions about outcomes among those lost. Explain what each analysis tests. A collection of arbitrary models is not a sensitivity strategy.
Avoid the Heckman reflex
Specialised selection models can be valuable in suitable econometric or latent-variable settings, but they require strong identification and distributional assumptions. Do not apply a named correction because software offers it. Explain why the method matches the selection process and what identifies the model.
A practical selection-bias workflow
- Define the exposure, outcome, estimand, source population, and target population.
- Draw the path from potential eligibility to the analysed dataset.
- List causes of invitation, eligibility, consent, completion, retention, and analysis.
- Add selection nodes and relevant causes to a causal diagram.
- Identify design changes that remove unnecessary selection stages.
- Prespecify exclusions, retention procedures, and the primary analysis population.
- Record participant numbers and reasons at each stage.
- Choose adjustment, weighting, imputation, or sensitivity methods only when their assumptions are defensible.
- Compare the achieved sample with the intended population using appropriate available data.
- Report likely direction, magnitude, and uncertainty of remaining bias.
How to report selection bias
The STROBE checklists ask observational researchers to describe eligibility, participant-selection methods, follow-up, efforts to address bias, participant numbers, reasons for non-participation, and relevant limitations. STROBE improves reporting; it is not a score proving that a study is free of bias.
Methods wording template
The source population comprised [population, place, and period]. Participants entered the study through [sampling and recruitment process]. Eligibility criteria were specified before recruitment. We mapped factors affecting consent, follow-up, and inclusion in the analytic sample using [theory, prior evidence, and a causal diagram]. The primary analysis included [population] because it estimates [estimand]. We addressed observed selection through [method and variables], assessed [diagnostics], and conducted [sensitivity analysis].
Results wording template
Of [number] potentially eligible individuals, [number] were invited, [number] consented, [number] completed baseline assessment, and [number] contributed to the primary analysis. The main reasons for exclusion or loss were [reasons]. Retained and non-retained participants differed on [variables] but were similar on [variables]. The primary estimate was [estimate and interval]; under [sensitivity condition], it was [estimate and interval].
Limitations wording template
Selection into the analysed sample may have depended on [mechanisms]. Because [factor] may relate to both [exposure] and [outcome], the observed association could be [inflated, attenuated, or uncertain]. Adjustment addressed measured differences in [variables], but unmeasured selection factors remain possible. The findings therefore apply most directly to [achieved population] and should not be assumed to represent [wider group].
Link the limitation to the actual claim rather than writing that “selection bias may have occurred.” The limitations guide shows how to discuss direction and consequence without treating uncertainty as failure.
Common mistakes and repairs
| Mistake | Why it is weak | Better practice |
|---|---|---|
| Calling every convenience sample biased | Representativeness depends on the estimand and selection mechanism | Define the target and explain the relevant pathway |
| Using sample size as proof of validity | Large selected samples can estimate the wrong quantity precisely | Evaluate how people entered and remained |
| Analysing completers without justification | Completion may depend on treatment and prognosis | Preserve planned comparison groups and examine attrition |
| Controlling for participation predictors automatically | A predictor may be a collider, mediator, or consequence | Choose variables from a causal model |
| Claiming weighting removes selection bias | Weights address only measured, modelled selection under assumptions | Report covariates, diagnostics, limitations, and sensitivity checks |
| Generalising beyond eligibility | Evidence does not directly cover excluded populations | State the achieved population and transportability limits |
Frequently asked questions
Is convenience sampling always selection bias?
No. Convenience sampling often limits population coverage, but bias in a particular estimate depends on why people are included and how selection relates to the variables and target. Describe the mechanism rather than applying a universal label.
Can random sampling prevent selection bias?
Random sampling from a suitable frame improves selection probabilities, but noncontact, refusal, attrition, frame omissions, and analysis exclusions can still matter. Document the whole pathway, not only the initial sampling method.
Is attrition bias the same as selection bias?
Attrition can be a form of selection bias when remaining under observation depends on relevant causes or consequences of exposure and outcome. Some dropout mainly reduces precision. Diagnose the mechanism and timing.
Should I compare completers with dropouts?
Yes, when baseline data are available and comparison is ethical. It can identify measured differences, but similarity on observed variables does not rule out differences on unmeasured variables or later outcomes.
Can propensity scores correct selection bias?
Propensity-style selection models or inverse probability weights can address measured selection mechanisms under strong assumptions. They cannot automatically correct unmeasured causes, poor overlap, model misspecification, or people absent from the sampling frame.
Where should participant flow appear?
Describe the process in methods, report counts and reasons in results, and use a flow diagram when it improves clarity. Detailed recruitment materials, exclusion logs, and supplementary diagnostics may be placed in appendices with privacy safeguards.
Conclusion
Selection bias is a causal and design problem, not a synonym for an imperfect sample. Define the target, map how participants enter and leave, identify whether conditioning can distort the focal relationship, prevent unnecessary exclusions, retain participants ethically, and select analysis methods whose assumptions can be defended. Report the achieved sample and remaining uncertainty with precision.
Get ethical support with participant-flow decisions
If your selection pathway is unclear, seek methodological feedback that preserves your authorship and follows institutional ethics and academic-integrity rules. Responsible support can review recruitment, eligibility, attrition, causal diagrams, weighting plans, sensitivity analyses, and reporting without inventing participants, hiding exclusions, altering data, or promising that a statistical correction eliminates bias.
