Psychology dissertation covariate balancing propensity score analysis combines a treatment-assignment model with explicit covariate-balance conditions. It can help students design an observational comparison when conventional propensity modelling produces poor balance, but it does not turn observational data into a randomised experiment.
This guide explains the estimand, assumptions, model choices, diagnostics, outcome analysis and reporting needed for a defensible dissertation. The main example is a binary exposure, such as receiving a campus wellbeing programme versus usual support. Extensions exist for other treatment structures, but they require their own theory and implementation.
Table of Contents
What a covariate balancing propensity score does
A propensity score is the conditional probability of exposure given measured pre-exposure covariates. Standard practice often estimates that probability with logistic regression and then checks whether matching or weighting balances the covariates. A model can predict exposure reasonably well yet still yield poor balance, particularly when its functional form is misspecified.
Covariate balancing propensity score, usually shortened to CBPS, addresses that tension during estimation. The foundational paper by Imai and Ratkovic treats the propensity score as both a treatment-assignment probability and a balancing score. Parameters are estimated through generalized method-of-moments or related conditions so that treatment prediction and covariate balance are considered together.
The method does not choose a causal question for you. After estimating propensity scores, the analyst still converts them into weights appropriate to an estimand such as the average treatment effect for the eligible population (ATE) or the average treatment effect among treated participants (ATT). Those targets answer different questions and can assign very different influence to the same participant.
| Approach | How balance enters | Typical output | Key distinction |
|---|---|---|---|
| Ordinary propensity model | Checked after prediction-model estimation | Estimated exposure probabilities | Likelihood fit does not directly enforce balance |
| CBPS | Balance conditions enter parameter estimation | Propensity scores and estimand-specific weights | Models assignment while targeting covariate balance |
| Entropy balancing | Selected moments are direct constraints | Calibration weights | Does not require the same assignment-model interpretation |
| Overlap weighting | Balance arises from a specified weighting scheme | Bounded ATO weights | Targets the population with greatest empirical overlap |
This distinction prevents cannibalisation with the site’s entropy balancing guide and overlap weighting guide. CBPS is principally about estimating a propensity model using balance conditions. Entropy balancing is a calibration method, while overlap weighting defines a particular target population and bounded weights.
When psychology dissertation CBPS is appropriate
CBPS can suit a dissertation using observational or quasi-experimental data with a clearly timed exposure and a defensible set of measured baseline confounders. For example, a student might compare changes in depressive symptoms between university students who chose therapist-guided digital support and those who chose self-guided support. Baseline symptoms, previous therapy, age, financial strain, study level and service accessibility may influence both pathway choice and later outcomes.
Another application might compare schools adopting a social-emotional learning programme with similar schools continuing usual provision. Baseline wellbeing, prior attainment, school size, staffing, region and earlier programme use could enter the assignment model. The analysis must define whether the target is all eligible schools, adopting schools or a different supported population before weights are generated.
CBPS is not a default rescue for every imbalanced dataset. It is unsuitable when the exposure is poorly defined, confounders occur after exposure, groups have no meaningful common support, or important common causes are unmeasured. A small sample with sparse categories can create unstable estimation and misleadingly precise-looking output. Longitudinal treatment decisions may instead require the framework described in the marginal structural models guide.
Define the estimand first
State the target contrast in words before fitting a model. An ATE asks about the mean effect across the eligible target population. An ATT asks about the mean effect for people who actually received the exposure. An ATO asks about people with the greatest overlap, but that target is normally paired with overlap weights rather than assumed from using CBPS alone.
The estimand determines the weight formula and interpretation. For a binary exposure A with propensity score e(X), conventional ATE weights are 1/e(X) for exposed participants and 1/{1 − e(X)} for unexposed participants. ATT weights are 1 for exposed participants and e(X)/{1 − e(X)} for unexposed participants. The CBPS estimates e(X); it does not erase the consequences of choosing one target rather than another.
Build a causal covariate set
Select covariates using temporal ordering, theory, prior evidence and a causal diagram. Include pre-exposure variables that are common causes of exposure and outcome, along with strong baseline predictors when they support precision and design. Do not select variables only because their univariable associations have small p-values.
Avoid mediators, descendants of exposure and colliders. If programme engagement is recorded after assignment, including it in the propensity model may condition on part of the exposure process. The site’s confounding variables guide explains why adding more controls can sometimes increase rather than reduce bias.
Continuous covariates should usually remain continuous. Arbitrary categories discard information and can leave imbalance within categories. Use justified nonlinear functions, such as splines or polynomial terms, and interactions where the assignment mechanism plausibly depends on combinations of variables. If you ask CBPS to balance transformed terms, explain why those terms matter and report their balance.
State the causal assumptions
A causal interpretation generally requires consistency, conditional exchangeability, positivity and correct handling of missingness and sampling. Consistency links observed exposure versions to the intervention being discussed. Conditional exchangeability assumes that the included covariates block all important exposure-outcome backdoor paths. Positivity requires a non-zero chance of each exposure at relevant covariate profiles.
CBPS can reduce imbalance caused by a misspecified conventional propensity model, but it cannot balance an unmeasured variable or create observations where one group is structurally absent. Balance among observed covariates is necessary for many weighting designs, not sufficient proof of exchangeability.
Choose between exact and over-identified CBPS
The original formulation distinguishes a just-identified or exactly identified approach based on balancing conditions from an over-identified approach that combines balance and treatment-assignment conditions. Terminology and defaults vary across software, so the dissertation should name the implementation, version, link function, estimand and option settings.
Do not select a version because it produces the smallest p-value in the outcome model. Compare specifications using pre-specified balance, overlap, convergence and weight-stability criteria while outcomes remain hidden where feasible. Outcome-free design limits researcher degrees of freedom and makes the analytic history easier to defend.
The official CBPS package documentation describes the cross-sectional estimator and its treatment-assignment and balancing objectives. The current WeightIt CBPS documentation supports estimand-specific weighting and options for higher moments, interactions and quantiles. These options are tools, not invitations to impose every possible balance condition in a modest dataset.
Check convergence and numerical behaviour
Record whether optimisation converged, whether warnings occurred and whether repeated runs with the same inputs reproduce the same solution. Examine coefficients and fitted probabilities for implausible values. Near separation, redundant covariates and a non-full-rank design matrix can cause numerical problems.
Standardisation may improve optimisation, but transformations should be documented. Sparse categories might need principled consolidation decided before outcome analysis. Do not quietly remove participants or covariates until the software returns a result. Every exclusion changes the target sample and needs a scientific justification.
| Design check | Evidence to retain | Warning sign | Response |
|---|---|---|---|
| Exposure timing | Clear baseline and follow-up dates | Covariate measured after exposure | Rebuild the temporal design |
| Optimisation | Convergence status and settings | Warnings or unstable solutions | Simplify justified terms and inspect coding |
| Overlap | Propensity distributions by group | Large regions containing one group only | Narrow the target or reconsider inference |
| Weights | Range, quantiles and group totals | A few records dominate | Check model, estimand and data errors |
| Balance | Before-and-after tables and plots | Important residual differences | Revise prespecified functional forms |
Diagnose balance after CBPS
Because balance entered estimation, students sometimes assume no further diagnosis is needed. That is incorrect. Report covariate balance before and after weighting for every term in the design, then examine clinically or psychologically important distributional features that were not directly constrained.
Standardised mean differences place continuous covariates on a common scale. A threshold such as 0.10 is a heuristic, not a validity certificate. The cobalt documentation explains adjusted differences, group summaries and effective sample sizes. For binary variables, raw proportion differences can be easier to interpret.
Include a Love plot and an exact balance table. Inspect the maximum absolute difference, not only an average. Consider variance ratios, quantiles, empirical cumulative distributions and interactions when they matter to the assignment process or outcome. Significance tests are poor balance diagnostics because they mix difference magnitude with sample size.
Inspect overlap and effective sample size
Plot the estimated propensity distributions by exposure group. Extreme probabilities can generate large inverse weights under ATE or ATT formulas. Summarise minimum, maximum, median and upper weight quantiles by group, along with the number of zero or near-zero weights if applicable.
Effective sample size is often calculated as the squared sum of weights divided by the sum of squared weights. It indicates information loss from unequal weighting, not a new participant count or proof of adequate power. A weighted analysis can retain every row while having substantially less precision than the nominal sample suggests.
Estimate outcomes without changing the question
Once the weighting design passes its criteria, estimate the outcome contrast on the scale specified in the protocol. A continuous psychology score might use a weighted mean difference. A binary outcome might use a weighted risk difference or risk ratio. Select the effect measure because it answers the research question, not because one scale crosses a significance threshold.
Uncertainty estimation must respect weighting and, when supported, propensity-score estimation. Naive standard errors that treat weights as ordinary replication counts may be inappropriate. Use a justified robust sandwich estimator, M-estimation procedure or bootstrap that repeats the relevant design steps. Report the exact method and software.
An outcome regression added after weighting can improve precision or provide an augmented estimator, but it introduces more modelling choices. State whether it is primary or sensitivity analysis, specify covariates and functional forms, and ensure the reported effect retains the intended estimand. Avoid describing a doubly robust estimator unless the estimator truly has that property under the cited conditions.
| Report section | Required information | Cautious interpretation |
|---|---|---|
| Design | Eligibility, exposure, time zero, estimand and covariates | Defines who and what the estimate concerns |
| CBPS model | Formula, identification type, link, software and convergence | Shows how assignment and balance were modelled |
| Diagnostics | Overlap, balance, weight distribution and effective sample size | Shows support among measured variables |
| Effect | Estimate, confidence interval, outcome scale and variance method | Quantifies uncertainty without relying on a p-value alone |
| Limitations | Unmeasured confounding, measurement, missingness and generalisability | Prevents balance from being mistaken for randomisation |
Plan informative sensitivity analyses
Compare the primary CBPS specification with a conventional propensity model using the same covariates and estimand. This comparison shows whether direct balancing conditions materially changed balance, weights or the effect estimate. It should not be framed as a contest in which the preferred result is the most statistically significant.
Test justified nonlinear terms and interactions, alternative handling of influential observations, and an augmented outcome model. If overlap is limited, consider whether a supported overlap-population question is more honest than an ATE requiring heavy extrapolation. That is a change of estimand and must be labelled clearly, not presented as a minor robustness tweak.
Address missing data before weighting. Complete-case analysis can select a different population and reintroduce imbalance. With multiple imputation, fit the weighting model within each imputed dataset, estimate the outcome contrast and combine estimates with a method appropriate to the complete procedure. Include exposure, outcome, covariates, important interactions and auxiliary variables in the imputation strategy as justified.
Quantitative bias analysis can assess the strength of unmeasured confounding or measurement error needed to alter a conclusion. Negative controls can sometimes reveal residual bias when valid controls exist. Neither method retroactively proves the CBPS design correct, but both can make uncertainty more explicit.
Common mistakes to avoid
Calling CBPS a balance guarantee
CBPS targets specified balance conditions. It does not guarantee every moment, interaction or unmeasured variable is balanced. Always inspect the realised weighted data and disclose remaining differences.
Ignoring the estimand
The same fitted scores can produce ATE, ATT or other weights. Each target answers a different question. Put the estimand in the methods, results and discussion rather than mentioning only the software function.
Using post-exposure covariates
Balancing a mediator or collider can bias the contrast even when its standardised difference becomes zero. Fix the causal timeline and defend every included variable.
Trimming weights after seeing outcomes
Trimming changes the analysed population and can alter bias and precision. Pre-specify rules, justify them, and report untrimmed and trimmed results when both are scientifically useful.
Equating measured balance with randomisation
Randomisation addresses measured and unmeasured baseline factors probabilistically. CBPS balances selected observed functions. Use conditional causal language and discuss residual confounding.
Reproducible dissertation workflow
- Define eligibility, exposure versions, time zero, follow-up and outcome.
- Write the estimand and target population in plain language.
- Use a causal diagram and evidence to select pre-exposure covariates.
- Document missingness, exclusions, coding and measurement quality.
- Inspect raw covariate support and propensity overlap.
- Fit the prespecified CBPS model and record all options and warnings.
- Generate estimand-specific weights and inspect their distribution.
- Assess means, proportions and relevant higher-order balance.
- Estimate the outcome effect with justified uncertainty.
- Run prespecified sensitivity analyses and preserve versioned code.
Keep a data dictionary, analysis log, session information and reproducible diagnostic outputs. Protect participant confidentiality, follow data-use agreements and remove direct identifiers. Reproducibility does not permit sharing restricted records.
Frequently asked questions
What is psychology dissertation covariate balancing propensity score analysis?
It is a propensity-score approach that estimates treatment-assignment parameters while directly incorporating covariate-balance conditions, then uses the estimated scores to construct weights for a stated estimand.
Is CBPS the same as entropy balancing?
No. CBPS combines an assignment model with balancing conditions. Entropy balancing directly calibrates weights to selected moments and has a different estimation framework.
Does CBPS remove unmeasured confounding?
No. It balances specified functions of observed covariates. A causal interpretation still depends on conditional exchangeability and other assumptions.
Can CBPS be used for ATT and ATE?
Yes, supported implementations can construct estimand-specific weights. The dissertation must state which estimand was chosen and why it matches the research question.
Should balance still be checked after CBPS?
Yes. Report realised mean and distributional balance, overlap, weight summaries and effective sample size. Estimation objectives do not replace diagnostics.
What if the CBPS model does not converge?
Inspect coding, sparse categories, redundant terms, scaling and overlap. Do not suppress warnings or delete variables solely to obtain a preferred outcome result.
Can CBPS make an observational study causal?
Only conditionally. Strong design, temporal clarity, measured confounding control, positivity, consistency and defensible missing-data handling are still required, and limitations should be explicit.
Conclusion
Psychology dissertation covariate balancing propensity score analysis is valuable when the research question calls for propensity weighting and balance should influence estimation rather than appear only as an afterthought. Its strength is the integration of assignment modelling and covariate-balance conditions. Its limits remain those of observational causal inference.
If you need support, use ethical dissertation coaching that helps you define the estimand, review code and interpret diagnostics while you retain authorship. Responsible assistance should never invent data, hide modelling decisions, promise significance or replace the independent work required by your university.
