ERIC ED518897: Multilevel Propensity Score Matching within pdf

ERIC ED518897: Multilevel Propensity Score Matching within

More Book Details

Description of the Book:

A central issue in nonexperimental studies is the identification of comparable individuals (e.g. students) to remove selection bias. One such increasingly common method to identify comparable individuals and address selection bias is the propensity score (PS). PS methods rely on a model of the treatment assignment to identify comparable individuals on the basis of similar probabilities of receiving treatment. To attend to how the treatment selection mechanism differs among schools, two principal approaches have been proposed. The first approach makes use of a canonical single level PS in conjunction with restricting comparisons to individuals within the same school (Rosenbaum, 1986). This approach addresses the varying influence of a school by blocking on school membership, and thus on observed and unobserved school covariates. A second approach to address varying school influence on treatment assignment is use of multilevel PSs (Hong & Raudenbush, 2006; Kim & Seltzer, 2007). This approach explicitly models how treatment assignment mechanisms differ among schools potentially making PSs comparable across school boundaries.

In this study, the author and his colleagues evaluated the performance of both of these approaches in nonexperimental studies resembling multisite randomized trials. In particular, they assessed the performance of matching students across schools based on multilevel PSs and matching students within schools based on single level PSs with specific focus on the sensitivity of the PSs to omitted covariates. More specifically, they examined the extent to which matching across groups using a multilevel PS produces a more effective estimator of the treatment effect as compared to matching within groups using a single level PS. They addressed the research question by assessing the performance of the treatment effect estimators in multiple scenarios using Monte Carlo simulation. For proposal brevity, they highlight four simple situations: (1) No unmeasured covariates that influence the treatment assignment; (2) One unmeasured school level covariate that influences the treatment assignment; (3) One unmeasured student level covariate that influences the treatment assignment; and (4) One unmeasured school level covariate and one unmeasured student level covariate; both of which influence the treatment assignment.

In summary, the results of this study suggest that matching across schools based on multilevel PSs has much to offer and that further study into its utility and the conditions under which it outperforms matching within schools based on a single level PS need to be explicated. In particular, the simulations presented address small covariate spaces with simple variable relations. Study of designs with more complex associations among both unobserved and observed confounders is warranted. Finally, omitted in this summary are more complex details concerning covariate balance. Specifically, obtaining and assessing covariate balance both within and across schools to build a strong basis for inference is much more complex. (Contains 4 tables

• Creator/s: ERIC
• Date: 2011
• Year: 2011
• Book Topics/Themes: ERIC Archive, Probability, Selection, Bias, Monte Carlo Methods, Schools, Students, Kelcey, Benjamin

An excerpt captured from the PDF book