The study, titled “Written in Black & White: Exploring Confirmation Bias in Racialized Perceptions of Writing Skills,” is well worth the short read.
We undertook this study with the hypothesis that unconscious confirmation bias in a supervising lawyer’s assessment of legal writing would result in a more negative rating if that writing was submitted by an African American lawyer in comparison to the same submission by a Caucasian lawyer. In order to create a study where we could control for enough variables to truly see the impact of confirmation bias, we did not study the potential variances that can be caused due to the intersection of race/ethnicity, gender, generational differences and other such salient identities. Thus, our conclusion is limited to the impact of confirmation bias in the evaluation of African American men in comparison to Caucasian men. We do not know (although we plan to study the issue in the very near future!) how this impact will splinter or strengthen when gender and/or other identities are introduced.
The data findings affirmed our hypothesis, but they also illustrated that the confirmation bias on the part of the evaluators occurred in the data collection phase of their evaluation processes – the identification of the errors – and not the final analysis phase. When expecting to find fewer errors, we find fewer errors. When expecting to find more errors, we find more errors. That is unconscious confirmation bias. Our evaluators unconsciously found more of the errors in the “African American” Thomas Meyer’s memo, but the final rating process was a conscious and unbiased analysis based on the number of errors found. When partners say that they are evaluating assignments without bias, they are probably right in believing that there is no bias in the assessment of the errors found; however, if there is bias in the finding of the errors, even a fair final analysis cannot, and will not, result in a fair result.