The SAT and the Pervasive Score Gaps: Time to Reevaluate

09 Jul The SAT and the Pervasive Score Gaps: Time to Reevaluate

By Addison Beer, Yale College Class of 2023, Summer 2020 Intern with SeeTheGaps.

In the wake of the coronavirus pandemic, recent unavailability of admissions tests and economic uncertainty has caused many colleges in the United States to move to a test-optional application cycle; some for the next admissions cycle only, others for the foreseeable future.   The entire University of California system, one of the largest and most successful university systems in the US, recently chose to become fully test optional until 2024, marking an influential change on America’s higher education landscape (Hubler, 2020). Given the rapidly shifting university admissions policies and ample scholarly research into standardized test score gaps, a reanalysis must be made on how effective the SAT truly is, and who it supports.

Standardized testing became a common practice in college admissions following WWI, when a military psychologist joined The College Board and transformed his Army intelligence test into the Scholastic Aptitude Test (SAT) (Rosales, 2018). The SAT was originally intended to measure innate aptitude for learning rather than memorized knowledge (Rosales, 2018), similar to the philosophy behind the IQ test. Today however, the test seems to favor white students from wealthy backgrounds and well-funded school districts who have more time and money to spend on private tutors and test-prep classes than their less-resourced peers, making the test at least partially based on income bracket rather than innate intelligence. Students from schools in wealthier districts are also more likely to qualify for additional time to take the SAT, due to over-identified learning disabilities. Some schools, such as Newton North Highschool near Boston, for example, have nearly one in three students qualify for additional time (Korn, 2019).

Gaming the system through liberally determined special needs diagnoses or hiring private tutors, however, are just a few of the contributors to a pervasive score-gap. The SAT has long been criticized for being closely linked to socioeconomic status and race (Schaeffer, 2019), as well as school district and home environment (Zwick, 2002). Deeper than household income or family resources, however, culturally biased questions that favor wealthy and white students may also lay behind score gaps (Jencks & Phillips, 2006). Tests that claim to measure “intelligence” or “aptitude” ignore the fact that distinct groups of students (whether it’s a racial distinction or economic one) have had different access and exposure to learning opportunities and experiences (like travel, upbringing, cultural outings and other kinds of inherited knowledge) (Dixon-Román, 2017), leading some groups to interpret standardized test questions differently (Freedle, 2003). This testing bias makes certain students more likely to succeed on the exam even before they walk into the testing center. For instance, Black students are less likely to take honors English and calculus in high school, and more often attend under-resourced schools in poorer districts than their white peers (Nettles et al., 2003). The College Board has made some efforts to lessen the cultural bias in their test questions. In 2016 they revamped the SAT by replacing the old flashcard-driven vocabulary items with questions that focus on context and textual evidence that, in theory, would be accessible to all students. (Balf, 2014). But if the score gaps stem from the nature of the test itself and a student’s background, rewriting the questions may not be enough. 

In addition to hidden biases, the psychological phenomenon of “stereotype threat”—the theory that students of color take standardized tests under the pressure that they will do poorly and thus reify the negative stereotypes of their race—may also contribute to the racial score gap (Claude Steele, 1997).  Negative stereotypes can cause any stereotyped group to underperform, potentially resulting in score gaps. Oftentimes, groups will under-perform themselves during a standardized testing environment whether or not the individual even believes in the stereotype, simply due to the added pressure that stereotypes hold (Claude Steele, 1997).  A qualitative study of Florida’s state test for elementary school students (FCAT), for example, found that high-achieving African American students were highly anxious surrounding the exam, and often-times conflated the purpose of their education itself with high-performance on the FCAT (Martin J. Wasserberg, 2017). The students were also overly concerned about the opinions of white people and were knowledgeable about the stereotypes surrounding their school and race. While stereotype threat can be a powerful influencer, and follow students into college no matter how well they did on the SATs (Owens & Massey, 2011), its effect can be reduced through better teaching methods that encourage growth and belonging (Claude Steele, 1997). 

Regardless of the potential causes, the SAT score disparity is still grossly apparent. For the 2019 class of SAT test-takers, 45% of white students and 55% of Asian American students scored over a 1200 on the 1600-point scale, whereas only 12% and 9% of Hispanic and Black students, respectively, performed the same. This racial score-gap has permeated the SAT’s history: a College Board annual report from 2000 cites Black students with nearly a 200-point average lower score, nearly one full standard deviation, compared to white students. Due to this disparity, many colleges have been distancing themselves from standardized testing for decades, with Bowdoin College as one of the first universities to become test optional in 1969 (Furuta, 2017), and many others following suit, with over 1200 colleges currently test optional.

The SAT is far from the first standardized test that students take and not the only one that perpetuates racial and socioeconomic disparities. In fact, many States, like the Florida test discussed above, have standardized tests for nearly every grade level (often following the Common Core standards developed by David Coleman, CEO of the SAT-managing company, The College Board). Some cities (such as NYC) use a completely test-score reliant system for admission to specialized public high schools (Mercedes E. Ebanks et al., 2012). This test (the SHSAT) can often be a ticket towards socioeconomic mobility.  While such a straightforward admissions process may seem meritocratic, NYC’s specialized high schools are struggling with diversity. Black and Hispanic students make up 70% of NYC’s public school enrollment, but only 10% of students at NYC’s nine specialized high schools (Veiga, 2018). At Stuyvesant High School in Manhattan, one of the most selective and prestigious of the specialized high schools, the percentage of Black or Hispanic students is about 3.5% (Stuyvesant High School, 2018). But while Black and Hispanic students are underrepresented in these specialized high schools, Asian students are attending in remarkable numbers. At Stuyvesant, over 70% of the student population comes from an Asian background, while the other high schools aren’t far behind. (Mercedes E. Ebanks et al., 2012). With a view to ameliorating these admissions disparities, NYC Mayor Bill DeBlasio outlined a plan to re-do the test.  Critics feared such a plan would hinder Asian students, who often overcome financial hurdles to attend these schools. Within the eight high schools that solely use the SHSAT for admission, 63% of Asian students come from a low-income background (Veiga, 2018). While these students have overcome financial adversity, the gaps plaguing Black and Hispanic students remain too vast to ignore, raising the question of how to support all groups fairly.

Even with ample evidence of the inherent unfairness of the SAT and other standardized tests, many college admissions offices still use the test as a metric for sorting and evaluating students. A large part of an admissions officer’s job is to quantify an applicant, whether through test scores, GPA, or a proprietary ranking system. Due to the number of applications that many colleges receive, aspects of a student are often rated into a numeric system, as revealed in the Harvard admissions lawsuit (Hartocollis, 2018). Admissions offices have started looking for more nuanced qualities in their students rather than what can be gleaned from test scores alone, such as “grit” and commitment to community service, but there remains a need to numerically categorize applicants (Fosnacht et al., 2019).  While wealthy and white students are more likely to reach higher score brackets, studies show the SAT is still a decent predictor of college academic performance, even when controlling for discrepancies in socioeconomic status (Sackett et al., 2009). Other studies have found the predictive reliability of the SAT to differ between white and Black students (Santelices, 2010).  The current infrastructure for higher education admissions and the sheer volume of applicants support some use of standardized testing, which makes effective reform challenging.

In an unequal society, there may be no such thing as a truly “standardized” test. But there are better metrics for college admissions officers to focus on than simply the SAT. The College Board is aware of the issues with diversity and the SAT, and they’ve made attempts to improve their flagship test. The College Board has attempted to quantify the obstacles students face into an “adversity index,” scored 1-100, that tries to contextualize a student’s background and hardships (Hartocollis, 2019). This metric ignores race however, and could simply be a band-aid to a much deeper problem.  The adversity index was short-lived and quickly replaced by a “landscape” dashboard for college use.  Another proposal came in a controversial 2003 paper by Roy Freedle who proposed a new method of scoring the SAT, which he called the Revised SAT score, or R-SAT score (Freedle, 2003). The R-SAT focused on scoring the “hard” portions of the verbal and math sections of the SAT, since Freedle’s research showed minority students and lower-income white students performing better on the “hard” test questions rather than the “easy” ones, which was attributed to cultural bias (Freedle, 2003). While Freedle’s paper faced backlash, future research seems to confirm that racial groups perform differently based on items in the verbal test (Santelices & Wilson, 2010). Students from different backgrounds will approach the SAT differently, which brings into question the SAT’s reputation as a “common yardstick.”

The University of California has been leading the trend away from reliance on SAT scores.  They found that for their applicant pool, one of the strongest predictors of success in college is a student’s high school GPA, rather than an SAT score (Geiser, 2009). Compared to GPA, the California study found that the SAT has a higher correlation with socioeconomic status, and, contrary to some studies, is a worse predictor of academic performance in college (Geiser, 2009). It should be noted that California, like Texas, is one of the few college systems to use a “top percent” admissions program as an alternative to race-based affirmative action. Students who graduate in the top 10% of their high school class are granted automatic admission into all state-funded colleges; this policy, on its own, did not impact state school diversity as intended (Barshay, 2019). Increased student outreach to underrepresented schools and districts could make a top 10% policy more effective (Barshay, 2019). Such GPA-based approaches are not entirely foolproof, however, as they do bear the risk of unequal high school grading systems (such as the prevalence of grade inflation in some wealthy school districts). The study also found that if admissions offices were to focus primarily on GPA, with a smaller emphasis on SAT scores, the score gaps would still remain, and wealthier students would still have an unfair advantage. See also, Zwick, Who Gets In, 2017.

The question remains as to whether a broader move to a test-optional college application system will make college admissions more equitable and campuses more diverse.  Recent studies are not encouraging. One study found zero connection between test optional policies, and racial or socioeconomic diversity on college campuses (Saboe, 2019). It did however find a slight increase in applicants following the announcement of a test-optional policy, implying that a test-optional shift could be a form of marketing on the behalf of the university, rather than a genuine strategy to change campus diversity.  The University of California system changes will be telling once there are several years of data to analyze. The recent U.S. News announcement to rank test-blind schools (Morse & Brooks, 2020) may also lessen the pressure on colleges to rely so heavily on SAT scores.

The pervasive and harmful effects of stereotype threat, potential cultural bias, and disparities in family and school resources, raise the question of whether or not the SAT and other college admissions tests could truly become meritocratic, or if meritocracy is even feasible. Some scholars like Pierre Bourdieu argued that the very structure of the education system and standardized testing supports the social reproduction of the elite, specifically wealthy white men (Bourdieau, 1977; Serna et al., 2017). Under Bourdieu’s social reproduction model, the dominant social class maintains power through limiting access (such as by requiring a standardized test to gain entry) to various forms of capital, economic or cultural (Bourdieu, 1986). Whether or not you subscribe to Bourdieu’s theoretical framework, the college admissions score gap is clearly deeper than tricky wording on a few questions or disparities in test-prep resources. Regardless of how the SAT and other standardized tests are eventually reformed, there will still be lingering inequalities plaguing our education system if there are societal gaps outside of the education sector. 

Admissions officers will always look towards an easily discernible metric for evaluating students, and in an ideal world we would have a functional meritocracy. If there was a truly equitable SAT, or any standardized test, racial and socioeconomic gaps would be nonexistent. Every student would have equal access to the tools needed to succeed, and those who work the hardest and are truly intellectually gifted would be able to obtain top scores. But in an inherently unequal society, is a true meritocracy, or a test that is “standardized” actually feasible?  The current state of the SAT—and perhaps even the entire college admissions process and higher education system—is nowhere close to being equitable. Once colleges begin to reassess after the coronavirus, it’s time to consider what’s worth keeping.

Work Cited

2018 | STUYVESANT HIGH SCHOOL – Enrollment Data | NYSED Data Site. (n.d.). Retrieved June 11, 2020, from https://data.nysed.gov/enrollment.php?year=2018&instid=800000046741

Balf, T. (2014, March 6). The Story Behind the SAT Overhaul. The New York Times. https://www.nytimes.com/2014/03/09/magazine/the-story-behind-the-sat-overhaul.html

Barshay, J. (2019, July 8). Texas top ten percent policy provides a cautionary lesson. The Hechinger Report. https://hechingerreport.org/texas-top-10-policy-didnt-expand-number-of-high-schools-feeding-students-to-top-universities/

Bourdieu, P. (1986). The Forms of Capital. Handbook of Theory and Research for the Sociology of Education, 241–258.

Claude Steele. (1997). A threat in the air: How stereotypes shape intellectual identity and performance. The American Psychologist, 52(6), 613. https://doi.org/10.1037//0003-066X.52.6.613

Dixon-Román, E. J. (2017). Inheriting Possibility: Social Reproduction and Quantification in Education. University of Minnesota Press; JSTOR. https://doi.org/10.5749/j.ctt1pwt7g0

Freedle, R. O. (2003). Correcting the SAT’s Ethnic and Social-Class Bias: A Method for Reestimating SAT Scores. Harvard Educational Review, 73(1), 1–43.

Furuta, J. (2017). Rationalization and Student/School Personhood in U.S. College Admissions: The Rise of Test-optional Policies, 1987 to 2015. Sociology of Education, 90(3), 236–254.

Geiser, S. (2009). Back to the Basics: In Defense of Achievement (and Achievement Tests) in College Admissions. Change: The Magazine of Higher Learning, 41(1), 16–23. https://doi.org/10.3200/CHNG.41.1.16-23

Hartocollis, A. (2019, May 16). SAT’s New ‘Adversity Score’ Will Take Students’ Hardships Into Account. The New York Times. https://www.nytimes.com/2019/05/16/us/sat-score.html

Hartocollis, A. (2018, October 19). Harvard’s Admissions Process, Once Secret, Is Unveiled in Affirmative Action Trial. The New York Times. https://www.nytimes.com/2018/10/19/us/harvard-admissions-affirmative-action.html

Hubler, Shawn. (2020, May 23). Why Is the SAT Falling Out of Favor? – The New York Times. The New York Times. https://www.nytimes.com/2020/05/23/us/SAT-ACT-abolish-debate-california.html

Hubler, S. (2020, May 21). University of California Will End Use of SAT and ACT in Admissions. The New York Times. https://www.nytimes.com/2020/05/21/us/university-california-sat-act.html

Jencks, C., & Phillips, M. (2006). The Black-White Test Score Gap. Brookings Institution Press. http://ebookcentral.proquest.com/lib/yale-ebooks/detail.action?docID=3004461

Korn, D. B., Jennifer Levitz and Melissa. (2019, May 21). Many More Students, Especially the Affluent, Get Extra Time to Take the SAT. Wall Street Journal. https://www.wsj.com/articles/many-more-students-especially-the-affluent-get-extra-time-to-take-the-sat-11558450347

Freedle, R. O. (2003). Correcting the SAT’s Ethnic and Social-Class Bias: A Method for Reestimating SAT Scores. Harvard Educational Review, 73(1), 1–43.

Fosnacht, K., Copridge, K., & Sarraf, S. A. (2019). How Valid is Grit in the Postsecondary Context? A Construct and Concurrent Validity Analysis. Research in Higher Education, 60(6), 803–822. https://doi.org/10.1007/s11162-018-9524-0

Mercedes E. Ebanks, Ivory A. Toldson, Soyini Richards, & Brianna P. Lemmons. (2012). Project 2011 and the Preparation of Black and Latino Students for Admission to Specialized High Schools in New York City. The Journal of Negro Education, 81(3), 241–251. JSTOR. https://doi.org/10.7709/jnegroeducation.81.3.0241

Morse, R., & Brooks, E. (n.d.). U.S. News to Start Ranking Test-Blind Schools. US News & World Report. Retrieved June 26, 2020, from https://www.usnews.com/education/blogs/college-rankings-blog/articles/2020-06-17/us-news-to-start-ranking-test-blind-schools

Nettles, M. T., Millett, C. M., Ready, D. D., Ludwig, J., Forman, J., & Millet, C. M. (2003). Attacking the African American: White Achievement Gap on College Admissions Tests. Brookings Papers on Education Policy, 6, 215–252. JSTOR.

Owens, J., & Massey, D. S. (2011). Stereotype threat and college academic performance: A latent variables approach. Social Science Research, 40(1), 150–166. https://doi.org/10.1016/j.ssresearch.2010.09.010

Rosales, J. (2018). The Racist Beginnings of Standardized Testing. National Education Association. http://www.nea.org//home/73288.htm

Sackett, P., Kuncel, N., & Arneson, J. (2009). Does Socioeconomic Status Explain the Relationship Between Admissions Tests and Post-Secondary Academic Performance? Psychological Bulletin, 135(1), 1–22. http://dx.doi.org/10.1037/a0013978

Saboe, M., & Terrizzi, S. (2019). SAT optional policies: Do they influence graduate quality, selectivity or diversity? Economics Letters, 174, 13–17. https://doi.org/10.1016/j.econlet.2018.10.017

Santelices, M. V., & Wilson, M. (2010). Unfair Treatment? The Case of Freedle, the SAT, and the Standardization Approach to Differential Item Functioning. Harvard Educational Review, 80(1), 106–134. https://doi.org/10.17763/haer.80.1.j94675w001329270

Schaeffer, B. (2019, September 24). 2019 SAT Scores: Gaps Between Demographic Groups Grows Larger | FairTest. https://www.fairtest.org/2019-sat-scores-gaps-between-demographic-groups-gr

The history of the SAT. (n.d.). Retrieved June 7, 2020, from https://www.manhattanreview.com/sat-history/

Veiga, C. (2018, August 13). Fair and objective or useless and biased? A Chalkbeat guide to the case for and against New York City’s specialized high school test. Chalkbeat New York. https://ny.chalkbeat.org/2018/8/13/21105525/fair-and-objective-or-useless-and-biased-a-chalkbeat-guide-to-the-case-for-and-against-new-york-city

Wasserberg, J., Martin. (2017). High-Achieving African American Elementary Students’ Perspectives on Standardized Testing and Stereotypes. The Journal of Negro Education, 86(1), 40–51. JSTOR. https://doi.org/10.7709/jnegroeducation.86.1.0040

Zwick, R. (2002). Is the SAT a ‘Wealth Test’? Phi Delta Kappa, 84(4), 307–311. https://doi.org/10.1177/003172170208400411

Zwick, R. (2017).  Who Gets In? Harvard University Press.