By Megan Stubbendeck, Posted January 22, 2016
Percentiles are a beast. And the beast is out in full force with the new PSAT. Why? Because it’s a new exam and everyone is wondering how it stacks up to the old test.
There’s been a lot of talk about scores, conversions, and selection indexes, but it all seems to focus now on percentiles.
Update (February 2016): The College Board announced that they will replace sample percentiles with percentiles from actual test takers in 2016. The current user percentiles are based on data from students who took the old exam in 2014 and the new exam in 2015. Once the percentiles are updated, students, parents, and educators will grasp a better understanding of how a student performed compared to other testers.
What is a percentile?
A percentile ranks your performance on the PSAT compared to the performance of other students. For example, if your score report says you are in the 76th percentile, you scored higher than 76% other students. The highest percentile is 99th.
What percentiles do we know?
There are three key percentiles on the redesigned PSAT:
1. Total Score Percentile (Nationally Representative Sample)
Where to find it: On every student’s online score report and paper score report. See it on a sample score report here.
Definition: Ranks you against ALL students in your grade nationwide. Includes even those students not on graduation and college-bound track. It was developed based on a small group of representative students tested by the College Board. This percentile is a new addition to the score reports.
2. Total Score Percentile (PSAT/NMSQT User Percentile)
Where to find it: Only available on a student’s online score report, not the paper version. See where to find it on the online score report.
Definition: Ranks you against students who traditionally take the PSAT. This percentile will be slightly lower than the percentile above because this group doesn’t include students who aren’t on track to go to college and thus normally don’t take the PSAT. It was developed based on a small group of representative students tested by the College Board. This is also a new percentile this year.
3. Selection Index Percentile
Where to find it: This percentile is not on any student’s score report. A student needs to calculate the conversion herself by using a chart released by the College Board. See the chart here.
Definition: Ranks you against students who take the PSAT as part of the National Merit Competition, the large majority of you reading this post. It also weights the sections a bit differently. This will likely be the lowest percentile because it is an even more selective group of students than the others. According to the College Board, this percentile should approximately be the same as the percentile from the old PSAT, thus it was the percentile we used in the chart below to make the most valid comparison of the new and old PSATs.
Are this year’s percentiles in line with previous years’?
The short answer is no. They are almost all higher on the new PSAT than on the old PSAT.
At ArborBridge we have poured over data to compare percentiles between the old and new PSAT.¹ We decided to focus on the Selection Index Percentiles because the College Board has said these should be the same or at least pretty close between the new PSAT and the old. What did we find? They aren’t the same, and sometimes they are far off.
Below is a chart of what we found for different Selection Indexes. It shows how many percentile points higher or lower a Selection Index earned this year. If the bar is positive (above 0), it means the percentile is higher this year than in the past. If the bar is negative, the percentile is lower this year.
Overall, the general trend indicates that the percentiles on the new PSAT are higher than on the old PSAT for a great majority of students.
A few other big takeaways:
- At the highest end (99th + percentile, Selection Indexes above 213), percentiles are consistent with past PSAT percentiles.
- However, the 98th through the 9th percentiles (Selection Indexes between 213 and 110)—which represents 90% of students—have higher percentiles on this year’s PSAT than in the past. Sometimes as many as 10 percentile points higher.
- The percentile inflation is shaped like a bell curve, growing most pronounced for the middle set of students, AKA: most college-bound students.
- The percentiles become more consistent again at the bottom from the 9th through the 1st percentiles (Selection Scores below 110). And in some places the percentiles even seem a bit lower than last year’s.
Why are percentiles important usually?
There is one reason we always care about percentiles:
Because they give you an idea of how you did on the test in comparison to other students. College admissions is largely about ensuring you’re a competitive applicant, so knowing how you compete on the exams helps you know how you stack up in one area of college admissions.
Why are percentiles so important this year?
Percentiles are getting much more focus this year than usual. But most people don’t understand what the hoopla is all about. There are three reasons why percentiles are so important this year:
1. Because the scoring of the new exam is so different from the past, percentiles matter more.
The PSAT scale is now out of 1520 instead of the old 240. There is no guessing penalty anymore. Sections carry different weights. The list of scoring changes goes on and on. But the one constant is percentiles: percentiles still rank you against other students and they are still on a 99-point scale. It’s the one stable metric in this year of change.
2. Because so far it’s the only way to compare how you might do on the ACT vs. the new SAT.
Many students use their PSAT scores to decide if they should take the SAT or the ACT for college admissions. With the new scoring system it’s pretty much impossible at this point to compare your PSAT score to how you might do on the ACT. Both tests, though, use the same percentile scale, so many students are focusing on percentiles to make their decisions. Whichever test you have a higher percentile on is likely the test you will take.
3. Because the College Board has used percentiles for its concordance tables.
The College Board, which writes and scores the SAT/PSAT, has released to everyone a preliminary concordance table (a fancy term for a chart that allows anyone to convert a new PSAT score to an old PSAT score). These tables are meant to help everyone better understand the new scores. According to the College Board, these concordance tables were based off of percentiles. Basically, if the table says a new 1300 is equal to an old 190 on the PSAT then the percentiles for both scores should be the same.
For all of these reasons, people are very focused on the percentiles the College Board released last week. But attention got even hotter when it became clear that the new percentiles actually aren’t lining up with the old test.
Why do the percentiles look so different this year?
It’s certainly not because the College Board is inept or doesn’t care. They’ve been working on this issue for months. Here are some of the reasons we’ve heard and the merits of each:
1. Different stakeholders are at the table this year.
This reason comes directly from a College Board representative. In the past, the biggest groups who cared about this data were college counselors, test prep experts, students, parents, and colleges. But after the College Board shifted focus a few years ago to align the SAT/PSAT with Common Core and replace states’ Common Core testing with the SAT, new (pretty powerful) players are at the table. These include boards of education, state and federal lawmakers, etc. The College Board has been heavily courting these groups to sign exclusive statewide contracts with the SAT, and it’s working. Many states have switched over to the SAT. These stakeholders need different data, though. Unlike college counselors and admissions departments who care only about percentiles for kids who actually plan to go to college, the new players at the table want to know more about all students. This is why the new, more inclusive percentiles (Nationally Representative Sample Percentile and PSAT/NMSQT User Percentile) have been added to student score reports and the Selection Index percentile no longer appears.
2. The test is entirely new.
The College Board has said this, as have counselors and test prep experts. We all were warned. Administering an entirely new test to over a million students worldwide is a tall order. The College Board reps have done their best, but they are human, too. Even they didn’t know for sure how scores would compare until after everyone took the exam. These are the risks of a new exam. Even the College Board has publically and repeatedly stressed that the percentiles and concordance tables are preliminary. They reserve the right to change them in May after students have taken the redesigned SAT in the spring.
Update (February 2016): The College Board recently announced that the preliminary user percentiles will be re-created using actual test-takers’ results in 2016.
3. The College Board is playing it safe.
This is a common reason you hear among counselors, college admissions officers, and test prep. When the exam changed back in 2005, there were some indications that scores were a bit inflated for the first few administrations of the SAT. As the story goes, this was because the College Board wanted to be sure about its scoring and percentiles without hurting students. The same impetus could be at play again.
4. It’s all just part of the SAT vs. ACT battle.
This reason is the loudest one out there right now. It also paints the College Board in the toughest light. The basic argument goes like this: the only way to figure out which test is better for you (ACT or SAT) is to look at percentiles. The two exams use very different scoring scales, so comparing scores isn’t all that helpful. But percentiles are percentiles and can be easily compared. The usual advice is if you have a higher percentile on the SAT, take the SAT. If you have a higher percentile on the ACT, take the ACT. We all know that the College Board’s revamp of the SAT this year is in part because the ACT has become the more popular test in the US. The rationale goes further, that to beat out the ACT, the College Board is intentionally inflating this year’s PSAT percentiles so more kids will choose the SAT instead. There is no direct or clear evidence conclusively proving this rumor. Yet some people point to the clear percentile inflation as indirect evidence. But, remember, that’s not necessarily a smoking gun.
Why does percentile inflation matter?
It primarily matters because it indicates that the College Board’s early data may be off, and changes will possibly come down the road.
It also matters because students may receive a false sense of how good they are at the redesigned SAT. They may think they have a better shot at certain colleges than they actually do. Or they may choose to take the SAT instead of the ACT based on this inflated percentile only.
Are the percentiles valid?
This is the million-dollar question. And there is no clear answer, except that there appears to be an inflation trend. The College Board has reserved the right to change percentiles in May. We won’t know anything for sure until they replace sample percentiles with percentiles from actual test takers.
What does this mean for National Merit?
Generally, National Merit finalists are the top scorers in their states (this means that only students in the highest percentiles qualify). Students often use their percentiles to get an early indication if they have a shot since the actual decisions don’t come out for months. Keep in mind that the cut off score is decided on a state-by-state basis. In some states you may need to be in the top 2 percentiles; in others it might be the top 4 percentiles.
This year, focus on your Selection Index Percentile to determine your chances. With the possible inflation of even this percentile though, you may need to be in a higher percentile than usual. But we won’t know until September when the National Merit Corporation sends out letters to accepted Semi-Finalists.
What does this mean for my test prep?
If you use your percentile to pick SAT or ACT, just know that there’s no foolproof plan this year.
If your percentile on PSAT is wildly higher than your ACT percentile, then SAT might be a good bet. If your PSAT percentiles is only slightly higher or the same, it’s up to you, but ACT is a known test with few changes and will likely be your safest option. If your PSAT percentile is lower than your ACT percentile, ACT all the way!
Whatever happens, as news comes out and the College Board makes changes, ArborBridge will be sure to stay on top of it for you.
¹To create our comparison, we selected a representative sample of Selection Index scores for the new PSAT, focusing on the 99th through the 1st percentile of the Selection Index Percentiles. The Selection Index scores selected included four specific scores for each block of 10 points between 228 and 83. Thus the scores covered were 228, 225, 223, 220, 118, 115, 113…95, 93, 90, 88, 85, 83. For each Selection Index score we created a group of hypothetical students covering possible subscore iterations from high-Verbal/low-Math, to even Verbal/Math, to low-Verbal/high-Math. To create a representative set of hypothetical students at each Selection Index, we modified each subscore by one point in order to cover the widest variety of students. For example, a Selection Index of 201 contained hypothetical students with the following Reading/Writing/Math subscore breakdowns: 38/38/28, 37/37/30, 36/36/32, 35/35/34, 34/34/36, 33/33/38. Using this approach, we created a sample set of 680 hypothetical students for our study. We then converted these subscores to old PSAT section scores and old Selection Indexes using the College Board’s released concordance tables. Finally, we compared the College Board’s published percentiles for the new Selection Index and that new Selection Index’s converted old Selection Index equivalent. The different between these two percentiles is reflected in the graph as both an average (orange bars) of the percentile differences for all hypothetical students at the given Selection Index and as a median percentile difference (red line) for all hypothetical students at a given Selection Index.