The Tests Aren't Sufficiently Content Based
Did you know the ACT Science sections requires no knowledge of any
science? Did you know that the hardest concepts tested on the SAT Math
are beginning Algebra II? Did you know that when grading the ACT and
SAT Essays, essay graders are forbidden from marking a student's essay
down for incorrect facts?
The Tests have a noble goal, (Predict a student's success!) but they
fail to address the concepts students are seeing in school and
increasingly rely on developing tricky questions instead of difficult,
content based material. With more and more students taking Calculus in
high school, how can we take seriously any exam that focuses on
Pre-Algebra?
Sign up for the OV Daily Newsletter
OV Social
The ACT science questions don't require you to have memorized the periodic table of elements, but they do require you to be able to analyze and interpret data to draw conclusions from it. It's a test of scientific reasoning, a much better prediction of success in scientific fields than rote memorization could ever be.
The highest level of math required for graduation in most high schools is the equivalent of beginning Algebra II. It makes sense for the standardized testing to go up only to that point. Students who reach higher levels of math take SAT II exams, or AP or IB exams, to demonstrate their knowledge.
I have no problem with essay graders focusing on a student's command of language, rather than whether they happened to use an anecdotal example that wasn't entirely true. Those essays don't even have to be provably true in the first place -- they can be about your personal hopes and goals, or about a vivid memory, or your role model. Fact-checking has nothing to do with what's being tested on this portion of the test. (Were you aware that the math portion of the SAT isn't affected by a student's grammar?)
More importantly, though: these points all sound like "the test is too easy." How is that a reason for not considering the results? If you do incredibly poorly on a very easy test, doesn't that predict a lack of success in college? At best this argument is completely irrelevant to the point you are supposedly making, but at worst (for you) it's an argument for the other side.
I'm not asserting that the test is simply "too easy." Instead, I'm noting that the tests aren't testing the concepts taught in schools.
At the very high and low ends of the tests, we learn quite a bit about students. Any student that can ace this exam is obviously well-qualified for college. Any student that struggles to complete Algebra I problems should obviously receive some remedial training. But what about the students that score in the middle? Are they unqualified for college or simply lost in content that they haven't been trained on?
Some kind of quantitative evaluation of students' abilities is necessary for admissions committees to have, if we want any kind of reasonable portion of the population going to college. Admissions committees can't spend a day with each applicant to learn everything about them and develop an accurate holistic evaluation of the student's capabilities.
Let's say a college has 10,000 applicants and they can admit 2,000. Perhaps the SAT or ACT is only a good predictor of the extreme cases. The admissions committee at this college can then rely on the numbers to accept 1,000 students and reject 1,000 students. That helps a lot. They now have only 8,000 applications to sift through in more detail, and a little more time to spend on each to pick the other 1,000 to admit. That's what we should do about the students who score in the middle -- we make sure they have also submitted letters of recommendation, and high school transcripts, and personal essays, and portfolios of their extracurricular activities. (In reality, admissions committees look over these things even for the people who score very well or very poorly on standardized tests, to make sure the test results weren't anomalous. But they can certainly spend less time on those applications because they have information you admit is reliable in their cases.)
No one is arguing that the SAT/ACT should be the sole determiner of college admission. No one is arguing that the tests are perfect. But in the absence of some better standardized test, and given that we agree they do provide some useful information, there's no reason not to include the test as one of many factors that admissions committees consider.
Just to be clear, I am not arguing against the SAT/ACT being used as a sole determinant for college because that's not what happens. Never does an SAT/ACT keep a student out of a college on the basis of the score alone.
I am arguing that the test shouldn't be used at all. We shouldn't accept them at any school. They shouldn't bear any weight because they are flawed beyond repair and fundamentally unneeded. Right now the scores are used in much the same way you describe and it's a significant problem.
The primary flaw with your example is that you are assuming that the distributions of the scores are even. That's unlikely. If a school with an average ACT score of 25 receives 10,000 applications, they will get lots of scores around the 25, with some amount above and below.
That means that the mushy middle remains important. With only a few students far above the average, most students fall in the position where we must continue to look at them holistically even though they took the test!
So skip it. Stop supporting it. Stop feeding a beast that doesn't aid the process. Look at the numbers and realize that GPA and Course Rigor are the best indicators of student success. Have kids focus on earning As in the classroom instead of acing a test that's not useful to the process.
The SAT and ACT are highly reliable tests, meaning that if a candidate takes the exam repeatedly, in the vast majority of cases, that person will receive essentially the same score over and over. So even though someone is in the "mushy middle", he or she is likely in the right place compared to peers. Knowing that adds to the information being considered by a college admissions officer as part of the process.
And I don't think that anyone would argue against GPA and Course Rigor as desirable measures--if those things were remotely comparable from student to student, school to school. There is no objective measure of how grades are assigned or how rigorous courses are. Even AP courses, that come with relatively constrained curricula, require a large-scale standardized test at the end to determine how effectively the material was learned by the students who nominally were all taught an equivalently rigorous and challenging course. GPA has suffered from massive grade inflation over the last few decades, and has become steadily less meaningful as a differentiator of individual academic strength.
Standardized tests are not perfect, but they do play a useful role.
GrayCat,
1) It is a highly reliable test. But that alone is not a good measurement of future student success. All that tells us is that the test is designed to produce consistent results.
You are making the common mistake: A broken test that returns the same wrong result every time is not accurate.
2) GPA is comparable. Studies have repeatedly shown that GPA is the single best indicator of future college success. Yes, grades have inflated. Yes, it's hard to judge from school to school. But in general, As are earned for working hard and turning in assignments. Fs are earned for not showing up to class and not doing your work.
The Standardized tests don't add anything of value!
I did not indicate that I believed that reliability was the only property these assessments have--they are also a valid measure of performance. I do not agree with your premise that the test returns a wrong result, so your conclusion is specious.
The test items are not selected solely to cover content--there you and many others make a common, albeit mistaken, assumption. The items are selected so that the content is relevant to the content taught in schools, yes, but they are not intended as a comprehensive survey of everything learned there. They aren't some grand comprehensive exit exam. Instead, the items are selected as indicators of content knowledge and understanding PLUS another important aspect--they correlate with whatever it is that predicts freshman GPA in students. This is why the elimination of the analogy items from the SAT was not a good idea. It may be true that this content is not a prominent feature of many college educational requirements; nonetheless, the items were a good predictor of freshman academic achievement and eliminating them in response to political pressure just reduced the predictive power of that exam. There is something about understanding how to complete an analogy that is related in some way to something that helps students achieve an acceptable GPA in the freshman year.
Neither the ACT nor the SAT predicts anything perfectly, but no one claims that they do. You appear to be equating imperfection in prediction with a complete lack of value--which is inappropriate. Imperfect measures still have value. If you don't believe that, next time you are at your doctor's for blood work, make sure he takes it all and not just a sample. The sample blood test result is correlated with your overall blood composition outcome if it were tested, but not perfectly! Nonetheless, a test of a sample of your blood has value in establishing probable information about your state of health...
You assert that grades are comparable. There is a considerable body of work that asserts just the opposite. The proportion of students reporting an "A" GPA has risen from less than 10% to more than 30% in the last few decades. Has overall student performance really increased that much? Are all the students reporting a GPA greater than 4.0--and there are a lot of them now--really so far above what was previously defined as "outstanding"? If this is the case, why has the proportion of students required to complete remedial work just to enter the standard freshman courses increased even more dramatically than the rate of those A averages? And to refer to one of your earlier issues with the tests, supposing we accept that the As and the Fs are useful indicators of academic qualities (questionable, but let's assume it). What about the "mushy middle" B-through-D students? How are they to be compared and sorted for admissions purposes?
I note with some amusement that you state that "As are earned for working hard and turning in assignments". Is that it? In your definition (and apparently in that of many of our schools), a student doesn't have to submit any actual correct answers or demonstrate mastery of the content to earn an A--just turn in the assignments. Interesting, especially in light of your claims about the properties and comparability of an A grade point average. I have little doubt you will claim that the correctness is implied--that's another point on which I suspect we disagree.
I'm willing to debate the tests predictor validity, I just don't think reliability is a strength...
I also never said that content was the issue. As a test prep tutor, I know all to well the role of "reasoning" in the test. But your factual statements are false. The elimination of analogies did not reduce the predictive power of the exam. By your line of logic, it should have. But it didn't.
http://www.omniaceducation.com/education-blog/bid/4863/Studies-Show-the-SAT-Still-Sucks-College-Board-Declares-Victory
However, I'm not asserting that the test holds no value at all. I'm asserting that the test holds less predictive value than grades do alone and that their predictive value isn't worth the price we pay to maintain the exams.
GPA is incredibly predictive. And one of the reasons it is so predictive is because that grade inflation happens at the collegiate level as well. There's nothing that renders professors immune to the same pressures!
So I suspect you and I agree more than we disagree outside of this conversation. I want students to be challenged and I'd like to see us raise our standards. Do you honestly think the SAT or ACT are helping us raise them?
Hmmm...don't know that a blog entry with a pretty clear viewpoint is definitive evidence of much. And since the entry refers to a point at which analogies were dropped and the writing component was added, it isn't clear that the loss of analogies didn't, in fact drop the predictive power but that the loss was compensated for by the addition of the essay (although a single-item test of anything is not a great approach).
Actually, GPA is not "incredibly predictive". Like the standardized tests, HS GPA is moderately predictive of college freshman GPA. And since everyone here states over and over that admission should not be decided by any single measure, let me also point out that, when you combine GPA and standardized test scores, you get more predictive power that either one alone! Ta-da--multiple measures work better than any single measure, and both have a place. Adding in standardized test score to the prediction provided by HS GPA alone does, in fact, increase the correlation and predictive power--so the HS GPA does not do the best job if used alone. Both HS GPA and standardized test scores contribute something unique to the prediction of college freshman GPA, and the combination is better than either by itself. Neither is perfect--neither is useless. Either measure alone is worse than using them both, so using them both seems the best approach.
Each measure provides value--and not the same value. If you'd like a reference that doesn't start with quite such an "in-your-face" viewpoint, Dr. Rebecca Zwick's book "Fair Game?: The Use of Standardized Admissions Tests in Higher Education" has a wealth of evidence gathered from empirical studies to support the use of multiple measures. And yes, I am aware that Dr. Zwick worked for ETS prior to her current university appointment; regardless of her employment history, the book is a balanced review of multiple studies from a variety of sources.
And of course I want students challenged and I want high standards--who can be against those things? However, I don't think it is the place of the SAT or the ACT or any test to raise our educational standards--they are measures of where our standards are, in one particular way for one particular group of students, not a driver of standards or reform. They are the mirror, not the face; the snapshot, not the scene. You may not like what you see, but they are just one image of the reality.
I am firmly against driving curriculum using a test, any test. Curriculum change should come from a variety of sources: parental input; student desires; research in the field; state standards; societal changes; innovative teacher approaches; college admissions needs; advances in technology; the list goes on. However, all of those approaches are hard and expensive if done well. Forcing classrooms to cover material by throwing it onto an exam then making the test required and high stakes is a poor approach. The teachers usually are not trained in the appropriate ways to teach the new material, and the pressure on students and teachers and administrators to get high scores leads to narrowing of instruction, teaching to the test, and a loss of breadth, depth, and creativity that is depressing to observe. If you want to raise educational standards, starting with the tests is, in my opinion, completely misguided. It is easy and cheap and fast--but still wrong.
Properly used, any test, including standardized ones, works fine as a measure of knowledge and skills. Standardized tests allow broad student comparisons that are not trivial without them. Students who are well-taught and who have learned a strong, rich curriculum should take tests with confidence and succeed with ease. Standardized tests are constructed to very high standards of quality, undergo rigorous review before use, and utilize strong analytic techniques to assure comparability over time and across groups--not claims that can be made of classroom tests or the HS GPAs that result from them. The consequences of misuse and overemphasis of test scores is not the fault of the test. The tests are a useful tool with a restricted, but nonetheless valuable, function. Put the fault where it belongs--in the hands of the mis-users. Most colleges use the SAT and ACT scores as one component of an admissions decision, a perfectly valid and appropriate use for which they do not deserve to be demonized.