Testing

I think most of us recognize the important role of data in decision making today, though this is one educator who thinks it has to done in ways reconciling of the importance of the qualitative assessment  and the significance of the unmeasurable.   I have written about this previously here.

But what data?   Standardized testing is the near-universal default for what is called measured "student achievement" but I want to take a look at what kind of testing we should be using, and could be using alternatively. 

I am following on the lead of Tony Wagner's Global Achievement Gap, which is greatly influential for me.   Wagner certainly acknowledges and accepts the importance of testing for accountability: "the fact that schools and districts are now being held accountable at all-- and accountable for the success of all of their students-- is a new and very important concept."

But the important caveat is, "do these state tests assess the skills that matter most for work, citizenship, and college?"    Employers surveyed did not rank mathematical skills as even being in the top ten list of most important.   It is not specific content knowledge that is important, but broader thinking and reasoning skills.   One international expert says in Wagner that "there are very different styles of assessment, but I would put the US at one end of the extreme, largely driven by efficient multiple-choice tests.... US students tend to be rather good in multiple choice tasks, when four choices are clearly laid out, [but] they have a much harder time when they're given open-ended tasks."  

When college students are surveyed about what they needed more of in preparation for college, they answer writing and research skills, time management, and learning to work with others in study groups, not more content knowledge.   

And the AP is no better, to Wagner.   It may have a reputation as a "gold standard of  rigor," but it is not deserved.  Studies have said it is not a predictor of college success.   It is textbook driven, dedicated to breadth rather than depth, lacking in critical thinking and analysis.  It is too much "focused on mastery of factual content-- at the expense of research, reasoning, and analysis."  It is not "merely old-fashioned... it is hopelessly obsolete."

So is there any alternative?  Yes.    There are many.  IB, PISA, CLA, ISkills, and Stenberg-Tufts, to name a few.  

To my disappointment, Wagner doesn't really engage with IB; this is all he says: "The IB has become more popular and demands more thinking from students in both course work and assessments, but it is far less prevalent than AP."    I am not IB expert, but have had two weeks of IB training and I want to venture that it is a much more substantial and rigorous in the right ways kind of assessment.   Tests are longer and richer; they employ exclusively open-ended questions with a great deal of choice, evaluating what kids know, not hunting out what they don't.   They measure strength of thinking and clarity of communication, not "having the right answer."   The structure of IB also requires community service and an extended essay.   

PISA (the Program for International Student Assessment) is an international test from the OECD, intended for 15 year olds.  To quote from its website, "In all cycles, the domains of reading, mathematical and scientific literacy are covered not merely in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life.In the PISA 2003 cycle, an additional domain of problem solving was introduced to continue the examination of cross-curriculum competencies."

CLA (College Learning Assessment) and the CWRA (College and Work Readiness Assessment) are two new related, and really exciting, programs to assess more authentically what really matters.   Both come from the CAE, the Council for Aid to Education; the first tests college freshman and seniors in an "open-ended, ninety minute performance assessment in which students have to demonstrate their reasoning, problem-solving, and writing skills while attempting to solve a "real-world" problem."  As an example, test takers are required to advise a mayoral candidate in a town with a high crime rate about which of several proposals for reducing crime might be most effective, reviewing documents and preparing a memo summarizing the pros and cons of different options.    Here is a link to more samples from the test. 

The CWRA is the CLA's extension to high school students, testing them in freshman fall and senior spring, and "teachers say it is the best test, by far, of the skills that matter most for college."  Speaking for myself, I think one of my first acts as a leader of a school including secondary grades would be to quickly adopt the CWRA. 

ISkills, from ETS, measures ICT, or critical thinking in the digital environment, how to navigate the plethora of digital information and appropriately evaluate and employ it.    It can be used for graduating high school students (I think, it was entirely clear) as a tool to test how effective kids are at managing information in the age of the internet.  Again, I'd love to jump on this myself as a school leader.   (The New York Times wrote about the on-line critical thinking topic in a popular article this summer, found here)

Robert Sternberg, whose work is essential to this topic, has ongoing projects considering how better to assess and evaluate for "successful intelligence"-- the stuff that really matters today.   His Rainbow Project, for the College Board, offered a better alternative, and was found to have demonstrated better college success prediction.  Unfortunately, it seems to have been put aside; Sternberg is onto new projects at Tufts.