STANDARDIZED TESTING INSANITY?

by Ron Willett

This writer was in the classroom for a quarter century, as an instructor and professor.  Even after leaving academia for industry, it turns out one of the functions of corporate leadership is promoting learning, by employees, customers, vendors, stockholders and stake holders, and even regulatory bodies.  That testing is more by boots on the ground, but the core principles of creating learning apply.  

As an academic, I designed multiple-choice tests, essay tests, problem-based tests, pop quizzes, open-book tests, take home tests, according to former students now friends or associates some nasty tests, and graded thousands of structured tests and blue books.  Scores were scaled, transformed, clustered, curved, et al., to try to properly assign metrics for achievement.  As a practicing statistician and researcher for most of that tenure, those tests were rarely out of a publisher's end-of-chapter boilerplate for teachers.

Reflecting, I also agonized over every one of those tests and their outcomes: Was the test a proper sample of the course contents; was it fair in difficulty and its language; was the grading fair and consistent; had multiple tests given across multiple sections, therefore changed to arrest cheating, been made equivalent, in contemporary parlance, “standardized;” if scaled, had results been correctly adjusted?  How would individual students respond to their grades and any comments?  Had I created a negative stimulus to learning?  Did the test create any learning?  Did the test results truly differentiate levels of student learning achievement?

In sum, achievement testing in education, K-12 or higher, is an essential part of assessing student learning performance.  K-12 public education characteristically coined another term, "formative assessment," to describe what good teaching has been doing for decades, but that in-process assessment and testing is a critical component of executing formal learning strategies and tactics.

So the issue and point of this blog is not whether there should be standardized testing in our schools, not even whether that testing should be reflected in high stakes assessment of student, school, teacher or jurisdictional performance, but precisely how you do that with integrity, and whether we have the knowledge and mastery of measurement to do that without causing damage greater than the good alleged?

There are strong indications that the current standardized testing vision, and its application to assessing student learning across schools, especially teachers, and longitudinally, are badly flawed.  That is the quest in this particular blog: What is standardized testing; when, where and how can it be employed for proper effect; how must its attributes be gauged to make it viable; and how is the present overreach likely to impact our public education system?

Adding some credence to these questions, the August 12, 2011 issue of the premier journal Science contains a well-documented review of the “value-added” approach to judging our teachers based on longitudinal standardized test scores.  Its author is a researcher at the University of Wisconsin’s Department of Educational Policy Studies.  Concluding on whether value-added can improve education, the review states:  “The statistical properties of value-added measures are unlikely to improve much.  According to Campbell’s law, using the measures in high stakes testing will likely distort the measures themselves and make matters worse.”  Then:  “Until researchers who have demonstrated the theoretical promise of value-added measures also demonstrate its effectiveness in practice, the vacuum of empirical evidence will continue to be filled by ideology and speculation.”  The writer might have added, and some really terrible decisions impacting our teachers where the process is misapplied.

In bolder terms, are we really in danger of further denigrating US public education, already damaged by a half century of dry rot and refusal to consider new ideas and ground breaking results from expanding neural biological research on learning?

What is standardized testing?

Sounds obvious, but in fact "standardized" is deceptive and the concept is quite complex.  Adding to the confusion especially among our policy makers, what is being called standardized testing is rarely standardized at all, with discrepant elements appearing through all aspects of that testing.  These discrepancies make up the basis of much of the current critique of NCLB, RttT, and the "value added" models being employed to assess and even fire teachers.  In fact, the sloppy and in some cases ideological use of flawed measurement is almost criminal, because of the abuses of measurement logic, incompetent administration, and flawed judgments being imposed on both students and teachers.

Start with simply definitions.  Standardized tests as they are being used are subsets of the generic category achievement testing.  Achievement testing has likely been with us as long as there has been a concept of formal teaching.  

Perhaps the most famous unknown in American testing is E. F. Lindquist at the University of Iowa.  In a prescient paper over a half-century ago Lindquist articulated the pros, cons and criteria for defensible standardized achievement testing.  Unknown except to the few deeply into testing theory, Lindquist was hardly anti-testing, and was one of the developers of the Iowa Tests of Basic Skills, the Iowa Tests of Educational Development, the ACT, the high school equivalency GED, and the National Merit Scholarship Qualifying Test.  His paper basically lays waste to most of the present standardized testing protocols.

Lindquist's arguments are highlighted and extended by Daniel Koretz, an education professor at Harvard, and one of the reigning experts on educational testing.  The substance of his 2008 book, Measuring Up:  What Educational Testing Really Tells Us, should be taught to every would be K-12 teacher and administrator, ironically with aggressive formative and summative testing of their grasp of its concepts before they are unleashed to staff a classroom or lead a school.

An additional caveat, standardized testing has come to be associated with multiple-choice tests, sometimes pejoratively referred to as “bubble tests,” but that is not a correct assumption.  Any test theoretically could be made a standardized test by using the protocols that seek to assure comparability of results across students, classrooms, schools, districts, time periods, and even states. 

At the level of grand design, a standardized test in K-12 is any test where student achievement being measured is attributable to that classroom’s practice, and free from competing explanations for the results across all of the causes, conditions and environments present.  Even with little specific knowledge of experimental logic, the reader can quickly sense that it could be a challenge to qualify a test as standardized when it is taken by a dynamic group of children or youth, then multiple groups, with different prior experiences and achievement, from different sub-cultures, with varying pre-test immediate experiences, for different knowledge areas, in various classroom physical environments, at different points in time, even in different inside room noise levels-temperatures-humidity, even whether the sun is shining to push the envelope.

There are multiple critical issues:  An assumption – and another dimension of whether that test result means anything – is whether the test itself and the individual items or questions elected measure what was intended or appropriate; another, establishing those protocols that will assure you really have standardization is metaphorically just a tad short of trying to find the Higgs boson given our present tools for creating tests and assessment. Virtually never mentioned, what we call knowledge is now doubling every decade, as well as being corrected for historical errors of interpretation, meaning that the sample of learning acquired in any test is getting more and more selective as time marches on.

You’re measuring what?

Put this into a real world context.  Imagine 30 eighth graders, reflecting an assortment of backgrounds, personalities, sub-cultures, parental interest, present moods, and other school exposure, are given a “standardized” test on science.  You are supplied from some distant test vendor, a 50 item multiple-choice test, created by an equally distant test designer who has never been in your classroom, perhaps any classroom recently, using an unknown sample of science knowledge and protocol to select 50 items to constitute that test, covering a fragment of the learning you believe you’ve stimulated over a term or year.  Across the city, town or county, another 30 eighth graders are administered the same test, but at a different time in a different environment, with those 30 students representing different backgrounds, personalities, cultures or sub-cultures, prior learning, and contemporary mindsets.  Employment of you and your peer across the way is to be determined by the results of that testing, allegedly measuring precisely and exclusively what either of you, uniquely, has managed to instill as science learning in your respective 30 kids.

Just a common sense question, that has a long history of measurement science and research at its roots, how confident should you be that when you’re fired because of that test’s scores you had a fair hearing?  Multiply that isolated testing challenge by a large multiple via accumulating those results to rate schools, jurisdictions and even states; room for error?  There’s an old saying for those of us who entered the computer age at its tender stage, before Bill Gates and Steve Jobs were out of their nappies, and a window was just a window:  Its acronym, GIGO; its definition, garbage in, garbage out.

The earlier cited work by Daniel Koretz is pretty dry reading, but it is a quality assessment by a competent testing professional on what we can measure and where those measurements can supply usable information.  The above “competing explanations,” which is simply an eloquent way of summarizing effects, explodes in your face when you open the box.

First, the question breaks into two questions:  The test itself; and how, where, when, and to whom it is applied. 

A test is almost invariably a specific sample of some aspect of learning or achievement from the universe of what is covered as instruction in the particular discipline.  To have it be otherwise would make the testing approximately the equivalent in scope and time as the original instruction.  Who selects what elements of knowledge or learning achievement are to be tested?  Who elects what specific property of what is learned to be the object of testing?  Is the test arbitrary or to be used to reference a student’s result (or accumulated results) against the norm of a larger universe of students’ results? Alternately is any result compared to some standard of learning, i.e., standards referenced?  Who selects the kind of test – i.e., the complexity or scope of each topic tested?  Does the test expression validly match what you wish to know, construct relevance?  The disconnect between the learning model(s) and the testing models becomes a yawning gap.

Then the issues really get hoary:  Validity – more than one type, construct underrepresentation, content versus performance standards, standard error of measurement (test-retest reliability, not traditional sampling error), sampling error resulting from who is tested and how they are specified, scaling or relating test scores to meaning, linear versus non-linear scale properties, bias, the real world tradeoff between the complexity of what is tested and reliability of results; and on.  Overarching the technical issues, present standardized testing has delegated answers to most of the above questions to a black box of test designers and psychometricians, all in a different place than the teacher in the classroom. 

Lastly, there is the classic wrecking ball of “correlation is not causation.”  One of the more common explanatory mistakes, even for professionals, is equating two or more sets of metric data because they demonstrate superficial association.   As test results proliferate, with pooling of data to move from usually defensible results within a classroom for specified materials, to comparisons of schools and longitudinal performance, the chance of visually suggestive associations increases.  Even most professional educators are inadequately trained in interpreting what is termed statistical correlation, much less multiple and partial correlation, factor analysis, and other tools that have been used at a research level to try to diagnose and verify such associations.  Even when there is a replicable statistical fit, the issue of causation is a logical evaluation, not just number manipulation.

A perspective comes from contrasting the models where measurement science is confident, with the kinds of testing involved in K-12.  It has been a challenge in social research to successfully install classic random assignment and experimental treatments with controls in the simplest economic and social experiments, and usually with few variables to make explanation robust.  In the kind of standardized testing called out here, throwing the results from millions of test takers, from thousands of schools, and an unknown mix of prior and environmental conditions, into crude classification “grades” for schools is an oar short of rowing to some reasonable destination.

One blossoming and terrible result for K-12 education, call it cart dragging the horse, what is taught and billed as learning is increasingly being determined by what is coming down the pike as testing, and there is a full disconnect between what that on-site learning effort should be and emphasize versus the origins of the tests.  The defensive albeit unethical result from our schools has been dubbed “teaching to the tests.”  It has even resulted in a caricature of serious testing, an abomination of more time devoted to trying to game the design of the options in those multiple-choice questions to try to increase scores, than had been devoted to creating usable learning.  Even more destructive of our education systems, full cheating via changing or manipulating actual test results is turning into an epidemic.

Question, why are we testing?

Most thoughtful critics of the present trend in testing, to force standardized testing into every aspect of K-12 change, make a highly valid point:  Present testing has done a tectonic shift, from being primarily a valid tool for diagnosing the effectiveness of classroom efforts to create learning, to being Thor’s Hammer, a “get tough on education” method of forcing accountability.  It offers little comfort to students impacted by flawed testing, or teachers fired for test results, or schools condemned on the basis of those results, that public education over a half century shot itself in the foot resulting in NCLB and its devil’s spawn.

But the shift has occurred, being aggressively promoted by our corporate business community on the premise that our schools should be more “businesslike,” and by a cabal of corporate test creators, vendors and scorers for profits. It is also being politically driven by conservatives because it has an aroma of control and discipline, and Federally because however crude, it is the only tool Constitutionally available.  Curiously, the one reason not promulgated is that it will demonstrate how to improve learning.  You judge, is it likely to enhance K-12 education, or create more problems than solutions?

The central issue is that at every stage of aggregation – from a student’s scores, to classroom level, to school level, to district, to state, across time periods – standardized does not necessarily mean equivalence.  It is in classic research parlance, conducting a massive set of experiments, with high stakes impacts, but without the controls that assure that student achievement or learning produced by its classroom mediator, and only that specific measurement is what is represented in the number tossed out with little regard for the underlying assumptions.   Arbitrary assignment of value judgments to results, such as Ohio’s (and other states’) crude school ratings now issuing is even more suspect.

Standardized testing has become an educational “wave,” almost a flash mob of rabid reformers, in railroading students and schools.  The comparisons claimed by those advocating high stakes testing and present standardized tests as the mechanism for improving learning simply don’t hold up as defensible when subjected to the standards that, for example, Koretz details.  We aren’t yet knowledgeable enough about learning as a process, or able to dictate the conditions for testing, or in possession of needed statistical discriminators, that allows absolute judgments of performance to support broadly changing educational infrastructure.

Does this take testing off the table as an educational tool?  Hardly, but it should place its use back into the hands of those in the classroom creating learning, for the purpose that makes most sense, diagnosing whether the classroom performance or any other medium is achieving desired learning, whether it is in process, or a summary effect needed to assess whether resources are being applied effectively.  A proposition is that the present trend in using testing to nationally penalize or award resources or systems is dysfunctional.

The counter to this is the large common sense question, if not such testing, then how do we hold our schools and educators accountable for doing their job, for producing needed student learning, and operating at desired levels of productivity?  Trust them? 

That’s a legitimate question, but it has a business counterpart in the distinction between traditional manufacturing quality control based on inspection, versus a contemporary concept of quality assurance based on process control.  In the latter model, by assuring the perfection of the inherent component process(es) (in this context, teachers, administrators, curricula, technology, assessment models) you ensure finished product quality.  You rarely inspect because the quality of the output is determined by the earlier control of the processes involved.

It is ironic and just short of condemnation, that the gang of school reformers smashing our public schools with standardized testing, are using last century’s obsolete management model of quality, while hypocritically trumpeting the need for sound business principles to be applied to education.  It appears that neither our reformers nor our public education establishment has managerial credibility.  Perhaps it is also that to establish educational process control means better teacher training, screening for competent administration, curricular reform, reforming school board oversight, understanding technology, and allocation of resources to learning first, to sports and bricks and mortar a distant second?  Called change, says easy, does hard.

But it also reflects a simplistic view of an enormously complex system, and the assumption that you can crank out a simple measure that will validly differentiate education that’s working from its obverse, and by cracking the whip.  Realty is that testing linked to the classroom, when it has been properly trained, works, even when there is a good deal of diversity in how it is handled teacher to teacher.  We should not be creating teachers with the notion that their skill level is limited to posturing at the front of a classroom, or on the side in a rocking chair in one of the more bizarre manifestations of pedagogical theory.

Where large scale results are necessary, testing like the National Assessment of Educational Progress (NAEP) has proven viable; the ACT and SAT testing less so but still effective.   Futuristically, but not very far out, there needs to be a “race to the moon” effort to create testing models that can get beyond simple memorization of facts or simplistic relationships, and test more complex integration of learned information as it applies to problem solving.  Paradoxically, Harvard’s Howard Gardner, creator of the concept of multiple intelligences, pioneered trial versions of such testing decades ago, but it was never followed by subsequent recognition and development.

Pointing up the movement

At the minimum, there needs to be some common sense in allowing the Federal government to develop some nationally comparable testing, ala NAEP, that can be used to with validity assess learning achievement across schools and our states.  Controversial, yes in the current political muddle, but the present testing game allowing states to choose to test what each considers knowledge is a fools’ paradise.  Present state deception, provinciality, ego, or belief in magic, by geography, simply isn’t supported by how knowledge is evolving. 

Lastly, the multiple-choice testing being saturated into K-12 is pushing precisely the wrong learning buttons, emphasizing isolated components of memory and narrow concepts rather than their integration into constructs that explain real phenomena and enable prediction, science’s sine qua non.  Ideology being vocalized, that we need to get tough and slam those public educators and their social engineering, can seem attractive because of the arrogance frequently demonstrated by our schools of education and self-righteous public educators/administrators in silos, but the game played that way has never in the history of education produced real winners.

Perhaps the best way to summarize a perspective about standardized testing as it is being used presently is to resort to a hoary old saying from bygone days of teaching and designing tests.  Present standardized testing, by those creating those tests in absentia, and heavy-handedly using their results to flog students, teachers and schools, as the way to improve K-12, are attempting “brain surgery with a meat ax.”

Nuance and incremental change in inducing positive change in our K-12 schools are not big in present reform circles.   They should be, because on its present course reform based primarily, or even heavily on use of standardized testing as it is being postured has the potential of pushing public education into a more fragile and dysfunctional position than present when it exited last century.  The reform movement should heed an ancient admonition, “first do no harm.”

References

For readers who haven’t yet challenged the literature of testing, the following books are entry points:

Daniel Koretz, Measuring Up:  What Educational Testing Really Tells Us. Cambridge, MA:  HARVARD UNIVERSITY PRESS, 2008.  ISBN 978-0-674-02805-0

Diane Ravitch, THE DEATH AND LIFE OF THE GREAT AMERICAN SCHOOL SYSTEM:  How Testing and Choice Are Undermining Education.  New York:  Basic Books, 2010.  ISBN 978-0-465-01491-0

 

Additionally, on a day-to-day basis, some of the very best commentary on K-12 reform and testing can be found in the blogs edited by The Washington Post’s Valerie Strauss, in its Education Section, "The Answer Sheet."  Many posts are by seasoned K-12 educators, education administrators and researchers.

EDUNATION - GETTING A GRIP

Next to jobs and the debt limit brouhaha, the status and future of U.S. public education has become one of America’s head-scratchers.  Attempting to make sense of the different positions on K-12 change reminds one of a TV gambit from the 1960s, a 60 second high speed presentation of graphics about some newsworthy item.  Too fast to register the significance of a single graphic, drinking from a fire hose, the medium was the message, conveying diversity and complexity, defying a normal mortal to comprehend.

Welcome to U.S. K-12 reform, charters, vouchers, Duncan’s duplicity, Gates’ gambits, misdirected acronyms, standardized test hyperventilation, value added (test) assessments to judge teachers, waivers, or whatever is trendy at the moment.   What is not obscure is that the end product of over 350 years of attempted education, starting on our shores before this was even a nation, is now a flat spot threatening to fracture the public education model.

The available lenses to view K-12 change are not countless, but so numerous that the next thousand words wouldn’t exhaust a précis of the approaches.  So, with a nod to practicality, let’s zero in on a couple of roots to frame the issues, and then move to current events.

Roots

If you are adverse to ambiguity, or nuance is elusive in your assessment processes, this is a good time to leave this blog.  For in our history of public education there are few straight lines, or simplistic connecting of the dots, or even universal goodness and prescience, including Horace Mann, father of the “common school” movement.  Indeed, there is reason to believe that few current teachers or administrators, unless they have pursued research tracks in education, have a full and clear view of how public education in the U.S. unfolded, or what it has become.

A single example may illuminate the point.  Simply Google “history of U.S. public education,” and prepare to be underwhelmed.  With the exception of a few stand out critics, what you will get is a vanilla, sugar-coated, pristine view of our public education system as goody two-shoes with a sports addiction and wrapped in the American flag.  If you dig a bit deeper, a far more nuanced picture emerges.

Virtually all education on what would become American soil prior to the 17^th Century was by and within families.   The principal shift in the 1600s started with the Boston Latin School in 1635, organized to educate clergy and public officials. Massachusetts was the center of gravity of all formal education in this period, culminating in The Massachusetts Law of 1647, decreeing that every town of 50 families hire a schoolmaster.

Through the 18^th Century primary and secondary education was open season for training provided by religious organizations, private contractors and companies, and virtually anyone who could set themselves up as a schoolmaster.   But two events in that century presage present education:  In 1787 the Northwest Ordinance stipulated that land in every new state be reserved for education; and The Bill of Rights was passed in 1791, that by exclusion of education made it the province of the states.  Our Founders demonstrated remarkable perspicacity in anticipating how politics might unfold in a new nation, but not unexpectedly, had little awareness of how science and the knowledge thus generated would change the need or format for education in the next 235 years.

An interesting sidelight, in 1779 Thomas Jefferson proposed a two-track education system, for “laboring and the learned.”

The pace quickened in the 19^th Century:  In 1821, the first public high school opened, Boston English High School; the McGuffey Readers were published; Horace Mann became Secretary of the new Massachusetts State Board of Education in 1837, launching what would become the “public common school“ movement; but Catholic immigration and schools raised public alarm and bigotry that actually helped popularize secular public school evolution.  It is worth noting that even as public schools proliferated private schools and church schools continued to serve education market segments, frequently being the high end of K-12 performance.

For the next hundred years both educational theory developments and structural evolution around two world wars, and increasing state government sophistication, shaped and hardened public school organization and operations.  The proliferation of teachers colleges and schools of education attached to our universities cemented the philosophies of public schools in place, but with genealogy that traces back to the common school ideologies. Contrary to the belief that teachers’ unions are a contemporary phenomenon, The National Teachers Association was formed in 1857 ultimately becoming the NEA.  The first mandatory student attendance law was enacted in 1852 (Massachusetts), and by 1918 all states had such laws.

Now politically incorrect, but factual, migration of students into K-12 collegiate education after WWII resulted in many of those students coming from the intellectual bottom one-third of the student barrel.  As late as the end of last century, and perhaps to this day, the assertion is that our schools of education are doing an inept job of training teachers, substituting indoctrination in rooted education beliefs for both substantive subject matter knowledge and the flood of neural learning research findings emerging, as well as failing to equip teachers to deal with classroom implementation.  A result has been the heavily hyped but overall inconsequential appearance of programs such as “Teach for America,” that are a drop in the bucket among almost 3.5 million public school teachers, handcuffed to frequently managerially unprepared education administration and highly variable school board preparedness and oversight.

Lastly, what is devaluing contemporary public education, and that set the scene for NCLB, our public schools over the last half of the 20^th Century became vehicles of retro education and weak STEM (science, technology, engineering and math) work, dropping the U.S. in world position.  This was complicated by union intransigence and teacher salary relative deprivation, finally precipitating a clumsy NCLB signed into law in January 2002, mandating high stakes student testing and school yearly progress requirements based on those tests.  America flopped from the frying pan into the fire.

Where are we?

Points of departure for inspecting the present debates about K-12 change are examining the wake-up call, assessing just what change entails, and how it should be enabled.

History provides a useful perspective in a nation contemporarily hobbled by the incapacity to simultaneously think and act strategically without acrid partisanship, and continuously pummeled by media that have even shorter attention spans than most of our citizens.  From the mid-17^th Century through the mid-18^th Century virtually every advance in education originated in Massachusetts.  Perhaps no surprise then, that Massachusetts has consistently been at the top of the heap in K-12 performance.  But worth reflection, it took over 100 years of experience to infuse that level of alacrity and competence.

Fast forward to the present debates about how to improve public education in the space of a few presidential terms.  Combine that naivety with the problem’s mass:  50 states each capable of going their own way on school curricula and governance, with politicized state boards of education and departments; 3.2MM public school teachers; 13,800 public systems; 13,500 local school boards, many fundamentally ignorant about real education; not only teachers’ unions, but principals’, superintendents’ and school board interest and lobbying groups; a deeply entrenched public education mentality steeped in visions of social engineering; and schools of education that tended to isolate themselves in the higher education community, partially because they were considered second-class academic citizens, but perhaps equally to protect a self-perceived identity and entitlement.

How public education evolved into a failing institutional mass the latter half of last century is also easier to understand if one looks at both the foregoing, and that history. 

Public school conceptualization started with elitism and well-intended self-righteousness, and with a likely genuine if debatable belief that the proper role of the public common school was to make your children wards of the state, and turn them into proper citizens.  Public education went through multiple phases of attempting social engineering in the 20^th Century succeeding principally in dumbing-down K-12, barely noticed by parents striving for their children’s education.  Couple this with low financial compensation of teachers.  Their rewards had to be internalized, and took the form of self-esteem, deep commitment to the profession, and even a sense of martyrdom for giving but receiving less financially for their education and investment than other professions.   The effect is at core no different than how any ideologically-driven group comes to play the game; tight identification within the genre, build defenses, avoid transparency of your actions as protection, develop self-righteousness, develop countervailing power groups, e.g., teachers’ unions.

History also teaches that these are properties that cause bureaucracies to evolve to protect the status quo, create risk aversion, and by definition suppress creativity and innovation that require risk taking and organizational openness.  Ask your local public school system to be transparent, display their education assumptions and theories, provide vita for all teachers and administrators, provide their textbooks and lesson plans for inspection, ante up their budgets beyond state minimal reporting, share system ACT and/or SAT results, show their plans for teacher development and research on classroom practices, show their technology plan, and respond with interest to ideas from parents and their taxpayer stake holders.  You won’t be pleasantly surprised.

By the mid-1980s the increased obsolescence and degraded performance of many public systems were broadly visible, juxtaposed against the performance of nationalized systems in European countries that had less diversity and more control.  This finally culminated in “A Nation at Risk,” a report commissioned by then President Reagan, who totally misread its outcomes.  Instead of a naïve assumption that the group would recommend restoration of school prayer, and dissolution of the U.S. Department of Education (in existence since 1867 to the surprise I suspect of many readers), the group correctly predicted the course of K-12 over the next couple of decades.  Mr. Reagan promptly dismissed the study group of some of our best and brightest and ignored the report.

The rest is also history but a pretty shabby version.  In existence under another banner for some time, No Child Left Behind (NCLB) was adopted in Mr. Bush’s administration, relying on standardized testing fostered by a control mentality and the cluster of corporations who saw profit in the construction and scoring of those tests, and in a captive market and draconian threats to public school systems that conservatives didn’t much like in the first place. 

NCLB accomplished one material thing; it exposed in clear relief in its first years of testing the public K-12 systems that were failing, dysfunctional, and dropout factories.  To that extent it merits applause. 

But instead of evolution of NCLB into a proactive strategy during the Bush years, or intellect taking over with Mr. Obama’s election -- the next step in-depth work on how learning could be enhanced in the classroom and adoption of both new neural learning findings and digital technologies -- reform came to be stylized by the same testing as NCLB but even more extensive.   Over $5B were thrown at Race to the Top (RttT) which was nothing more than warmed over protocol from last century, perhaps with the belief that if enough schools followed the drills improvement might happen. 

There is almost no escaping the conclusion, that while both Mr. Obama and Mr. Duncan intellectually know that standardized testing fails in measuring genuine learning – they have publicly stated as much – it and Federal funds are what they have to manipulate public schools.  Also political points from trying to claim some short term gains, and reluctance to offend a liberal public education base including the teachers’ unions, scored far higher than properly setting a new course and long term strategic plan for the nation’s public education.

Not widely known, the U.S. Department of Education supports a small army of PhD resources in its National Center for Education Research (NCER), and funds an array of academic research efforts on education.  The issue is that the majority of these projects are probably of quality, but also highly fragmented research that never gets knitted together into coherent K-12 strategies.  Its voice, an Internet web site called “Doing What Works,” produces fragments and tactics for improving teaching, but seems boxed-in by Mr. Duncan’s and the Department’s tunnel vision.  The one NCER research area that could maximally serve public education, on organization and education leadership, is ignorantly managed and short of needed research on an area that may be far more responsible for U.S. public K-12 mediocrity and malfeasance than training of teachers or gaps in curricula.

Complexity and dogmatism

These two words to a large extent characterize the buzz saw public education has invited by management mediocrity and the prevailing process of circling the wagons, rather than redefining missions and employing the intellect it claims to innovate.  The very first phases of any strategic plan need to be some hard questions:  How did we get so far along without someone shouting, “time out;” who defined the problems; who defined and how was the organization of solutions parceled out; who defined the standards of performance, and who is keeping score?

Our public education system is also by no stretch monolithic, but highly fragmented in both educational environments and oversight.  If nothing else, leaving education to our states, compounded by a fierce if unthinking commitment to local control, ensures that any one-size-fits-all policy will be a train wreck.  But there has been little if any attempt to segment both the environments for change, nor attempt to see if there is a match between the diverse forms of underachievement and prescriptions for change.  The poorest urban school is subject to the same treatments as relatively affluent rural school systems, yet failures regularly occur in the latter though the underlying bases for failure are quite different.

Given the rigidity and pragmatism of the present solution set – test, test, test, then penalize for inadequate progress – who devised this model?  There are clues that it didn’t originate with the champions of genuine learning in this nation, but was set in motion by a tortured and misguided analogy between our economy’s private sector production systems and the process of churning out students with certain achievement properties.  Would NCLB and RttT have been different if the guiding hands were those doing contemporary neural research and those resources had greater political clout?  Would they have been different if the prime movers had been acknowledged high-level students of institutionalized learning with full awareness of our present public K-12 infrastructure?  Would they have been different if they had been based on critical thinking rather than politics and rigid ideologies?

Enter local control, introducing a major conundrum.  There are sound arguments for local control of public schools because of how education is funded and because of their need to reflect in their operations some elements of local culture.  But how reconcile the concept of broad local control, including what happens in the classroom because of control of administrative and teacher hiring, with knowledge expansion and universality?  Math, physics, history, reading, et al., in Massachusetts are still math, physics, history, reading, et al., in Texas, even if the latter’s governor and legislators believe it takes divine guidance to define the subjects.  Instances of local boards and systems trying to chase America’s schools back to magic by reintroducing religious mythology into curricula, and as noted in the last issue of Science, small groups of extremist parents turning into a “hate lynch mob” -- trying to block even use of the words “climate change” in factual courses on environmental science, with cowardly school administration capitulating -- are two examples of the damage local forcing of education can create.  Others are gutting of academic standards, cheating and “teaching to the tests,” throwing an iron curtain around a school’s true performance and finances, and cynical use of sports and boosterism to deflect dissent.

Beating up local school boards, while it seems so totally justified by their frequent ignorance and intransigence, won’t suddenly turn our schools into learning communities.  Coincidentally, unless the Constitution is changed, this mechanism will still control much of future local educational process, and those boards would have to become more aware of real learning and discover how to interact with their stake holders in a collaborative way to advance public K-12.  Present propensities are pulling up the drawbridge to information about what a school system is really doing and delivering, as peak school administrators’ tactics of choice to protect their views from assessment and their positions.  That suggests that post-secondary education and at least functional literacy be required to run for board positions, and subsequent educational awareness training be required of the elected as a condition of being seated and serving.

Lastly, a convergence of the one place our nation is still innovating – highly refined digital technologies that are changing the very meaning of knowledge and its use – with an unsustainable funding model for public K-12 education, has redefined the learning playing field.  The whole “seat time” model of U.S. public education, created by the Carnegie Foundation for the Advancement of Teaching in 1905 (the roots of the “production” model of education inappropriately pushed in that period), persists to this day, deforming just about every attempt to install and validate more productive learning approaches.   Public education, intended or not, has managed to embed its feet in clay that has paralyzed both education and the economic creativity dependent on its delivery of better learning; U.S. public education crippled itself, and seems incapable or lethargic in finding the buttons or courage to refresh beliefs and select a new vector and gear.

Futures?  

A conclusion perpetually ignored by those seeking quick or simplistic answers, there is no silver bullet for remaking U.S. schools.  Increasingly draconian and invalid testing now being used to hire and fire teachers will simply ramp up resistance to all change, already evident in Duncan’s proposed NCLB waivers (with strings attached).  Simultaneously, there is a massive need for testing protocols that test for critical thinking and problem solving, and to measure the productivity of key resources poured into our youths’ education.  That means projectable research at the classroom level and new and better models for measurement of more complex learning than memorization of facts, not just a bigger test ball bat or more times at bat.

Issues go even deeper into basic philosophy.  Is the express purpose of our 9-12 or post-secondary schooling now preparation for an entry-level job?  In the present environment of knee-jerk reaction to both Keynesian and structural job displacement, are we creating a sub-optimal education system that may be irreversible and that we’ll regret?  Have we unintentionally fulfilled Thomas Jefferson’s 18^th Century proposal, a two tier education system for the “laboring and the learned?”  If so, what does this say about American values of egalitarianism, equal opportunity, and class mobility, much less understanding of our society and its civic values, and our evolution as a species? 

At the output end of K-12, a growing criticism of its function has been failure to prepare students for transition to post-secondary work.  In fact, there has been a two-century disconnect between U.S. primary/secondary education and higher education, each acting as if they were just ships passing in the night.  But in the wake of escalating mainstream collegiate tuition, there has been an explosion of so-called community colleges and satellite campuses.  Frequently employing both unqualified faculty and academic administration, lacking quality controls for curricula and teaching, are they more than high school two (or too, your choice) than higher education?  Even when they are attached to an accredited university they may fit into that box.  Their administrations, in turn, appear more dedicated to acquiring public construction dollars, and building sustainable empire, than creating even credible education.  Perhaps an alternative is a nationally accredited cluster of online colleges chartered by groups of states -- thereby conforming to the Constitution -- that can use the best of our curricula, and leverage competent faculty, rather than continue the dumbing-down of U.S. higher education.

Meanwhile in K-12 systems, rejecting or firing teachers for narrowly conceived poor performance art won’t improve learning.  Simultaneously, teachers have to be educated and trained for classroom effectiveness in some formal way; our alleged schools of education may need to be taken apart and reassembled with better parts and leadership, and greater requirements for subject matter knowledge.  At the moment the majority of those schools are in denial and our university leaderships appear too wimpy or politically correct to make the call.

And while the nation is gearing up to beat up our K-12 teachers, many of whom are genuine "Mr. Chips" and in the game by the pure motivation to teach and serve their charges, the currently worst K-12 culprits are likely its administrators, poorly trained managerially, lacking oversight by boards they regularly seek to manipulate for the precise reason of avoiding oversight, and in the process demonstrating the universality of Lord Acton’s dictum about how power corrupts.  Our public K-12 systems could probably take a giant step forward if all of its superintendents had to be recertified as capable of leadership, or that specific function was replaced by a new organizational model and human resources with generic managerial expertise -- principals remaining chief academic officers -- rather than alleged educators with a yen to command and control, or simply pursue the higher dollars.

There appear to be ample topics for future blogs and debate if one can forgive the understatement.  Perhaps the most discouraging aspect of our current national K-12 education machinations is the tendency for otherwise intelligent and learned leadership to cheapen these debates by reducing the issues to one-liners, or resorting to simplistic slogans rather than defensible argument.  Opportunists trying to put their stamp on our nation, winding up merely deepening the demagoguery surrounding K-12, for example, the well funded but frequently inept and arrogant initiatives of Bill Gates, may match that.  We need some gateways and a segue to better education problem solving, but maybe fewer gates?

Next blog, into the teeth of the beast:  Standardized (not really) testing; why, who, what effects, and what options?

Ron Willett

COMING SOON...

A web site that will seek to illuminate some of the issues that surround K-12 reform efforts, and that have created rifts between those seeking reform and our public education establishment.

The advocacy of more standardized testing – tied to a rewrite of NCLB and the provisions of RttT – colliding with the concerns from both teachers and most legitimate educational professionals, has begun to appear almost as contentious as the recent debt limit brouhaha.  Adding to the debate have been the elections by some states to directly tie teacher assessment, rewards, and even employment to the results of that testing. 

Legitimate students of K-12 learning have been both vocal and persuasive in arguing that the kind of learning enforced by reliance on high stakes testing contributes minimally to the development of critical thought, problem solving and creativity.

On the side of those unilaterally pushing such testing, the arguments are less intellectual than pragmatic, driven by realities:  How U.S. public education has slipped in world positioning over the last decades; the sheer magnitude of the challenge to upgrade thousands of systems while lacking the power to override local control of education; and by frequent failure of public education bureaucracies to adjust standards and police their own profession.

Some of the issues to be addressed include but aren’t limited to:     

• Synthesis of arguments about K-12 reform.
• Opposing views on standardized testing dominance.
• New neural findings on learning and their application.
• Questions parents and taxpayers should be asking their systems and boards.
• Opportunity for teachers and parents to express their experiences and views about any contemporary aspect of learning improvement.
• A platform for reporting examples of systems that have created successful learning change, and that have been able to straddle substantive education change and still meet standardized testing challenges.
• A platform for teachers or parents to anonymously report systems cheating or teaching to the test and covering up those performances.
• Lastly, but it might lead the list, what do today’s students as early as sixth grade think of, and want that primary and secondary education to be?  They are surprisingly vocal and perceptive even if their values are seldom reflected in the debates.

The site will be structured to invite crowdsourcing of experiences in K-12, critique of present models, ideas for change, and examples of both great schools and ones not so great. 

A mission component is to invite conversations among parents, teachers, administrators, and including present students and those recently thrust into the real world by graduation and job seeking.  That eclectic exchange seems missing in many blogs that are excellent but attract only a few segments of the total audiences for education action.

Please keep checking on the site – launch should occur before our schools reconvene for the 2011-2012 school year.  In the interim, this is an invitation to any viewer to input ideas on how the site might make the most useful contribution to current reform deliberations – direct ideas and comments to Ron.


Ron Willett