A Mile Wide, A Mile Deep, Part 3: Assessment

Preface

Somewhere down the longitudinal trail a brave historian is going to ask the question:  How could a generally literate society not stand up and protest the destruction of real learning in its public K-12 schools:  By a U.S. Secretary of Education observing learning through a straw, while his chain is being yanked by Bill Gates; by a cabal of socially irresponsible testing corporations; by a generation of for-profit psychometricians operating in corporate bubbles; and by states’ political infrastructure putting ideology ahead of a nation’s next generations of citizens?

The companion more localized question is, how could our collegiate schools of education, and virtually entire local K-12 public education establishments, express that ignorance, or be sycophant to alleged reform pivoting on a naïve form of testing?

If you believe that improving U.S. public K-12 education goes beyond alleged standardized multiple-choice testing, and are a fan of Malcolm Gladwell (The Tipping Point), take solace in some recent developments.

Straws in the Wind

Unknown to many Americans, the U.S. emulated the UK in pushing standardized testing, notably because of Margaret Thatcher’s advocacy in the 1980s.  Other countries then emulated those U.S. public primary and secondary education strategies.  Specifically, the use of standardized testing has been prevalent in Israel and in other parts of Europe, and long enough for its efficacy to be assessed.

Domestically, California has now voted to drop most K-12 standardized testing.  New York State has now stated it will reduce its use of that testing, and critics in that State are advocating even broader cuts.  Given that CA and NY are generally where trends begin, perhaps we are seeing approach of that “tipping point?”

Internationally, Scotland discontinued that testing in 2003.  Israel has now discontinued the testing.  Wales recently rescinded most standardized testing, and the reasons are notable: 

“What do Welsh teachers use instead of the tests? With government guidance, teachers come up with their own assessments and report the results to parents, local education authorities, and the Welsh government each year. Freed from the need to prepare students for narrow tests, secondary school teachers employ out-of-school experiences, in-depth research, and presentations, emphasizing applied learning in secondary school and underscoring the importance of play in early childhood education.

Brian Lightman, head teacher at a secondary school outside Cardiff, Wales, helped pilot some of the new approaches and is impressed with the results. ‘Our students now are so much more independent and capable of organizing and analyzing what they're doing, and they're able to improve as a result of that,’ he said. ‘They are very different in the way they go about their learning.’”

Only the U.S. appears still fully in the grip of something close to mass hysteria – or perverse dedication at our state levels to extreme conservative school ideologies executed pretty much without critical thought.  Unfortunately, this tunnel vision extends all the way to Arne Duncan, and one step beyond to the gross hypocrisy of President Obama.  Mr. Obama, in virtually every speech touching public education, churns out the right words about the need for K-12 understanding and the limitations of present testing obsessions, but then blesses the actions of Duncan and the U.S. Department of Education doubling-down on testing imposition.

With the above bits and pieces suggesting emerging challenge of the standardized testing orgy, why is it so deeply entrenched?  By the ignorance and political extremism of current Republican state education bureaucracies and legislators?  Or by the combination of naïveté and cowardice of too many of our public schools in America’s “Pleasantvilles,” lacking intellectually and managerially competent administration and better training than being turned out by our collegiate schools of education?  Or by generations of parents, products of the same systems, lacking criteria other than local ego, splendor of physical plant, and sports obsessions to guide local schools?  Or notably, because of frequent election to local school boards of members lacking the competence or experience to provide oversight, or in some cases those with other agendas.

Award-winning NYC principal Carol Burris, in a recent piece in The Washington Post’s “The Answer Sheet,” offered another perspective that addresses the top down issues:

"What is equally disconcerting is that these reforms are being pursued with little or no evidentiary grounding. There is, for instance, zero sound research that demonstrates that if you raise a student’s score into the new proficiency range, the chances of the student successfully completing college increases. New York’s new cut scores are an attempt to benchmark state scores to the proficiency rates attached to the National Assessment of Educational Progress, or, NAEP. Yet the connections between NAEP scores and college performance are so spurious that researchers have yet to claim that NAEP scores have any predictive value at all when it comes to college and career readiness."

"The bottom line is that there are tremendous financial interests driving the agenda about our schools — from test makers, to publishers, to data management corporations — all making tremendous profits from the chaotic change. When the scores drop, they prosper. When the tests change, they prosper. When schools scramble to buy materials to raise scores, they prosper. There are curriculum developers earning millions to created scripted lessons to turn teachers into deliverers of modules in alignment with the Common Core (or to replace teachers with computer software carefully designed for such alignment).  This is all to be enforced by their principals, who must attend calibration events run by network teams.”

Obfuscation and Myths

Normally this would be the place to launch a spirited defense of those challenging standardized testing.  If the audience is educationally literate that isn’t an issue.  The limitations of alleged standardized multiple-choice tests have been well documented for decades.  What is troubling is that multiple empirical studies of their limitations were prominent in the U.S., by respected academic institutions, before NCLB was launched by the Bush Administration.  Even more studies and critique were available before the bureaucratically-driven debacle of RTTT was launched with billions of dollars in bribes to state governments.  If our Congressional Republicans wanted to plant a scandal-bomb under the White House, it might better be shaped to open and reveal RTTT’s waste rather than Benghazi or ACA.

A second bit of misdirection is the classic tactic of attacking the critic, in the case of present reform, with the usually smirking question:  Why are you against testing; don’t we have to have some way of holding our tax-supported public schools accountable?  One would suspect that genre might be smart enough to know the answer is both out there, and has been in place for most of the tenure of public education.  Rejecting present dominance of that testing has zero to do with the need to assess.  Of course assessment of many types, and testing of many flavors are needed, and have been the process material of public K-12 excellence for over a century.

A third topic that may not be visible in these debates is the checkered history of the hero/villain of present reform, the ubiquitous multiple-choice test.  Surprise to even many educators, the multiple-choice test format, next year, will mark its 100^th birthday; precursors existed at the beginning of that century.

The multiple-choice test was created by Frederick J. Kelly, a byproduct of his doctoral dissertation at Emporia State University (formerly Kansas State Teachers’ College).  Allegedly he was motivated by the desire to eliminate the subjectivity of teachers’ judgments at the time and to acquire “uniform results.”  The approach was perceived as “…the assembly-line model of dependability and standardization.” 

Kelly was circumspect about the testing model.  Attributed to him, a quote about the model that today would generate a Twitter firestorm of political correctness.  Kelly said:  “This is a test of lower order thinking for the lower orders.”  The testing was commissioned by the Army in WWI to evaluate recruits, but the story is not unexpectedly not quite that simple:

“Most of us have experienced a multiple-choice test. Our children undergo them, you've certainly taken them, your parents probably did, and for some, even their grandparents had to endure them. All of us have given them the power to decide our destiny. But what most of us do not know is that multiple-choice tests resulted from an attempt to legitimize the field of psychology, with a dash of xenophobia and scientific racism. Stephen Jay Gould spells out the dark past of these tests in his aptly titled book The Mismeasure of Man. This highly recommended read reveals all the gory details of IQ testing. Gould explains that the development of IQ testing was used to identify feeble-mindedness in ‘unwanted’ groups (usually determined by race or country of origin).

Multiple-choice tests had their origin in World War I, when Dr. Robert Yerkes, President of the American Psychological Association (APA), convinced the Army to commission them to test the intelligence of recruits. The Army's goal was to improve the efficiency of evaluating men by moving away from time-consuming written and oral examinations. Yerkes' motives were to make psychiatry a more scientific field and move it away from its affiliation with philosophy.

A total of 1.7 million recruits were tested, giving the multiple-choice test an air of legitimacy, but in the end, the Army found no value in the results. Yerkes omitted that part of the story when he sold this idea to educational testing outfits. The validity of the test was not questioned. The rest is an unfortunate history.”

The ironic conclusion to Professor Kelly’s odyssey:

“A few years later, as President of the University of Idaho, Kelly disowned the idea, pointing out that it was an appropriate method to test only a tiny portion of what is actually taught and should be abandoned. The industrialists and the mass educators revolted and he was fired.”

The story gives further reflective meaning to the old saw, no good deed goes unpunished in American society.

The last piece of the puzzle about our testing trajectory has not been well aired, that is, the role that the psychometric subset of psychology has played in creating the present mess.  The field is focused on the construction and validation of measurement instruments, including tests and personality instruments.  Attributed as launch of a discipline to Sir Francis Galton (1822-1911), the field developed with some distinction from the latter part of the 19^th century through the present.  The field is not a household word, and even surveying the history is well beyond this post.  There are two key points, however, that at least at the level of a chapter title merit comment.

In this century psychometric modeling and math were greatly extended, applicable to present standardized testing design through item analysis.   Item analysis is a class of analysis that broke out of relative obscurity when our testing companies and their vended tests were identified as the source of large scale or unexpected shifts in results of state testing.  There is little question that item analysis is valid as a mechanism for identifying testing that discriminates human responses and can create gradients and clustering. 

Point one is that the models could be applicable in any measure of a human population that displays discrete gradients of performance on some set of attributes.  The models say nothing about the concept validity of the property being measured.  In sum, as sophisticated as the techniques for deriving present test components, the results have no intrinsic claim to measuring understanding.  Thus to the extent that much of present testing cannot be linked to clear statements of how test scores explain high order thinking and understanding, the use of test results as the definition of whether testing is of value is pure tautology and not a basis for claiming reform.

Secondly, there is even greater harm in the role our testing companies have assumed, with some hubris, designating what is knowledge, without transparency.  Massively pervasive standardized testing, driving out of classrooms traditional attention to critical thought, de facto by that testing defines a nation's first 12 years of formative knowledge.  Psychometric input deserves its provenance as expertise on selective test creation.  The creators and keepers of knowledge have been excluded, public education disavowed interest in content over a half century ago becoming classroom mechanics, and by default the nation's knowledge is now being devised by amateurs in all except selective disciplines.  That would not appear to project a bright future for our national intellect?

So, How Assess?

Somewhat cynically, I suspect the shadow version of this question is, how assess our students’ performances without working too hard?  If that is the basis for much of public education’s slavish acceptance of present standardized testing, we have indeed evolved a pretty sick public education system.

If instead, the basis is that our education community simply knows no better, then a recent article referencing educator training, by The New York Times’ writer Bill Keller (“An Industry of Mediocrity”) merits your reading.

A third possibility is that our public school systems have been so intimidated (or bought off) by Federal initiatives, by state controls, or their school boards and administrators are too fearful to actually operate on the basis of communities’ desires for local education control. The answer then is at the ballot box, if a community’s school board representation by free election hasn’t already been rigged by incumbents, one of the key sources of local public school corruption of democratic process.  The clues aren’t hard to identify; try a ballot with three candidates, vote for three.  Democratic process in action, or election fraud?

Assessment that evolves from teaching that recognizes and emphasizes understanding and learning is hardly a mystery.  Here is simply a topic list of some proposed assessment methods:

• Classic Socratic questioning
• Mastery learning
• Project application of constructs
• Student progress reports (a’ la Gardner or Boyer)
• Performances
• Authentic assessment (usually with PBL)
• Embed standardized tests as pragmatism
• SCALE (Stanford Center for Assessment, Learning and Equity)
• Old fashioned quizzes
• Product related outputs
• Process related outputs
• Writing, essays!
• Use authentic audiences for demonstration of performance
• Role-based (in PBL)
• The flipped classroom engaging parents
• Use Bloom’s and Marzano’s taxonomies
• Dynamic testing (integrated with teaching)
• Indirect assessment (with formative and summative assessment)
• Interactive analysis
• Mathematical thinking              

               -Fault finding and fixing           

               -Plausible estimation

               -Creating measures

               -Convincing and proving

               -Reasoning from evidence

• Conceptual diagnostic tests
• Attitude surveys
• Concept mapping
• Exhibitions
• Portfolios
• Self- and peer-evaluation
• Gaming outcomes
• Simulations assessing performance
• Artificial intelligence (expert systems)
• Longitudinal performance tracking 

In a recent communication, master educator and author Dr. Marion Brady (who was inventing education before most of you were born) proposed an assessment philosophy that he commented may not be ready for prime time.  I believe it is “just in time” if you will pardon my reversion to a prior profession.  Marion’s take:

“The reform cart is in front of the horse. Its initial assumption is faulty. The aim isn’t to teach the core subjects well, but to rear smart kids. If I’m right, then the first step in a proper reform effort is creating tests. Tests first, not last—tests that evaluate what Einstein said should be our first priority—the ability to imagine alternative futures and deal with the problems those futures create.

That done, tell teachers to teach to have at it. If it’s thought that standards are needed, let teachers write them, but keep them in electronic form so they can continuously evolve as professional dialogue expands expertise.

.

.

.

My evaluation-related assumptions: (1) Evaluation tasks should require kids to apply what they know in a not-previously studied situation; (2) the best tasks are concrete rather than abstract, real-world rather than theoretical, ‘supra-disciplinary’ rather than tied to a single school-subject; (3) there’s no good reason for a test to be timed; (4) a good task requires no security measures, no honor code, no anti-plagiarizing strategy, no vigilant watching for evidence of cheating. The response to a good task will be so idiosyncratic that any teacher in charge of a reasonably-sized class for more than a very few weeks will know who wrote what.

Many years ago, when I first read Alfred North Whitehead’s 1916 Presidential Address to the Mathematical Association of England, I was mystified by his insistence that ‘no educational system is possible unless every question directly asked of a pupil at any examination is either framed or modified by the actual teacher of that pupil in that subject.’

It took me many more years to see the wisdom in that requirement. Now, I can see no acceptable alternative.”

Bottom Lines

As a way of summing, consider this quote from a high school student featured in the Lucas Educational Foundation site, “edutopia:”

“And yet, in the world of education, the "next big thing" is merit pay for teachers and boosting test scores. Do our policymakers not understand that the world is going through a revolution in the way we live, interact and learn?

Our education system is stuck in paralysis. We have tried doing the same thing over and over again with the expectation of a different result. This is insanity at its finest. The way we educate is based on the tenets of the Industrial Revolution -- conformity and standardization.

For instance, creativity is virtually extinguished as a child goes through his or her schooling. In their 1998 book Breakpoint and Beyond, George Land and Beth Jarman refer to a study in which 1,500 kindergartners between three and five years old were given a divergent thinking test. Divergent thinking tests don't measure creativity, but rather one's propensity for creativity. The test asks questions such as ‘How many ways could you use this paperclip?’ or ‘How many ways could you improve this toy fire truck?’ -- questions designed to encourage creative thought rather than elicit right-or-wrong answers. Ninety-eight percent of kindergarteners tested at genius level. The kids were tested every few years. By the end of post-secondary education, only two percent of students tested at genius level.

So, if you're trying to produce compliant, dead-brained, formulaic workers, our system is doing exactly what it was designed for. (I should add ‘grade-obsessed’ to that cadre of properties.) But in a society where innovation is simply everything, it is a cultural and moral failure to encourage this compliance.”

There is when all denial is purged, and when all preconceptions and pretensions are deflated, still the belief that there is some ‘magic sauce’ that will transform a U.S. public K-12 education system that lost its will to excel and its capacity for servant leadership some decades ago.  When confronted with examples of Finland’s comparable systems, or Singapore’s, or Shanghai’s, and their successes relative to the U.S., the prototypical domestic response is some form of “but they’re smaller, or more homogenous, or more socialistic.”  The savvy observer of our society, and NYT writer and author Tom Friedman, recently visited Shanghai in search of the ‘magic sauce;’ there isn’t any, but there is a master class lesson in K-12 education.

Part 4

The next blog goes down into the trenches.  The challenge:  Speculate why too many of our public schools, unwatched outside their bubbles and unheralded by the media, are not able to either conceive of or implement the kind of self-reform that might have prevented the quagmire of present test-based alleged reform.  There is reason to expect that the answers will raise hackles, and will not fit into the neat confines of politically correct caveats.

A Mile Wide, A Mile Deep -- Part 2

Preface

Briefly reviewing the last post, this series is an attempt to start describing the salient features of the U.S. public K-12 education system potentially impacted by misdirection and error in current corporate reform.   The point of the last post was that our public school universe is a systemically connected but not homogenous mass of 100,000 schools and thousands of oversight entities, intersecting still more types of organizations and institutions exerting influences on their operations.

The resultant interactions are beggared by 49.5MM students served, in turn represented by high double-digit millions of parents.  Then, periodically, at places where representation and levies are on the ballot, these numbers rocket to a couple hundred million actors.

Obvious core questions are:  (1) How to define and describe this complex a public education system in a fashion that enables analysis; (2) what are the functional linkages among parts and players in those systems that must be understood; (3) how do you keep score in this milieu?

Part 2 opens that discussion with the proposition that understanding the resultant systemic structure of public K-12 is critical to mediating it.  A logical consequence of that proposal is that present corporate reform lacks credibility, and has already damaged U.S. public schools by focusing wholly on one narrow and partial measurement of learning emanating from black boxes of unspecified students and teachers.

“Dem Bones”

We live in a hyper-connected universe, where material is neither created nor destroyed, and where every atom has had a precursor history and is on its way to another assignment.  Simply mouthing “education” with serious demeanor does not communicate a sense of full understanding.  Nor does it disconnect the contents of that noun from any other idea, force, energy, awareness, device, structure, human entity, etc., that intersects the concept as the mechanism by which we organize and transfer knowledge.

“The foot bone connected to the ankle bone, The ankle bone connected to the shin bone, The shin bone connected to the knee bone,…, The shoulder bone connected to the neck bone, The neck bone connected to the head bone…”

The first observation expresses the spirit of the old nursery rhyme.  In a bare bones form that describes public education’s layers:  Launch with parents with a flood of antecedent states and behaviors connected to the K-12 student (with circularity defined by parents’ tenures in the same generic systems with whatever learning was achieved, or not), connected to a grade band's student learning component, connected to a body of subject matter or process, connected to a teacher (that teacher a product of some system of teacher education, certification and renewal), connected to a classroom, connected to any externally mandated classroom rubrics, connected to a building’s physical enablers or constraints, connected to a system’s model of management and leadership (also connected to unions, vendors and third party sources of knowledge), connected to an organizational model, connected to the culture of a district, connected to taxpayers/voters, connected to a school board supposedly representative of a school’s constituencies, connected to various county educational services or intermediaries, connected to a state education bureaucracy, connected to a governor and a legislature, connected to an army of education lobbyists, test and textbook marketers, connected to the Federal education bureaucracy and selective Federal laws, connected to our Congress, connected to international measurements of national school performance, connected to multi-national private sector demand for educated human resources, and on.

Every “connected” above expresses a complex functional relationship at the root of connectedness, along with inputs, arrangements of the human resources at the boundaries of connected components, and performance outputs and their assessment.  Every linkage may require some model or process that expresses role and operations that make a system function.  Isn't this unnecessarily complicating what's obvious?

The complication is that what's obvious may not be how the game works, and why many of these links become dysfunctional.  Additionally, many of the above linkages need not approach the status of needed formal rules.  Many needed interactions have evolved over time with mutually acceptable, largely internalized ground rules, that when they work simply become in effect common law. When challenges arise because what used to work doesn't anymore, there is a logical void and players frequently retreat to entrenched positions.  

An example:  In the heart of public education's current reform challenges sits a key link to public K-12 success or mediocrity -- the eponymous school board. It sits generically between the public constituency and tax sources, and a school system's management, as well as between upstream state and Federal oversight and a system's compliance.  The role is a tough one under any conditions, but in most states there are few general professional requirements for board service, and in many not even functional literacy is required.  Few board members at least before being seated could pass a comprehensive oral or written exam on the principles and requirements of service on any "board," or on contemporary theory and operations of a public school system, or have served in senior managerial roles where professionals are sourced, vetted and hired.  Folklore and local preconceptions usually form the basis for role definition.  This linkage has been conveniently bypassed by corporate reform, by denial, or perhaps because it was deemed easier to intimidate children and teachers deprived of the countervailing power of teachers' unions than adults who may also be politically connected and sub-cultures that applaud ignorance.

This by-the-way, is not an endorsement of teachers' unions, that generally deserve present attacks because of decades of self-centric and dogmatic refusal to recognize and respond to, and have blocked public K-12's need for change. The role of and need for countervailing power in democratically-driven societies is well documented, even if it is hard to see its expression in the current U.S. Congress and the resolution of other contemporary U.S. societal and economic issues.

Education Quantum Mechanics

How characterize this highly complex reality in some common sense terms?

As a market theorist, convenient shorthand is depicting it as a large n-dimensional multivariate system subject to some of the magic of statistical modeling when you can put dimensions and numbers on the parts.  Unfortunately, that ethereal depiction does not help much.  Another conceptualization is to visualize our public schools nationally with the kinds of counts that characterize our U.S. Census; good counts of schools stratified by size, by location, by the socioeconomic and cultural properties of their location, by learning strategies employed, by their costs of delivering their function, by their educational results. 

Perhaps the most graphic description, if one is digitally current, would be to see the grand system as a “cloud,” a' la the servers in the virtual sky that now exist in anonymity warehousing your digital everything.  That depiction has virtue; a sprawling collection of organizations, not truly homogeneous, with complex interactions of the set pieces, spread across 50 states and thousands of communities, semi-autonomously governed, and not historically prone to collaborations as much as competition.

A repeat of the earlier question, why does moving beyond just common sense and what we can see and experience have to be injected into depicting public K-12?  We think we know what our schools are, they all basically look alike once past the bricks and mortar.  Aren’t they essentially doing pretty much the same thing, the same way, given similar teacher and administrator training, similar certification, similar textbooks, possessing pretty much the same knowledge?  The answer is the distinction between the tricks our minds play in creating what we want or expect to see, versus the way those crazy little fundamental particles behave via hard laws to create our and every other object’s substance and properties.  When those components result in a sentient being, add intellect, attitudes, opinions, beliefs, and free will.  Education in any formal setting, and all of the preparation and paraphernalia leading to its accomplishment, are the product of human endeavors that range from random to despotic through the Goldilocks zone of reasonable and intellectually defensible propositions. The photons our eyes physically register are not what our brain “sees.”

Hence the argument, as in any version of the science of knowledge, is that facts, schemes for categorizing and appraising them, precise definitions, measurement, and rules for understanding what we measure are the sine quo non of actually “knowing.”  While adopting any analytical approach depicted to study education may seem mind-boggling, it really is not if there were already in place the mechanisms to put some substantive descriptive and data meat on those bones.  That selective failure in the case of our public K-12 schools is a national disgrace, also in part precipitated by our public systems’ sclerosis of transparency for decades, the reluctance to open systems for public critique.

The need is a national standardized census (rather than depiction by narrow and flawed standardized testing and “grades”) of our public K-12 systems, gathering instead of the politically correct and bureaucratic Federal NCES parameters, a competent survey of what each system is, how it is functioning, embedded values and culture, its resource qualities, and how it assesses performance.  This is hardly rocket science, being the guts of market analysis by the private sector that has been solidly in place for over six decades and repetitively refined to guide businesses' market strategies.  This is also a failure that could be corrected, and that should have preceded the deployment of NCLB, RTTT, billions of Federal dollars to bribe states, the testing imposition, state grading, and the damage all have inflicted on competent higher order K-12 learning in the U.S.

Assessing Our Public Systems

The framework implied above is what would be termed in explanatory theory as a cross-sectional versus dynamic model, the latter additionally portraying the functional activities and interdependencies that create a system’s actions and performances.  That action view is made more complex by introducing time and longitudinal change in performances and results.  Lastly those sequences in turn are governed by how the organization defines its mission, employs and stages resources, and how it values and measures both inputs and assesses its performances.  In traditional parlance, our systems are mediated by both macro (market/large system) and micro (school and local system) variables and forces.

Reminiscent of the grandiose but flawed rhetoric for the weeks of Federal governmental closure, public education catcalls from the sidelines have proclaimed public education is failing, or it isn’t failing, or it has failed historically, or feature states’ self-congratulation for cranking out school system letter grades that offer little valid school assessment.  After activating some neurons, ask:  How, given the interactive complexity of our public system as defined above, coupled to the diversity of those component systems in virtually every aspect of schools' functions, and reflecting student diversity, do you assign a simplistic letter grade to a school?  How given that same diversity, is there any credibility in comparing even a pair of schools in unequal settings, much less 100,000 schools across 50 states.  It seems doubtful an inflexible grading model could be successfully applied to differentially rating ubiquitous supermarket dressed chickens?

Recently published, Professor Diane Ravitch’s new book, Reign of Error, “…focuses on what she sees as hoaxes aimed at winning private control of education and suggests solutions, many of them addressing the challenges of racial segregation and poverty.  ‘Public education is not broken,’ she writes. ‘It is not failing or declining. The diagnosis is wrong, and the solutions of the corporate reformers are wrong. Our urban public schools are in trouble because of the concentrated poverty and racial segregation. ... Public education is in a crisis only so far as society is and only so far as this new narrative of crisis has destabilized it.’”

This writer believes that Dr. Ravitch, as much as her history in education and expertise are admired, is as wrong in her recent assessments as are the opposite judgments coming from the corporate reform movement she rips.  Does Dr. Ravitch really believe she speaks for 100,000 schools, holding pragmatically less objective information on that super system than even some of the reformers.  At least many states, even the testing companies, and Bill Gates’ funded efforts, have made some attempt to gather K-12 performance data.  In drawing inferences about our composite public K-12 education system, virtually all of the critics are operating without reference to representative samples of performance; they’re navigating on the basis of haphazard observations that also occur with the inevitability of human bias.  The argument is that both sides of this debate, if not wrong, are subject to a “Scotch verdict,” not proven.

Sampling a few systems does not prove our public K-12 schools as a grand system are failing, or have failed.  Unfortunately, the U.S. NAEP studies and international PISA studies suggest they are underperforming for the resources committed, and the recent OECD study suggests that has been the case for some time.  Are these studies adequate diagnostically to support upending U.S. public education?  Rationally, probably not.  But do they also suggest that the U.S. has been remiss in not doing that diagnostic work on the grand system, and that a vast majority of our public K-12 schools has been remiss in blocking transparency, practicing denial, and rejecting self-reform?  

Yes, on both counts.  The “yes” is backed up by admittedly small numbers, but in depth observations of systems that display the superficial trappings of acceptable public K-12 performance, but digging deeper reveal areas of incompetence, unethical operation, or simply never being equipped to properly recognize the complete public education mission or how they must perform to support it. 

The additional property of U.S. public education, product of American simplistic views of governance, visceral rejection of some national learning standards, and socialism paranoia, is “local control.”  In the real world, local control of public schools has long been nibbled away by needed reforms to bolster equitable function and satisfy state/Federal mandates.  But also, reality, local control in bubble communities leaves enough room to make ignorance of contemporary knowledge, sports obsessions, inherited social customs, and community-centric thinking the most prominent precursors of local school strategies.  When the four are combined even the most reasonable interventions to modernize K-12 learning are generally rejected.  The nation’s current reformers have heavily targeted public schools serving the disenfranchised where major learning gaps occur; however, major contributors by count to U.S. public educational mediocrity among the world’s developed countries are equally likely many of America’s “Pleasantvilles.”

Lastly, relating these arguments to present reform philosophy, the question that needed to be asked as early as launch of NCLB was:  Can narrow, focused, and punitive emphasis on only one class of assessment of public school performance effectively (and responsibly) modify the upstream interactive series of strategies, choices, resources and practices of a school system to produce sustainable learning performance?  Making that reform mantra a monolithic tactic, ignoring all of the other processes and components contributing to learning change, even if it forced some adaptive behavior, was a highly risky strategy.  It left no wiggle room to recognize the internal mechanisms that had to change, nor acknowledged the inputs to regroup if the approach failed.

One of those mechanisms critical to change has been attacked by alleged reform -- our public school teachers -- but with such a blunt instrument (VAM) that our systems are being denigrated rather than improved.  The other side of that coin is whether, given the present models of teacher training, our K-12 teachers are being supplied the cognitive awareness to be professionally self-aware, and given pragmatically the self-analytical tools to either improve classroom operations on their own, or to recognize the research and modeling from third party sources that offer that opportunity.  Too frequently, the image of many teachers is adherence to history, or inherited techniques, or rolling over to accept the latest administrative approach to surviving the testing and state grading assaults.  Metaphorically, it invites teaching that becomes the equivalent of "painting by the numbers."  In fairness many excellent and committed teachers do recognize the issue, but are bullied or succumb to interventions to support a system's administrative judgments.  The latter have even less credibility in many public schools – for reasons of inadequate recruitment and training – than the teacher products of present collegiate schools of education.

Both sides of the public school reform debate are likely wrong; there are material public K-12 numbers of systems guilty of under achievement, and some failures; but the present reform model based on testing terror tactics arguably will never sustainably modify the causes.  Making better decisions on both sides of the argument means having a valid and more robust conceptual model of school organization and schools' functions, better data, working with better tools for creating and testing classroom tactics for learning, and operating pragmatically rather than driven by passion and ideology.

Hit Pause

Part Three of this series will focus on the topic crucial to enabling competent changes in public K-12 in the trenches – identifying the mission, and recognizing when you’ve succeeded or have drifted off course.  If the raison d'etre is not to be the scores on standardized tests, or surviving a state’s arbitrary letter grading scheme, what is success in achieving learning?  And critically, how do you validly and reliably measure schools’ learning successes (or failures), and aggregate such observations into some composite school rating scheme?  

Part Four will probe the factors that inhibit or derail internal K-12 public school reform – where change will have the greatest acceptance and impact – and what are the school and external factors that inhibit self-motivated local reform?

Lastly, Part five will try some national divination, projecting for roughly the next decade system environmental factors that might drive public K-12 change:  The trajectory of the need for K-12 learning, and how its specifications might change; how the causal factors for school success may shift or allow factor substitutability; how technology could impact public schools; and how national civic changes that may occur, whether applauded or not, might effect policy changes throughout our public grand system?

A Mile Wide, A Mile Deep, and Murky

Preface

A favorite anthem of those applauding alleged corporate reform, and the alleged “common core,” is that public K-12 is usefully moving via those initiatives from education “a mile wide and an inch deep” to learning “an inch wide and a mile deep.”  Unfortunately, the only real content in that simplistic assertion may be that our educators may now need hip boots.

The debates over NCLB, RTTT, the “common core,” standardized testing, VAM, and a virtual landslide of barely coherent state grading of public schools, mercifully fall short of the intensity of our recent Congressional war on budgets and the debt limit.  Also mercifully, the debates have never truly touched the issue of widespread shutdowns or privatization of public schools in spite of the rhetoric.

Perhaps the closest present reform has come to provoking some real energy – it certainly hasn’t happened in our public schools, where our ‘edusheeple’ have generally capitulated to testing, VAM, and related demands – emerged in Buffalo, NY, where 2,500 teachers, parents and administrators recently turned out to protest more highstakes testing.

At the other end of the polar energy scale, is our U.S. Secretary of Education, who recently used a large measure of his ‘state of education’ report to attack his critics.  As Valerie Strauss – The Answer Sheet, The Washington Post – pointed out in a review, “…the education secretary still doesn’t seem to understand what many of his critics are saying.”  Curious, our alleged national guru of learning doesn’t get the essence of the gig.

But the whole corporate reform movement has managed to oversimplify virtually every material issue surrounding U.S. public K-12 performance. That oversimplification to a large extent accounts for the blizzard of alternative versions of “public education’s problem,” equally becoming a roadblock to fashioning either consensus or durable solutions.

This post of Edunationredux is the first of a multi-part series seeking to probe the full U.S. public K-12 reform jigsaw puzzle.

The Eye of the Beholder

Incredibly the largest stumbling block to moving forward to shore up public education and create rational true reform was articulated in the 9^th century; the parable of the blind men and the elephant.  Most adult Americans have at some time been exposed to the lesson imparted.  In the Jain version of the parable, the king responds:  “All of you are right. The reason every one of you is telling it differently is because each one of you touched the different part of the elephant. So, actually the elephant has all the features you mentioned.”  It says much about both our educators and their critics, that as simple a lesson has been ignored in most of the rhetoric of the last dozen years about public K-12 performance and change.

Let’s take a better look at that assertion; certainly all of the bright people in America’s education systems couldn’t be that prone to tunnel vision? 

There is a constant stream of comments to online press stories from usually parents, in denial, trying to put some feel good spin on the reform news, by claiming that our public schools have no problems, that the alleged reforms are bogus or politically motivated.  Years of results from the U.S. NAEP, from the international PISA testing, and just issued results from OECD testing are not perceived.  Nor is the polling effect noted that has respondents condemning Congress, but fully supporting their local Congressional representative even if they’re a jerk; the same effect applies to views of “public education” versus Pollyannaish assessments of a local school.

Too many of our public K-12 schools, tuned over decades to never being challenged, now facing critique, exhibit behaviors either paranoid, or circling the wagons, or denying transparency of what they are actually teaching and how, or simply rolling over and in many cases denying their own values and integrity by simply slavishly putting testing ahead of learning.  Where there is a modicum of courage it may manifest itself in the guts to aggressively teach “for” the tests to get state education bureaucracies off their case, then going back to real learning.  Related, America’s collegiate schools of education are hiding in the weeds, making every effort to deflect the reality that their failures in creating fully competent teachers and administrators are part of the cause of public education’s stall.

The combined mess – of NCLB, RTTT and its wasted double-digit billions of dollars, Obama-Duncan acquiescence to the mass of standard testing, the CCSSI and its alleged knowledge standards conjured up in secrecy, and testing companies driven by profit and de facto specifying what is knowledge – lights up the scam meter.  An hypothesis is that Obama is pragmatically willing to see public K-12 tortured with testing and VAM in the idealistic to delusional belief that the universal testing challenge might force greater integration of low income, racially discriminated, and culturally deprived children into mainstream school performance.   What appears to be missing, in what is a legitimate if brutal trade-off, is recognition that the testing/VAM strategies might collide with the law of unintended consequences.

The mainstream of the reform movement is the in-your-face ideologically and politically inspired goal of privatizing public education.  As in the case of the conservative effort to scuttle ACA, it is delusion that privatization could be pulled off in any foreseeable time span, or without simply despotically riding over democratic process.  The issue is that the ideologies breed destructive tactics, precisely what has occurred with the imposition of unproven testing logic, Draconian and unfair teacher assessment, and opening the door to waves of demagoguery on public education that benefit a few, much of that for profit or to acquire power.

Motivations of other reformers, especially the cabal of billionaires dabbling in public K-12 reform, are particularly curious.  Some like Bill Gates, reflecting intellect and a history via his Foundation of tackling major societal and technical issues, are predictable.  For others less transparent there appear multiple hidden agendas.  Some reflect strategic thinking, albeit not praiseworthy, for example, seeking to influence public K-12 education to suppress the teaching of evolution, or seeking to suppress consideration of climate change in K-12 curricula to cynically protect business profits.  Even in Gates’ case, tens of millions of dollars have been invested to promote standardized testing and VAM, a case where an argument might be that the knowledge is missing to impose that assumption on a nation’s school system.  Parenthetically, no one invited him to that party.

Students of our systems are, of course, ignored as sources of insight about questionable learning.  Parents in turn are suspended in a state where legitimate education assessment is hard to acquire.   They believe what local systems pump out as propaganda.  The media is almost as vacant of insight about public education, only rising to the bait when some school issue can be sensationalized.  The chance that any local public school system will currently have at its head a human resource advocating servant leadership and participative management is not robust.  How public system administrators are trained, then hired, vetted and overseen by amateurs virtually assures that. 

To add to that roadblock to change, public school superintendents are literally unaccountable to anyone if they can bully or con a school board.  In that environment, there is no check and balance on the evolution of self-righteous behavior, or dysfunctional ego, or poor leadership compared to our better private sector organizations where stakeholders can’t be easily frozen out.  Public school administrations, even the better brands, find it inviting to invoke executive privilege or misstate privacy concerns to block system transparency. 

Lastly, a small army of professionals who should know better have bought off on the belief that poverty and culture really explain all test results, thereby asserting that the grand fix is reinvigorating the middle class and assuaging poverty.   The schools and the teachers and the administrators are just fine.  This issue is important, because many allegedly data-based research findings and correlations seem to support the contention, but pretty much ignoring a major admonition of that statistical genre, that correlation is not causation. That finding also depends on the unit of analysis and level of aggregation of data.  Does it explain all deficiencies measured in learning performance?  Hardly, because as the analytical focus goes from macro to micro, that next level of explanatory variables kicks in.

Summing up, all are blind to some parts of our massive educational complex.  Thus problem definitions and fixes get defined by what is locally perceived.  Words so often spoken, but rarely fully registering, there are no silver bullets for changing public K-12, and no single-cause system that encompasses why schools perform sub-optimally.  Because public K-12 is a system of systems, nested in variable environments, engaging diverse human educators, there are complex interactions among the determinants of good learning issuing from a school.  That means that success factors can be additive, substitutable, canceling, with great explanatory difficulty in sorting out causes and effects without some hard science for testing effects.  Unfortunately, doing good research on public K-12, testing assumptions, using experimental design and multivariate models, has rarely been in the teachers’ or administrators’ tool kits for professional practice.

Where does this leave the potential public school reformer?  A short answer is, either doing more harm than good, or up the proverbial creek without a paddle.

How Do You Eat An Elephant?

Is there a way out of the conundrum suggested above; a way to address a system as massive as public K-12 – 100,000 schools, over 14,000 school boards, state variability, and a now misdirected and massively intrusive Federal invasion – with some common and replicable sets of variables that when massaged improve the end product? 

One approach is obviously what is being loaded unto our public schools, assume that demanding highly stylized quality control of the end product, down and dirty quantification, with major consequences (high stakes), will force change in the upstream determinants of classrooms’ and human resources’ action sets.  The people pushing this have a cerebellum – they know that the process has concomitant effects, and that they are injurious.  The argument has to be the end justifies the means.  Add that the groups pushing present reform formats must have concluded any other approach is far too slow, or evolutionary for political goals sought.  The crux of the issue is whether the entire reform model simply tumbled out of disparate political events and ideology, experienced little logical or empirical testing, attracted the private sector testing vultures, and was simply picked up and advanced by every entity that saw a stake for personal gain?

Our states have simply performed solely as their political funding and national attachment dictate.  The testing strategies, purchased tests, corrupted lobbying surrounding that function, flawed and simplistic state grades that distort educational achievement, with state educational bureaucracies that represent politics more than critical thought, have become the wholesalers of the ersatz reform movement.  Most appear clueless what the words mean for the measures being imposed on local schools.

Is there an option?  Ideally there are many, but the Catch 22 is in the “ideally.”  There is no ideal core to U.S. public education in spite of the lip service given to its being one system.  It is not homogenous in spite of the similarities of practice enforced by history, unions, or states’ similar models for funding and flogging the function. 

One nasty element of the public model is the local school board, the third rail in public education.  In no other American human endeavor are so many assets, and so much national and personal portent, delegated to so few human resources competent to offer that oversight (if you discount superior competition from the U.S. Congress).  This may be the most egregious act of our state governments, failure for decades to correct the way public school oversight is determined, and even retaining the local board solution, refusing to update the requirements for serving in that role.  There is apparently no obvious research that has ever been permitted or created to definitively assess this issue, but the hypothesis is that incompetent school boards are a root cause for a major portion of U.S. public K-12 underachievement.

About that metaphorical elephant; by getting it into much smaller and manageable pieces.

Hit the Pause Button

The next Edunationredux will try to identify the variables sets, structural linkages, and layers of influence in our public system, and also try to show how the large components of public education interact or are contingent, to suggest some pressure or action points for achieving productive change with less damage to our human players.  The assumptions, variables, the environments, and the human resources that are stirred together in U.S. public education are truly a mile wide, a mile deep, and with daunting entanglement.

Change and Coleridge*

One of the K-12 organizational innovations, appearing in the more specialized education journals for some time, has now gone mainstream.  Appearing prominently in The New York Times yesterday, that is the "flipped classroom," story linked here.  

The report is timely, coming on the heels of the just issued report from the international Organization for Economic Cooperation and Development (OECD), of a cross-national study that revealed material deficits in the competence of U.S. adults.  

Drop however briefly the denial and hypocrisy that permeate too many public K-12 bureaucracies -- the adult deficits cited are the product of U.S. public K-12 education that for decades either fumbled or ignored coherent self reform.  Sickening irony, those same schools, boards, and administrators now lemming-like are in lock step with ideologically-driven state bureaucracy and corporate reform, where testing obsessions and increasing focus on trivia and methods mumbo-jumbo are destroying real learning.

Pathways away from even more U.S. public educational mediocrity are problem solving and creativity that work in the real world of business and science and other settings where its practitioners are motivated to create innovation.  The isolated education example above is just a snippet.

An example that may not be familiar to many of you, but is rooted in the site of this blog, is the innovative materials handling equipment manufacturer, Crown Equipment Corporation. This industry leading producer has massively innovated by employing fuel cell energy technology, and creatively combining digital technologies with their fork trucks to change the face of supply chain execution and optimization.  Should our public K-12 schools be at the tail end of creative change in our society, its "late adopters" in sociologist Everett Rogers' model of diffusion of innovation?

Soaring beyond the flipped classroom, and out there in pilot form, is what might be termed the one-to-one classroom; curricula, lesson plans, MOOC, constructivism, homework, and even day-to-day assessment designed for the individual student, implemented with advanced computer and software capabilities.  Clearly this innovation, that depends on a major increase in the digital competence of present teacher education, and equally a wake-up call to public school oversight and administration, won't see exposition in the NYT or The Washington Post or Education Week for some time.  But it is indicative that public K-12 education will face disruptive change.

Part of that change will manifest itself in demands for more effective physical infrastructure, new school buildings that reflect a departure from almost a century of stagnation in design concepts.  Education civilian and parental eyes tend to glaze over when this form of K-12 innovation is broached.  But those same eyes gain sharp focus when, for example, they're informed a school system will hang around their neck and on their school tax bill a three decades' albatross of tens of millions of bond dollars for an obsolete structure. Attitudes and choices can quickly change when boards and educators are revealed educationally retro, or reactionary, or too bureaucratic, or lack the skill to improve platforms for learning coming out of taxpayer pockets.

Bricks and mortar are only part of the downstream need for change, and they interact with K-12 organization and technology.  Below is just a brief sampling of some ideas recently floated for our future schools and their leadership needs:

http://www.linkedin.com/today/post/article/20131009220227-2906843-why-leaders-matter 

http://www.digitalpromise.org/what-makes-an-innovative-school-district-leader/ 

http://ew.edweek.org/nxtbooks/epe/ew_sr_10022013/index.php#/2

http://www.prnewswire.com/news-releases/follett-challenge-to-recognize-k-12-innovation-at-15-schools-226019211.html

http://www.marketwatch.com/story/desire2learn-offers-the-first-fully-integrated-learning-platform-to-higher-education-k-12-and-enterprises-worldwide-2013-10-08-101735656?reflink=MW_news_stmp

http://www.huffingtonpost.com/michael-seo/leadership-in-education_b_4054059.html

http://www.berkshireeagle.com/news/ci_24245223/mclas-feigenbaum-center-science-and-innovation-boon-entire

http://www.prweb.com/releases/2013/9/prweb11179489.htm

http://www.marionbrady.com

Schools' leadership should be making the charge to identify, create, and pilot test these and related concepts.  New facility and technology investment in turn should reflect the future, not obsolescence and pedestrian housekeeping.

* For those who have forgotten, "The Rime of the Ancient Mariner," and the infamous albatross.