Assessing US Higher Education: Information, Intimidation, Ignorance, or Insanity?

The last post of Edunationredux offered a partial critique of the Obama/Duncan scheme to rate America's colleges and universities. Prior national critique reflected almost a "you gotta be kidding" ambience, illuminating the perceived chasm between what Arne Duncan and the US Department of Education are proposing, and anything resembling intelligent social science applied to the measurement task.  Today’s post extends the prior critique, exploring the real measurement chores needed to create valid and reliable ratings of America's colleges and universities.

That chasm between the proposal and reality is so great it raises major questions; what conceptual malaise and what leadership degradation have occurred in that Department, who is steering this measurement debacle, and what resources are executing the work.  Is the proposal chain-rattling just to get the attention of higher education leadership?  If the intent is to actually carry through the scheme, is this another Federal agency that has now lost steerage, and mismatched the resources needed to actually conduct competent education work?

Post Critique, Critique

One tiny slip in the pronouncement of a functionary in the Department of Education may have given away the naïveté and slanted thinking footing the current proposal:  One of the factors allegedly being considered was how to treat "improvement" as a variable, and presumably as a simple metric.  The statement infers that the designers of this scheme may see the assessment of our colleges and universities occupying the same conceptual space as improving test scores in a public school system.  There are likely a few community college-scale institutions, close to being simply extensions of high school level performance, where this may be applicable, but any resources knowledgeable about the functions within a major university would deservedly see this as bizarre.

A last retrospective issue is further scrutiny of the misguided proposal to use beginning salaries of graduating students as a basis for institutional assessment.  This component of the proposal has some serious logic issues.  Aside from the nearly impossible chore of equilibrating the professional destinations of students across institutions to create one valid metric (or even multiple metrics), and the cognitive error of relating quality to profession sought, a peek at the distributions of those starting salaries poses an even more daunting issue.  Starting salaries are not distributed normally, but are skewed to the high end. The overwhelming body of starting salaries is so constrained, the distribution leptokurtic, that little or any discrimination among most salaries attributable to institutions could be detected. 

A pretty cynical outcome of using the proposed metric(s) for salaries, aside from all other faults, is that success in that venue would come from maximizing an institution's output of petroleum engineers, and wiping out the education of all PreK-12 teachers.  If the underlying intent of this scheme is some social engineering to equalize higher education opportunity, and social and economic states, its extreme liberal designers need to go back to the drawing board, or better, acquire some higher education.

Fair Challenge

The classic, and legitimate challenge to last post's critique of what's proposed -- that it is a loser -- is provide a more effective system for assessing our institutions.  The remainder of this post takes a stab at that challenge.

Dimensions

The starting point in this quest is identical to every legitimate research effort since the Enlightenment:  What is the goal, what hypotheses are to be tested, what question or questions are being posed for answers; what is the universe from which measurements are sought; what are the variables or factors requiring measurement, and what are their functional relationships to the criterion question(s); what are the properties of the variables, in this instance measurements wanted, i.e., nominal, ordinal, interval, cardinal; what are the hypothesized or measurable distributions of measurements sought; how do the error terms intrinsic to all variables fall out, intra-institutional variance versus inter-institutional variance, driving the comparisons of institutions or institutional subsets sought; what are the weights of contributing variables in forming then informing about the differential effectiveness or qualities of institutions being assessed; and critically, with a finite set of candidates for positioning, how may the units in the universe need to be stratified or clustered to minimize confounding of results attributable to basically different higher education systems being appraised?

Given a US universe of 4,140 institutions of higher education, with internal partitioning that may multiply the actual units of analysis by levels of magnitude, with hypothetically complex variable sets driving the criterion effect, the project is not the simplistic vision of the US Department of Education, revolving around already extant data, but what is now colloquially termed "big data:" "...an all-encompassing term for any collection of data sets so large and complex that it becomes difficult to process them using traditional data processing applications. The challenges include analysis, capture, curation, search, sharing, storage, transfer, visualization, and privacy violations."  The mission here, assigning performance ratings to America's colleges and universities, is arguably the very definition of the analysis challenge described.

Department of Education thinking is apparently to measure some amalgam of institutional functional performance and contribution to social goals.  Both become subdivided into constituent goals that complicate what is proposed and currently measured:  For performance, institutional graduation rates overall versus by students' degree tracks, as well as longitudinally by how the process is finally achieved and the time involved; the learning effectiveness of what's been acquired along the way (made more complex when apportioned among multiple disciplines and degree tracks); the complexity of devising true costs of education delivered, plus the cogent issue of the productivity of all of the assets and operations incurred to produce a graduate; and close to the most salient first use of any assessment, whether the results actually materially impact via improvement the choice processes of prospects seeking higher education.  Also ignored in the Department's rhetoric, the longitudinal complexity of worth of prior learning at exit from the institution, versus its worth at the various career stages the graduate experiences.

Measurement Factor Complications

The performance of our institutions in creating equitable student access may be slightly easier to access in principle, but introduces major problems in execution:  A large multivariate causal set of determinants of schools screened, preceding the issue of differential institutional compliance with equitable admissions, is problematic; the reality that acceptance of those who might be discriminated is also based on the failures or successes of our public K-12 systems, long before an institution's action effecting equity kicks in; and a major barrier to measurement at the level of the individual student/family is driven by confidentiality considerations.

A pre-collegiate experience case in point, familial relationship to this writer, is a collegiate freshman at a major university, majoring in an engineering specialty.  Partially because of the 9-12 work in an effective science high school, this soon-to-be second semester freshman will be moving into second semester sophomore level academic work with perfect "A" grades, primed by the prior high school work.  Adding to the analysis challenge of assessing institutional performance, then, are the assets/deficits that precede and impact acceptance.  The remedial work impeding, or prior learning permitting accelerated collegiate work, becomes another complication in assessing collegiate end-game contribution.

Another set of factors in judging performance is the subjectivity of protocols of collegiate grading, variable among institutions, among schools, among departments, and even among individual faculty.  Without some national, standardized achievement testing, by specific disciplines or academic track of students, the comparative use of even grades and point averages as measures of institutional performance add complexity to any rating scheme.

The prior Edunationredux blog also unfolded another major constraint, comparison of institutions based on the proper unit of analysis as well as assuring comparability, rendering the simplistic measurement chore inferred in the Obama/Duncan thinking the height of amateurism. 

Still another factor ignored in the current conceptualization is the role played by geographic and location factors, perhaps even highly specific location factors related to the population and cultural composition surrounding a student's residential assignment, influencing institutional outcomes.

But there is another gut issue that will at present -- and in the absence of never executed benchmark research on our colleges/universities -- blind side and hamstring the proposal.  That is the core pattern of variance of any variable or factor used as a basis of measurement.  In virtually all diversified and complex systems (precisely what every major college/university is) there is leveling of outputs based on de facto competition.  In common sense terms, there may be more variation of performance within an organization, than among similar organizations, where an attempt is made to sum or average overall experience.  The practical significance, with a small bit of coaching, human experts on higher education can likely identify the better or worse extremities of “high performing” and "low performing" colleges/universities.  The in-the-middle thousands may blur because their performances tend to regress to each stratum's universe mean.  Consider that in the last half century no credible college or university has been put out of business because their outputs were wholly without merit, or their graduates could not acquire employment.

Rank Versus Supply Real Information for Choice

The commercially hyped collegiate rating schemes -- U.S. News, Forbes, Princeton, and et al. -- have been widely criticized for their simplistic foundations, and the reality that they are minimal discrimination of a complex product.  But they, along with such counter productive ratings of “best party school,” are still allegedly used for input to a critical life decision, an American tragedy.  That prompts the leading question:  Is the Obama/Duncan strategy embodied in the proposed rankings one of the worst decisions of this administration, matching or exceeding even the core ignorance of present punitive-based testing in public K-12?  Would far better choices have been, for example, the long view with strategic research to field a legitimate comprehensive rating scheme for our institutions’ multidimensional areas of performance, call it the 'value-rating' model; or a non-punitive and affirmative alternative 'value-choice' model, the mission, providing comprehensive valid and comparable information on all public higher education institutions, letting the user supply their own criteria for use of the information for choice of school? 

Both example approaches start with the same research roots:  A priori judgments of the factors considered central to the quality and equity of higher education delivered, irrespective of whether those factors are presently quantified; next the development work is executed to convert those multidimensional factors, by algorithm or by scaling techniques to create digital metrics for factors.  At this point the approaches bifurcate, value-choice becoming the issue of creating easily accessible and universal databases, placing them in "the cloud" readily available online, searchable via criteria pertinent to the individual collegiate wannabe, or in another possible form as the material for use of simulation to derive optimal choices for a student.  The rest of our real world is inundated with clever "apps," available for even the ubiquitous smart phone.  Publicly accessible digitally, online, the system offers at low or no cost the structured information to personally search possible school choices.  The values or experiences available from a candidate school remain the elections of the potential student and parents, not predetermined by big brother.

The second approach -- value-rating -- does carry out the intent of the Obama/Duncan vision, ordinal rating of institutions, but based on the constituent properties of collegiate value delivery noted for the first approach.  What changes, what additional research is needed?  One model for the second approach might be structured as follows:  The starting point is a quota sample from America's colleges/universities serving as the development base, the sample reflecting meaningful categorizations of our institutions; for factors presumed causal for quality and equitable delivery by an institution, break out programs or tracks that constitute legitimate units of analysis; use a "human expert model" of decision making to create criterion positioning of the sample organizations, for the unit of analysis, by the various factors; then the goodness of fit is tested between metrics devised and expert positioning of all factors/units of analysis, mathematically determining the salience and weighting of factors that fit expert prediction.  Lastly, the metrics proving predictive are tested on a second comparable sample of our institutions for verification.

There are already out there, in the mass of college/university data banked on institutions' web sites, and made available in detail by a plethora of both public and private sector organizations, the raw data to start building either of the above approaches.  Most of our institutions are working with their own game plans, but the composite of data generated could be a starting point, for example, for building a universal higher education database serving the value-choice approach.  A tragedy of our present society is that a Bill Gates, instead of funding programs designed to beat on our public schools with testing, apparently lacked the perspicacity to pursue even his own suite of digital experiences to fund and guide the assembly of a suitable higher education national database?

Can the value-ranking model actually be executed?  It is arguable that it already has been in part, that the logic employed by Tom Peters and his associates in creating the corporate effort, In Search of Excellence, is an early precursor to that approach; it stopped short of seeking to quantify determinants of excellence, but the core idea was successful.  Using the power of that same Federal funding to our colleges/universities serves as an incentive to engage our universities in needed research.  That is a far better use of the incentive than seeking to intimidate our institutions into change by ranking linked to punitive reductions in funding.  Lastly, you are developing metrics that are defined by the real measurement challenge, and not by what was developed for other purposes or is simply convenient.

Conclusion

Historically, toward the end of last century, one of the Presidential Commissions on Higher Education offered the White House and our higher education community very practical recommendations.  They encompassed reducing higher education costs, reforming funding of tuition and other costs of a degree, and cooperation among all of our post-secondary schools to adopt a common set of parameters making available to America's families uniform ways to assess collegiate choice.  Both our college/university leaderships, and our political system quickly rejected all three sets of well-reasoned recommendations.  Clearly, moving either of the above approaches, or anything resembling them to a productive destination would require some new mindsets, among our higher education institutions, and in Federal education leadership's sensitivity to genuine national needs over liberal dreaming.

Counterpoint is that some of our colleges and universities, presumably “reading the room,” have already initiated innovative changes in their collegiate instruction.   Reported in Saturday’s New York Times, changes are occurring in B-schools’ MBA programs -- to emulate the rapidity of change and experimentation from Silicon Valley – and in basic collegiate science courses to move from lecture modes to high student involvement and problem solving.  Long valid patterns of diffusion of innovation will change higher education, even as the critically deficient Obama/Duncan rating scheme is stumbling out of the starting gate.  Perhaps merely the threat of that Federal ‘Franken data’ has stimulated collegiate action?  Incredibly cynical albeit clever if true; but if accurate the rest of program should be given a quick burial.

On real inspection the proposed Department of Education rating scheme regardless of intentions simply reeks of ignorance and flawed understanding of both complex academic organizational behavior, of advanced learning, and of the most basic principles of inquiry and social science explanation.  Their scheme could, analogically, be compared to trying to build a quantum computer using some AA batteries, a photo transistor, a couple of resistors/capacitors, and some wire scrounged from the ties used on garbage bags.  The present scheme, even if Machiavellian, as well as mirroring the mental set that any solution has to be punitive, is wholly unworthy of a Federal education function critical to our nation, and is condemnation of the current resources managing that agency.

Epilog

The next issues of Educationredux will move into challenges and opportunities throughout US higher education that might be areas for measured change along with possible innovations.  First out of the chute will be the footers for more productive higher education experiences -- bridging the chasm between our K-12, especially 9-12 school outputs, and the incoming requirements for collegiate success -- allowing passage through collegiate work with greater learning effect, in shorter periods of time, and therefore with less investment.

US Higher Education: The Light Versus Enlightenment?

The Obama Administration, fronted by Secretary of Education Arne Duncan, having virtually emasculated the chances for intelligent reform of US public schools by dogmatically and despotically backing test-based alleged “corporate reform” — from its inception a cultural throwback view of our schools’ issues, and dedicated to ‘test and punish’ — is now switching venues. Spoiler alert: Our system of US higher education may need to erect real battlements around their academic enclaves to fend off a horde of metric trolls.

For readers who have been preoccupied with trying to survive Black Friday and the need to gift those dear, what the US Department of Education is proposing to launch, allegedly in 2015, is a rating scheme for America’s 4,140 colleges and universities.  Here are the available details about that intent:

The Rating Scheme

On the table thus far from Duncan and company:

• Schools would be rated as “high performers,” or “low performers,” or “in the middle.”  (Note:  The critique invited by the overwhelming sophistication of this scheme is an immediate temptation, but assessment will wait for the whole story.)

• The reasoning, and justification for Federal intervention, is allegedly assessment of institutions with students receiving federal student aid.

• Allegedly being considered:  Which metrics; how to give credit for improvement; meaning and span of “in the middle;” a single composite rating, or multiple ratings for an institution?

• Factors in the scheme:  Accessibility — number of Pell Grant students, family contributions to tuition, student share whose parents did not attend college (what these have to do with educational performance seems a mystery); affordability — average net price, and ANP for families by income level; outcomes — graduation rates, transfer rates, grad school attendance, loan repayment, and “labor market success” (the latter apparently meaning graduates' beginning earnings, but a better index in today's economy might be time to acquire initial employment, or the discount in salary taken from the target profession's norm to acquire any job).

The last item to drop, ratings will allegedly be calculated separately for institutions segmented into homogeneous clusters.  An immediate observation is that even the simple factors noted — apparently there not because they are the most salient measures, but happen to be available as data — if they are probed are not at all simple.  Established in prior work by our institutions themselves, what seems straightforward, e. g., even average net price for an institutions’ students varies, with real import depending on how costs are staged or offset, and services delivered.

Many of the leaders of our colleges and universities have already weighed in on the cogency of these proposals. Not unexpectedly, most of the comments, while critical of the proposed mechanisms, have been constrained or politically correct.  Our colleges and universities in top tiers are virtually unanimously led by smart people; it is a reasonable proposition that were the faculty/research smarts encompassed by our best 100, or even a dozen excellent institutions, let loose on the validity and reliability of this proposal, the results might be a little fly ash remaining.

Educationredux readers will have to be content with a quick pass at the issues embedded in this scheme; Christmas would intervene were the whole enchilada attempted in one sitting.  Titles for the issues perceived include: Purpose of the ratings, and the core relevance of the proposed rankings; using ‘what’s out there,’ versus researching and designing metrics that are specific and valid; and the troubled path this proposal will encounter if its creators comprehend and apply the concept of “unit of analysis” that foots all science.

Purpose

The alleged purpose of the ratings is what; education quality, social equality, turning out the right human resources, deeply informed candidates, and all with a vague ordinal depiction of our colleges/universities?  The scheme as outlined so far is a patchwork of opportunistic measures, actually a crude multi-dimensional conceptualization; but purporting to offer information suitable for real world discrimination for life-modifying choices.  This scheme’s scope beggars the well developed work in marketing to develop multi-dimensional scaling of single brands.  To even consider simple ordinal positioning or ranking, i.e., comparative assignment of institutions of the complexity we have, ranges from magical thinking to a fool’s errand.

Using New Graduates' Earnings

This item gobsmacks even common sense, and raises the question of the core competence of those developing this scheme.  The determinants of beginning graduate salaries are complex, are a function of the subject matter specialization marketed, and are variably impacted by transitory demand versus supply of workplace candidates.  Beginning salaries are related to mid-career earnings, but not perfectly, and will this Duncanian dysfunctional rating factor wait for promulgation until the next 20 years’ experience of those graduates is logged?  Lastly, but critically, those salaries may have nothing to contribute to assessing the worth of either the graduates, or their preparation for practice, to our economy or society.  

How many more of the finance droids, that brought the US the financial meltdown, does our nation really need?  Or how many CEOs can the system support?  Versus how many more really good teachers, K-12, and post-secondary instructors, does this nation really need?  The proposed ratings scheme flips the world upside down.  It also says that Mr. Duncan, who has never graced a real classroom, or had an education about education, or has questionably matured beyond an extreme liberal visitor to “Alice in (education) Wonderland,” needs to find a new quixotic pursuit.  Perhaps he could link arms with Bill Gates, extinguish two misdirected blow-torches destroying rational US public education.

Unit of Analysis and Those Clusters

The readiness of this concept for prime time is already questionable simply based on the above issues.  The notion of creating a compensatory fix for inequities, by assigning metrics to clusters of institutions judged to be comparable, may constitute the most unreasonable part of the scheme among a litany of the unreasonable.  There are two issues:  What is the proper “unit of analysis” for assembling metrics; and what happens to the set when that unit becomes a valid one?

Saying you are going to rate a higher education institution on a few metrics is roughly the equivalent of saying you are going to assign one measure of assessment to, for example, the qualities of products in an Amazon warehouse. A newly minted college graduate may walk through a common commencement exercise, but the education represented issued from some specific track within that academic labyrinth.  Each track could be considered the proper unit of analysis, accumulated by a scheme and weighting that would metaphorically mirror putting a human on Mars.  Even going up another level of aggregation may work for valid metrics, but the reality of that analysis challenge doesn’t assuage much.  Here’s one example of the challenge you face in trying to decide how to assess one institution — it is one intimately familiar, but also representative of many in the US — Indiana University.

Indiana University (IU) has two main campuses, Bloomington and Indianapolis, different academic environments.  It has six regional campuses. The Bloomington campus has 14 separate schools plus a College of Arts and Science.  All 15 major units have multiple departments, multiple faculties, heterogeneous curricula (and some institutions differential tuition) — that factually determine the quality of a degree — with 180 majors, in 157 departments, representing 330 degree programs.  The other campuses have variable presence of the same venues, plus where a campus is a joint IU-Purdue campus, there may be additional departments representing engineering, nursing, et al.

So the question is:  What is the effective and defensible unit of analysis?  If it is the substantive track the student takes, and if even our 629 public 4-year institutions have an approximation of the above internal structure, the analysis chore for that subset masses up to over 200,000 unique entities to be judged.

But perhaps the most elemental critique of this Obama/Duncan odyssey is a classic used in virtually every operations research course ever offered, what is termed “the drunkard’s search.”  Referenced by philosopher Abraham Kaplan (author of a text used extensively in higher education research courses, The Conduct of Inquiry), it is his observation that:  “Much effort…in behavioral science itself, is vitiated, in my opinion, by the principle of the drunkard’s search:  There is the story of a drunkard, searching under a lamp for his house key, which he dropped some distance away.   Asked why he didn’t look where he dropped it, he replied ‘It’s lighter here!’”

Lastly, a challenge to the creators of this scheme to actually employ some of the science of measurement that has accumulated since Descartes, LaPlace, Pascal, Fermat, et al., roamed the historical halls of academe, through contemporary expertise:  Will the team developing this scheme even tap the most rudimentary pretest of its metrics; putting a test run of their results up against the expert judgements of a panel of our best and brightest, to see if their metrics can replicate the arguably informed and sophisticated professional judgements of quality of a cross section of institutions?  The prudent advice is, don’t try to hold your breath for the pretest.

Tentative Conclusions

American post-secondary institutions, especially the two-year and four-year variety that lack quality accreditation, or are isolated academically from primary campuses, and that lack the internal controls on faculty quality that are embedded in mainstream institutions, are most in need of assessment for the quality of outputs.  But a material fraction, of our almost 2,500 4-year public and private colleges/universities, probably internally does more work on maintaining learning quality than the US Department of Education does to police their own cognitive integrity.

All of America’s colleges and universities, however, may be candidates for inspection for symptoms of “Baumol’s cost disease,” referencing failure to aggressively seek functional productivity increases over decades.  And some of the mainstream campuses we all relate to may have components that have decayed, or are still fielding bricks-and-mortar excesses.  But what appears very clear is, this scheme by the Obama Administration is not a viable cure for any part of America's post-secondary education assessment needs; it comes closer to being another dose of Federal snake-oil.

US Post-Secondary Education: Higher or Space Cadets?

So begins a journey to try to understand how America’s very diverse institutions of higher education are positioned to bridge the chasm between what issues from US public high schools, and our nation’s demand for knowledge evolution and educational prowess for the next decades.  The attempt is footed by a quarter century in its classrooms and councils, another dozen years hiring its products, but still reflects the humility of always being a student of those complex learning communities that have evolved over a millennium (the first university attributed to the University of Bologna, 1088).

Worth contrast in this blog’s transition from public K-12, consider that our US public schools, even stretching their formation to seeds in the late 1800s, are less than 120-130 years old. 

In a prior post, copied to some of you, educator Dr. Grant Wiggins rather aggressively took to task our high school teachers for what he termed “dereliction of duty” in preparing their students for the college academic experience.  While his facts from an ACT survey may well be broadly representative of too much of present public 9-12 curricula, and provisionally much pedagogy employed, Wiggins’ arguments fatally ignored that there are two sides to the argument.  Personally experienced and noted in the writer’s early faculty years, our colleges and universities are as fully culpable in those periodic failures as our public high schools, perhaps more so.  It probably isn’t too strong an assertion, that our college and university denizens are contemptuous of, or simply ignore both public K-12 and even 9-12 processes, as well as our prospective K-12 educators launching from frequently demeaned collegiate schools of education.

Whether this chasm between higher education and public schools is rooted in differential history, or a function of the knowledge and pedagogy differentials, the social gap is disconnect that has damaged the mission of creating an educated citizenry.  There is another theory that what is taught at the 9-12 level is inherently incomplete or insufficient hence wrong as learning, because that environment and student maturation preclude necessary depth and complexity of explanation; the argument proceeds that is expected, to be modified in subsequent learning.   Irrespective, this failed bridge between secondary and post-secondary is just one of the issues to be pursued in future posts.

How to get a handle on any reasonable assessment of American colleges and universities is, of course, a bit of a bear; a procession of presidential commissions on higher education, dating from their inauguration by former President Harry Truman, post WWII, plus numerous other commissions and national study groups, have tried with limited success to encompass the genre.  In turn, the post WWII influx of new numbers of college matriculants from the GI Bill changed the game, as has the evolution of community colleges and regional campuses that frequently lack the quality assurance processes built into traditional academic faculties. 

Since that same time period some major shifts in policy, and especially funding of higher education have profoundly effected all public institutions.  More to be said on that issue, but a broad effect over a half century has been the eroded link between funding and oversight of those institutions:  Federal funding, especially of research has been a game changer; declines in state funding of originally state institutions have allowed evasion of much state governmental oversight; and increasingly both private sector endowments, and cooperative corporate research and private sector education programs have created a new and potent stake holder that impacts institutional policies.  All are possible future topics.

Seeking that “handle,” one approach is to go back to basics.  At the most elemental level, our higher education institutions can be viewed as a black box, receiving inputs of students, running a gamut of processes, and hopefully ejecting a modified human resource equipped better to perform as a productive citizen.  That model is a bit primitive, a bit like building a modern vehicle out of Legos; or for a controversial contemporary education example, using VAM (value added measurement) of student scores on standardized tests to assess K-12 teacher performance in creating learning experiences.

The problems with the model:  The inputs are diverse, ranging from every cultural and socioeconomic variant through the preparedness for post-secondary work; the black box is a very complex organizational form, that doesn’t conform to any simplistic management model, and contains sub-organizations, within sub-organizations, all varying with disciplinary contents and mission, e.g., education versus research versus public service; all with variance in sub-group governance and values; the processes for creating learning are equally diverse, differing materially from K-12 because there are few unifying controls on curricula, or preparation for the classroom, or in management of professorial resources, or even in values across disciplines; collegiate organization is typically by discipline, those divisions becoming cultural islands; hence, the processes that originate in the classroom can be as divergent as the individual faculty member.  The criteria for burping out a graduate vary with students’ occupational or further education destinations.  There is no common learning or graduation test.

How have these overall models panned out?  Criticism is obviously not hard to come by:  UNC, recently reported, created fake courses for 18 years to support sports teams; in a 2011 book, Academically Adrift, sociologists Arum and Roksa reported tracking over 1,600 students during college, and over 1,000 for a subsequent two years – their overall conclusion was that in four years students’ acquired knowledge changed little. 

The reader can judge.  Tuition at US four-year institutions overall, for the last dozen years, has increased +69 percent in current dollars compared to an overall CPI change of +27 percent in the same period.  Only 56 percent of US college/university students currently graduate within six years.  From the aforementioned 1946 Truman Presidential Commission on Higher Education, roughly two-thirds of its recommendations have never been adopted by our institutions in almost 70 years, fewer from subsequent commissions.  The presidential commission of former President H. W. George Bush, produced a 1990 draft report that was subsequently quashed, never to be seen again.  Its concluding paragraph may have been the reason; from a learned group that included multiple Nobel Laureates, that paragraph stated, “American colleges and universities are riddled with dry rot.”  Add from the work by Arum and Roksa, cited above:  “…dismaying: Of the students who didn't go immediately into graduate school, slightly more than a quarter earned above $40,000 a year in a full-time job two years after graduation. Nearly three-quarters relied on their parents for at least some financial assistance.”

But simultaneously our colleges and universities have graduated millions of graduates who have been equipped to professionally succeed, despite the exceptional glitches to be expected among 4,140 institutions currently annually graduating 1.8MM with four-year degrees, and another 1MM with two-year associate degrees.  Add, American higher education institutions are responsible for 14 percent of total US R&D expenditures, arguably equivalent invention, and far greater levels of if not most of contemporary knowledge development. 

Reform our system of higher education; a question up for grabs?  Our top universities still score eight places in the world’s top ten list, and represent a fifth of the top 100 in the world.  The key implications then for reform appear to focus on:  The equitable accessibility of our collegiate schools to all qualified; performance in holding and graduating those who enter; the productivity of those institutions’ deployment of assets; finding mechanisms to reflect the interdependencies between higher education and institutions both feeding it and employing its graduates; given the growth in size and complexity of our institutions, whether traditional academic organizational structure needs updating; whether present board/trustee oversight is now intellectually adequate for oversight, and whether other infrastructures are needed; and the wisdom of present academic leadership scenarios for future demands on higher learning.

One assertion that appears defensible, reforming our colleges and universities if necessary does not appear to be the stuff of public K-12 reform; indeed, the prospect of the crudity of present reform tactics being employed in that venue, including the proposed but ill-conceived Obama/Duncan ranking schemes, is venal.

There appears grist for future Edunationredux blogs, and room for debate.

Public School Reform: Micro – Macro?

Introduction

The progression of large-scale social changes as in natural systems tends to move from the relatively simple to the complex and disorganized.  With that complexity comes a form of cultural myopia:  “…information entering into the colonized mind is focused solely through a limited worldview, and anything existing outside of that limited worldview cannot be seen with clarity.” 

In the case of the 35 years of alleged “corporate reform” of America’s public K-12 schools, this has translated over time into two perceptual tendencies.  One has been to accept as given and let drift into the background the core issues that prompted what was essentially a revolt against public education as it had evolved through the early 1980s.  The second developed systemically with the increasing participation of those seeking prosecution of our schools, and in the consequent expansion of issues that are seen as included in that quest for reform.  In short, the proliferation of isolated or one-note items embroiled in reform quickly drives out or numerically submerges the core things that originally footed reform.

Today’s post dives under that barrage of localized or parochial material that has accumulated documenting the case for school change, as well as the now increasing evidence of public and system member pushback.  Too much of current rhetoric, by becoming highly relevant to only subsets of the whole school reform picture, diverts attention from those macro issues that have driven the attacks on US public schools.

Core Questions

There are five questions or footers for virtually all of current detail:  (1) Why did “corporate reform” launch, recognizing the key early players; (2) how did the current cast of reform advocates develop, and even polar values come to drive present tactics; (3) why standardized testing fails as the weapon of choice to try to force school change; (4) how have US public systems overall responded to the attack on their performance; and (5) how did a very large US human resource sub-population of teachers (3.1+MM) become the reform targets, versus statutorily accountable state education departments, public school administrators (215+K), and somewhere in excess of 80K sworn BOE members?

Why Reform?

In practically complete evasion of this most basic question, the apparent answer is, because the private sector consumers of the human resources being churned out by our public schools concluded that the public systems were failing their educational mission.  Retrospectively, there were few particulars offered by the early leaders of a reform charge, just ramped up presentation of an ideology built around inadequate education coupled with public systems’ alleged refusal of accountability for learning deficits.  A serious subtext, even now spoken only in restrained tones, was an alleged belief that our overall public systems were explicitly advocating extreme liberal values, counter to many private sector beliefs.

On the other side of the future skirmish line, our nation’s overall public education enclaves were doing pretty much all of that.  (Parenthetically, the counterpoint is what is occurring today in a Colorado BOE, where three extreme right wing board members are dogmatically trying to install an opposite, self-gratuitous political correctness to an AP curriculum for history; no less egregious than earlier overall public system attempts to install certain liberal values.)  That, in turn, was a legacy of schools of education that had by that point chosen self-righteousness over the nation’s learning missions. Absorbing and relaying to nascent teachers contemporary views of learning, and experimentally-derived models of "what works," were suppressed or ignored; instead those programs continued to float deductive concepts and install a teacher self-image.  It surfaces the old anthem, “you gets what you pays for.”

The initiation acts of the reformers were detailed in an earlier blog post, but key players were the Business Roundtable, and a former IBM CEO, with a selling effort that brought on board the National Governors Association (NGA), dominated by right wing views, and ALEC, the conservative legislative lobbying organization creating legislation for conservative state legislatures.  Subsequently, the Common Core State Standards Initiative (CCSSI) issued via the NGA, with an anonymous group of unknown academic competence creating an alleged “Common Core.”  What is now known is that the group creating those alleged standards was populated with few educators, many political representatives, and chaired by the present CEO of a testing company.  A group supposed to review the “Core” was identified, and consisted of some of our educational representation at best riddled with professional mediocrity.  In sum, the nation’s “best and brightest” were not enlisted to either provide or vet those standards.

Following years of under the table lobbying, and under the Reagan Administration, that reform agenda finally issued first as a call to action, “A Nation at Risk” (ANAR); then with the Bush Administration as the “No Child Left Behind” (NCLB) update of the “Elementary and Secondary Education Act” (ESEA).  A reading of the original language of NCLB leaves little doubt that the titling was pure hype and deceptive, and the purpose of the model was creating failure modes for public schools that would allow conscription of their assets for conversion into vouchers and charter schools using public tax dollars.  NCLB was a highly political act, the first round in attempts to privatize our public systems.

Even by the middle of last decade, the public was barely cognizant of what had been imposed on our nation, and even less celebratory, many of our public schools and their administrations were in their own parochial and self-righteous zones, totally oblivious to what was coming at their students and teachers.

Reform Adversaries?

“Corporate reform” started with the top 100 CEOs in the US, but quickly moved into a political venue, with the Cheneys prominent in the early attacks. Then advocacy was fully linked politically, frequently quietly at the time, by the plurality of Republican governors.  One example is Ohio, where a back door into Ohio’s Department of Education was accessible for those pioneering charter schools, and as early as the 1990s and virtually unseen by most, that format was being peddled to a gullible Ohio public as “community schools.”  Overwhelmingly, these schools were accompanied by corrupted promotion and payments to “charter consultants.”  Some audaciously claimed actual identification with the ODOE.  Many of these charters are now either failed, or have seen managements prosecuted for corrupt practices, and most constitute the bottom of the barrel of Ohio K-12 education performance.

Nationally, a platoon of opportunists joined the reform bandwagon, including Michelle Rhee, Jeb Bush, Joel Klein, Wendy Kopp with an allegedly righteous but misdirected “Teach for America” (TFA), and to America’s misfortune, scion of the “billionaires boys club,” Bill Gates.  It is arguable that Mr. Gates (who paradoxically never actually finished an education) has operated with the sincere belief that his intrusive intervention and use of wealth -- funding reform initiatives, standardized testing expansion, VAM attacks on teachers, the CCSSI “Common Core” installation among the states, and now pushy advocacy of at best a “CliffsNotes” view of the history of the universe -- were in the nation’s interest.  It is also arguable that Mr. Gates’ ignorance, arrogance, and misplaced intrusion in public education has deepened the school-reform divide, and complicated rather than assuaged the process for constructive US public school change.

Lastly, the dark underbelly of present reform is the phalanx of private sector textbook and testing companies.  The latter have been culpable in creating the need for public school reform for at least a half century, unchecked by any real oversight, institutional or governmental.  This set of players must be labeled as some of the most destructive corporate entities in our society.  Worse, they have become uninvited surrogates for legitimate scholarship in defining what is being tested K-12, and by virtue of that intrusion defining what is considered knowledge.  That is outrageous for a society honoring reason, science, and objectivity of inquiry, threatening America’s future standing as an educated society.

The list of destructive players continues, however, and goes bipartisan.   Covered in prior posts, Mr. Obama has been blatantly hypocritical in prescriptions for reform, and has virtually destroyed the US Department of Education with the continued appointment of Arne Duncan as its Secretary.  (Parenthetically and ironically, former President Reagan would have been proud of the service.)  It remains something of a mystery why Obama/Duncan have continued (linking arms with the worst of the right wing) to prosecute alleged reform, prosecution of NCLB, and a hopelessly bureaucratic and deceptive “Race to the Top” with billion dollar bribery of our states to continue to press standardized testing.

Testing, Testing, Testing?

There has been so much reported on the standardized testing being employed K-12 that it likely can’t be embellished.  All of the critique demonstrating that the testing model is either ineffective in creating higher order learning, or even destructive of it, is unassailable.  The logic for present testing is simply wrong.  But there are two different negatives operating at the roots of testing-driven alleged reform.

First, there is an elephant in the room that is being ignored in all of the minutiae – the core logic of the entire test motif as the backbone of school accountability and alleged search for performance.  That is:  Why is the entire test logic being employed, presumed the vehicle for accounting for performance quality, using quality assessment logic that was obsolete decades ago to achieve quality assurance, and that wouldn’t be employed by any contemporary US private sector enterprise in this century?

The resultant hypocrisy of every niche of reform prosecution is mind boggling.  Quality assurance in this century is based on controlling the processes that create entity quality, not on destructive or post-creation testing of a product long after it is timely or efficient to catch failures of quality in action.  Bottom line, it has never been about testing per se, but about the intelligence in how testing is employed.  Simply, even present formulaic testing is kosher were it being employed at the locus of learning, designed by those responsible for that learning, and being employed as a formative device.  The ignorance of present reform leadership is breathtaking and challenges credulity.

The second testing issue simply destroys what is assumed to be the Administration’s semi-delusional motivation for reform – trying to in any tactical time frame erase learning deficits attributable to racial, economic, familial, and cultural differences among the nation’s children.  Multiple studies have demonstrated that especially early grade standardized testing of children reflects more frequently their incoming and residual backgrounds defined above, versus assessing classroom effects.  The result is that effective testing of children with those differential attributes would need to follow a different course; one size does not fit all. 

Hence, present undifferentiated standardized testing becomes a counter-productive, if not destructive monolithic device in trying to boost learning among disadvantaged students.  This is not a new or original concept; see the extensive work of Harvard education professor, Howard Gardner, on multiple intelligences, earlier sidetracked by more educational naivety, but now being revisited in its proper not revisionist form.

When what is tested with present modeling, now massively deployed, defines what is being taught, and that in turn defines what is knowledge in America, the nation is in far greater trouble than an unfavorable comparison with other nations in the results of the PISA testing of students.  The only good news is that objections to testing overkill are gaining national traction.

Public School Sponges

A factor so obvious, but seemingly oblivious to our reform remonstraters, is the capacity our public systems have demonstrated to go from simply ignoring or denying that “corporate reform” exists, and that they are still under attack, to going venal by exhibiting more creativity in devising ways to cheat on that testing than it would have taken to launch self-reform.

In this milieu, there are a few exemplary public systems, that with courage and a ‘stick it somewhere’ attitude toward the reform vultures, that have creatively changed their own classroom models, and advanced critical thinking and learning.  The vast majority of our public schools have simply hunkered down, some too thick or self-centric to even acknowledge they’re under attack.  A strategy has been to simply game the testing to make it something doable without triggering any self-assessment of why they are being bombed with the testing weapon.  The resultant administrative malfeasance, not primarily our public system teachers, is the culprit but still unrecognized, or at least only grudgingly acknowledged and virtually untouched by alleged reform.

Our overall public K-12 systems precipitated the present reform war, and now are extending it by reticence or dogmatism.  Their failure overall to get around cultural myopia, and offset a century of self-righteousness and perceived entitlement, is now the fuel that is extending reform threats.  That this is happening, while part of our public, increasingly our parents, even their students, are showing more awareness of the threat to future learning, is wholesale indictment of much present public school leadership.

Roots of School Dysfunction?

When you dig for answers to why America’s public schools created the environment for “corporate reform,” then dig some more, the least referenced causal factor in this 35 year societal debacle is likely the most important. That factor is our obsolete and dysfunctional schools of education, not universally, but with only few exceptions the schools with roots in the original “normal” schools of early last century, or most states’ university-associated schools of education.

As one critic put it, our “schools of education think they own America’s public K-12 schools.”  In fact, most of our schools of education should be candidates for dissolution, and a new start.  They have created and fostered a faulty logic of learning for a century, have both failed to adopt the results of accumulating neural research to amend flawed deductive methods, and created faculties not competent to research learning.  It takes very little research to find that other nations' educators, for example in the key reform targets of literacy and numeracy, have gone way beyond the US in developing the underlying theories of human perception and cognition that foot teaching to achieve those goals.

But the indictment gets worse.  Factually, our schools of education have consistently attracted the intellectual bottom one-third of the barrel of college students, pragmatically defining most of our present teachers.  Without almost immediate change in that teacher recruiting and education system, or some drastic change in local systems’ further education of their teachers, it will also define the next generation of US public school teachers.

Having failed that basic training, along with failure to create preparation for competent school administration with awareness of contemporary organizational behavior, they have set up the present mess of public schools being unethically and ineptly guided.  The not politically correct bottom line; our public schools are staffed with the intellectually weakest outputs of our colleges. They are being managed by resources with no better and frequently worse credentials, and who may be motivated by factors that have nothing to do with most teachers’ still wholly sincere reasons for choosing the profession.

The place where public school reform should have begun is with the leadership of our public systems, the real responsible and accountable for school failure to educate for this century and beyond.  The proverb, “the fish rots from the head down,” could be applied to a fair fraction of current public school leadership.

Digging Very Deep

The OECD’s PISA international testing of public school secondary students is real enough, and likely accurately predicts that overall America’s public systems are turning out products who are now behind some other nations in learning.  That was not true historically, and it may well be that America’s perceived decline is attributable to other nations’ progress as well as need for US absolute improvement.  But as both economic and scientific performance world-wide becomes more homogeneous, it still demands that US schools find paths to create their own internal improvements in learning.

One wishes that improvement in America’s capacity to compete and function with societal excellence was the overriding thrust of present “corporate reform.”  Possibly early on in the challenges to public K-12 it may have been, with the better of our corporate community initiating the charge.  But as the movement became more politicized in the last and this decade, there is a suspicion that motivations for reform became blurred.

It is a cynical point of view, but it is also a credible assessment: That far too much of present public school reform is being pushed simplistically to try to reassert American exceptionalism in world testing.  To the extent that is occurring, it is ignorant selfishness.  The latter because major gains are being sought in 'our reformers’ time,' with little or no strategic awareness of the downstream strategic costs to the nation and its children, potentially intellectually and professionally depreciated by the present standardized testing to achieve myopic learning gains. 

If there is a kernel of truth in this point of view, there is even a stronger case to assert that any real public school reform will need to blossom grass roots, propagated from within our local systems, or at least be coextensive with different reform modes and from better angels in our states, in the USDOE, and from our universities.

Some Old and New Public School Reform “Straight Talk”

Forward

Over two decades of not terribly tranquil senior management — spanning the academic version learning you don’t herd professors, through a private sector turn-around and a high tech start-up — some critical experiential learning was installed. That was, that too many managements spent too much energy and time pushing minutiae.  The concept applies to present public school alleged reform; witness reform zealots as well as inept school management stomping on test and bureaucratic ants while metaphorically rogue elephants as well as some intellectually challenged jackasses roam the halls.

There are a small number of genuinely critical factors in the present public school reform war; they are:  A chunk of education history, that failing retrieval, is like the proverb doomed to be repeated; the critically deficient system for oversight of most US public systems; and the lessons from properly viewing any complex system, that change is neither for the timid nor those seeking instant gratification.  A few more biggies are out there deferred for now, but they don’t include standardized testing, VAM inaccuracy, phony accountability, or firing teachers.

Following are amended versions of two prior 2012 Edunationredux posts, plus note of the above third cited factor.

Factor One:  Edunationredux April 4, 2012 - Where Are We?

Not in a good place, and pushback isn't happening fast enough to potentially brake before the cliff.

Saturday's (March 31, 2012) The Washington Post, "The Answer Sheet," presented contrasting tones. Stanford Professor of Education Linda Darling-Hammond, an alleged champion of public education, and advisor to Mr. Obama before his election, mechanically repeated the platitudinous squibs well known for over a decade, that income and cultural differences among schools and students have more to do with performance than what are presently being institutionalized as reforms.  Apparently, neither Mr. Obama nor his subsequent attack leader, Mr. Duncan, registered the message, then, or since.

Churned up in a related search, Indiana University's "Center for Evaluation and Education Policy" at this century's turn reported the results of a comprehensive quantitative assessment of the income/culture-performance question, inferring that most of the variation in public schools' performances could be statistically linked to demographic, socioeconomic, and cultural attributes of their participants.  Also reported at the old stomping grounds, and also over a decade ago, a meticulous review of research on classroom teaching methods, indicating that little of the experimental results purported to be the definitive "methods" guides for public K-12 classrooms reflected fully defensible research or analytical techniques.

But NCLB, RTTT, standardized testing, and VAM have moved the goalposts and improved scoring, right?  Another tidbit reported that hasn't received much attention by its state's press, dated April 2012: "...25 percent of Indiana's students passing its Core 40 testing (Indiana's graduation test) require remedial course work entering postsecondary work."

If that isn't straight enough, continue reading.

The Duncan Disaster

At the top of the present mess is an educationally delusional, politically hardened, or just hypocritical Arne Duncan, who should be removed from the U.S. Department of Education’s head shed, forthwith, and with prejudice.  “The Answer Sheet” made the case for impeaching his leadership.

Naïve thinking?

A property of virtually all of the critique of standardized testing being floated revolves around the value judgment that it is testing various wrong things – test validity – or that the testing is unreliable.  Legitimate research of present standardized testing suggests both cases can be made, but the roots of an entire dysfunctional movement, imposed so aggressively and with so little intelligent deliberation, go much deeper than failure or perversion of the psychometrics that allegedly underpin the tests.  One has to start with basic causes.

The first observation will certainly not be popular with many of those in public education finally awakening and registering current reality, that one intent of the testing mechanisms being hustled is to push public K-12 out of the education business. If you are one of those late bloomers, let's be clear, that is one of the goals of some fraction of the shadowy and diverse origins of present testing; the goal is privatization of K-12 in the misguided belief that “markets” will cause competition to perfect our schools.  That naïve view is the alleged wisdom of neoconservatives lacking a functioning cerebral cortex, and arguably much economic education.

What triggered this point of view?  This will also not be appreciated:  What created the present devolution of US public K-12 is the broad and historically extended failure of public education self-assessment, and it's teacher feedstock creator, our fumbling schools of education.  Public K-12 has had at least a half century to start understanding learning, moderating the ignorant dependence on deduced "methods” with neural biology supplying real answers.  Also contributing, liberal political correctness that infiltrated virtually every US public school last century.  Remedial steps -- including reforming curricula, and adopting the organizational knowledge and managerial theories and practices that drive ethical private sector performance and creativity -- never made it across the starting line. Those failures, translating into some fraction of public education school boards, administrators, and teachers ranging from self-righteous to myopic even when ethical and educationally competent, are what initiated and have fueled the present disaster.  

Somewhere in the halls of public sector leadership the belief was formed that public K-12 could or would never reform itself, and that the only Federal tool out there was the testing hammer of NCLB, perpetuating the most simplistic and punitive properties of ESEA.

Beating a dead horse?

If this seems pejorative, one only has to observe in real time some of our heartland’s public systems, including where based on only test scores, excellence is trumpeted. Two local systems in this blog's backyard stand out. 

The first has elected over two decades four superintendents not role models: The earliest was subsequently convicted of felony trafficking in child pornography; the next in line was seemingly educationally competent, but unethical and guilty of educational fraud; next an allegedly borderline sociopathic system leader seeing contemporary school management as "command and control," and allegedly maliciously and discriminatorily blocking any taxpayer who might challenge the model -- Ohio's sanction for that educational performance, appointment as a county superintendent, embedded in Ohio's most educationally corrupted infrastructure, its so-called education service centers; and the most recent leader installation, guilty of plagiarism and dishonest manipulation of school issues, and allegedly lacking educational competence. The school's curricula are still peppered with mediocrity, some contents and methods simply wrong.  Its digital strategy started with fraud, and in spite of the gift of extraordinary internet bandwidth, remains a deficit for its students.  Consequently, with perverse logic, it declares itself excellent.  

The system's BOE, in turn, has squandered every opportunity for transparency and broad community support, losing two recent major facility levy votes (unheard of until these votes) by trying to scam the community. Ohio's open records and meetings acts have been repetitively violated.  The BOE's latest act of electoral good faith, was allegedly tinkering with the last board election to deflect competition for the BOE and seat an 'inside board.'  This is a system, that on the surface and in its proclamations of excellence looks good, but smells bad; a Texas idiom, "all hat and no cattle."

The second system may be worse, but perhaps with guilt, less glib it its propaganda.  The overall system appears mediocre with occasional teacher bright spots, but unsupported; its BOE is now chaotic, the result of misguided board leadership that embodies everything that has become negative about too many US public schools:  Self-righteousness and ignorance of BOE responsibilities; the belief that due and democratic process just gets in the way of control by an inside cabal; disrespect for Ohio law; literally all of the behaviors that in larger measure characterize institution performance that can become dictatorship.  Also dominating are examples of arrogance and the lack of competent oversight, with administrative leadership that is equally malfeasant.  All of these factors have some icing on top; a majority of its BOE members too cowardly, or ignorant, or phlegmatic to step up and execute the oversight that they swore to uphold with their oaths of elected office.  A role model for the system's children?  Meanwhile most of the community remains blissfully unaware of or disinterested in most of these deficits.  

Some of the above characters' performances, both systems, should be addressed by Ohio's Department of Education, but that politicized department as earlier noted acts with virtual total disinterest in local school performance (if it isn't embodied in standardized test scores and school grades), or local system administrative integrity, or genuine Ohio K-12 learning.  There are obviously educationally competent and civically meritorious public school systems in Ohio, but they may be developing about the same incidence as the occurrence in the US House of Representatives of members who are intelligent, politically centric, and uncorrupted.  

Is this the public K-12 education it’s worth saving with double-digit billions of tax dollars, or by going to the mat with the USDOE, an Arne Duncan, or a Bill Gates, or with our politicized state education departments, or the corporate testing and charter lobbying enclaves?  In sophisticated language, education professor Linda Darling-Hammond suggests we simply need to play nice and bring together all of the social/institutional components effecting public K-12. The answer, or a serious academic reality deficit?  The conundrum represented, draconian attack of public schools with narrow testing, local retro public K-12 and their BOE, two-faced Federal leadership, and an electorate with the capacity for strategic time scales and delayed gratification of our 21 year old cat; all challenge finding quick or one-note solutions that can actually advance public US K-12 schools and real learning.

The infamous accountability challenge

Accountability is like “ahhhh, Bach;” who can be against accountability? 

But let’s put it in place where it most effects genuine learning:  Congress kicking the EASA/NCLB can down the trail; moving on to Arne Duncan and an extreme liberal but now operationally nearly despotic U.S. Department of Education; a couple of generations of scattered but embedded incompetent or dogmatic public K-12 superintendents and principals; an equivalent swath of incompetent to worse local school boards; some fraction of the products of our schools of education who really are incapable of intellectualism and competent classroom teaching/coaching; let’s not forget the politicized and educationally questionable denizens of many state departments of education; the farm-team academics or anonymous text publisher back rooms typically producing most public K-12 texts, and the corporations aggressively lobbying every state to continue to peddle them; schools misunderstanding and rejecting proper technology adoption; and maybe lastly, let’s not ignore a wide swath of parents clueless or desultory about the full role they are supposed to play in seeing their children educated.

Accountability?  Damn straight we need it, but who’s delivering?

The testing mysteries

Moving on, the majority critique of alleged standardized testing revolves around its relevance to real learning, then the tests’ specific validity and reliability, each having multiple parts.  All three points are relevant, but peek behind the wizards’ curtains.

Who, specifically, is writing the tests’ questions, based on what logic, with what credentials, supervised by what expertise, with oversight by which subject matter experts, with what inputs from real classrooms, using which psychometric principles, field tested when and where before being rolled-out nationally, and subject to what mechanisms for critique and resolution of test failure or performance distortion?  Who is watching the corporate creators of these multi-billion dollar businesses – the answer of course is no one supported by regulations or with the power to create confidence in what’s being delivered.  Does this scenario conjure an aroma of financial meltdown déjà vu?

Have you ever heard of the JCSEE?  The acronym stands for the "Joint Committee on Standards for Educational Evaluation," representing the major national associations that have anything to do with K-12, spanning 18 associations from the American Association of School Administrators to the National School Boards Association.  Since 1988, the JCSEE has for K-12 schools published and updated every five years:  Personnel Evaluation Standards; Program Evaluation Standards; Student Evaluation Standards; and accredited by the American National Standards Institute (ANSI, that every technical professional will recognize), making those standards “American National Standards.”

Give or take a little Kentucky windage, the current standardized tests being bought with billions of tax dollars, to allegedly metrically gauge manufactured recall and goad teachers, violate virtually every caveat of these standards.  To add insult to injury, the testing, to the extent it is transparent, appears to violate most of the caveats articulated for test design specified by the psychometric literature.  In any discussion of that testing to date, we have yet to hear which theory of measurement, if any, has been used to create the test items:  Item Response Theory, Item Characteristic Curve Theory, Latent Trait Theory, Rasch Model, 2PL Model, 3PL Model, the Birnbaum Model, or just some teaching professionalism; all of the above; none of the above?  Listening...

In fact, we don’t know how these tests are being conceived, or who is by indirection calling out what has become sadly de rigueur by state for all subject matter to be tested, hence learned?  If this is a “black box,” who is defining learning in US public K-12 schools, with what logic, with what purpose, with what demonstrated expertise, and by what authority?  Is the game to support our uncontrolled text/test oligopoly of barely visible corporate profit machines, manipulating both what they with uncertified portfolio declare to be knowledge, then enforcing that omniscience by also writing its tests?

If so, fools are determining the contents of US public K-12 education.

A small part of the above puzzle was addressed in an April 27 post to The Washington Post’s “The Answer Sheet,” “Pearson and how 2012 standardized tests were designed.”  It is only a partial answer, but on its face already a basis for challenging the entire standardized testing strategy underpinning alleged but seriously flawed reform.

Where the radical left joins the radical right, centrism isn't the product, and everyone loses

The next issue is the product of literally years of puzzling:  Specifically, why is the Obama Administration in bed with the most zealous enemies of public education?  Who are the players behind closed doors who have executed what tactics, paid what amounts, and used what intimidation or extortion to blend Obama oil and extreme right wing swamp water.

If the argument is, this testing and VAM are the only tools available to Federal enforcement under the Constitution, there is at least some logic in the tactics; simultaneously, failure to recognize or callous disregard of present tactics’ unintended consequences is egregious intellectually and ethically.  But the extended question is whether the actions being pushed by Mr. Obama and Mr. Duncan actually represent an extreme liberal delusion that was never subjected to proper vetting?  How is what is being inflicted on U.S. public education by this Administration any less destructive than how public K-12 brought down its own house, or the educational Darwinism advocated by the worst of political and educational retrograde thinking on the right?

Lastly, by failing to challenge the National Governors Association, and short-circuit CCSSI perversion of knowledge standards to ramp up ignorant state testing and curriculum requirements, the Administration has set up virtually all K-12 to be nickel-and-dimed into strategic mediocrity and reversion to a past century.  The stupidity of state actions is already being unfolded, for example, launching in Indiana and Ohio with state administrative bully-driven and retrogressive testing and school grading, even while nationally protests are building to cease the testing blitz.

A small, still material unintended consequence, and a huge one

There are multiple negatives strewn across the US by the present manipulation and dissembling being called reform, most already called out by our best students of K-12 history and prospect.  There is another unintended consequence:  Discouraging students from challenging themselves and choosing higher level courses when they may have fewer normal grade implications, if the course test results are factored into rating their teachers.

The huge one, however,  that has not received attention is the consequence of the diversion of attention, intellect, and energy to firefights on testing, versus addressing an educational tsunami:  Playing political or utopian games with last century’s bases for learning, while those very conceptual bases are undergoing massive transformation.

Churning just beneath the surface of applicability is a mushrooming universe of technologies and expanded knowledge bought with the R&D investments of the last half-century. What the philosopher terms knowledge, and pragmatically new understanding are changing core beliefs of how everything works, universe to elementary particles through individual human and social behavior, and aggregating and doubling at a rate exceeding Moore's Law.  We are already at the boundary – primarily effecting secondary education, but inching toward the baccalaureate level – where what we've assumed for over a half-century can be trimmed or surveyed and introduced in 7-12 or 9-12, simply won't fit anymore.  Indeed, unless there is a sea change in the re-education of public K-12 teachers, or rethinking the seat-time model, or rethinking even the entire basis of pre-postsecondary education, the whole alleged learning package becomes retro smoke and mirrors.

You either craft a new theory of what public K-12 and especially 9-12 should be, or the subject matter issuing in future US classrooms and presently tested, is not just useless but destructive of this society’s economic growth and sustainability.  

Footnote:  In the 26 April 2012 issue of the world science journal, Nature, neuroscience author, science journal publisher, and professor Michael Shermer, as part of a book review noted:

"It has been estimated that, from the beginning of civilization -- 5,000 years ago or more -- until 2003, humanity created a total five exabytes (billion gigabytes) of information.  From 2003 to 2010, we created this amount every two days.  By 2013 we will be doing so every ten minutes, exceeding within hours all the information contained in all books ever written."  He concludes:  "...the mountain of facts is now so vast that we cannot hope to learn, let alone remember them."

Thus, it just makes sense to repeat last century's production model of education; memorize facts, with disconnected fragments of knowledge, and flog with standardized tests, that engender no questions nor creativity?  Add that future manufacturing may more nearly resemble 3-D printing.  Then "corporate reform" is America's thrust for its future?  Well, not so much.

Factor Two:  Public K-12 – Is School Board Reform an Oxymoron?  

(Update 3/13/2012)

There are allegedly around 14,000 of them.  Politically, like death and taxes, they will likely  be with us in perpetuity in one organizational arrangement or another. 

At the same time, though earning widespread disrespect, they are also rarely mentioned in the ongoing assault and alleged reform of public schools.  They are the frequently elected, sometimes appointed, sometimes qualified, rarely properly vetted electorally, rarely trained/prepared for their functions after installation, rarely perform transparently or are made accountable, and the alleged community representatives we love to hate -- your local school board. How does our nation strategize and implement public school change that must depend on oversight and educational literacy of our BOE?

This topic is easily worth books; in fact, there are 37 relevant books on school board governance in the first dozen pages of Amazon listings for a search on “school board reform,” and some multiple of that in relevant journal articles, few of which if any have ever been read by school board members seen to date in this neck of the woods.  Many boards do not know their responsibilities much less exercise them.

To keep this post manageable, the topics are restricted to some high- or perhaps, low-lights:  Empirical knowledge about our boards; what happens to school boards; possible board reforms; boards as factors in K-12 reform.

What do we know?

The first item; what do we know about board performance?  

The answer, virtually nothing based on good research methodology.  Compounding the issue of how to gauge board performance is the need to cover years of actions to assess the concordance of board behavior with related school performance.  Further complicating assessment, the effects of a board’s function are played out in the diverse operations within a school or district and may show up as delayed effects.  One obvious, politically incorrect example, is the board that insidiously puts a school’s sports’ values ahead of learning, even to the extent of choking off spending for learning infrastructure in favor of sports complexes to feed parental and community sports egos.  It may take years for the cultural impact of such a value system to be seen in graduation rates, or meaningful assessment of real learning.  When it occurs, the board that spawned the degradation of real education is frequently long gone, the link erased to protect the ignorant and guilty.  Most schools and their boards aren’t believers in “double-loop learning.”

Board research also needs to be longitudinal, and the cost to secure sustainability of current longitudinal research is high, both in maintaining organizational relationships with systems to allow study and the lack of funding for such research, versus the episodic issues seen as central to classroom function.  It takes strategic perspective, not big at any recent time in US public K-12 education.

What is it about school boards?

Other boards work, why not school boards?  This question puts you into the heart of the issue.  For example, how can five or seven literate, intelligent, frequently professional human resources seen individually, turn into a board that becomes paranoid, secretive, unresponsive to those who elected them, possibly micromanagers of a system, or alternately so intimidated by a superintendent that they have little effective oversight of that system?  Witness to the latter syndrome, boards where the minutes of a board meeting are prepared by a superintendent in their totality and never amended – well in advance of the meeting in question – or where responsibility for strategic issues is simply delegated to a superintendent because board members are risk averse.  All it takes, referencing the earlier system example, is bad leadership and board members too timorous or lacking the integrity to honor their oaths of office.

The above immediately directs the discussion to the roles of the board, versus roles of a superintendent.  Related, whether many school boards, even when they represent reasonable elected membership, are equipped without further professional counsel to hire a superintendent.  One reality is that even when you have a competent board, matched with a competent superintendent, the roles to be played are not simply ones that can be easily codified, but represent a subtle dance of the two entities and sets of functions.  

One of the most frequent criticisms of boards that are populated by the generally competent is that they are still predisposed to micromanage, or focus on minutiae instead of policy and larger issues.  This speaks to whether most boards, even consisting of competent members, have the organizational awareness or coaching to fashion the playbook to stay out of most school operations and within the agreed board policy and decision boundaries.

The other side of this coin is whether a board is predisposed to get between school leadership and its community to protect a system from funding and other topics that take on community-wide disagreement.  That is one of the roles, but one that is a hard sell when a board lacks confidence in its policy positions, or is more interested in re-election than supporting learning.  

Another of those realities is that too frequently school board seats are sought for reasons other than service to K-12 education:  For ego and social self-promotion; to pursue special interests, or in many cases a prior grievance with a school; as a stepping stone to other public office; and even as a way to practice nepotism or award the “good old boys’ (or girls’) network” in bringing human resources into a system.   As there is little oversight of a school board once installed, unless there has been state reform to enable a malfeasant member or even board removal from office, who watches the watchers?

At the end of the trail in trying to assess board quality, the issue comes down to a combination of how human resources are chosen for any material assignment, and whether after they are chosen, there is in place the necessary developmental work to create the expertise for the role, akin to the fashion of boards in other venues.  In sum, you don’t invite the incompetent to become the basis of organizational oversight in good corporations, or in boards of professional associations, or in pubic sector organizations where legitimate oversight is sought; or the illiterate or naïve to serve as oversight of public K-12 education.

Reform possibilities?

Is school board reform possible?  A raft of optimistic educational researchers, pundits, the National School Boards Association, and related assets still believe it is.  Below is an abbreviated list of proposals that have been floated for school board reform:

• Move to appointed boards, or a mix of elected and appointed boards, where qualifications of appointed members can be required.

• Change the electoral patterns for school boards, requiring the testing and debate in the public square characterizing most elective competition. 

• Statutorily increase the educational requirements to run for a school board.

• Require mandatory training for elected board members; possibly even certification by testing before a board member can be seated.  Add mandatory periodic developmental training for currency.

• Require a code of ethics and conflict-of-interest policy for all boards.

• Statutorily provide for removal of a board member, or an entire board for cause. 

• Better define the roles of a board versus a superintendent, even express these contractually. 

• Specifically define the duties of board members, with provision for requiring performance to maintain position. 

• Create a school board report card, with annual assessments; a recommendation of the NSBA. 

• Merge districts for board representation, to reduce the number of boards, increase the pool of competent candidates for election. 

• Pay board members at a sufficient level to create performance incentives and provide disincentives for malfeasance. 

• Organizational training in addition to educational indoctrination, to improve the actual conduct of board operations, including awareness of the needed transparency and communications relationships of a board with its constituent community. 

• Require qualified advisory groups from a community be used to provide professional assessments of superintendent hires, forecasting and budgeting, school design and construction, and social and behavioral issues within a system. 

• Take on the voter educational task of explaining K-12 pedagogy and reform needs to parents and the community, because a board is an intermediary between system and those funding it.

• Establish a solid pattern of communicating with parents and the community; one strategy that automatically improves both the contents of school board meetings, and the community’s interest and attention to education, is using the CATS provisions of local cable operation.  Put your meetings online, in real time; where this is employed the whole spectrum of quality of content through quality of board deportment improves, and a community in turn learns why there are school challenges, and why their support is important.

Do any of these recommendations, drawn from many sources including ones representing school boards, have a chance in the present reform environment?  They are all pretty rational, none really extreme judged against the contents of professional standards expected in other venues that have a lesser impact on American society.  Answer:  Highly unlikely in the present US education environment.

Bitter addendum from the search

In the process of researching this post, an opinion piece by nationally known educator Larry Cuban was noted.  Always informative, this one captured the writer’s attention, not by its erudition that was substantial, but by the large number and the contents of comments it had elicited from parents with children in our public schools, including many parents who were also educational professionals in some capacity.

The parental comments went beyond troubling, indicating broad frustration and discontent with their own public schools, even myopic teachers, but especially dogmatic, myopic and self-righteous boards, principals and superintendents, more concerned with rules, risk aversion and deflecting transparency and critique, than whether the children involved were ever being educated beyond achieving on standardized tests to keep school images, and their own reputations intact.

Astounding was the sameness of the critiques of public K-12 systems widely scattered across the US, reflecting vitriol for public education professionals who just wanted those parents to go away, let them practice what they knew, even if it was last century’s education, and expressing either disinterest in or contempt for internal creativity and change in any facet of their systems including greater teacher involvement in the core processes.  If there is any question why the bizarre combination of a liberal President and a profiteering and potentially dirty segment of the corporate sector, with a few narrow or billionaire do-gooders thrown in, are the merged driving force of alleged public K-12 reform, our public education establishment doesn’t have to look beyond some its own door jambs.

A postscript

From a PhD researcher at the Intercultural Development Research Association, a bipartisan Texas-based organization working on quality of teaching and learning, the prescriptions for school boards’ efforts to improve performance were logical and fit school reform needs:

“1. Become better informed of community assets and needs, student characteristics, and implications for a quality educational program. Although most states require that their school board members receive training during their tenure, the training rarely targets knowing their communities (assets, needs, student characteristics) or basic knowledge about a quality education program. How can we entrust the education of our children to persons who are responsible for school policy but who have a limited knowledge of quality education and quality teaching?

Unfortunately, it is not uncommon for school board members to become totally disconnected from their role and the duty that they are elected or appointed to carry out. The community that elected them should demand greater interest, action and leadership from them.

2. Engage in constant dialogue with community leaders and parents to ensure that schools work in partnership with community members and parents to enrich the quality of education to be provided. Successful school boards meaningfully engage their communities in periodic forums, meetings and reflection sessions to check the pulse of schools in graduating students who are ready for college, in ensuring that schools are holding on to students, and in creating school environments that are safe and responsive to the needs of all students.

Building community consensus and support for school transformations based on research and compassion are powerful methods. It also can neutralize the effects of political rivalry and enmity that cause school board paralysis, deadlock and inappropriate action. Too often school boards engage community only during election times.

3. Promote and facilitate partnerships with community members and parents as a powerful way of creating and sustaining educational change. Recently, a leading school superintendent was lamenting the lack of knowledge and commitment of school administrators to value and partner with their communities and parents to create a learning community that works and supports a quality educational program.

Effective school boards are strong advocates of meaningful engagement. They promote and facilitate partnerships with community and parents as a powerful way of creating and sustaining change that leads to student engagement and success. School administrators must realize that total student success will not be achieved until the school partners with all sectors of the community and parents and has the full confidence of students.

4. Become an integral part of a leadership team responsible for designing school reform efforts. Many times, school boards underestimate their contributions as citizens and elected representatives of the general public in school reform efforts. They bring different, essential perspectives into the planning and design phase of school reform. They are in a position to change policies to enable schools to make the necessary changes.

The total disengagement of school board members from school reform efforts can have a detrimental impact on schools’ success. By disengaging, board members abdicate the power and responsibility entrusted to them through the democratic process.

5. Be accountable to the community for excellence and equity in the provision of services and the resultant academic accomplishments. If systemic changes were well-defined, understood and supported by an informed school board, they would be less vulnerable to disruption of educational services to students created by school leadership changes like a new superintendent or new principals. Many times, leadership vacuums left by superintendents’ or administrators’ sudden departure lead to complete school disarray and dysfunction.” 

The above are immediately a source of frustration; for they are what most smaller system school boards work diligently to avoid.

Factor Three:  Zap Them or Think Strategically?

Lastly, coming contemporarily from an outstanding legislator and student of schools in multiple cultures, the wisdom that in any large system the rate and scope of education change within short time spans can be dysfunctional.  The thoughts were aired recently in The Washington Post, "The Answer Sheet."

The author is Andy Hargreaves, Brennan Chair in Education, at Boston College.  He is an advisor to the premier and minister of education of Ontario, Canada; and in the last year participated with the OECD (source of the PISA international tests that are used as the rationale for much of the US public school reform agenda) in reviewing a Welsh strategy to simultaneously raise student literacy and numeracy in a brief time span.

His and the OECD's counsel; don't.  Hargreaves' assessment:

"Large-scale literacy reform has been in vogue in the United States and elsewhere for two decades now. It has been one of the driving forces of educational change across the country and many other parts of the world. One of the places it began was in New York District 2 in the mid 1990s. There, the Chancellor of Schools, Anthony Alvarado, and his staff, imposed a literacy program across the whole system, linked to measurable achievement gains, and backed up with detailed new materials and intensive one-on-one in-classroom coaching.

Harvard professor Richard Elmore and his school superintendent coauthor Deanne Burney articulated and applauded the reform design and its impact on results. Diane Ravitch later took some of the edge off the achievement gains by arguing that some of them were a result of gentrification of the community, not of the change strategy. But the more important point is that when the San Diego school district became enamored of the model, and transplanted Alvarado and many of his team members to implement it on the other side of America in a fraction of the timescale, the results were catastrophic. Gains were not sustainable and open warfare broke out between district factions as teachers and principals buckled under impossible high stakes pressure for short-term results. What was the lesson to be learned? Large-scale literacy reform has to be grown gradually. It cannot be imposed impatiently."

But that is precisely what NCLB has attempted, then doubled down by the Obama administration with RttT and throwing dollars at systems, arguably seeking a quick education win to burnish the two presidential terms.  That the strategy has failed is becoming obvious, and in the manner experienced in the far wiser OECD read of those needs for change.  Hargreaves sums up the experience: Problems were "...massive teacher burnout and professional disillusionment that led to a crisis of recruitment and retention of teachers.  Simultaneous imposition of literacy and math reform requires teachers to change all their practice all at once and this is so overwhelming that it threatens the basic capacity of the profession to maintain its quality."

Reflect those effects occur in education systems dwarfed by the US, and likely with far greater national control of the vagaries that inflict and differentiate American public schools.

It is a cynical point of view, but it is also a credible assessment: That far too much of present public school reform, whether flogged by Obama and Duncan, or right-wing ideology, or some cabal of our corporate community, is being pushed simplistically to try to reassert American exceptionalism in world primary and secondary education competition, driven by nationalistic hubris, and candidly, ignorant selfishness.  The latter because major gains are being sought in 'their time,' with little or no strategic awareness of the downstream strategic costs to the nation, and to the children potentially intellectually and professionally depreciated by the present standardized testing stupidity and creation of myopic short term learning gains.  Ego and myopia are an educationally destructive combination.

Add to the deficits, our teacher population and the foundations of an egalitarian public school philosophy become collateral damage, and that overarching mission of "corporate reform," creating a better and more creative labor force, will see virtually the opposite proceeds.  The presently derided predictions of even highly knowledgeable business strategy gurus may prove not fiction at all; that within some of your lifetimes a combination of robotics and artificial intelligence (AI) may further decimate our middle majority of knowledge workers, for cause!

The bottom line on things that are still crippling US public school strategic reform:  "Plus ça change, plus c'est la même chose."