Wednesday, February 25, 2015

How the Testing Grinch Stole Learning

Every US public school student, and their parents, by now have experienced the joys of NCLB’s standardized testing.  In the spirit of testing, why?

An answer is a curvy and pothole-filled path, but the only way to enlightenment.  So let’s assume you overhear the séance of a parent with the departed spirits of two architects of US public schools, Horace Mann and John Dewey.  The dialogue snatched went as follows...

Parent:  Hi Horace and John, please, help me!  Why is some massive “corporate reform” movement beating up our children and public schools you created?

HM & JD:  Good morning; while we try to avoid dealing directly with your present society, we’ll make an exception and take your questions.  Recognize that I, John the humanist, and fellow spirit Horace, more authoritarian, come philosophically from different places – a bit like your present Congress.

But to your question, why?  Four reasons:  (1) Because what we would call the “nanny state” now trusts neither you, nor your teachers to prepare your progeny for life and work; (2) because your “advanced” society apparently believes all children are widgets and need to be produced to a common knowledge blueprint; (3) because some segment of your society believes there can be a perfect human condition, fully defined by a “metric” (we don’t fully understand today’s term though one of us invented the "Dewey Decimal System" for libraries; and (4) because something called the Business Roundtable, and a person labeled a CEO from something called IBM, once decided your schools weren’t adequately preparing their future employees.

Parent:  Whoa, I’m deeply offended; you are saying we are failed parents?  Are you saying our public schools – that both of you helped configure – are failing?  Are you saying our leaders don’t trust us?

HM & JD:  Unfortunately, precisely.  What was the last non-fiction book you read that addressed the basis and trajectory of learning?  What was the last adult education class you attended?  When was the last time you addressed your BOE with tough questions about your school’s curriculum, teacher quality or leadership, or challenged your BOE’s secretive deliberations and choices?  In fact, when was the last time you demanded BOE candidates answer some real education questions before an election, or demanded they be required to undergo some training after election before being seated? 

Parent:  Really hurtful, but reform is working, right, based on standardized testing?  Our kids will be able to cope with a different future than you faced?

HM & JD:  You wish to move to the heart of that matter?  Admirable. 

First, testing is not just what your contemporary tongue calls “testing.”  Your press has made the term synonymous by analogy with a generic thing.  Your standardized testing is:  One type of testing; assumes there is one correct answer; emphasizes memorization of alleged facts or small packets of knowledge; uses something called “multiple choice” putting more emphasis on tactics in answering than mastery; provides little diagnostic value; based on neural research we see blossoming will quickly be forgotten; won’t solve complex problems; and looking down at your naïve society, has a small cabal of profiteering testing companies deciding for a nation what constitutes knowledge.  To us, that seems preposterous, as there are many types of assessment and all must be considered.  Add, we see what you call knowledge doubling every few months because your science discovered what you term “digital technology," inferring that you will never keep up using that simplistic version of learning. 

Lastly, no, the reform approach is failing.  Indeed, your present society’s unbalanced and discriminatory social, economic and cultural properties now rival the tableaus we mercifully departed.  Those properties have more to do with your test results than your classrooms.

Parent:  Why don’t I know all of this, and what can I do about it?

HM & JD:  This is complicated.  Your “reform” has been going on for 35 years; but in that time your nation’s press has pretty much ignored gritty details.  Your – our – public schools have also ignored modernization, and retreated into comfortable inbred enclaves that do not like change, complicated by failure of your schools of education to see learning as an ongoing process of discovery and properly educate both your teachers and schoolmasters.  Instead of creative public school change we see only dogmatism and greater capacities for opacity and cheating.

What can you do?  That is a tough question – you’re learning.  The representation of the reformers is that without your standardized testing, one, you wouldn’t know what your children are achieving, or two, children will “fall between the cracks” whatever that means.  We find this confusing.  Many decades ago parents knew what their children were achieving because teachers developed and gave tests, they heard recitation, they gave out report cards, and parents who cared talked to both their children and their teachers directly and with purpose.  The national outlook is blurred.  I, John, have advocated liberal policies, but this “nanny state” and belief there can be a uniform child are worrisome.  Right now your Congress is on the verge of renewing what you call ESEA/NCLB with continuation of present or even more standardized testing.  A national leader called Obama, and an underling called Duncan, alleged education advocates, appear naïve ideologues.  Your states are equal disappointments.

Parent:  (Unprintable), so what do we do; you are supposed to be the inspiration?

HM & JD:  We disagree on many things, but we both believe that local control of education is still the best path for learning.  Looking down on your mess, we see across the nation thousands of parents now courageously opting their children out of that testing, with no Federal prohibition of that choice.  We must believe in, but better educate those who teach  in and direct public schools.  The reform testing is improving, slightly, adding more reasoning to questions.  But the present formats will never measure a child’s capacity for critical thought and complex problem solving, and social and civic competence, which is after all, what school and learning are supposed to be about.

Parent:  But isn’t it all about getting a job on graduating?

Concluding HM & JD:  No.  That is the rhetoric that recently drifted up from a lightly educated political dilettante named Scott Walker, trying to re-write the eloquent mission of a venerable institution of higher education of our vintage.  

Sorry, but we must go; a chorus of undiscriminating voices from below with labels like CCSSI, ALEC, NGA, RNC, TFA, a "Gates," etc., are starting to assert that perhaps the two of us don’t belong up here, versus, er, the other place…

Sunday, February 8, 2015

Public PreK-12 Reform: A Baker's Dozen Inconvenient Truths

This is the last post on public school reform before some blog hiatus on that topic.  After nearly 100 posts on various issues embedded in that alleged reform’s history, on standardized testing and VAM issues, and on other challenges in and surrounding our schools courtesy of “corporate reform” and NCLB, there seemed little to add.  But the outbreak of political correctness, and myopia permeating current rhetoric on the above called for a reality check.

The current status of test-based reform is fluid.  Senator Lamar Alexander is chairing the Senate’s education committee, taking testimony, and apparently in line to propose changes to ESEA (NCLB).  It is unclear whether the pendulum is coming down on the side of some sanity that would reduce destructive testing and at minimum cause the CCSSI debacle to be reviewed?  What is apparent is that defenders of continuation of present and proposed testing are ramping up rhetoric in response to public protests of testing, offering some marginal to idiotic reasoning why this testing is necessary to save our discriminated or disadvantaged children from “falling through the cracks,” whatever scientifically that is supposed to mean.

As egregious as the above technically disputable wisdom, the principal argument the advocates seem to muster is that the nation’s public school students will not be evaluated in grades three to eight, and again in high school, if present testing is not pursued.  Do over three million teachers, the vast majority more committed to real education than the cabal of testing companies lobbying every state legislature or the US Department of Education, simply vanish from the classroom when the time comes to do some formative or summative testing of their efforts?  Get serious.

Straight talk on public education is increasingly hard to find.  Some is proposed here, in the form of perceived truths however inconvenient to both reformers and anti-reformers:

Truth #1

Both the reformers and the public school establishment are wrong, both culpable for the state of American public PreK-12, and self-righteously turning out the least prepared generation in a half century to deal with a nation’s survival problem-solving.  

How did you get there?  Give us at least one fact that is a legitimate assertion?  Top down:  A U.S. President and Education Secretary who believe they have the high ground, but are ideologically so liberally twisted or delusional that the utopian obsession with tactically elevating all discriminated or disadvantaged children's education is rationalized as a legitimate basis for destroying a century-old public school system; the present U.S. Congress (need there be more elaboration?); and a cabal of testing companies motivated by distorted business theory and greed, given carte blanche to define what constitutes contemporary knowledge.  Bottom up: the marginal to dismal performances of some fraction of 15,000 systems and 90,000 schools, in both international testing, allegedly per the private sector in preparation for employment, and in lack of preparation for postsecondary work based on remedial work; the fumbling of some fraction of 15,000 BOE; the ineptitude to demagoguery of some large fraction of 15,000 superintendents who should not be there; and some fraction of over three million teachers unprepared for their job description.

Not complex or detailed argument, but usually conceded to be general knowledge:  That 35 years of highly involved reform challenges would not have endured if there was not some basis for deficits traceable to the public schools' performances (over and above deficits attributable to the income and cultural discrepancies among the nation's children); and on the flip side, over those same 35 years failure of testing- and VAM-based "reform" to actually produce measurable public school process and behavior changes not negative to genuine learning.


Truth #2

The present motif for reform – hammering both students and teachers after the fact with simplistic learning logic and convoluted tests – is so bizarre in the 21st century it defies societal sanity.  Truth:  America is turning out a generation of its youth with an inventory of disaggregated facts that will be neurally extinguished with disuse, so unbalancing legitimate critical thought and problem solving capacities that the nation will be populated with a constituency neither capable nor creative in discriminating among increasingly complex and risky options in every civil venue.  Truly bizarre:  Calling for every child to be “college ready” literally from kindergarten; juxtaposed against the content of that college readiness based on standardized testing; juxtaposed against the near irrelevance and even dysfunction of that alleged learning to success in the college/university mission parroted?  Does this actually go beyond simple ideology or idealism to outright stupidity?

Truth #3

The values for those fractions cited in Truth #1 are?  Truth, we know virtually nothing definitive about the full condition of America’s public systems, because – with the exception of Dr. John Goodlad’s earlier research spanning 22,000 public school students – our national leaderships have not chosen to invest in that knowledge.  The quick retort from the dissenter, that is not practical for that massive universe.  Response:  What’s needed is not necessarily census, but a valid and reliable model for assessing those systems based on data from a projectable sample of our schools, verified, then made available to our states and systems as a nationally required DIY format for self-assessment and benchmarking. 

Truth #4

America’s colleges and universities should have been in the forefront of any public system reform, because they have ignored primary/secondary education for a century, and because they have the stewardship for the training of our nation’s public school teachers.  Those schools of education have failed, for lack of intellect, and dogmatic pursuit of the wrong learning rubrics.  Our colleges and universities in turn have been too cowardly to address that higher education failure. 

The least known, but most egregious contemporary reform act, involved the creation of common K-12 STEM standards.  That chore, undertaken by legitimate scientists in higher education in concert with the American Association for the Advancement of Science (AAAS), was advanced with a proposed process to make that knowledge accessible for critical thinking and problem solving.  Required, however, the hand-off of that intelligence to the CCSSI crowd; the standards were promptly trashed and subverted to become more disconnected fact chaff for support of standardized testing.

Embedded in this truth, both higher education and our public systems could have demonstrable gains from breaking through the wall of mutual distrust or contempt that separates them.  Higher education could materially reduce its costs of delivery if public K-12 delivered true college-ready students (not the ersatz version insanely advocated by Arne Duncan and the unthinking), making a four- or even three-year degree the norm.

Truth #5

Some fraction of America’s BOE is a disaster.  Present methods of electing those supposed to provide oversight are not uncommonly failures of democracy.  Frequently there is no competition for joining a BOE, there is electoral manipulation by a system’s administration to promote compliant BOE members, there is ineptitude in knowledge of educational theory and practice, there is no requirement for their training on education before being seated, and in too many cases service is sought for all of the wrong reasons.  Below the radar, this may be our nation’s weakest electoral office, and until 50 states upgrade that system for assigning local school oversight, it is a controlling roadblock to any genuine change in our public systems.

Truth #6

An arguable material fraction of our public systems’ superintendents has obsolete managerial education, is poorly selected by incompetent BOE, is poorly vetted, and should be re-educated or booted from public education.  A fraction increasingly earns jail time.

Perhaps west central Ohio is an anomaly, but its schools also feature some of the most incompetent, venal, and arrogant alleged education administrators seen in 15 years of research.  Ohio has virtually no valid system for removal of such administrators from office unless they commit a high order felony; even theft of resources, failure to perform, insubordination, and violation of education open records laws are challenged as a basis for dismissal.  Just plain educational ignorance, managerial ineptitude, despotism and power seeking, and even sociopathic behavior barely tip the scale.  Incredibly, those illustrious attributes can result in promotion to broader superintendent responsibility, a sick case in this area.  One egregious example of system venality is a case where three of its BOE members cannot be conceptually viewed as having been elected, the result of system manipulation of nominations, and three candidates for three BOE seats.  They effectively elected themselves if they voted, and voted for themselves – school democracy in action?

In another large public system in a sister state, its superintendent (already on the record as educationally naive and a leadership failure) is currently seed funding an attempt to enact state law that would restrict public system transparency and reporting.  Say again?  Its BOE is complicit in the quest, and the community’s taxpayers, parents, and its press appear as dumb as rocks in response to this effort.

Truth #7

Ultimately the truth is that there are good US public schools, but no way currently in place of comprehensively identifying and classifying those successes; equivalently, the difficulty in singling out the systems that actually should be reformed.  Another truth is that the testing army can succeed in beating on our public systems for the next decade, and they will not measurably reform or genuinely improve learning in those systems.  Because any real change in the complexity and culture that is a public school system will have to be executed from within the organization, and have to actively engage all of its critical human resources.  That has been borne out by decades of sophisticated business practice, but ignorantly or deliberately ignored by the reform horde.

Truth #8

Another truth:  Present school grade bands were an invention of the early Carnegie attempts to manipulate public education; present school organization is a century or more old; both are arguably obsolete in our present society and world.  Virtually no effort has been made by any educational authority to innovate these infrastructures that are confining and misdirecting real learning.

Truth #9

The most righteous in our present uncharacterized mass of public schools, differentiated by 50 states with varying levels of educational oversight credibility, are likely its teachers.  An accompanying reality, that is because they are mostly in the profession by self-selection, and rooted in empathy that makes them valid in the classroom.  Simultaneously, that focus on the children they must support, with what is regularly now formulaic to despotic school administration, results in their retreat to their own space, rendering them incapable of leading any real reform charge.

Truth #10

Reality is that little of the substance of the genuine challenges, debates, and information that surround present school reform manages to appear in our general press, allegedly guided by journalistic integrity to see some truth.  For whatever reasons, ignorance of the detailed questions, desire to report only good news, fear of offending local systems' educators or parents, or that school learning deficits are just not as newsworthy as a good killing or scandal, our press seem incapable of informing the public of what's driving testing, VAM, and other assaults on their systems.  The most blatant lie regularly allowed past the "Pinocchio Test" is, that without present standardized testing, parents would have no idea whether their children are succeeding in their schools.  Our schools, our teachers, do not test any aspect of the learning process they conduct?  Even the most ill-informed parent couldn't swallow that.  In turn, the so-called education pages of your average newspaper report primarily the feel good propaganda put out by local schools.  That there is literally a war with public schools underway is lost or unwanted intelligence to most press.  Some editors go so far as total denial, or censorship, to deflect that knowledge from their readers.


Truth #11

Really inconvenient reality:  Too many of America’s parents and taxpayers, victims of Truth #10, are products of the same school concepts being aggressively attacked for 35 years as inadequate, and perhaps because of that education, are blind to or incapable of critically thinking about their local schools, or too timorous to object to local education failures or system malfeasance.  A perversion of the mantra “local control:”  As the costs of local public education are increasingly diverted by our states to local funding, that shift with electorate disinterest or evaluative deficits in assessing local system performance, further complicates any positive change.

Truth #12

Call this summative assessment of the truth about America’s public system attacks:  The 35 years of targeting public education did not originate out of thin air; the performances overall of our public systems in the last several decades of last century were the trigger.  Schools of education, and public systems taking their cues from that platform, adopted a series of silly liberal motifs, ignored innovation, and evolving from managerial weakness and lack of proper teacher education, led to systems dropping the learning ball.  The build-up of private sector resentment finally led to the proactive reform events that started long before NCLB, factually in 1980 spearheaded by The Business Roundtable and the National Governors Association (NGA).  This quietly stayed under our general population’s awareness until "A Nation at Risk" (ANAR) issued, formulated to panic the nation.  That Commission perverted its findings to support NCLB and a market give-away by the Bush Administration to our testing companies already deeply rooted in control of school texts.

Our public schools with any intelligence responded as expected to the testing onslaught; they did whatever it took in the short run to execute a testing work-around.   First teach to the tests as quickly as possible, then in a few quality cases also create legitimate learning.  In the not so quality cases, teach to the tests, if that came up short cheat on the testing, and if that was inadequate manipulate who was tested to control scores.  In this decade a runaway test load has in many cases invalidated even better schools’ attempts to weave in real learning because teachers are intimidated or the time simply doesn’t exist.

Truth #13

Lastly, it is almost unfathomable how an army of reformers and established educators, who must have some intellect, have managed to ignore virtually every precept of the science of explanation and increasingly sophisticated understanding of human behavior and neural processes forging learning, and wagered all on fraudulent and ignorant process for forcing change.  Is this ideology overtaking every vestige of critical thought?  Is it naïve belief in single cause systems?  Are these value systems that are truly warped to self-centric beliefs that override even common sense?  Is it all of these?  Perhaps at the most macro level our public education fabric is fragmenting into factions with only myopic self-interest, or into some subtle level of national insanity?  That is really serious inconvenient truth; because there seems no pat prescription for disrupting the present reform trajectory generating public system fragmentation and entropy.

Saturday, January 31, 2015

US Higher Education: Quo Vadis?

An opinion piece in Friday's Dayton Daily News, by the president of Ohio's Antioch College, jump-started today's post.  The promised offering of alternative futures for America's colleges and universities has been delayed, and a puzzle.  The puzzle, however, was not for reasons one might suspect.

Antioch's president addressed the core need for higher education to achieve not just pro forma housecleaning, but material basic reductions in the cost of a college education, citing the now prominently displayed findings that over 50 percent of America's public PreK-12 students now live in poverty.  That finding doesn't magically improve when those constituents try to educationally move into the next level of education, where they encounter a level of magnitude greater costs.

Reflecting on Antioch president Mark Roosevelt's common sense statement of need, its contrast with a reality became writer’s block.  That reality:  Few if any of the contemporary depictions of why the cost of higher education is what it is capture the full scope and depth of the issues, and there is little candor in describing prospects that those costs can even be nudged.

Prevailing Wisdom

The most common assertion – true – is that the last decades’ college tuition escalation reflects long term reductions in state support of those institutions. Next in line is the plea that human resource costs have soared, both required salaries to hold quality faculty, and the costs of health insurance and pension reserves.  Kept low key in reporting, the cost of debt incurred to support bursts of campus construction, some justified, but much designed to dress those campuses to compete for students.  Next, legitimate, some fragment of those costs is attributable to meeting regulatory requirements.  The net costs of sports to our institutions is rarely transparent – a mixed bag, in some cases profitable, in other cases football and basketball revenues offset the costs of other sports. Lastly, add the soaring costs of research assigned to our universities to support the nation's technology needs, some moved from the private sector and imposed on our research universities.

Collegiate administration, an increasingly savvy lot in this century, squirm or bob and weave, but rarely find the industry or courage to try to re-write their strategic game plans.  The dirty little secret that has blocked higher education reform for decades is, with rare exceptions, they have no need to change.  A virtually isolated exception in this last decade was the University of Virginia, where an initial brouhaha slowly dissipated moving the dial back to virtually where the revolt started.

Reality is that our traditional colleges and universities have few natural predators as a check and balance, pragmatically receive perfunctory oversight, and increasingly corral a captive audience of sports-intoxicated supporters, and alumni who where successful and with nostalgia for their campus salad days feed those institutions endowment dollars.  That wake, funding diverted to student recreational infrastructure, doubling of bureaucratic resources, naive over-compensation of privileged faculty in the guise of staying competitive, and pious refusal to entertain the use of various learning innovations (MOOC) pleading they will reduce education quality.

The Riddle That Is US Higher Education

The list goes on for any willing to dismiss the hype, and look deeply into what the academic legions are and have been doing for the last 25-30 years.  Complexity, however, is that the academy is not an organized entity aligned with private sector organizational design.  It is fragmented, with mixed levels of faculty versus administrative governance, with operating rules and values installed over most of last century.  Inside the whole, every discipline can also be an organizational subunit reflecting a different set of management and performance criteria.  The presence of faculty tenure virtually ensures that there will be little slack to remove unproductive human resources at the academic level; lack of administrative courage in turn virtually assures that once bureaucracy is planted it will mightily resist uprooting, even in the face of reduced need or obsolescence.

Vivid in memory of being a collegiate administrator, was the realization after a few months in the saddle, that the faculty for whom I was responsible did not typically see themselves as employees of the university; rather with uncharacteristic boldness they perceived themselves as independent contractors to the institution.  Tenure meant you could not terminate a poor teacher, or faculty member who would rather wander the campus smelling the flowers than do research, or write, or even regularly meet their classes.  There was great diversity of organizational behavior depending on the colleges/schools/departments, and at some complex level based on the nature of the disciplines represented.

Because of the writer’s prior academic venue, better known cases in point are US schools of business.  Last century, circa 1960, spurred by widespread private sector critique and scathing criticism by two major foundations, those schools were forced to retool their curricula.  What had been a practical but simply descriptive view of American business was prodded to find disciplinary roots for business as a legitimate social science.  In the early 1960s that widespread curricular change occurred, creating a new B-school model, one driven by psychology, sociology, research methods, mathematical modeling, economic theory, and computer technology and computational business solutions.

That regimen legitimized our B-schools academically, but had an opposite effect than the private sector anticipated.  Some sound but exotic conceptual research blossomed, but the manner in which basic disciplines were incorporated damped interest and application of emerging social science to real, street level and especially bottom line oriented teaching and problem solving.  By the late 1970s that approach had again disenchanted the business community, but it had created a bipolar business education revolution. 

Business teaching incorporated some of the science bases of explanation of market and business organization phenomena; simultaneously B-schools’ imports of faculty from more liberally oriented disciplines to reach the earlier reform goals had instilled in those schools – at least for a time – the roots of societal values and ethics to accompany harder edged business practice.  That included 20th century consumerism, and the notion that business had social responsibilities distinct from simply unleashing market-based forces.

By the late 1970s because of corporate voices, and even in B-school internal debate, there was forced evaluation of whether the “social science” missions of that education had diffused and weakened the need for business teaching to be paired with usable applications of theory.  At this point, reminiscent of the reform of the 1960s, a constructive result might have been another update of curricula to marry contemporary theory and business practice.  That need was lost to emerging B-school leadership, pumped up by ramping faculty salaries, emergence of successful prior graduates bringing in endowment dollars, and the misconception that teaching management imbued one with the capacity to practice the best of it.  That preferred result did occur in a few specific business disciplines, notably in organizational behavior and selected research applications, but the overall myopic result was narrowing of focus of educational preparation for business.

For the subsequent 35 years our B-schools adapted to business’ needs by sharply refocusing learning on working to maximize bottom lines, by aggressive development of MBA work and executive education.  Prior sensitivity to business ethics and societal responsibility gave way to current conservative beliefs and myths about the supremacy of “the market” in resolving management decisions – that also created the present public PreK-12 reform debacle.  Bizarre to those of us who practiced through the reform period of the early 1960s, our B-schools began to evolve similarly to our public school system, into learning “factories” premised on standardization, highly programmed curricula, and the ritual MBA.

One might argue that in the last quarter of last century our B-schools created too many myopic marketers; in this century, too many myopic finance graduates, many who helped bring us the prior financial meltdown.  No mistake, tactically the MBA phenomenon brought our B-schools major success, mega endowment dollars, and escalated faculty salaries frequently without justification.  Simultaneously, the last 25-30 years of that academic progress has been a business research wasteland, and has contributed virtually nothing to our advanced understanding of business organization or market behavior.  Only the fairly recent emergence of behavioral economics has added any intellectual accomplishment to business academia.  Even digital applications, once embryonic in our B-schools, were quickly eclipsed by businesses willing to innovate and assume risk.   Paradoxically, assuming risk, and willingness to make mistakes were never tolerated in the academic places supposed to be teaching those arts to the private sector.

Borrowing the term, the ‘bottom line’ is that academic business is long overdue for self-assessment and curricular and learning methods reform.  With the present leadership of most of our B-schools, and without an external force majeure, that is not likely.  To some extent, with the possible exception of our hard and biological sciences, similar critique can apply to most parts of the academy.  Most egregious, among the already questionable, is an obsolete conceptual model for teaching the teacher, the worst of the collegiate breed, our schools of education.  Even that disgrace is not sufficient motivation to prompt collegiate leaderships to enact reform; perhaps because of the belief that would set in motion a view that more of higher education should also be subject to real change?

This is an all too brief survey of why much of higher education needs reform in the worst way; also too brief in part because paradoxically, there is precious little research on higher education that would allow comprehensive diagnosis.

Gridlock

This part of the argument could form the basis for a book or two.  To compress an answer into a few words, the assertion is that there is little threat to higher education, therefore little incentive to generate major internal debate or change, and enduring for the rest of this decade and perhaps the next. 

The Obama/Duncan rating scheme, to shame(?) or with a financial wrist-slap force the institutions into strategy change is so lame it merits no further mention. 

Our states long ago lost effective control of state higher education institutions when majority funding was transferred to tuition and corporatization. An example cited in the last post, the Indiana University system currently received only 24 percent of recent annual revenues from the state.  Collegiate sports in turn have become the armored columns protecting the academic franchise. 

Business practice as an institution is not a current threat, but promises to further compromise academic values by transferring initial training for future hires back to our colleges and universities. 

Too many collegiate boards of trustees or regents are either politically inspired or lack the intellect to exercise that oversight.  Peering into the hazy future, what threats or events could force higher education leaderships to move – to date there appear none.

What Would It Take?

The first answer to that question is, an epiphany by an army of collegiate leaderships that is bright, keeps its heads down, hides behind alumni-bureaucracy-sports, appeals to a swath of America’s middle class parents, and has a formidable if undeserved reputation for being the backbone of American future invention, industry, and prosperity.  In sum, not likely.

The second answer then is almost irrelevant, but still food for thought:  What would that change look like if one could wave a magic wand and scare the bejeebers out of a few thousand collegiate presidents, and cause a sudden internal assessment of their institutions’ missions and methods?  A rough try at an answer:

  • One, it would take the assembly of the full financial statements of a projectable sample of our institutions to understand the financial components that are susceptible to change, and longitudinally, their demand and organizational elasticity.
  • Two, it would require rebuilding the conceptual model of higher education into a major departure from a millennium of history; recognizing that the very nature of knowledge and access thereto has undergone a fundamental change, displacing the core concept of “university,”
  • Three, it would require acceptance of the reality that our institutions are packed with tenured and tenure track faculty who are being over compensated, are not infrequently either subpar classroom teachers or minimally committed to the classroom, and unless they perform a needed research function aren’t really needed to execute the higher education learning mission.  It may even raise the question of whether it is time to scrap out the concept of collegiate tenure.  A majority of higher education classes – for better or worse – is now being taught by part-time and non-tenure track faculty.  Egregiously, on many distance and community campuses, alleged collegiate–level work is being taught by unprepared teachers who could not pass those courses in legitimate university work.
  • Four, it would require coming to grips with the reality that much of now heavily hyped higher education’s lack of on-time graduation performance is attributable to the failure of public PreK-12, and especially its high schools to fundamentally equip their students to operate successfully in higher education.  That should open the door to a new model of education years 9-16, breaking down the disconnect between public education and higher education.  One form that might take is redefinition of current grades 11 and 12, and collegiate 13 and 14, eliminating the grade bands, regularly allowing higher education courses to be more fluidly applied and double counted toward high school completion as well as degree progress.  One simple (but major) factor that could reduce the recipient’s cost of higher education without major challenge of the establishment, is to materially shorten the time a student takes to complete a degree. 
  • Five, arguably every collegiate academic discipline would be tasked to assess and revise as necessary both curricula and how that knowledge is imparted/induced, with some form of oversight of the resultant work by a national academic board for each discipline.
  • Six, a major part of the cost model for higher education is the extended residential environment.  One concept is a staged learning procession that goes one better than tying loan repayment to subsequent employment, but makes some combination of on-campus learning and earlier professional employment the mainstream model.  That in turn would require new processes to allow MOOC to supplement campus work, and/or see that knowledge sourcing tied to a new level of inter-institutional cooperation to cross-recognize academic work.

What’s It All About?

Even the above short list, if one has been sensitive to the arc required to change any major institutional system, immediately becomes discouraging.  If the perpetually more optimistic would dispute that, consider the product of present public PreK-12 alleged reform – 35 years, acts of Congress, high double-digit billions of dollars, an army of idealistic zealots, an underclass army of professionals intimidated to change some undesignated behavior, states slavishly applying and wallowing in test data they usually do not understand, teachers and children degraded by the insensitive application of ESEA without addition of common sense, a procession of Bill Gates’ intrusions and flubs, now emerging protest movements still unable to create sane reassessment of PreK-12 education reform tactics – and a national system barely nudged.

Is the better learning factory the answer, or is the basic theoretical structure being forced onto our public systems simply wrong, based on false assumptions about learning, driven by ideology rather than the learning sought and wisdom, and now peppered with corporate and market self-interest, and political goals that drive out positive organizational learning of a century?

For all of the above reasons, there has to be great macro skepticism that change will occur in higher education venues over less than decades, and without the appearance of the metaphorical “black swan” that disrupts national beliefs and infrastructure.  But in the trenches, America is still graced with great and committed teachers.  For anyone who has spent serious time in the classroom, the event that makes it all worthwhile can be a single episode. 

Years ago the writer, in close to the last class taught before exchanging the classroom for corporate leadership, was offering an advanced MBA course in marketing research.  This was an unusually talented, but also pretty prideful group of about to become MBAs.  The course by design employed both traditional methods exposure and a constructivist approach.  The latter; three real-time marketing research issues underway at Piper Aircraft Corporation.  The class, knowing all that an MBA could possibly want, was pretty dismissive of the projects’ challenges.  They vocally branded the work a no-brainer.  Meanwhile Piper granted funding to buy just about any professional survey resources required to carry out the projects and cover any expenses, and shared proprietary data about past, present and prospective customers for framing the research.

Shortened story, the class had to be dragged kicking and screaming into the awareness that they really didn’t yet comprehend project management, or the market behaviors that had to be assessed.  Prodding but permissive, the class was allowed to seek its own timing levels, with the understanding that performance not pro forma procedure was the mission and test.  As the class approached the end of the term, and graduation for most, their past experiences predicted that if they didn’t finish the work, the worst case was an “Incomplete,” in virtually every case allowing their graduation.

As “fish or cut bait” time neared, it was made clear to the group there would be no Incompletes, rather an F because the course grade was premised on their actual performance; an F would have blocked graduation for all.  We’ll skip the rest of that session which became a bit emotional.  To give all an opportunity to succeed, a classroom was exceptionally sequestered for 24/7 use.  Funding for renting hotel rooms and related expenses was extended for any who were losing their resident housing.  School services were arranged 24/7 as support, and the writer was on call 24/7 for consultation.

The mission:  Piper sent a cabin class twin to pick up the class, and the results of the (hopefully) completed research for all three projects were to be presented in Lock Haven, PA to a full complement of Piper corporate vice-presidents and department/product managers.  The teams completed their reports at roughly 3-4 AM the morning of the flight, just in time to suit up.  All three teams had to practice their presentations for the first time in flight.

The performances:  A bit shaky out of the hopper, but all three teams pulled it together and did an excellent job of reporting their findings, generating from the Piper group praise for the work, a highly respectable congruity of the teams’ findings with Piper’s own professional research results, and praise for the IU MBA.  The flight home was smooth and initially quiet.  The writer was co-pilot in the right seat, the flight compartment separated from the rest of the cabin, but any conversation was audible.  About halfway through the flight home, the buzz started; all three teams congratulating each other, individuals doing the same, all to a person vocalizing how they had made all three projects work, impressed a corporate enclave, and aced their de facto test.  That is why you teach.

Perhaps, as another education writer with major K-12 credentials recently put it, that principle is where present public PreK-12 reform circuses should have started, where any higher education reform should focus, and where its resources and positive reinforcement should be directed -- not to punitive factory quality control logic that was obsolete when the scourge was launched?

Thursday, January 22, 2015

Rating US Colleges and Universities: An Inconvenient Reality

The US Department of Education/Duncan proposal (Postsecondary Institutions Rating System or PIRS) to grade America's colleges and universities -- at the moment into three still vague performance categories -- has not yet issued in any detail.  Representations have been that three factors are involved:  Affordability, access, and results.  Implicit has been that the three factors will need to be measured using data already Federally available, byproducts of various Federal programs, including ones not directly involved in the various Federal education "Title" authorities.

If one had just landed on earth from a distant planet, with the technological prowess that implies, the notion that over 4,000 diverse higher education institutions could be successfully characterized and rated by those three factors might actually seem to make sense.  What could be simpler:  Do a nation’s applicable citizens have equal access to those institutions; can they afford the price of attendance; and what has been the value added by their participation?

After a few trips around the societal track, that visitor from another place becomes linguistically proficient and starts to understand organizational behavior and our societal hangups, concluding the proposed scheme for characterizing an educational institution, by analogy, has the credibility of studying earth's life and its behavior by simply designating it bacteria, archaea, or eukaryota.  (We as multicellular organisms are constructed of eukaryotes, microbes, et al., but that true depiction falls a bit short of characterizing the sentient human.) 

In fact, the scheme proposed by the US Department of Education is a total whack job, calling into question Secretary Duncan’s intellectual competence, or surfacing the question of what values and ideological excursion precipitated the proposal?

Both the rating scheme, and in fairness this writer’s challenge, fit the trope  “says easy, does hard.”  Let the reader be the judge, based on the reality of the behaviors, factors, and rating process being proposed for PIRS.

What are the issues?

The US Department of Education/Duncan depiction of the need for this scheme remains vague; what are the reasons the proposal has been floated now, and how do they hold up under scrutiny:

  • Are the proposed ratings – even if valid and reliable – needed?
  • What is the valid unit of analysis, i.e., the total institution, intra-institutional colleges and schools (there may be great variance inside an institution.)
  • On any factor requiring differentiation to constitute a rating basis, is there greater intra-organizational variation than variation among institutions?
  • Will the ratings differentiate institutions judged deficient in providing equitable access?
  • Will the ratings differentiate institutions based on cost of delivery of a degree; will those costs be comparable based on the quality of the degree delivered?
  • Subsumed in the above, how are the times for delivery of a degree accurately determined?
  • How is it determined that ratings of institutions are based on valid assessments of comparable institutions?
  • How will the punitive measure proposed change institutional behaviors?
  • How does a limited number of ad hoc measures of existing variables translate into a rational scheme to measure performance of any institution that is, de facto, a system and complex layered organization?
  • Are the variables proposed up to the task of alleged measurement: Genuine accessibility–true net cost–education value added, and valid comparisons?

In the wake of undercutting of genuine learning experiences by dogmatic Federal pursuit of standardized testing as the backbone of US public school reform, it seems fair to propose that future initiatives be judged by one of the same standards as medical practice – first, do no harm.

Ratings needed?

There are currently in excess of 20 US web sites devoted to search that can mate a collegiate prospect and a college or university, and multiple ratings already published, e.g., US News, Forbes, Princeton, et al.  Add the online sites of virtually every credible college and university.

The categories of information may not be comparable among these sources, and they have variable credibility; however, the proposed assignment of multi-thousand institutions into three crudely defined hoppers, even if those assignments were valid, appears destined to add nothing to a prospect’s effective discrimination in choosing a collegiate destination.

The unit of analysis?

A fancy phrase for a core issue; what measure of homogeneity, or level of disaggregation makes institutions being assessed comparable?

This is an example previously used, but it makes the point:

“Indiana University (IU) has two main campuses, Bloomington and Indianapolis, different academic environments.  It has six regional campuses. The Bloomington campus has 14 separate schools plus a College of Arts and Science.  All 15 major units have multiple departments, multiple faculties, heterogeneous curricula (and some institutions differential tuition) — that factually determine the quality of a degree — with 180 majors, in 157 departments, representing 330 degree programs.  The other campuses have variable presence of the same venues, plus where a campus is a joint IU-Purdue campus, there may be additional departments representing engineering, nursing, et al.”

What is the appropriate unit for measurement:  The composite institution; each campus location; the college(s) embedded in each campus; the various schools; even subject matter departments that may be as large in student enrollment as some small colleges?  Those differences in programs and enrollees may produce very different results for the variables proposed as the basis for ratings.

Foreknowledge of the universe?

Is there any a priori basis for the Department/Duncan proposals based on even sample research of how ratings factors show dispersion across institutions, or within institutions and across the above potential units of analysis?  Thus far the Department has offered no evidence of prior or ongoing research that would foot any rational proposal of this magnitude and potential for negative effects.

The second factor impacting validity is comparability.  Are any two institutions of higher education comparable given their capacity for independence of action and complexity of offerings?  What research on multidimensional properties has been executed to provide categories of institutions that can arguably be comparable?  The factors allegedly being rated are intrinsically linked to many of those properties, therefore have a potential of being misinterpreted as performance gradients rather than just concomitant effects of those properties.

A college/university is a complex organization.

In the rush to rate higher education institutions a fatal error is failure to recognize that every college and university, even the most austere, is a level of magnitude more complex as an organization than, for example, a public school that has narrower roots, fewer human resources, and relatively a fairly simple organizational structure; even with those similarities our public schools are not automatically comparable in assessing learning performance or even test-based metrics.

Breathtaking is the naïveté to believe any organization, and ones as complex as a college or university, could be assessed for quality based on a handful of incomplete or flawed variables (if that is the true motivation, venality if it is not).

The scope of measurement of organizational performance – especially for an entity as layered and complex as a college or university – is impossibly beyond the scope of this blog.  Many assessment models exist, and the real factors, variables, functions, actors, and internal behaviors that foot an organization’s true performance are massive.  Just one example of such a guide to determinants of performance is linked here.   The Department/Duncan model is roughly the equivalent of trying to build a real operating system with Legos.

Assessing student access to higher education?

As complex as every other factor footing the proposed rating scheme, this one is presently categorically blocked by both a lack of longitudinal research on how admittance is sought and played out in real time, and confidentiality law installed by Congress.  To answer this question would require comprehensive access to college applicant records leading to acceptance or rejection, not permitted by law except at the moment available to the applicant. 

The latter access was just exploited by a cluster of Stanford University undergraduates, who demanded and received their full files.  The results underscore the complexity and nuances of the admissions process; such full disclosure would be needed to assign faults for failures to admit, and to attribute that failure to some form of discrimination other than student performance criteria.

Time to acquire a diploma as a performance factor?

On its face this factor appears one, that coupled with the cost of the educational experience, might be defensible.

In 2011 a group within the US Department of Education was tasked with assessing the factors that might be measured for rating colleges/universities, initially targeting two-year institutions.  Of the multiple factors noted above, only one was thoroughly vetted – the time required to acquire a degree/diploma.

At the moment the only data the Department has to quantify that factor is the measurement of the number of years taken to acquire a degree or diploma, by a first-time, full-time degree seeking student.  As focus shifted to four-year as well as two-year programs, it is from that narrow data concept that the various alerts have come, stating that some material percent of BS/BA level students fails to get a degree within the nominal four years, and now six years.

The Department’s own report, citing the errors in that measure, because it did not track transfers and possible degree completion or subsequent degree pursuit and acquisition after the initial drop out, has seemingly been ignored in the PIRS ratings quest.  In short, that six-year figure for a four-year degree, popularized by our press, is likely a misrepresentation of reality with little or no research undertaken to rectify that to pursue the ratings.

Still another idea floated, use of Federal job placement data of new graduates as a surrogate for quality of education delivered.  Your average eighth grader could slam that rendering of uncritical thought; at the most basic level, starting salaries of new graduates are tightly linked to job and professional service type, and our institutions are diverse in occupational preparation supplied, therefore salaries are confounded with job type.  As the occupational types number in the hundreds, type would have to be held constant to impute a salary quality indicator.  The universe of college and universities categorically can’t support the data logically needed.

Punish to change?

First question is, to change what; the time to degree, the net cost, the quality of learning generated?  The first item is unresolved, the second subject to measurement of a total cost to the student as yet undefined, and the third will allegedly not be attempted.  One hammer proposed is tying availability of Pell Grants to a college's or university's rating.  Other public critiques of PIRS suggest, that because of the crude reasoning and categories footing the scheme, redirecting Pell Grants may actually worsen support for collegiate candidates most needing support. 

Next, will the crude ratings being proposed by the Department/Duncan affect the behaviors and performance of the institutions targeted?  Because of the complexity of decision making in present higher education, with the layers of stake holders, it is highly questionable even if the ratings induce greater deliberation.  Using the prior IU example for a moment, student financial aid measures roughly seven percent of composite cash flow associated with annual operations, and that does not include the influence of endowment funds flowing to the institution.  Presently, the departure or hire of a handful of sports coaches in some quarters might have greater impact than everything the US Department of Education can use to put a brand on an institution.

The list goes on, to where?

Pre-dating NCLB, and blossoming in the period immediately prior to the Obama Administration’s installation, there was a small explosion of studies and conferences addressing the core issues surrounding change in America’s colleges and universities.  Some of the most comprehensive work, now simply being repeated in most discourse on higher education change, was originated by the Association of American Colleges and Universities, and by a small number of states, the latter focusing on measurement of the quality of community college outputs.  This work was seemingly lost in what subsequently became, it is asserted, an unthinking and unreasonable commitment by the Department/Duncan to ideologically driven postsecondary reform tactics.

This generic topic is only scratched by the above observations.  There is cause to argue that America’s colleges and universities should be assessed for mission, and for operating performances that miss or contradict the mission. Staying with the former academic stomping grounds for an example, and with a prior small window into IU’s 2014 strategic planning for its Bloomington campus, the resultant plan was narrow in perspective, institutionally self-centric, virtually void of any recognition of the national and strategic issues that vex present higher education.  Procedurally the planning process was less than inclusive, literally taking properly credentialed faculty representation out of the loop, substituting a set-piece of submissive faculty for broader campus faculty input.  Change is arguably needed in present US higher education organizational leadership as well as in the mechanisms of pursuit of student learning.

But overall, the present US Department of Education/Duncan initiative is arguably the flimsiest and most disingenuous proposal thus far for the purpose of producing positive change in our collegiate institutions.

There is lastly also obvious room to argue that none of the narrow and simplistic reform designs currently being floated for higher education, irrespective of the origin, should be permitted to advance without some meaningful research that first codifies key characteristics and performance indicators for all 4,000 plus institutions, or minimally a projectable sample of those institutions.  Sequentially, that likely is not possible without creativity currently evading higher education, and a new level of inter-institutional conversation and cooperation among university leaderships, along with comparable states’ cooperation via perhaps the National Governors Association (NGA).  The assumption is that the present US Congress is unlikely to grant such power for discovery to the present White House.

Conclusions?

Viewed against the common sense of most of Tuesday’s SOTU address by Mr. Obama, this proposal simply doesn’t satisfy a “sniff test.”  The complexity of the mission, juxtaposed against the ignorance and ad hoc tactics proposed to rate higher education, has to be viewed as failed logic and programming.  Compared to pragmatically failing testing-only based alleged reform being impressed on public schools, this proposal is not the product of competence that should guide national education advocacy. 

American public higher education that was formerly dominated by state funding and occasionally adequate oversight has executed a 180 over the last several decades.  For example, using IU again for convenience, that university system’s funding from the State of Indiana is now less than 24 percent of total annual revenue.   There is an inevitable loss of practical public control of oversight of institutions that must retool to support themselves.

Our collegiate managements reflect intelligent and highly educated human resources, but are as vulnerable as any private sector firm to managerial failure; perhaps to a greater extent in many institutions where leadership has come through the academic ranks and lacks the managerial expertise demanded in the private sector.  That has become increasingly evident in higher education leadership’s emulation of corporate leadership that formerly dismissed strategic thinking.  In short, our collegiate leaderships can learn something from our private sectors and from resources who have pioneered change in management thought; the question is whether leaderships will register that in time?

America’s colleges and universities are also vulnerable to obsolescence in spite of the intellectual capital they inventory.  Change is needed, as suggested in a prior post, to:  Prioritize the real missions; get on the same page in providing information for potential students; make the process of accepting students as transparent as possible within the context of existing confidentiality laws; address the phenomenon of substituting part-time faculty for tenured and tenure-track teachers, or verify that the former’s vetting equals traditional scrutiny; combine cost effectiveness initiatives with learning output assessment to increase productivity; get back to four years (or two years) means “four years;” consider the possibility that “lean” techniques applied to industry do have a role in education; and move beyond present institutionalization of curricula to aggressive updating of knowledge being offered.

Lastly, it is impossible to avoid the reality (provocative to the guilty) that a whole lot of America’s higher education shortfalls do not spring from higher education, at least tactically, but because US public schools, and especially the secondary grades are simply not performing.  Over a dozen years NCLB, in spite of the hype, has produced from a quarter to a third of America’s children that have been “left behind” in spite of the hype, and will struggle to get beyond that fate.

There is really no mystery why America is still in a form of educational crisis – you only have to pull cognitive function out of where it has been slumbering. Look critically at too many of our local schools still dug in to last century’s rituals and knowledge obsolescence, refusing change, exhibiting administrative venality, and BOE that are unprepared or misdirected. That is amplified by inadequate teacher training by our schools of education, offset only by the better fraction of US teachers who have internalized stronger academic values and taken the initiative to advance their own learning and classroom skills.

Perhaps there is discovery afoot precipitated by a shift in emphasis to higher education:  That a century, of disassociating US public PreK-12 systems and practices from the post-secondary education function, has to come to an end, or will at least begin to register educational and legislative awareness?