Culture, standards, and program qualities

Abstract

The simplest, unfortunately least effective, “defense” against Deputy Governor Danforth is, of course, to say “well, I just don't see things like that”. In Salem there was no mediating device between the wielders of judgment and those whose lives were to be judged, for the judges were putative agents of a higher, immutable logic. Hence, there was no appeal against the arbitrariness of the theocratic judgment — “As God have not empowered me like Joshua to stop this sun from rising, so I cannot withhold from them the perfection of their punishment” (Miller, 1976, p. 104), bemoaned the hapless Danforth. The standard is the standard. This, in fact — the absence of mediating structures — partially defines a theocratic state, a state that asserts a unitary system of meaning. But the liberal democratic state is partially defined by precisely its opposite — the presence of such mediating structures and the possibility of some distance between those who judge and those who are judged. An element of a liberal state is the live practice of holding logic of government publicly accountable; the increasing absence of this condition in Western democratic states is corrosive of liberalism.Evaluators inhabit that mediation territory — which is why it is so important for us to maintain some distance from each side of the judgment-equation (and, hence, Democratic Evaluation — in its best expression — refuses to make recommendations). Contemporary Western political cultures are fearful of such arbitrary power and vest some vestigial democratic authority in intermediary agencies. Evaluation — to greater and lesser degrees politically neutral — has thrived on that. Perhaps, as our democracies grow more liberalized and less liberal that ethical space in which we conduct our business becomes more straitened — perhaps, too, we are, as a community, increasingly co-opted into political ambitions. Nonetheless, there is at least an opportunity at the margins to play a role in enhancing the self-determination of people and increasing the accountability of policies and programs to that aim. Central to the task of such an intermediary role is resistance to the passion of precision. There are many ways to define standards and there are many approaches to understanding quality, and it is the responsibility of the impartial evaluator to resist the arbitrary dismissal of alternatives. We should treat with professional scepticism the increasingly common claims that such concepts as “national standards”, “best practices”, “quality control criteria”, “benchmarks” and “excellence in performance” are meaningful in the context of professional and social action.Glass (1978) warned evaluators and assessors against ascribing a level of precision in judgment to a subject matter that has less of it in itself — here evaluation contaminates its subject. Programs are rarely as exact in their aspirations, processes, impacts or meanings as our characterisations of them and as our measurements of their success and failure. Glass urged evaluation to avoid absolute statements and to stay, at best, with comparative statements — rough indications of movement, what makes one state or position distinct from another, distinguishing ascendance from decline, etc. Given the 20-odd staff at the Rafael Hernandez School, the 200-or-so pupils, the range of languages and social backgrounds, the plurality of meanings perceived in a curriculum statement — given all of these where is the boundary between the exact and the arbitrary? And if we are to settle for the arbitrary, why commission someone as expensive and potentially explosive as an evaluator? Ask a cleric.

Citation

Kushner, S. (2001), "Culture, standards, and program qualities", Benson, A.P., Michelle Hinn, D. and Lloyd, C. (Ed.) Vision of Quality: How Evaluators Define, Understand and Represent Program Quality (Advances in Program Evaluation, Vol. 7), Emerald Group Publishing Limited, Leeds, pp. 121-134. https://doi.org/10.1016/S1474-7863(01)80069-7