Glenn L. Immegart, Based at the University of Rochester, New York, USA
In noting the human being′s propensity to value and evaluate, explores five levels of assessment from the casual to the very serious and rigorous. A distinction is made between management or normative appraisals and leadership or discretionary evaluations for organizational improvement and making a difference. The importance of rigorous assessment of the school by the school head or principal is underscored in the discussion of the why, what, and how of school evaluation for management and leadership by the head or principal.
Assessment; Education; Evaluation; Performance appraisal; Schools; Teachers.
International Journal of Educational Management
MCB UP Ltd
A unique aspect or characteristic of members of the human species is their ability and propensity to value – to value virtually everything and to value differently from others of the species. In fact, we humans not only assess or evaluate everything, but also do so regardless of whether or not what we evaluate is important to us, and, we assess or evaluate things within our personal structures of values, preferences and goals. We assess the weather, the news, local and international developments, the ecology, human relations in the workplace, and relations between nations. We evaluate restaurants, the theatre, our clothing, the performance of musicians and politicians, the clergy and speakers, as well as the meetings or conferences we attend.
Reasoning for Evaluating
Why do we do this, or what are our reasons for assessing and evaluating? In the first place, being valuing beings and caring more or less about almost everything, we just cannot seem to avoid it. We react in an evaluative way to what we see, hear, and experience, and we are constantly subjected to the assessments of others. If we have a fine meal at an excellent restaurant, we often remark to others about the goodness or pleasantness of the experience. In passing a theatre or concert hall, we hear others raving about the play or concert they just experienced. We complain or express our feelings about the temperature in an overly warm or cold room, and we and others are quite free about assessments of the weather or season of the year. We are simply beings who value and evaluate, and we send and receive information of an assessment or evaluative nature regardless of whether it is asked for or not and whether it is at a conscious or unconscious level.
Also, we are truly curious, and as a result, we are information seeking beings. Not only do we see, hear, and experience as well as get information from others about what they see, hear, and experience, but we and others also seek or ask for information about things that are important to us or about which we are just curious. Wheat did you think of that movie? What do you think of the collapse of the Soviet Union? Is it warm or cold outside? Is the train ride from Paris to Geneva beautiful? Was the meal at the hotel cafeteria tasty? All of these are very common and typical queries for evaluative information. As curious and interested beings, we thus actively solicit information and assessments or evaluations from others, and we seek information in order to make our own evaluations and assessments.
Beyond this, with respect to things or matters that affect us or with which, or about which, we are concerned, we pursue both targeted and precise information in order to get general appraisals of situations or states of affairs. For example, if one is about to depart on an automobile trip over roads that have been under repair, one may wish to know whether the work is finished or if there could be delays. If one plans to purchase a coat or suit, one may want to known whether there is a sale at a clothing store. Should one be working with students who need help in solving mathematical problems, one might want to know whether the students understand or not, or if they can or cannot solve certain kinds of problems, and school heads, directors or principals might be concerned about whether the teachers, parents, students, and others on the school or advisory committee feel the school goals for the year have been reached or whether the group itself is functioning effectively. Such appraisals are often general and are used to determine adequacy versus the lack of adequacy, whether something is working or not, or whether operations or activities are or are not effective. These evaluations are typically of an “on-off” or “yes-no” nature, but they clearly seek evaluative information or feedback on matters that affect one or about which one is concerned, and in the sense of gross or general measurement.
When one has responsibility for an activity, group, or organization, one often needs even more precise information or more rigorous assessment or feedback. Such evaluative information is desired in order to ensure proper operations or functioning, or it is sought in order to alleviate deficiencies or correct problems.
In order to oversee, manage, or direct an enterprise, its operations and activities, systematic assessment is necessary to resolve problems, make corrections, and attend to malfunctions and deficiencies. For example, if a school is doing two or three things designed to increase the attendance of students, it is essential to monitor attendance rates. If the desired effect is not being realized, it is important to evaluate or assess each of the two or three interventions to ascertain whether they are operating, are operating effectively, or are in need of adjustment or correction in order to have the desired effect. If one of the school′s yearly goals for social studies or civics classes is to increase student reading of current periodicals and newspapers as well as listening to radio and television news, classroom assessment of the results of encouraging these activities must be undertaken. Should students not be meeting the desired expectations, or should there be wide variation from class to class or teacher to teacher, further evaluation by class and teacher would be called for in order to correct problems or ensure reaching desired expectations. In the sense of this type of evaluative information or feedback, assessment is focused or precisely targeted in terms of standards such as goals, objectives, and behaviours or in terms of a normative appraisal such as realizing improved attitudes, developing specified skills, or resulting in desired student performance. In any event, this is systematic, planned measurement and analysis (or assessment) directed toward needed corrections, problems, or malfunctions in order to ensure proper operations or work in an organization.
At times in one′s work or in fulfilling one′s responsibilities, one is interested in improvement, in making a difference, or in providing leadership. In this respect, perhaps the highest level of assessment is called for, and the quality of the assessment is most critical. Further, in this respect evaluation must be concerned not only with what is, but also with what ought to be. Put another way, comprehensive assessment of goals, operations, activities, and results in addition to the assessment of needs and the future are necessary to develop and improve, to making something better, and to making a difference – or leading. To illustrate, if the school head, director, or principal wishes to improve the effective outcomes of the school, an assessment of current activities and efforts toward such ends along with current results and perceptions in this regard must be made. What is desired (needs, expectations) should also be appraised as well as appropriate means to realize what is desired and how to implement them in the school. Such evaluation is essential prior to effectively implementing new and improved efforts intended to have greater impact on the effective development of students in the school. In improving or in moving to better programmes, both an evaluative information base on what is in place and an assessment of what is needed and how best to achieve desired changes and improvements are essential. Evaluations in this regard are, of course, both more comprehensive and rigorous, and thus clearly go beyond general or simply corrective feedback or evaluative information.
This analysis of why people in general evaluate or assess things has identified five basic reasons:
- We just cannot avoid valuing and evaluating.
- We seek information or appraisals because we are curious.
- We seek appraisals or general evaluative information about what affects us or about things with which we are concerned.
- We seek normative evaluative information or feedback about those things for which we are responsible.
- We seek evaluative information and a range of assessments about those things we hope to improve or with which we hope to make a difference.
These five reasons can be represented as a series of levels of assessment or evaluation ranging from the casual and unplanned to the most comprehensive and rigorous. In this sense, the levels of reasons for assessment or evaluation in general can be depicted as follows:
- We assess in order to improve or make a difference.
- We evaluate that for which we are responsible.
- We appraise what affects us and our concerns.
- People seek information because of curiosity.
- People cannot avoid evaluating.
From the analysis, the lower levels (1 and 2) are casual, general, and subjective. The higher levels (3, 4, and 5) represent evaluative appraisals or feedback of an increasingly more focused, systematic, and rigorous nature, means, and measurement sophistication. From the general but focused appraisal of what works or does not work or what is effective or not at level 3, to the systematic and rigorous evaluation of problems or deficiencies for correction and in order to ensure proper functioning at level 4, to the comprehensive assessment of performance, needs, and the future at level 5, increasing methodological, measurement, and analytical rigour are called for and used.
In a hierarchical sense, the lowest level (1) is not only casual and subjective, but also often accidental, chance, and happenstance evaluation. At level 2, information or appraisal is sought but it is very general, casual, and subjective, while at level 3 the sought after information or appraisal is more focused and represents at least dichotomous measurement at an overall level based on predetermined standards or criteria (works or does not, is effective or not). Evaluation at level 4 is more systematic and rigorous as well as focused on operations and needed corrections and involves comparative appraisals in terms of precise standards or in a normative sense, and at level 5, appraisal is clearly criterion and performance referenced as well as need based and future oriented for broad and detailed analysis related to improvement and development.
In another sense, level 1 represents assessments that are unplanned and unavoidable, levels 2 and 3 are general and solicited appraisals about things that concern, affect, or interest people, and levels 4 and 5 are consciously planned and rigorous evaluations and appraisals of tasks and responsibilities of people directed toward correction, improvement, or making a difference.
These levels are also in a sense cumulative, not discrete. Thus, one may focus on operating at one level or another, but whenever one functions at a given level, activity at lower levels usually can be assumed to be going on simultaneously, or have the potential to do so. Thus, even if one is getting corrective, evaluative information (level 4), one might as well be getting assessments or appraisals at levels 1, 2, and 3. (The implications in this regard are rather obvious.) However, operating at level 2 one cannot hope for evaluative information of the quality or fidelity of levels 4 or 5. To no small degree, the effectiveness and worth of assessment or evaluation is related to the level at which one operates because the level determines the means, method, rigour, and quality of the assessment or evaluation employed. Clearly the use (what one plans to do with an evaluation), value (merit or quality), and worth of any assessment is likewise related to the level of the evaluation in the above general sense.
The notions set forth in this section can be summarized in Table I. Included in the table is the level number, the reason or motivation at each level, what is involved at each level, and the general method employed at each level.
School Heads, Directors, Principals and School Assessment
If the background material on the reasons why people assess and evaluate things so much and so extensively, but for such differing reasons and in such differing ways, is at all accurate, it is clear that principals, cannot avoid involving themselves in school assessment. Important questions are how, and how effectively, do they do this? These questions are, nevertheless, contingent upon an even more important pair of questions: Why? And why should school heads assess schools?
There is really little question that school heads assess schools. Whether they admit it or not, they do! Being human (one hopes) they cannot avoid doing this; but also being human, school heads or principals might be more or less open about why they evaluate something, and even more or less open yet about how they in fact do it. For example, the director might freely point out that he or she assesses the appearance and cleanliness of the school. Why? In order to ensure a clean and pleasant environment for the work or teachers and students. How? By personally touring the school and monitoring the cleanliness of students and teachers as well as the cleanliness of the physical building and the work of the custodians. This is an easy, straightforward task!
Another example could involve that of the school programme or curriculum. The school head might have determined – by more than casual assessment – that there is a real need to improve the foreign language programme of the school, but the curriculum co-ordinator, who has a maths-science bent, clearly is more interested in further developing and refining upper-level maths and science courses. In order not to cause a conflict or stir up a controversy, the principal may be reluctant to push this matter, let alone get too deeply into the why or how of his assessment, even if prodded by teachers, parents, or students interested in a better foreign language programme. Here assessment and the use of it becomes a bit more touchy!
Where are school heads or directors really reluctant to get into the matter of assessment or evaluation? One such area all too often is that of the appraisal or assessment of teachers or teaching. This is, indeed, very tenuous and uncertain turf. It is like trying to evaluate a gourmet chef and telling the chef that a dish needs more sauce or oregano! All too often teachers are just left alone, and they are not evaluated openly or directly, or are even open to evaluation by other teachers or the school director. We do not, then, in this respect even get close to the questions of why or how. Rather, discussion borders on the taboo. But this is a curious situation because lots of people, including school heads, other teachers and students, do assess teachers and their work whether or not they are willing to admit it. To say otherwise is like saying that those eating at a fine restaurant do not evaluate the work or cooking of the chef. Come now! We will return to this matter (the evaluation of teachers) in due course, but the point is that we are more or less open about why and how we assess teachers in schools. This indeed is a much avoided or explicit matter.
Perhaps enough said. We should now turn to why should schools or aspects of schools be assessed. There is little question that we evaluate a number of things related to schools (including, in all honesty, teachers!), but why we should do it is a very important question; and, inevitably, if one is interested in the effectiveness of one′s evaluations, the level at which one operates in assessment is critical.
It is easy with the assessment of things like teachers, which is often difficult, problematic, and even taboo, to render assessments to lower levels and not to discuss them, particularly if the assessments are unfavourable. (Of course, this is not necessarily so with favourable evaluations. It is easy to say, “You are a great teacher”. It is a lot less easy to say, “Your teaching is not so good” or is “in great need of improvement”!) Very critical, when we choose to employ lower level assessments are the matters of the accuracy, value, potential use, and the possible effects of such assessments. In a sense, evaluating important things in a casual, subjective or less than rigorous way is like trying to measure wind speed by how quickly your hat blows off or measuring temperature by holding your hand out of the window or door. A basic point: if it is important or if it makes a difference, it should be evaluated and an appropriate level or rigour of measurement is essential.
Whether a movie, concert, or football match is potentially good or not usually is of little critical import. On the other hand, the quality of school experiences and the effectiveness of teachers – like the resources available to school and governmental priorities for education – are extremely important to youth, society, and national interests. The point is that these kinds of assessments are so important that they should neither be casual, accidental, or subjective, nor avoided or kept secret.
Thus, one of the real reasons for evaluation or “why” in any sense of the word, including in the assessment of schools, is that if it is important, some evaluation is called for and any such appraisal effort should be at an appropriate level. Assessment is needed to see how things are going, to determine needed corrections or adjustments, and to move toward desired improvements and the future. Put another way, feedback or evaluative information is required to ensure performance and to lead to a better state of affairs. From the perspective of systems theory, evaluative information from assessment is crucial to the proper ongoing functioning and sound development of any system.
This line of argument has led to asserting the general importance of assessment in organizations and to pointing out the differences between casual and rigorous evaluations. Whether one follows the leads from systems theory or general organization theory the importance of information, and more particularly of evaluative information that serves as the basis for sound organizational decision making, has been increasingly underscored. Information acquisition and utilization are essential to communication, planning, decision making, and leadership[4, p. 253-4].
By linking the discussion of assessment to decision making and leadership we are moving into the important realm of organizational heads, managers and leaders, as well as to some very important evidence and critical reasons why heads, managers, leaders, and school principals need to be concerned with assessment. Management is essentially a normative activity involving work and operations that are goal directed and necessary in ensuring organizational performance and desired outcomes. Managerial work includes planning and problem solving, working with human and material resources, and directing operations within the context of internal influences, environmental demands, and normative role requirements[4, p. 258]. As a result, managers need evaluative information on operations and problems or malfunctions. To ensure operations and needed corrections or adjustments, managers require more than casual or random assessments and more than general appraisals at levels 1, 2, and 3 as set forth above. Clearly, they require assessment at level 4 in order to fulfil the duties and responsibilities of sound and proper management.
Leadership is, however, yet another different kind of activity. It is a discretionary activity[4, p. 258] involving strategic choice or decision-making directed toward improvement, development, change, and making a difference. Leadership work includes information scanning, assessment, and evaluation; vision and a future orientation; goal setting; directing action; implementing actions; influencing goal attainment; and realizing improvement and change[4, p. 258]. As a result, leaders need evaluative information on the existing organization and operations (as do managers), but in addition they also require assessment of needs, assessment of resource prospects, appraisal of external forces, knowledge of trends and the environment, and consideration or estimation of the future as well as an analysis of a variety of possible courses of action. To lead, the information and evaluation requirements for the discretionary activity of the leader are greater than those for managers. Thus, leaders require assessments beyond level 4, and they must also operate at level 5 as sketched out earlier.
Given certain differences between management and leadership, there has been a persistent series of efforts over the years to separate and make distinctions between these arenas. Typically, management in such views is status quo oriented, and leadership involves change or doing things differently. Recent evidence, however, indicates that such distinctions and separation may not be useful. In fact, it appears that sound management is an essential component of leadership[4, p.275]. In that sense, leadership is dependent upon effective management. (From the intensive case study of educational leaders, it is interesting to note that all subjects reported on the import of an orderly functioning organization as a requisite to the freedom to lead and the ability to do so effectively.) Thus, if organizations are to adapt and change effectively, it seems only reasonable that managers need also to be able to lead. Indeed, this is not an either-or matter: management and leadership are, in fact, intertwined and mutually dependent.
Thus, regardless of how a position of headship is defined, the most important reason for school assessment is to provide the information base necessary for proper management and sound leadership. Managers and leaders need to know where they are going in order to get results. They should also be aware of operations and current outcomes. Further, they must assess needs, look to the future, and provide a vision for goals, activities, and improvement whether they are in private sector organizations or schools. The basis of all of this is information, and most assuredly evaluative information. Management is more than just maintaining operations, and leadership is more than charisma and transactions. Both involve choice or decision making, planning, intelligent action, and work in order to get results, and the basis of this is sound and appropriate assessment.
Other Issues for School Heads, Directors and Principals in School Assessment
The issues of why evaluate and why should school directors assess schools are basic. Having suggested that not only can school directors not avoid assessing schools, but also that there is mounting and compelling evidence of the importance of information, evaluative information, and assessment of what is, as well as of needs and trends for managers and leaders, it is then necessary to deal with more detailed aspects of what, how, who and when in order to ensure sound and appropriate assessment. This discussion can then be concluded with a brief treatment of criteria for evaluations.
What? Once the value or import of assessment has been established, one can turn to the matter of what should be evaluated. Given the complexity and multi-faceted reality of schools, this is no small matter. There are teachers, students, programmes of study, student learnings, materials, classrooms, the school building and grounds, fiscal resources, technology, other staff, parents, school goals, regional educational authorities and policies, school committees, the management team, collective effects of the school, and expectations for schooling to illustrate only the major categories of what could be assessed. And many of these categories can be broken into even more subcategories. For example, the school programme consists of objectives, content, and materials; there are cognitive, affective, and psychomotor skills and objectives; the programme can be subdivided by levels, disciplines or areas, and tracks; and there are courses, units, and classes, all adding further to the complexity and possible extent of school evaluation.
At this point it would be nice if we knew precisely what ought to be assessed in order to get the best picture or evaluation of a school, or even if we were aware of the key indicators of quality or effectiveness of schools. Although there is much in the current literature on characteristics of effective schools, some caution must be advised in limiting assessment to such characteristics. Much on this topic deals with the differences between effective and less effective schools, unfortunately not necessarily with what makes some schools effective or what should be looked at to best assess a school. (One simply cannot always assume causation!) Short of clear or definitive leads to what should be assessed in evaluating schools, two guidelines can, however, be offered.
First, when dealing with complex and multi-faceted things like schools, assessment must go beyond one or two, easily evaluated, safe or non-controversial aspects of schooling, or it cannot just focus on one thing, not matter how important it may be. On the one hand, emphasis on straightforward, easy, and safe dimensions such as examining courses of studies, materials, or student performance in terms of teacher marks simply will not give a very comprehensive or useful assessment of a school. In a similar sense, even an annual goal-results assessment in schools is really not a wide-ranging, powerful assessment of a school; it deals only with a sub-set of goals (yearly ones) and related activity and results, a very small and selective slice of what goes on in schools.
On the other hand, and at the very least, school assessment should deal with:
- School inputs such as the school programme, materials, students, fiscal resources, school facilities, and yes, teachers.
- What is happening in classrooms and learning situations as well as in individual student work.
- School outcomes such as learning, criterion referenced student performance, the effective development of students, student effectiveness in subsequent learning or work, and perceptions of the school′s effectiveness.
It is interesting that, although there is somewhat general agreement on such variables, the evaluation literature in the US has been rather silent about the inclusion of teachers in the consideration of “what”. Recently, however, an important treatment in this regard concluded: “It is impossible to exclude personnel evaluation from a sound evaluation of a programme (or school)”. This is only logical because how could one leave out a major facet of a school such as either students, teachers, courses of study, student performance, or learning in assessing the school? Put more directly, one must look at many components, all major or important components, and their inter-relationships in school assessment.
Second, the purpose or intent behind an assessment or evaluation affects the “what” in this regard. If the intent is to assess a programme or the results of a yearly set of goals, such activities should be appropriately labelled the assessment of goal activity and results, or the evaluation of such and such a programme, not school assessment. This is not school assessment, and it cannot stand in place of broader appraisal. However, if assessment is to ascertain whether things are working or working properly, one could operate at a level 3 type of comprehensive evaluation. But it must be cautioned that such assessment should look at many facets or variables in order to ensure a truly comprehensive overall appraisal. In addition, standards or comparative assessment should be employed to the extent possible at this level of evaluation.
If assessment is for managerial purposes in order to correct deficiencies and ensure proper operations, the evaluation should focus on the above input, process, and outcome variables employing systematic, rigorous appraisal and analysis of the evaluative information obtained. It would conform to a level 4 type of normative evaluation and would result in precise, targeted information, corrections, problem alleviation, and needed adjustments. But if assessment is for purposes of leading or making a difference, assessment would include level 4 evaluation considerations plus level 5 assessment features. Assessment would be comprehensive using measurable performance, criterion referenced data or information, and it would include current operations and outcomes as well as an assessment of needs, the appraisal of trends and the environment, and some forecasting of the future. The thrust in this respect would be not only for management purposes, but also for improvement, development, and school betterment.
What to look at in assessing schools is an issue and an emerging activity. We are clearly moving from the all too prevalent level 1 and 2 appraisals of the past to more rigorous and accurate kinds of evaluations needed by educational heads, managers, and leaders today. What to look at regarding schools is just as complex as schools are themselves, but schools are important and so are sound school evaluations for many purposes, including betterment.
How? There are, in fact, a variety of methods at and across the different levels of assessment for purposes of appraisal or evaluation. As has been implied, real world evaluations and assessments, even if planned and conscious, all too often are casual, subjective, and not very rigorous in any sense of the wors. It is, however, increasingly recognized that formal assessments and the evaluative information gathered and analysed needs to be at higher levels than simply personal or casual judgements. Indeed, it is clear that good assessment and evaluation draws on the same methodologies, procedures, techniques, and expertise as does good research[8, p. 572]. Although the goals and purposes of research and evaluation are different, both rely on common methods and techniques, and sound inquiry methods contribute to the quality of assessments.
With respect to school assessment, an obviously important thing to evaluate is student outcomes or results. The typical method used in this regard is some form of student testing. Testing involves both formal tests and teacher-made tests. In the first respect, achievement testing is a very popular way to assess student outcomes. Such tests are normative in nature and are available at a number of levels, from specific content areas to national and international tests, covering one to many areas of content. These tests have been criticized because they give comparative results and do not always test what is taught in certain localities. This has given rise to domain-referenced analysis of such tests in order to link test results better to local objectives. Even better, if rigorously devised, are teacher-made tests either at school (across teachers) or departmental levels. Such tests can better get at the content or curriculum taught, but they often lack the rigour of more formal or professionally prepared tests. The most promising and useful type of test in this regard is the criterion referenced test that measures performance in terms of instructional objectives. The value of this approach involves the determination of measurable objectives (criteria), instruction directed toward objective achievement, and the criterion related measurement of performance. These tests can be developed at the classroom, school, regional, and national levels. Despite the cautions of not setting performance levels too low and avoiding blatant “teaching to the test”, the criterion referenced test is optimal for getting at results with pupils.
In addition, there are a range of other teacher tests and approaches that can be used to assess student performance. Typical individual classroom tests and student marks are not, however, the best ways to get reliable and accurate evaluative information. Better, in this respect would be teacher observation in terms of expectations, standards, or objectives; written reports with anecdotal evidence; and subsequent analysis and review. Although subjective, such an approach is more systematic and analytical than typical teacher class tests. Better still would be pre-test/post-test situations where teachers diagnose learning needs, intervene, and then test the results of their intervention. This enables expectations and objectives to be set up beforehand and to ascertain what happens with each student. Also, when assessment and evaluation are embedded in an instructional approach, such as with mastery learning, valuable and sound evaluative information on student outcomes can be obtained. Of course, the best classroom level data that can be obtained is by using performance objectives and using criterion referenced measures to evaluate results. Both teachers and students can assess instructional outcomes in this way.
Equally as important, in the assessment of schools at the classroom level, as the performance of students is the performance of teachers. As the direct deliverers of learning, the work of teachers in the classroom is clearly related to student results. As such, teachers and their work warrant evaluation in order to ensure proper teacher performance and student learning. There are a variety of approaches to teacher appraisal, from structured observation, to target setting and subjective, open-ended narrative reports for analysis. It is incumbent on any teacher evaluation approach or system to focus on the work of the teacher and in so doing to seek to correct or remedy deficiencies and, in general, to seek teacher growth and improvement. Although narrative appraisals and target setting are sometimes easier and more acceptable to all (and can be effective), much can fall between the cracks with either of these approaches. Considerable experience in working with administrators in the field over the years suggests that some structured observation, combined with narrative evidence or anecdotal evidence, and supplemented with target setting directed toward growth and improvement is best. Structured observation can focus on what makes a difference in teacher performance from the perspectives of research and experience, as well as what is valued in a school. Narrative or anecdotal evidence can illuminate and extend such observation, and target setting is an invaluable approach in correcting deficiencies and leading to improvement. And, with adults, joint consideration by a teacher (self-appraisal) and by knowledgeable others (department heads or school principals) ensures teacher growth and development.
Going beyond learning outcomes and teaching, other aspects of the classroom can be assessed through observation and the use of protocols or structured observation schedules. There are a number of extant procedures to get at classroom interactions, teacher-pupil verbal and non-verbal exchanges, and behaviour. Further, the classroom can be inspected for cleanliness and safety, and the entire physical plant can be so appraised visually in terms of safety, cleanliness, and the need for repairs or renovations.
The school programme, curricula, and specific courses can be assessed in terms of the extent to which they meet standards and expectations. Standards in this regard typically come from both regional and local levels. Thus, state or ministry officials can assess the degree to which programmes, curricula, and courses meet regional or state expectations and standards, and teachers and advisory committees can assess the degree to which they meet local expectations and standards. In addition to such comparative appraisals, student success at subsequent educational levels or in the world of work (or their problems in these respects) can be analysed and compared with school programmes, curricula, and even specific courses.
Some things in schools – such as the work of managers and supervisors, the deliberations of groups or committees, or teachers′ efforts towards self-improvement – cannot be evaluated as rigorously as some of the other things discussed above. In these cases, comprehensive and sufficiently detailed narrative description including outcomes can be developed and employed in assessment. Of course, narratives as such are basically descriptive and not evaluative. Nonetheless, through the use of audits which involve reviews by others and external analysts, as well as expert analysis and appraisal, narrative descriptions can be very effective in any programme of school review and assessment. In addition, audits can be used with extant data bases such as student marks or test results, school records, or budgetary or fiscal accounting. The advantage of the audit is the possibility for review and appraisal by others, including external sources and experts, which add both independent views and expertise to evaluation, thus expanding the base of assessment.
It is also sometimes possible to use actual research or experimentation in school assessments. University researchers and faculty are involved in inquiry, and sometimes their interests mesh with the needs of schools for appraisal. This cannot always be counted on, sometimes interests do not mesh, and at best this is somewhat up to chance. But when such possibilities exist, they should be explored because of their potential value, rigour, and low cost to the school.
Up to this point the emphasis has been on rigour and quite frankly ensuring objectivity in how to assess aspects of schools. The reason clearly in this regard is that this is directly related to the question of how effective assessments are. It is not, though, to rule out or ignore subjective or qualitative appraisals. Some such appraisals can be invaluable in school assessment. Not only do these kinds of evaluations deal with perceptions (the basis of much human behaviour), but also they can provide leads for more precise and rigorous appraisals. Feelings of parents, students, teachers, and school groups or committees can enlighten more formal and rigorous evaluative information. Such subjective assessment should, of course, be planned, systematic, and employ appropriate procedures. In order to extend the value of such evaluations, comparative, profile, or “on-off” analyses in terms of standards or expectations, or even the context at hand, represent ways to effectively analyse and consider subjective, evaluative information. These approaches can both add to more objective assessments and indicate areas for further or future appraisal.
In response to the issue of how, and more importantly how effectively, to assess schools, it is clear that multiple, proven means (methods and procedures) designed to fit the many variables to be assessed are essential. These inevitably must be determined at the local school level, at a particular point in time, and relative to the purposes at hand. Better approaches are characterized by methodological rigour, analytical sophistication, accuracy, and involve “on-off” measurement or the measurement of criterion referenced performance objectives, or both. Realizing the “why” and “what” of assessment successfully is contingent on how effectively evaluation is done.
Who? Sound school assessment requires broad participation in evaluation activities, and successful educational evaluations have demonstrated that fact over the years[8, pp. 574-5]. Broad involvement is essential if comprehensive and rigorous appraisals and evaluations are to take place. Participation must go well beyond one or a few people and, ideally, should also be viewed as a part of the overall process of the development and growth of a school and its staff.
Both internal (from students, teachers, the management team, and school committees) and external participants (parents, inspectors, ministry officials, and other experts) have a role in good school assessment. Major actors are certainly teachers, the management team, and members of school committees, but expertise from outside the school including ministry officials, university faculty, and other educational experts is necessary for independent analysis, audits, and the general collection of evaluative information.
Illustrative of roles, approaches, and types of evaluative information appropriate for different participants are the following: teachers and experts can be involved in the evaluation of student outcomes through tests and the criterion referenced measurement of student performance. School administrators, school groups or committees, supervisors or inspectors, and parents can participate in the evaluation of programmes and courses. The principal, curriculum specialists, department heads, and teachers themselves can appraise teaching and classroom instruction. State or ministry officials, supervisors or inspectors, and university faculty can be involved in audits of records or existing databases. Students can help appraise the effectiveness of courses and the school programme through self reactions (even subjective ones), self evaluation of personal achievement and meeting school and learning objectives. Then school groups and school administrators can collate, analyse, and move toward planning for corrections and/or improvement. Others would most assuredly be involved in planning and implementation, but one school level role would be to bring the evaluative information and assessment components together for a total appraisal and determination of the possible outcomes of evaluation, a task that would largely fall on the school principal.
Ultimately, who is involved in evaluation is contingent on the purposes of appraisal, what is to be assessed and how it is to be assessed. But, just as there are ideally multiple methods, there should as well be multiple perspectives and multiple views in assessment.
When? Despite the continued discussion and debate over the relative benefits and appropriateness of formative and summative evaluations, much of the heat in this regard is perhaps misplaced[8, p. 579].
The point rather simply is that some things call for continuous appraisal and other things are more conducive to terminal evaluations. For example, one could well argue for continuous evaluation of students through a maths course and for a terminal assessment of students′ knowledge of history. And, in either case, some form of database of evaluative information is necessary.
With respect to continuous or periodical assessment, some things might be evaluated during or at the end of a unit, a course, or a school year. Often pre-and post-testing determines when. However, other things are best assessed yearly, every so many years, or when needed (e.g. when problems arise, deficiencies are noted, there are malfunctions, things are not going well, or whatever). But some things cannot be assessed “early on” or at the beginning of the year, while long delays in evaluation can pose problems (although later follow-up can be invaluable). And there are some things that are best not appraised “in process”.
In addressing when, it is obvious that this hinges on the other decisions about assessment. Nevertheless, some appraisal must be continuous and some terminal. Common sense also says consideration of when should fit the situation, and it is wise not to overload appraisals in particular time frames or for individuals at any given time. Evaluators need sufficient time in order to collect information and to analyse and report evaluative information. In planning comprehensive, multi-faceted assessments, overall timing and the development of a schedule for all components of the assessment are beneficial to all involved and to ensure completion in an orderly fashion prior to overall analysis and use of the assessment for the betterment of the school.
Adequacy in school assessments. A word or two is appropriate at this point regarding what makes for or ensures a good or adequate evaluation or assessment. In this respect, those responsible for, or involved in school assessment should be guided by the three concepts of soundness, selectivity, and systematic appraisal. First of all, any assessment should be sound. In this sense the methodology and techniques employed should be appropriate, rigorous, and ensure accuracy and quality (value or merit and worth or utility). Second, in adhering to selectivity, the evaluation should concentrate on key aspects and indicators that fit the assessment purpose(s). Assessment should also deal with the relationships of aspects and components of the school as well as attend to other problems or deficiencies at whatever level as identified in the course of assessment. Also, appraisals should be multi-perspective (as well as multi-method) to the highest degree possible. Third, assessments should be systematic. That is, they should be planned, orderly, and professionally accomplished.
The above criteria of adequacy are well in tune with existing standards for evaluations in education such as the often used standards of utility, feasibility, propriety, and accuracy[8, p. 582]. In any event, those responsible for and involved in the assessment of schools must make efforts toward ensuring the adequacy of their work.
The Role of the School Director in School Assessment
Although not handled in great detail, the above issues and aspects of evaluation and school assessment attest to the fact that school appraisal is a very large task. However, the importance of evaluation, rigorous appraisal, and a critical posture in order to ensure school functions and operations and to provide a basis for improvement and betterment are underscored by the literature on management and leadership. School heads – however defined or legitimized – are the key in school assessment, and assessment (or evaluative information) is the basis for realizing effective schools and increasing the effectiveness of schools. Further, there is much evidence that real change in education and educational improvement is achieved at the school level.
Heads of schools as managers and leaders have not only a logical and empirical basis for engaging in assessment, but as professionals also have a moral obligation to use sound, accurate, evaluative information in their own work. Because of the import of evaluative information to managerial and leadership activity, assessment for school heads must be a mode of operating and the basis for thinking in terms of school goals and objectives as well as school improvement. It is high time to get rid of taboos, “hang-ups”, looking the other way, ignoring obvious problems and whatever, and it is high time to take appropriate action based on assessments of the full complexity of schools and their components and operations.
Accordingly, the role of the school director in this regard is clearly to:
- promote school assessment and evaluation;
- encourage assessment and participation in assessment;
- organize and implement school assessment activities;
- involve others actively in school assessment;
- ensure the adequacy of all school assessment efforts;
- see that school assessment results are summarized and considered by all appropriate decision makers;
- see that school assessment results in necessary corrections or adjustments and improvement or development.
School assessment can be done consciously or not, systematically or not, soundly or not, sensibly or not, and effectively or not. It is clear that the better we assess schools, the better is our evaluative information for improving schools and education. This clearly is an important but underemphasized responsibility of school heads, directors, or principals. It is, without doubt, crucial to their success as well as the success of teachers and students.
Table I. Levels of Assessment and Evaluation.
Meckling, W.H. (1979), "Management of Educational Organizations and Implications for Research", in Immegart, G., Boyd, W. (Eds),Problem Finding in Educational Administration, D.C. Heath, Lexington, Mass., .
Davis, K. (1958), Human Society, Macmillan, New York, NY, .
Immegart, G., Pilecki, F. (1979), An Introduction to Systems for the Educational Administrator, Addison-Wesley, Reading, MA, pp.55-62.
Fleishman, E.A. (1991), "Taxonomic Efforts in the Description of Leader Behaviour", et al., Vol. 2 No.4, pp.245-88.
Immegart, G. research currently being conducted with selected educational leaders, .
Gorham, R. (1991), "Transformational Leadership of Middle Grade Level Principals Involved in Policy Implementation", Ed.D. Dissertation, University of Rochester, .
Clark, D.L. (1984), "Effective Schools and School Improvement: A Comparative Analysis of Two Lines of Inquiry", Educational Administration Quarterly, Vol. 20 No.3, pp.41-46.
Stufflebeam, D., Webster, W.J. (1988), "Evaluation as an Administrative Function", in N.J. Boyan (Eds),Handbook of Research on Educational Administration, Longman, New York, NY, pp.585.
Goodlad, J.I. (1984), A Place Called School: Prospects for the Future, McGraw-Hill, New York, NY, .