Background and Methodology Undergirding the Development of the Star Urban Principals Interview

Martin Haberman
Distinguished Professor
School of Education
University of Wisconsin Milwaukee

Background
The Urban Teacher Selection Interview was developed by identifying functions which discriminate between the behaviors of star teachers and quitter/failure teachers. The number of star teachers (app. 8%) and the number of quitter/failure teachers (app. 40%) in the 120 major urban school districts was both sufficiently large and accessible. Stars were willing to be interviewed because they had been identified on seven criteria as being extraordinary teachers. Quitter/failure teachers were equally willing to be interviewed because they believed that the problems they encountered with students were the fault of the students, their parents or the schools systems. Indeed, they were generally eager to conduct exit interviews and explain in detail the reasons why “good” teachers such as themselves could not continue to teach in these systems.

After identifying the functions which discriminate between the two groups questions were developed which required subjects to respond to whether they would perform these functions and the degree to which they would perform them. In effect, subjects’ answers are assessed in terms of the proximity or distance of their responses from the responses made by stars or quitter/failures. The best answers are closest to how star teachers respond. The poorest answers are closest to how quitter/failure teachers respond.

The interview predicts who will remain in teaching in highly bureaucratic school systems, relate positively to diverse students in poverty and be effective versus those who will quit or fail. This interview is now used in over 220 major urban school districts any of which can provide additional evidence of its predictive validity.

The development of the Star Urban Principal Selection Interview could not be developed following the same procedures as the star teacher interview. Quitter/failure principals are reluctant to be interviewed for a variety of reasons. Some have been told they will get a good reference if they leave the district quietly. Others are promised other jobs in the district if they keep quiet about the causes of their stepping down. Some have been given lucrative buyouts to leave quietly. Some have relatives who still work in the district and fear retribution, while others save face by claiming they were simply taking early retirement or leaving for health reasons. While classroom teachers fail more quietly in their classrooms, principals fail publicly. Because it is more difficult for them to pretend they have been successful they are reluctant to be interviewed. This limits the number of those who would cooperate in the development of an instrument to those who would not be representative of the total group of quitter/failure principals.

The method used to develop the Star Urban Principal Selection therefore drew upon both the written knowledge base and the best practice cited by star principals to explain their effectiveness. In this sense, the star principal methodology was more complete than the process used to develop the star teacher interview.

The Knowledge and Skill Base
The Danforth and the Geraldine Rockefeller Dodge foundations supported a year-long project to summarize the best practice of school principals.The Eli Lilly Foundation also contributed several forms of support and expertise. With the cooperation of all the major professional associations that represent school administrators, the National Policy Board for Educational Administration compiled Principals for Changing Schools: Knowledge and Skill Base. Fairfax,VA: 1992.

The process the group followed was to define the scope of the principal’s role as covering 21 domains of knowledge and skill. Teams of 6-12 academic and practitioner scholars then summarized the knowledge base in each of these domains including critical behaviors to be performed, ways of measuring effectiveness and the supporting bibliography. The domains of the principal’s knowledge and skill base covered were: leadership, information collection, problem analysis, judgment, organizational oversight, implementation, delegation, instruction, curriculum design, guidance, staff development, measurement and evaluation, resources allocation, motivation, interpersonal sensitivity, oral and non-verbal expression, written expression, contextual domains, legal and regulatory applications, policy and political influences, and public relations.

Selecting the Outstanding Principals
The practitioners selected to serve as resources and jurors were drawn from the Milwaukee Public Schools, the Chicago Public Schools and the Houston Independent School District. The method used to identify star principals was as follows. Principals with a minimum of five years of service were evaluated by their superiors on the basis of their demonstrated performance of the following functions: creating a common vision, team building, securing teacher buy in, staff development, instructional leadership, parental and community relations, performing within the contractual obligations of the various unions representing the school staff, implementing innovative programs, budgeting, and garnering resources for their schools. Principals were also asked to evaluate themselves against the same criteria used by their superiors. An anonymous survey was made of assistant principals asking them to assess these same principals on the same criteria. Traditional criteria such as student learning, absenteeism, suspensions, expulsions, and graduation rates were also assessed. All principals identified were leading schools making annual improvement and growth for the preceding five years on these criteria. In addition, some less traditional means were used to select star principals. Surveys of principals identifying colleagues they regarded as stars were used to rank those most frequently named. These surveys were also used to rank principals most frequently cited as being the most desired principal coaches. Finally, supplemental data were collected on those considered as possible stars: these data included, the number of the transfer requests made by teachers in their buildings who had been rated as effective, teacher absenteeism and the degree to which their schools were used after school, weekends and summers. The result of this extensive process yielded 27 star principals in Milwaukee, 36 in Chicago and 18 in Houston.

Developing the Practitioners’ Knowledge Base
This phase of the work, including the identification of star principals described above, covered a period of three years from 1993 through 1996. Daylong workshops of the 81 star principals were held in each of the three cities. The principals engaged in a process of team deliberations which yielded their perceptions of 14 functions they performed that they believed accounted for their success. They also produced specific behaviors which were behavioral manifestations of each of these functions.

Working with a team of doctoral students in the 1993-94 school year, the 21 domains of the research knowledge base in Principles for Changing Schools were merged with the 14 functions identified by outstanding principals as the explanation for their success. The 21 Milwaukee principals serving as a jury agreed that the merged document preserved the intent and the functions of both the literature and the practitioners’ knowledge bases.

The 1994-95 year was spent developing questions that could assess the functions identified. Through an iterative process trial questions were tested for their ability to orally communicate the functions correctly to a variety of constituencies; principals, acting principals and principals in training. Respondents were not asked to answer the questions but were asked to explain what they believed the question was asking them. These trials also produced the prompts interviewers might use to ensure that respondents understood and answered the questions being asked. At the end of these year-long trials the questions and prompts were worded so that 100 per cent of English speaking respondents agreed upon what was being asked in the questions.

During 1996 in-person day-long meetings were again held with the three star principal groups in Milwaukee, Chicago and Houston. Of the 81 original participants 63 were available and participated. Acting independently, the three panels agreed that the functions they had developed earlier were reflected in the questions developed to assess them. The juries also agreed that the behavioral manifestations of each of the functions used on the draft of the questionnaire were accurate reflections of the functions they were intended to assess.

Essentially, the interview seeks to assess what respondents regard as good practice and why they would engage in those practices. In the course of this development it became apparent that the questions were not mutually exclusive. Some of the content used in answering some of the questions are likely to be repeated in answering others. As a result, during this next phase of development three of the questions were identified as redundant and dropped. At this point the interview was developed in its present form: i.e. questions related to eleven functions and requiring approximately one hour to administer.

Field Trials
For the next five years the Star Principals Selection Interview was used in ways that could both assist school districts and assure that the interview would be tested as a valid predictor of principals’ effectiveness. In these trials the interview was used by districts as one criteria of principal selection but not as the determining one. This practice provided an opportunity for the scores of respondents who were hired to be compared with their subsequent performance as practicing principals. This process was used to assess the interview’s power to predict respondents’ behavior from their total scores, as well as from their scores on each of the eleven specific functions assessed by the interview. In other words, if Principal Z was evaluated more highly than Principal B then Z’s total score should be higher. Even more importantly, if Principal B performed more poorly than Z with parents and community, or on creating a common vision among his faculty, his score on those specific questions would have to be lower. These validation trials demonstrated the power of the interview to predict subsequent behavior. At that point school districts began using the interview as a critical or even deciding factor in principal selection.

Training Interviewers
There is a one day workshop offered for training those who would conduct the interview. It is recommended that at least two and not more than three conduct the interview and no more than two others be present as silent observers. In most cases, the two interviewers are the only ones present. It is also common for interviews to be audio recorded or video taped. Confidentiality and Bias

It is necessary for all those trained to use the interview sign a confidentiality agreement. There is no “train the trainer” model: those trained cannot train others. Once trained, an individual and school district may use the interview as frequently as desired with no further cost or retraining. As a non-profit foundation, the Haberman Educational Foundation uses funds district pay for training to train districts that are bankrupt or under fiscal takeovers which prevent them from paying for training.

There is an on-going problem of individuals who have been trained breaking confidentiality to help colleagues secure positions in districts which use the interview. This practice is engaged in by both school administrators and college faculty. There is also the problem of bias. Frequently, central office administrators know and favor particular individuals and do not score according to the rubrics as trained. The Haberman Educational Foundation provides free consultation to prevent and cut down on breaches of confidentiality.

A Final Test
In 2001 I interviewed all 167 individuals functioning as principals in the Milwaukee Public Schools. This is the only documented instance of an entire principal corps in a major urban school district being interviewed. As a result of this effort the district began a system of principal coaching which is still operating. The particular weaknesses of the principals identified to receive coaching matched their low total scores and their low specific scores on those same functions in the initial interview.

Dissemination
The list of school districts using the Star Urban Principals Selection Interview are available from the Haberman Educational Foundation. Each district can also provide its own data on the predictive value of the instrument.

4018 Martinshire Drive
Houston, Texas, 77025-3918
Fax/Phone (713) 667-6185