David Berliner presents a compelling argument for the engagement of educational psychologists in policy research, analysis, argumentation, and commentary. He sees this activity as a professional obligation of senior educational psychologists. Providing examples of educational psychology constructs and research that could contribute to policy debates, he suggests that there are warranted truths to be told to policy makers that could affect educational policies and eventually impact the education of our students. Of course, he also states that warranted truths ("is data" rather than "ought data") may not necessarily have much of an impact on policy. Thus it is worth looking closely at what Berliner is asking of senior educational psychologists.

Throughout the paper, Berliner calls for the following activities: policy analysis, speaking the truth to power, using or creating data that will help to clarify an issue, developing arguments to clarify an issue, entering policy debates and challenging proposed (or existing) policies, publicly professing, and conducting studies with policy implications. As I understand his view on this topic, he sees most educational psychologists as engaging in what they consider to be value-free, objective research, whereas he is calling for what he labels "partisan research". While he feels that partisan research can be reasonably objective, it is not value free. We can think of this characterization of educational psychological research as lying along a continuum from perceived-as-value-free and objective research to value-laden partisan research. He is asking that more effort be devoted towards partisan research. In the next sections of this response, I will address 1) values in research, 2) a friendly critique of partisan research, and 3) a suggestion for the development of guidelines for good partisan research.

1. Values in Research. Berliner suggests that educational psychologists have been trained to "check their values" at the door as they head into research. He also states that this may be impossible. However, he is not really sure about this since he later states: "Our data should not be tainted by our values". But of course they are. We should be aware of the effect of values, ideology, beliefs and goals on the decisions concerning which data to collect, the wording of the survey, the organization of the data, the sample, and so on. Data in social science research are not value-free. One might think, for example, that data such as demographics are pure, untainted, and value free. Yet, all we have to do is to consider the various definitions and categories of ethnicity to realize the degree to which values enter into the data. In addition to the data, values permeate all other aspects of the research: analysis, choice of constructs used to lump data together in order to conduct relationship studies, deciding which findings to present to the reader/listener, and the ways in which the data and analyses are represented, interpreted and explained.

Berliner also suggests that it is incumbent upon partisan researchers to be aware of how their values affect their research. Given the sense that all research is loaded with values, this should really apply to everyone. I would add that not only should partisan researchers be aware of how values affect their research, they should also be striving for quality, rigor, fairness, and honesty in the conduct of that research and its representation. How do these qualities play out in partisan research?

2. Partisan Research. I strongly agree with Berliner that research can and should be used to improve the quality of policy debates, and that educational psychologists are in an excellent position to enter into these dialogues. However, issues mentioned above concerning the quality and rigor of partisan research should probably be addressed prior to pushing it very much further. As more senior researchers enter policy dialogues, it will strongly affect a new generation of educational researchers who are now working on their Ph.D. degrees. And thus, it is well worth the effort to identify some issues and guidelines for such research.

One issue concerns the search for and conduct of research to support a belief. Partisan research appears to differ from the typical policy analysis genre in which all sides of the argument are presented with some carefully crafted concluding remarks that may lead the reader in one or another direction. There is a strong hint in Berliner’s paper that one begins with the values, concerns, perhaps answers, and musters the appropriate research--existing or new--to support the researcher’s arguments. How, then, does one remain objective and semi-skeptical when one already has the answer? What if the newly collected data do not back the researcher’s values? Are they thrown away? Reanalyzed using a different category system? A number of years ago, I asked a similar question of a European sociologist who was a critical theorist: "If you already have the answer, why are you conducting research?" Her answer was, "for purposes of convincing others. Some people will only listen if you have data to back your arguments." Within a relatively traditional set of values concerning the nature of research, I questioned whether such an approach actually produced what could be labeled research.

Of course, a senior educational psychologist has been involved with research for many years, and the belief of interest may have developed on the basis of a strong storehouse of knowledge and understanding of the field. The belief may be warranted, and the partisan effort becomes one of pulling material together to support it. And while a piece of partisan research may not convince someone to change his/her mind, a policy argument may be completely ignored if it makes empirical claims that are not supported with research. However, I am sure that we all worry about this process becoming dishonest, becoming embedded in a strongly ideological bias that affects the validity of the research and its representation. Although the purpose of this form of research is to convince others, all should not be thought of as fair in partisan research as it is in love and war.

An additional set of issues involving the conduct of partisan research may be illustrated in one of the examples used by Berliner in his paper. This was the study of teacher certification vs. noncertification in Arizona, with one focus being on the noncertified Teacher For America teachers.

Teach for America (TFA) prepares college graduates to teach for two years in areas of great need for classroom teachers, using a short summer program (five weeks), followed by internship and mentoring programs that support the teachers once they are in the classroom. TFA students become teachers through emergency certification in large urban and rural areas that request their presence because of a great demand for new teachers. It receives a considerable amount of press, and is seen by the founders as a successful teacher education program alternative to the typical "traditional" program offered in colleges and universities.

Two recent partisan research studies around Teach For America have come to seemingly opposite conclusions. One was conducted by The Center for Research on Educational Outcomes (CREDO), Hoover Institute, Stanford University. This study used data from the Houston, Texas, school district, and compared the scores of students in classrooms with new (first two years) Teach For America teachers with the scores of students in all other classrooms in which the teachers were new, as well as with all other teachers in the district. In conducting the study, the researchers controlled for such aspects as degree of school poverty, experience level of teachers, percentage of minority students, etc. The study suggests that TFA teachers perform modestly better in some subjects, although not always significantly, than other teachers in the Houston system. As a group, they don’t perform any worse.

The second study was described by Berliner in his paper. He and Laczko-Kerr (2002) examined the differences in achievement scores of the students of a sample of novice (first two years) teachers who were fully certified in the State of Arizona with a matched sample of those who were not certified. The latter category included teachers who had gone through the Teach For America program. They found that the students of the certified teachers did better than those who had been in the classrooms of non-certified teachers. They then looked at the small sample of TFA teachers, and found them to be no different than the other non-certified teachers in terms of their students’ test scores; their students still obtained lower test scores than those of the certified teachers.

Both of these pieces of research are, to a greater or lesser degree, partisan. CREDO was funded by the Thomas B. Fordham Foundation, a conservative think-tank that questions teacher certification and promotes alternative teacher preparation programs such as TFA. The CREDO authors do attempt to maintain some independence from the Foundation, and disagreed, in writing, with a statement made by the Foundation in the Forward to the report (http://credo.stanford.edu/AACTEresponse.htm ). Nonetheless, they also state that they agree with much of the Foundation work.

It is not clear whether Laczko-Kerr (2002) would call the study described by Berliner as partisan research. After all, it is a dissertation, and Berliner suggests that partisan research should be conducted only by senior researchers. However, it is described by Berliner as partisan research in his article. For many, TFA is seen as a threat to traditional teacher education programs. For example, Berliner suggests that TFA is being used by conservative "opinion-tanks" to attack university teacher education programs, and therefore the foundations courses such as educational psychology that are required in most of these programs. It is also used as an argument to eliminate state-level certification procedures. The Berliner/Laczko-Kerr study was designed to show that certified teachers do better by their students than uncertified teachers, including the TFA teachers.

As with any large-scale, policy study such as the two described here, there are questions related to sample selection and size, analysis designs, and interpretation and representation of results. Both studies used school district or state data--often incomplete and undoubtedly quite crude--and had deal with a teacher quality indicator that is questionable and ambiguous. For example, while a teacher may have attended a school of education and received certification in another state, s/he might be considered uncertified in Arizona, at least until s/he takes a course in Arizona history, phonics across the curriculum, or some other required course. At the same time, in the CREDO study, TFA teachers were being compared with new teachers in Houston, quite a few of which did not hold a B.A. or B.S. degree. Both sets of researchers were interested in how TFA teachers performed in relationship to other groups of teachers. But while they appeared to be interested in the same questions, they really weren’t, and the teacher groups chosen to compare with TFA indicated this difference. The desired outcome of the Berliner/Laczko-Kerr study was that certified teachers–largely educated in schools of education–outperformed uncertified teachers, particularly those prepared by TFA. CREDO, on the other hand, was interested in finding that Teach For America teachers do the same or better than all other teachers, whether certified or not.

As I perused both reports and Berliner’s description of his study, I had many unanswered questions. However, I would like to avoid the typical quibbling with sample, research design, etc., and focus, instead, on the educational psychologist’s obligations to his/her field and the general public in conducting and presenting partisan research. While the call for partisan research is important in bringing research into the policy debates, it probably should be accompanied by a set of guidelines concerning the conduct of such research, and its representation to the readers.

An example of a guideline relates to the "setting-the-argument" section of the representation of a piece of partisan research. This connects to a need to be careful and fair about the representation of research in the literature, even if it does not match the researcher’s values. It is quite amazing to see the considerable space taken up in a "literature" section of a partisan policy article with a critique of a study that does not match the researcher’s values, while briefly and uncritically referring to a deeply flawed but supportive study as evidence for the researcher’s position. Further, in a seeming scramble to find references to back an empirical claim, one finds a mixture of speculative, analytic, research review, and perhaps one empirical study in the list of references.

In addition to guidelines related to awareness of values, rigor, maintenance of a modicum of skepticism, honesty and fairness, I believe that we also have to pick our battles carefully and with an eye on significance. For example, I question whether educational psychologists, alone, can make deep inroads into the improvement of education through policy. In fact, I doubt that any one discipline can. Educational psychologists certainly can contribute, but there may always be an issue concerning the focus of the research. For example, the TFA partisan research suggests that TFA teachers are a little better or a little worse than certified teachers in terms of their students’ scores on questionable tests. Is this something that we should be worrying about? Yes, we need to remind conservative thinkers from time to time that preservice teachers learn a great deal in university-based teacher education programs that helps them become quality teachers. However, our major problem relates to teacher retention, particularly in large urban school districts. The fact that such school districts hire more non-certified teachers who teach subjects in which they have little expertise has been documented (Darling-Hammond, et al., 1999). The Houston Public Schools data in the CREDO study were frightening. In 1999 and 2000, Houston hired 414 and 354 new grade 4-8 teachers, respectively, and of those, 70% (1999), and 65% (2000) held bachelor’s degrees. As many as 35% of the first year teachers left their positions prior to the beginning of the second year.

The problem, as Ingersoll (2001) so compellingly pointed out, is not one of supply. It is a retention problem. Suppose, for example, that we increased by 25% the number of students in our university-based teacher education programs such that we could eliminate all non-university based alternative programs, would the problem of teacher quality be resolved? Probably not. Quality teachers (or others for that matter) do not want to stay for long in urban schools that treat them and their students with little respect and remarkably little support. Thus, large urban school districts would be compelled to continue to scramble to place a teacher in every classroom.

The issue, then, is how we make the work of teaching more interesting and supported, particularly in large urban school districts, such that quality teachers remain in these schools and in the profession. This is a problem for which no one disciplinary group has the answer or the wherewithal within their discipline to develop the answer. From time to time, we need those engaged in educational practice--including teacher educators from universities and Teach for America, as well as educational psychologists, anthropologists, economists, sociologists, and organizational theorists--to set aside partisan aspirations and work together on what is the most intractable problem that we face: education in large urban areas.

This analysis suggests that partisan research may be helpful in presenting an alternative argument in a policy debate. However, because of its potential worth as well as Berliner’s suggestion that more people engage in the activity, it is also worth considering guidelines for its practice. For if research is used in partisan argumentation without rigor, honesty, fairness, or significance, its worth will be questioned. Given that this could happen in a large and public policy arena, I believe that such questioning could lead to the disparagement of the concept of educational research in general, thus threatening the enterprise.

In concluding this response, I feel a need to state that considering this topic has been both interesting and helpful. Needless to say, my concerns about partisan research emanate, in part, from my own involvement in it from time to time. As I considered the various criteria mentioned here--knowledge of values, rigor, skepticism, fairness, honesty--I became aware of the many times I have not met these criteria because of a push for an argument that supported my values. Thus, I feel that this special issue of provides an excellent foundation for beginning to develop further the criteria for quality partisan research such that it becomes recognized as an important element of the professional life of the education professorate.

References

Darling-Hammond, L., Berry, B. & Thoreson, A. (2001). Does teacher certification matter? Evaluating the evidence. Educational Evaluation and Policy Analysis, 23(1), 57-77.

Ingersoll, Richard (2001). Teacher turnover and teacher shortages: An organizational analysis. American Educational Research Journal, 38(3), 499-534.

Laczko-Kerr, Ildiko (2002). The effects of teacher certification on student achievement: An analysis of Stanford Nine achievement for students with emergency and standard certified teachers. Paper presented at the annual meeting of the AERA, New Orleans.