After a long summer of travel, conferences, and some vacation, I am finally back in the saddle in Auburn. In late June I had a chance to talk about the Personal Impact Factor (PIF) at the International Material Handling Research Colloquium in Gardanne, France. After the talk, I was approached by an academic administrator about how one might bring the concepts behind the PIF to bear at a department, college, or university level.
It has since occurred to me that there are two related questions:
- How can an academic unit (department, college, or university) better evaluate its faculty? and
- How should an academic unit assess its performance as an organization?
When I first posted my thoughts on the PIF, I had the opportunity to discuss the first question at length with a dean of engineering who was concerned that the PIF as I described it was “gameable.” I quickly conceded the point, but replied that the purpose of the PIF was to allow a professor to better to evaluate his or her own performance. Someone who games a personal evaluation metric is in serious trouble!
My discussion with the administrator in France led me to propose that he require of his faculty a 1-2 page document each year describing how each had “made an impact” in teaching, research, and service. The document would be in addition to ordinary documentation listing papers, seminars, and so on. Before requiring such a document, of course, he would have to articulate to his faculty just what he considers “impact.” This he could do in a formal document, presentation, or workshop (better: do all three). The goal is to allow faculty to talk about forms of contribution not traditionally rewarded, such as implementations in practice, press coverage, influence on government policy, and so on. Administrators could put whatever weight they like on this portion of the faculty report, and they might give it more weight over time.
I think there are three benefits to this approach: (1) traditional measures are still part of the evaluation scheme, so we haven’t upset the apple cart (yet), (2) faculty will be encouraged to and rewarded for making an impact on academia, industry, and society, not just for producing means, and (3) department heads and deans would have a means with which to give feedback on what faculty think it means to make an impact. Those who’ve made a significant impact could be held up as examples; those who don’t quite get it could be given guidance.
The second question—how should academic units evaluate themselves?—is beyond my experience, but I think the basic model of the PIF applies: Articulate Grand Ideals and Ends, specify Means to accomplish the Ends, then develop Metrics that indicate whether or not Ends are being accomplished. The model seems flexible enough to let academic units focus on what they’re good at, without feeling they have to conform to someone else’s view of “a quality program.” Most of all, they can punt the dreaded U.S. News & World Report rankings if they want to!