Sunday, February 18, 2007

More on Rankings

I've been thinking more about rankings after reading some comments on Berit Brogaard's blog.

One question is whether there's an alternative to the Leiter Report. Now, some people do love the Leiter Report, but others find it hopelessly biased.

Here's some food for thought. I hadn't heard of this Faculty Scholarly Productivity Index before. It measures productivity by # of books and articles published and also by the # of citations. The methodology looks sound to me. Or at least as sound as these things go.

So what are the highest ranking philosophy departments? Here they are:

1. Michigan St.
3. Princeton
4. UVA
5. Rutgers
6. UC San Diego
7. Penn State
8. Texas
9. SUNY Stony Brook
10. Rice

That list doesn't look anything like the Leiter list. Of course one response is to argue that Princeton philosophers may publish less but what they do publish is higher quality than, say, Michigan State. But that won't entirely hold up. The survey also measures "Percentage of Faculty With Journal Article Cited By Another Work." This would seem to be a measure of quality (orat least it normally taken to be). Michigan State's percentage? 33%. Princeton's percentage? Hold on to your hats: 7%


Blogger hilde said...

Is a graduate student looking for a good graduate teacher? If so, then the measure of a good graduate teacher may not be how much they publish, but how many hours they are available to their students, how many comments they put on their papers, theses, etc. A good teacher is "productive" in a way unlike the ones mentioned in this ranking, and I'd argue they're as important...if not more.

12:13 AM  
Blogger dr. wentzel said...

the best teacher would have the ability to be a good scholar and/or creative philosopher herself. I believe this enables her to better encourage these skills in others. publications demonstrate these talents. Of course, this doesn't mean that people who don't publish much don't have these talents as well. But we just don't have publically available reasons to think that they do. A good publication history provides assurance that this condition is met. Its true that this sort of ranking doesn't demonstrate that the professors at the ranked program have the ability to assist students in developing their own talents. An ideal ranking would somehow measure both.

10:53 AM  
Blogger hilde said...

Sorry, but I just don't see why being a productive scholar proves anything about being a good graduate teacher. Scholars can be idiosyncratic in the way they work, plus they can be self interested, selfish with their time, and totally lacking in the practical skills about how to instruct about the skills of scholarship. Consider how well most academics manage themselves (dept. meetings, faculty assembly, etc.) and you start to get a sense of how intellectual acumen in one area can be totally disconnected from other areas.

My experience in graduate school showed that the more a professor published, the more he/she was both disinterested and/or unable to train graduate students.

12:38 AM  
Blogger Khadimir said...

I would have to agree with you, hilde.

As a graduate student finishing up coursework, I have noted that some of the best "publically verifiable" scholars are the worst teachers, and vice versa.

The skills that lead to scholarly publications are not the same as those of a mentor. In fact, many of the skills, i.e. social and personal, are not even related to the professional skills.

Lastly, selfless teachers often have less time for publications. I have in mind a professor who has 3-4 young children, mentors numerous students inside and outside the department, yet has little time at the end of the day to publish much. Likewise, some professors on 1-1 or 1-0 schedules publish much but mentor very little.

9:49 AM  
Blogger dr. wentzel said...

I did not say that being a productive scholar proves or is a sufficient condition for being a good graduate teacher.

It seems being a good graduate teacher requires at least two sets of skills, one relating to the ability to recognize and produce good scholarship and the other set (time, willingness, good communication skills, good people skills) relating to the ability to effectively communicate these skills so that others might hone them.

Good publications demonstrate that one has the first set of skills (but note that no publications does not demonstrate the lack of them as in no evidence for A is not evidence for -A). The ranking can, therefore, tell us that at a given ranked program, at least some professors meet one of the conditions for being a good graduate educator (and following from my earlier point, it cannot tell us that unranked or low ranked programs have faculty lacking either of these skills. Humans being the fallible reasoners that they often are, may mistakenly make this faulty inference. This is one problematic aspect of looking to the rankings, though it is a problem with interpretation of rankings rather than the rankings themselves. We should recognize how limited the information provided by the rankings is).

Perhaps you see a correlation between ability to do good work, and a deficit in the other set of required skills, such that professors who produce more are on average, worse teachers than professors who produce less. I'm not sure about this. It doesn't seem to hold.

10:24 AM  
Blogger dr. wentzel said...

i am aware that some rankings are advertised as providing a list of the *best* graduate programs. I do think this is false advertising (if best means where one will have opportunities gor getting the best training) given what I have already noted about the sort of information that the current ranking can and cannot provide. it is wrong to see these as telling us which graduate programs are *strong* in terms of graduate training, and which are *weak*.

12:37 PM  
Blogger hilde said...

I guess I misinterpreted Wentzel.

But didn't Wentzel say, initially:

"the best teacher would HAVE the ability to be a good scholar and/or creative philosopher herself. I believe THIS enables her to better encourage these skills in others."

Later, Wentzel corrects the record by saying

"I did not say that being a productive scholar proves or is a sufficient condition for being a good graduate teacher."

So we're bogging down over "necessary" vs. "sufficient".

But note, I never said Wentzel said "sufficient." I simply questioned whether there was a necessary link--I was questioning the "HAVE."

That said, I think Wentzel's follow up remark about there being two sets of skills is exactly what I want to say, and that missing set implicates the rankings as being askew of what graduate students need, qua students.

9:47 AM  
Blogger dr. wentzel said...

right, having an ability to do publishable work does not entail that one must be demonstrating that ability by doing the work. In addition, the fact that this is something that enables a person to be a good teacher of graduate students does not mean that other things are not also required (for example, it is also a requireent that one's heart be beating, as well as that one is able to communicate at all).

11:02 AM  
Blogger GF-A said...

Hi --

I just stumbled across this blog; hope you don't mind me barging in.

That data from the survey you cite is just not accurate for philosophy. It says that only 3 Princeton faculty members published an article anytime during 2003-05. If you just look at faculty webpages at Princeton you can see that that's wrong.

What happened? The survey you cite uses Scopus to count up ow many publications each faculty member has. Scopus does not yet cover the humanities. So the only articles that were counted were ones that appear in scientific or mathematical databases (e.g. philosophy of science, or logic, or...).

So while the survey's methodology is quite good for scientific disciplines, it's not very useful for philosophy.


4:47 PM  

Post a Comment

<< Home