Does your institution accept pseudojournals? Mine does.
Today, I got an invitation to publish in the new journal, “Expert Opinion in Environmental Biology.” Then it provided a list of the “High Profile Editorial Board” members. I usually don’t discuss spam at breakfast (nor eat it), but this morning my family had fun inventing names of prestigious journals. I could go through my spambox and find a couple dozen more.
These journals exist because there are people out there whose jobs require some sort of external validation of their scholarship. Long ago, the Who’s Who series profited from people who needed to show their names in a bound volume. They’re still making a mint, I think. Now they are joined by a small army of “peer-reviewed” journals. Any website claiming to have a peer review process can magically add fresh meat to your publications list on your CV.
Who would be involved with such an outfit? After all, if my 9-year-old kid can see through the name of a silly pseudojournal, shouldn’t professors in the field? Wouldn’t they actually make you look worse? The answer is, apparently, no.
Universities require that faculty coming up for tenure and promotion are demonstrably scholars within their field. At teaching schools, in which most people do little research, how is scholarship evaluated? Within a department where new faculty coming up for tenure, the evaluators may have by been tenured long ago, unfamiliar with the current norms in the field.
Standards might be locked in time from when faculty were active scholars in grad school. For example, a former colleague of mine was convinced that a having paper in Ecological Entomology would be a much bigger accomplishment one in Ecology Letters. That’s because he hadn’t heard of Ecology Letters, even if it had recently become the ISI top-ranked journal in the field.
Some scholars realize that things change, which maybe is why pseudojournals may be so easily accepted. Some might see through the sham of the pseudojournal, but decide to not care about the deceit because scholarly prominence may not be a priority.
At my university, I attended a session about tenure file evaluations. There was a discussion about the perrenial problem: how can faculty evaluate people in different subfields, especially in diverse disciplines?
It was a disappointing conversation. The outcomes affirmed the following policies: Committees are not allowed to request external evaluation of an academic record or CV. They are allowed to subjectively evaluate journal quality, but are specifically forbidden from referencing specific metrics such as impact factor, h-scores or ISI indexing. They are not encouraged to search for information regarding the validity of a journal, and any specific facts or evidence that a journal is of poor quality, or has sham peer review, should not be included in an evaluation.It is okay to report that you have not heard of a journal, but you can’t report whether your investigation has shows it to be a good journal. So, the only other ecologist at my old job would say, “I’ve never heard of Ecology Letters before” but he wouldn’t be allowed to say that ISI ranks it is the top journal in my field. Ecology Letters would be on par with Expert Opinion in Environmental Biology.
These policies allow the continued persistence of pseudojournals. This lets the institution check off the scholarship box on the tenure file without caring about the reality or quality of the scholarship. That said, people have been denied tenure for inadequate scholarship. However, it appears that this can only happen to the scholars who have too much pride to publish in pseudojournals. Heck, they could join the editorial boards of several of them, if they so chose.
I suspect, or at least hope, that many teaching schools actually do ferret out pseudojournals. However, the proliferation of these venues – and the willingness of faculty to lend their names to editorial boards – suggests that they indeed have an audience. Otherwise, who would go to the trouble and how could you make a profit without a customer base? Who really does see these things and think they’re real? I am honestly confused.
As academic publishing is moving towards more transparency, open access and data sharing, it will be interesting to see how the perception or measurement of scholarly activity shifts. The acceptance rate at PLoS ONE, for example, is over 2/3. Anybody can publish in this journal, and even more importantly, anybody can read this journal. I imagine that this trend will continue, and I’m excited for the notion that citation, and all that comes with it, will be more of a meritocracy, not managed by for-profit publishers keeping findings away from the public.
Small Pond Science 
Surprised your school doesn’t ask for external reference letters for tenure. Is it common in your experience for small, teaching-oriented schools not to?
Re: the notion that citation will become more meritocratic in a future in which everyone publishes in Plos One or similar journals, don’t be so sure:
http://dynamicecology.wordpress.com/2012/07/16/citation-concentration-filtering-incentives-and-green-beards/
I think the same effect I describe in that post applies just as much, if not more, to Plos One papers. Citations are viral; people read and cite what everyone else reads and cites. In a future in which we all publish everything in Plos One, I think citation counts will be about as meritocratic as YouTube video views.
Actually, I’ve never personally heard of a teaching school that expected outside reviews. I imagine highly-ranked liberal arts schools – with teaching loads of 2 courses per semester or less – probably do, but I don’t have direct experience with them. I’d love to hear more about this.
When I submitted my tenure file at my last job, I was told that external evaluations were optional, and I would have to solicit them and turn in the letters. I did that, with a few outside letters. I learned later that this actually hurt me (though it’s a secret process, one person filled me in after it was all over), because the college-level and university-level committees thought I was showing off. If untenured faculty at my university had outside letters solicited by their tenure committees, I suspect a sizable fraction of these letters would read, essentially, “Who?” Many struggle to get a few papers out before tenure, which is quite understandable given our teaching load.
As for the citation/meritocracy thing, I think there are different dynamics on opposite ends of the scale. I agree that some papers are more likely to go viral. Also, that scientists who have good PR and social media mojo are going to get cited more often. What goes viral and what doesn’t probably has little to do with the importance or quality of the find. On the other hand, I think at the other end of the scale, some great science that would otherwise go totally unread, will get at least some reads and citations. I’ve discovered papers that are awesome and totally cool buried in a journal (e.g., http://www.ncbi.nlm.nih.gov/pmc/articles/PMC355916/), that are important for a variety of things, but nobody’s found them. I think that might happen less. For example, a horrible, totally pointless video on youtube might get almost no views at all. But if a video is really cool, then it will at least get a modest number of views. That might only happen because the person that posted it told friends about it, and those friends watched it, and at least one or two of them shared it on. It didn’t go viral, but people saw it.
Once I saw how often people were looking at my website, I realized I needed to have a much better one, and because of that my papers are getting downloaded and read a lot more often than if I didn’t have the site. I’m essentially making all my papers open access, as long as people can find them (presumably via google scholar). Even if I still have a paper that the publisher might have behind a paywall, I don’t honor that and put it up. It’s my form of civil disobedience since my work is funded by taxpayers, taxpayers deserve to see it.
The other ‘meritocracy’ part that I think might hold is that, if papers aren’t judged by journal ‘quality,’ then papers that otherwise would be totally ignored will at least get perused. And papers that make it into high profile journals that are wholly silly, which is far too often, will be laughed off more readily. (The other thing which will make things more of a meritocracy is that the amount of time that teaching faculty have to play the publication game will drop. I sometimes don’t submit to high tier journals, even if I think it’s a very cool paper, because I don’t have the hours in the day to deal with the cycle of submissions.) So faculty with cool ideas but without the time and energy, or the professional network, to pump them up into fancy journals, will still have their papers read by somebody, rather than get buried in low-impact journal. In all, I do agree that the same fadishness and silliness of what get cited will persist.
At Williams College I’m sure external letters were expected, but yeah, Williams was a pretty high-ranked place and so probably atypical of small undergraduate colleges in general. I can understand why external letters aren’t required not at most teaching schools. But on the other hand it seems like one or two external evaluations of the entire tenure dossier, evaluating teaching and service as well as research, might be a useful input into the process. Faculty at other teaching-oriented colleges would a be a good source for such evaluations, I’d have thought. I don’t know, what do you think?
Interesting suggestion that a move to open-access publishing might ensure at least modest readership for most papers, while perhaps not doing much to reduce the overall degree of citation concentration. That seems plausible to me, but it also seems possible that it might work the other way–that papers in low-impact journals at least get read by the (small) audiences of those journals, while in an open-access, everything’s-in-Plos-One future, most papers only get read by the author’s closest colleagues. It would be interesting to collect data on this, but unfortunately I can’t see that happening.
Glad to hear you’re not one of the open access advocates (and I’ve encountered a few…) who think that the open-access future will be one of total meritocracy, and much less citation concentration. There are plenty of good arguments for open access, but “making readership and citations much more meritocratic” and “equalizing readership and citations among papers” are not among them!
It think it would be great if all campuses would have external review of portfolios. Whenever it’s been brought up, though, the trope is pulled out, “outsider reviewers wouldn’t understand the circumstances unique to our campus.”
And there is some truth to that. I think it’s true that most outsiders wouldn’t have an appreciation of what happens at my campus. (For starters, the student: permanent faculty ratio for our majors is now about 100. I could go on and on, and on.) Given our circumstances, an expectation of any kind of faculty publication, in the sciences, is actually quite absurd. I don’t think there’s a way that an external reviewer could appreciate how dire our lack of resources is, that constrains our teaching and research. I don’t want to go into specifics, because frankly it would be so embarrassing to my university. To use a metaphor: we don’t have a shoestring budget, because we ate the shoestring out of hunger two years ago.
Another reason that teaching campuses often don’t want external review, is that campuses try to mold faculty to what they think is good, which often doesn’t fit broadly accepted norms. (For example, at my former institution, whenever faculty observed my teaching I would get criticized for not lecturing enough. I had a variety of discovery-based and interactive lessons that rocked, and had the students engaged and they learned a ton. In my reviews, I was scolded and told to be “less Socratic” in my methods. In their view, good teaching was to teach like them, which is to zoom through a jazillion powerpoint slides and speak quickly enough that nobody has time to discuss, ask questions, or process information. I believe that my teaching materials, if subjected to outside review, would have fared quite well, especially if sent to those actively working in science education. They wouldn’t want that, though, because the departmental culture involved ritual mocking of science education research.)
So, external review would be great. However, for many campuses it would be embarrassing because it would show the disparity between local idiosyncratic standards and what is expected outside. External review of candidates for tenure would bring outside standards inside the department, and tenured faculty don’t want that because that would require keeping up with the times. This is, in part, why some people hate accreditation.
My wife wanted to add her two favorite pseudojournal titles she invented: “What’s What in Biology” and “Smarty-Pants Scientist”
You should see if you can get them to count a paper in Annals of Improbable Research.
http://www.improbable.com/
Although I hear that their rejection rate is even higher than Nature’s or Science’s.
On the other hand, folks at your school won’t run into the problem some people at research universities have run into, where senior faculty insist that papers in Plos One “don’t count” as publications. Like, not just “don’t count as much as Nature or Ecology Letters papers” or “don’t count for much”, but “don’t count, period”!