Academia Obscura

by Glen Wright

  Some journals go to great lengths to avoid using the ‘p’ word. One said that several passages from another paper ‘could be viewed as a form of plagiarism’, another noted that a paper had an ‘originality issue’.

  Why so coy?

  Journals tell us that lawyers play an outsized role in all of this – they sometimes take an aggressive stance and journals back down because they don’t want to deal with excessive legal costs. Accused scientists have been suing institutions, journals, and even commenters on PubPeer (a website that allows users to discuss and review scientific research). We are keeping an eye on it.

  Bloody lawyers. What’s the future of retractions?

  We’d be happy if we didn’t have retractions at all, they are the nuclear option. Instead we need a solid correction mechanism and to stop thinking of papers as immutable. Science is an iterative and incremental process and papers should reflect that. There are a lot of initiatives being developed to take this forward, like PubPeer and CrossMark. Nonetheless, I don’t think retractions are going anywhere in the near future.

  Do you have a personal favourite?

  There’s always some interesting news or a baffling story that makes it fun for us. I have a favourite category – fake peer reviews. One researcher has notched up 28 retractions because he did almost all of his own peer reviews. His system was ultimately foiled because all the reviews came back in under 24 hours. The editor became suspicious because he did not believe that real reviewers would have turned the papers around so quickly! Those papers probably should have been published anyway, but I guess getting your reviews back in 24 hours with guaranteed acceptance is a pretty good insurance policy.

  Is the pressure to publish leading to increased misconduct, or are we just getting better at spotting bad behaviour?

  The rise in retractions is dramatic – the rate increased tenfold between 2001–10. However, we must have a sense of perspective. There are millions of published papers, so hundreds of retractions is still not that many. I think the rise is mostly down to the fact that we are getting better at finding misconduct. We now have plagiarism detection software rooting out the most flagrant cases; there are many more readers of papers because everything is online; new tools and communities are poring over papers to find inconsistencies and problems, so it is no surprise that the rate is going up.

  I do however think that the pressures on researchers and the incentive structures must contribute in some way: you have to publish papers to get tenure, grants, promotion – i.e. everything you need to have a successful career in science. Everyone does what they think they need to do. For some that means working incredibly hard, a few cut corners, while a tiny minority simply start making things up.

  You got a lot of attention during the LaCour scandal. How have such high-profile cases affected Retraction Watch?

  We broke the LaCour story, and it had a dramatic impact on us. I got a tip via Twitter – it was very early in the morning and I happened to be awake. Once I confirmed that a senior author was requesting a retraction we broke the story. It crashed our server – I had to pay $300 to upgrade that day to cope with the traffic. The interviews were constant. I did one from South Korea at 1 a.m., then NPR, then somewhere else. The New York Times profiled us and published our op-ed on the case. It was an incredible boost for us.

  Do authors often self-retract?

  Self-submitted retractions are not even a large minority yet, but we do have a category on the site called ‘Doing the Right Thing’. We try to highlight and praise authors that do self-correct – about a hundred posts so far. Retracting still has a stigma, and no one likes to see their work go to waste, but we think it is better to hear the story from the authors themselves.

  Most retractions?

  We have a leader board, there are currently around 30 people on it. They shift around as new information comes in. Yoshitaka Fujii is currently number one with 183 papers and shows no signs of budging any time soon. Fujii is a good example of the new tools and communities that are sniffing out bad science – he was caught out by peers who meticulously ran the numbers on his papers.

  Funniest retractions?

  The paper on plagiarism guidelines that had been plagiarised. Or the two cases where hidden cameras were used to catch researchers tampering with experiments – twice in 2 million papers really does make these one-in-a-million! Best intentions and safeguards will never stop those that are determined to cheat.

  On your merchandise page there is a Retraction Watch clock. Can I buy a Retraction Watch watch?

  You aren’t the first to suggest that …

  Damn, I thought I was being funny. If people want to support Retraction Watch how can they do that?

  We appreciate any and all support – reading our site, commenting on posts, sending us tips, telling your colleagues about us. If people are able to make a financial contribution, we are a registered non-profit and they can do so via our site. Thanks for helping us spread the word!

  THE HOAXES WITH THE MOSTEST

  In 1996, Alan Sokal, a physics professor at New York University, became infamous as the instigator of the best-known hoax in academic publishing history. At the height of postmodernism’s popularity, Sokal submitted a paper to the journal Social Text entitled ‘Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity’.54 The paper proposed that quantum gravity is a social and linguistic construct, and ostensibly demonstrated how ‘postmodern science provides a powerful refutation of the authoritarianism and elitism inherent in traditional science’.55

  Sokal did not write the paper as a genuine work of critical theory, but as, in his own words, ‘A pastiche of left-wing cant, fawning references, grandiose quotations and outright nonsense.’* Sokal wanted to test whether a leading journal of cultural studies would publish an article ‘liberally salted with nonsense if it sounded good and it flattered the editors’ ideological preconceptions.’ The answer was a resounding yes.

  The journal did not have a peer-review process at the time, so the paper wasn’t reviewed by an external expert, much less a physicist. Sokal revealed his hoax on publication day, igniting a debate about the scholarly merit of humanistic commentary on the physical sciences, as well as on academic ethics (i.e. whether Sokal was wrong to deceive, and conversely whether the journal erred in its lack of academic oversight).

  The Social Text editors said they thought Sokal was honestly seeking ‘some kind of affirmation from postmodern philosophy for developments in his field’ and that the paper was a ‘change of heart, or a folding of his intellectual resolve’.56 None of the editors suspected that the piece was a parody, and even once they learned it was a hoax, they argued that it was still of interest as a ‘symptomatic document’ (i.e. as an example of how awkwardly a natural scientist might approach postmodern epistemology). Sokal was probably further amused that the glaring absurdity was not patently obvious. Indeed, in just the second paragraph he claims that physical reality is just a social and linguistic construct. ‘Fair enough,’ he says. ‘Anyone who believes that the laws of physics are mere social conventions is invited to try transgressing those conventions from the windows of my apartment. I live on the twenty-first floor.’57

  The editors of Social Text won the Ig Nobel Prize for literature that year, for ‘Publishing research that they could not understand, that the author said was meaningless, and which claimed that reality does not exist.’58

  The best hoaxes have a serious point to make. Three enterprising MIT graduate students, wanted to expose the daylight robbery that is shoddy academic conferences (see page 182), so they created SCIgen. SCIgen is a nifty piece of software that seamlessly weaves together gobbledegook into grammatical sentences and presents it in a familiar format, ready to be submitted to conferences. They generated a couple of papers, stuck their names on them, and sent them off to the World Multiconference on Systemics, Cybernetics and Informatics (WMSCI), a conference that Maxwell Krohn, one of the creators of
SCIgen, says was notorious for ‘being spammy and having loose standards’.59

  Their paper ‘Rooter: A Methodology for the Typical Unification of Access Points and Redundancy’ was immediately accepted as a non-reviewed paper (because reviews had not been received by the deadline).* They accepted in style, with an email containing no less than three smileys.60

  The three planned to attend the conference, but the organisers eventually got wind of what was going on and withdrew their invitation amidst growing international media attention. The organisers sent the authors a four-page letter that one professor described as ‘a mind-boggling, rambling rationalization, written in full-bore buzzwordia academic’.61 The students were not easily deterred. Capitalising on the building momentum, they raised $2,500 in just 72 hours to travel to Orlando (not bad given that this was before the golden era of viral videos and crowdfunding). They rented out a room at the same hotel as the conference and proceeded to hold their own session, which consisted of randomly generated talks by academics with fake names, fake business cards, and fake moustaches.

  SCIgen is free to download, and has taken on a life of its own as scientists have used it to have a bit of fun and further expose poor publishing practices in the process. There are plenty of examples, though one published SCIgen paper stands out for its unusual author list: Marge Simpson, Kim Jong Fun, and Edna Krabappel. Alex Smolyanitsky, a researcher at the US National Institute of Standards and Technology, refused to pay the Aperito Journal of Nanoscience Technology $459 to publish it, but they did anyway. The paper remains freely available on the journal’s website.62 In addition to the atypical author list, SCIgen churned out some preposterous passages, such as:

  Is it possible to justify the great pains we took in our implementation? No. With these considerations in mind, we ran four novel experiments . . . We deployed 98 Motorola bag telephones across the Internet-2 network, and tested our flipflop gates accordingly.

  In December 2013, computer scientist Navin Kabra had his bogus paper, ‘Use of Cloud-Computing and Social Media to Determine Box Office Performance’,63 accepted to a conference. He was trying to highlight the pitfalls of policies at his university that forced students to publish, usually in the proceedings of low- or no-standard conferences. In the introduction, Kabra (claiming to be from the ‘Sokal Institute of Technology’) explicitly warns the reader that what follows is meaningless drivel:

  You should read any paragraph that starts with the first 4 words in bold and italics – those have been written by the author in painstaking detail. However, if a paragraph does not start with bold and italics, feel free to skip it because it is gibberish auto-generated by the good folks at SCIGen.

  The paper occasionally pretends to discuss the purported topic, including discussion of UIB and AAF algorithms (later revealed to be ‘Use IMDB.com via a Browser’ and ‘Ask a Friend’ respectively). The paper includes nineteen lines about the 1970s Bollywood film Sholay, and another nineteen taken directly from the 1992 Hollywood film My Cousin Vinny. Following one remarkably nonsensical passage, the paper states: ‘The motivated reader is encouraged to not read too much into the previous paragraph, because it was copy-pasted from a random document on the internet.’ The organisers claimed that the paper was one of only 60 submissions accepted of the 130 received and that all papers were double-blind reviewed by international experts.64

  In a similar bid to expose junk journals piggybacking on university publication requirements, Mikhail Gelfand from the Russian Academy of Sciences translated the original SCIgen paper into Russian and submitted it to the Russian language Journal of Scientific Publications of Aspirants and Doctorants. The journal accepted Gelfand’s paper and charged 4,000 Rubles (£40) for publication. However, his protest hit the mark and the government revoked their accreditation of the journal two weeks later.

  French researcher Cyril Labbé of Joseph Fourier University in Grenoble catalogued SCIgen papers that had made it into over thirty published conference proceedings between 2008–13. His work revealed that 16 nonsense papers had been published by the publishing giant Springer, while the US Institute of Electrical and Electronic Engineers (IEEE) had published over a hundred. Labbé privately informed the publishers, who subsequently took steps to remove the offending papers. Labbé has since developed a program to spot fake papers by comparing an uploaded manuscript to papers known to have been generated using SCIgen.*65

  One of the creators of SCIgen notes that Labbé’s work revealed just how deep this problem runs, stating that he is proud of the program and the fact that it continues to expose weaknesses in the world of science. ‘I’m psyched,’ he said in an interview with the Guardian. ‘It’s so great. These papers are so funny; you read them and can’t help but laugh. They are total bullshit. And I don’t see this going away.’66

  In a systematic study of sketchy publishing practices, Science corres­pondent John Bohannon published ‘Who’s Afraid of Peer Review?’, an investigation into the peer-review processes among fee-charging, open access journals. Between January and August 2013, he submitted a fake scientific paper to 304 journals. The paper was considerably more plausible than anything SCIgen spews out, but was nevertheless written with such serious and self-evident scientific flaws that editors and peer reviewers should have summarily rejected it.† Nonetheless, 60% of the journals accepted it. The Economist dubbed it ‘Science’s Sokal moment’.67

  Bohannon used Beall’s List of predatory publishers and the Directory of Open Access Journals to build a list of 304 targets.‡ Journals accepting the paper were not only the usual suspects, but also included those from big names like Elsevier, Sage, Wolters Kluwer, and several universities. India emerged as the largest base for such publications, with 64 publishers – over 90% of them – accepting the paper. The US came in second with 29 publishers accepting the paper and 26 rejecting it. Nigeria was the largest African offender, with all of the journals there accepting the paper.

  Because Bohannon’s exposé focused only on open access publishers, it quickly became part of the polarising debate around the evolution and future of scientific publishing, with open access advocate Michael Eisen commenting that accusing the open access model of enabling internet scamming is ‘like saying that the problem with the international finance system is that it enables Nigerian wire transfer scams.’*68

  As is often the case in academia (and in basically all my romantic relationships to date), we agree on much of the substance, but argue vehemently about specifics and semantics. Here’s my summary: let’s not condemn all open access journals because of a few unscrupulous actors, but let’s also be careful not to shoot the messenger when studies call out bad practices.

  Unsubscribe

  David Mazières and Eddie Kohler submitted a paper entitled ‘Get me off Your Fucking Mailing List’ to WMSCI 2005 (the same conference that accepted the original SCIgen paper). The paper consists of the title sentence, repeated over and over.

  Figure 10: Get me off Your Fucking Mailing List

  The paper didn’t make it, but it got a second chance in 2014 when Peter Vamplew of Federation University Australia forwarded it to the International Journal of Advanced Computer Technology as a retort to their spam email.69 The paper was then ‘reviewed’, rated as ‘excellent’, and accepted for publication (though the reviewer did ask Vamplew to update the references). Vamplew declined to pay the $150 article-processing fee and so the paper was ultimately not published.

  It is not known whether he was removed from the mailing list.

  MALE, MAD AND MUDDLE-HEADED ACADEMICS IN KIDS’ BOOKS

  Melissa Terras is the Director of the Centre for Digital Humanities and a Professor of Digital Humanities at University College London. She is also an expert on the portrayal of academics in kids’ books, having analysed almost 300 titles.

  How did you end up with a library of kids’ books featuring academics?

  I’m keen to share my love of books with my three kids, so we read a lot. One week I
came across two different professors in children’s books in quick succession. I thought it’d be a fun project to see how academics are portrayed. This turned out to be both an excuse to buy more books and a way to explain to my kids what Mummy actually does.

  How do you find the books?

  For four years I searched for new finds in the little bits of spare time I get throughout the day. Often academics appearing in books are not named in the title and therefore don’t turn up easily via electronic searches, so I also began to obsessively search the shelves at our local library and friends’ houses, and waiting rooms at doctors and dentists.

  Fortunately I don’t always have to do the digging myself as librarians from all over the world send me leads. People occasionally sidle up to me after a guest lecture and whisper, ‘I have a good professor for you …’

  What’s the oldest example you’ve found?

  The earliest goes all the way back to 1850 – a time when the world had far fewer higher education institutions. Indeed, given the exponential growth of universities and the publication of 1.6m English language kids’ books in the intervening 150 years, 281 academics seems disappointingly low.

  What are the academics in children’s books like?

  I usually summarise them as ‘male, mad and muddle-headed’. There is a lot of lazy stereotyping. Academics tend to be either crazy evil egotists (such as ‘Mad Professor Erasmus’, the maddest evil professor in the world) or kindly, but baffled – obsessive eggheads who don’t quite function normally.