Looks Like PLOS ONE Screwed Up the “Creator” Retraction, Too

March 4, 2016 jonfwilkins Leave a comment

Okay, that “Creator” paper has officially been retracted by PLOS ONE (previously, and here). Based on what we now know, that looks like the wrong decision — at once unfair to the authors and completely failing to address the actual issue.

When PLOS ONE first announced its intention to retract the article, they stated that “the peer review process did not adequately evaluate several aspects of the work”, which makes it sound like they found problems other than inclusion of the “Creator” language that meant it should not have been published. Now that the formal retraction has happened, here’s the official statement:

Upon receiving these concerns, the PLOS ONE editors have carried out an evaluation of the manuscript and the pre-publication process, and they sought further advice on the work from experts in the editorial board. This evaluation confirmed concerns with the scientific rationale, presentation and language, which were not adequately addressed during peer review.

Consequently, the PLOS ONE editors consider that the work cannot be relied upon and retract this publication.

The editors apologize to readers for the inappropriate language in the article and the errors during the evaluation process.

This is infuriatingly vague, but it makes it sound as if the primary issue was the “Creator” language. The authors have insisted that this was a translation problem. In the context of the rest of the paper, that seems entirely plausible to me. In support of this explanation, check out this comment from over at Complex Roots (spelling corrected):

I am so surprised that so many people assert that there is no way a translation error though they don’t speak any Chinese.

In fact, there is special phrase in Chinese, which is “zao wu zhe”. If we translate it literally and directly into English, it is “the one who creates” or ‘creator’. Ancient Chinese people use it a lot in poems, way long before Christian is introduced in China. The meaning is same as “nature” because they believe that nature ‘creates’ everything, not a special man, or a God. There is a sentence in a poem written in Song Dynasty (more than 1000 years ago) by Su Shi, which saying that ‘we can enjoy the the breeze of the river, the moon between the mountain; this is the inexhaustible treasure that the creator have, and all of us can appreciate them together’. So here ‘creator’ means nature. (poem link: http://www.rthk.org.hk/chiculture/chilit/dy05_1205.htm)

Or you can use google translator to check this page (a Chinese dictionary): https://www.moedict.tw/%E9%80%A0%E7%89%A9%E8%80%85
It will tell you that ‘zao wu zhe’, which means who created all things. It refers to nature.

However, in English, Creator is epithet of God because people firstly say it believe God creates everything. That’s the difference. The author used capitalized ‘Creator’ because he thought that the underling meaning of this idiom in Chinese and English is same.

Unless there were technical issues with the science, the authors should have been given the opportunity to edit the paper to correct the offending language.

As I argued previously, the fact that this error slipped through is troubling, not because it plays into some creationist agenda, but because it reveals a review and editorial process that involved absolutely no care or effort.

Now, it seems that PLOS has responded to the twitter/comment outrage by throwing the authors under the bus, while giving no reason to believe that any other manuscripts, present or future, are going to receive any more care and attention than this one did.

academia, biology, evolution, religion

“Creator” Paper Retracted at PLOS One

March 3, 2016 jonfwilkins Leave a comment

Well, true to their word, the editorial staff at PLOS ONE acted quickly to review that paper from January that interpreted their study of biomechanical characteristics of hand coordination as evidence of “proper design by the Creator”. (Look here for background.) They issued this statement today:

The PLOS ONE editors have followed up on the concerns raised about this publication. We have completed an evaluation of the history of the submission and received advice from two experts in our editorial board. Our internal review and the advice we have received have confirmed the concerns about the article and revealed that the peer review process did not adequately evaluate several aspects of the work.

In light of the concerns identified, the PLOS ONE editors have decided to retract the article, the retraction is being processed and will be posted as soon as possible. We apologize for the errors and oversight leading to the publication of this paper.

The paper’s first author, Ming-Jin Liu, has posted multiple comments asserting that there was no creationist agenda, and that this was simply an issue of non-native English speakers misunderstanding the implications of using “the Creator” when they had meant “natural selection”.

Personally, I’m inclined to believe this explanation, and if this were the only problem with the paper, I would let them make a correction. If, in each of the three places where the Creator is credited, the authors were to cite their findings as “evidence of exquisite adaptation” or some such thing, the meaning would be largely unchanged, and no eyebrows would be raised.

Here’s the thing, though: at this point, I have no confidence that there is not something else dreadfully wrong with the paper. Including three references to “the Creator” — one in the abstract — raises such an obvious red flags that even a cursory read should have identified this as a problem. The capital C makes the word jump out if you even scan the abstract.

I think I would feel the same way if the paper were littered with errors involving there, their, and they’re: it’s a mistake a non-native speaker could make, and it would not make the science wrong. But the only way those errors make it all the way through to publication is if multiple people fail to do their jobs.

So what this says to me is that none of the people involved in the editorial and review process put in even a modest effort. I don’t know if there are major, even fatal, technical flaws with the paper. However, I am confident that if there are major flaws, the careless review process applied to this paper would never have identified them.

The question, then, is how much of an outlier this was. Can we trust that the rest of the articles at PLOS ONE are actually going through a legitimate review process (as imperfect as that is under the best of circumstances)? Or should we assume it has slid into the predatory open-access model of publishing?

In short, I don’t really care whether or not this particular paper is retracted. I do care whether or not PLOS can do something to shore up its review process.

Or is this another piece of evidence in favor of post-publication peer review? It is certainly true that an advantage of that model is that avoids creating a false sense of authority.

One bright side of the controversy is that it provides an excuse to revisit this piece of awesomeness from the New York Dolls:

academia, science

“Mystery of the Creator’s Invention” at PLOS One

March 2, 2016 jonfwilkins Leave a comment

Oops!

PLOS One published a paper in January with the title “Biomechanical Characteristics of Hand Coordination in Grasping Activities of Daily Living“. And the abstract contains this line:

The explicit functional link indicates that the biomechanical characteristic of tendinous connective architecture between muscles and articulations is the proper design by the Creator to perform a multitude of daily tasks in a comfortable way.

The main text contains two more references to “the Creator”. The Introduction notes that

Hand coordination should indicate the mystery of the Creator’s invention.

And the end of the Discussion:

In conclusion, our study can improve the understanding of the human hand and confirm that the mechanical architecture is the proper design by the Creator for dexterous performance of numerous functions following the evolutionary remodeling of the ancestral hand for millions of years.

Today, someone seems to have noticed this “Creator” stuff and brought it to the attention of the journal, which issued a statement that they’re looking into it.

So what happened?

This does not read to me like it is part of some sort of conspiracy to infiltrate the biology literature with intelligent design propaganda. However, it is a good illustration of the issue with the PLOS One model as it is implemented in practice.

PLOS One is based on an interesting idea. The papers are peer-reviewed, but evaluation is explicitly supposed to focus on technical accuracy, ignoring “impact”. This was a brilliant idea. Historically limited space (in print) and a pathological pursuit of citation metrics means that lots of good science has a hard time getting published, either because it is not flashy enough, or because journals are reluctant to publish things that do not fit squarely in the domain of what they imagine their readers’ interests to be.

In a sense, PLOS One aims to split the difference between traditional publishing and the preprint / post-pub-peer-review model. It is someplace where you can publish interdisciplinary work, weird little fun studies, negative results, etc. But in principle you get the benefit of knowing that the work itself has been vetted.

Or at least as vetted as you ever get with peer review, which is to say, imperfectly and highly variably.

As a result, PLOS One publishes lots of cool stuff, and it provides a valuable service to the community. But when something like this happens, it makes it seem like the editorial policy in practice is something more along the lines of “just make sure the check clears”.

Politics, Uncategorized

How Should Democrats Allocate Primary Delegates Among States?

March 2, 2016 jonfwilkins Leave a comment

The US Presidential election system is weird, but the primary system is really weird. Contests are strung out over the course of five months, with the rules varying from state to state — from who can vote to how the voting happens to how the delegates are allocated among the candidates.

For the Democratic Party, there are 4051 “pledged” delegates, whose first-ballot votes at the convention are determined by the results of state-wide (or state-equivalent-wide) primaries, and 714 “superdelegates”, party leaders and insiders of various sorts who can vote for whomever they want.

Allocation of Delegates

The exact number of delegates assigned to a given jurisdiction is determined in a number of steps (details here). First, each jurisdiction is assigned a base number of delegate votes. For the 50 states and DC, this base is 3200 time an “allocation factor” that is the average of two quantities. Half of the allocation factor is set by the fraction of electoral college votes for the jurisdiction (e.g., 3/538 for DC and 29/538 for Florida). The other half is the fraction of the nationwide popular vote for the Democratic presidential candidate that came from that state over the previous three elections.

That second factor does two things. First, it makes the delegate allocation more proportional to population — reducing the advantage given by the electoral college system to smaller states. Second, it rewards states that tend to vote Democratic.

In this figure, blue dots indicate states that have gone for the Democratic candidate in each of the three previous elections, while red dots have gone for the Republican candidate in all three. Gray dots are states that have voted for the Democrat in either one or two of the three most recent elections. You can see that the slope of the red points is lower than that of the blue points.

You can also see how this scheme down-weights the electoral value of the small states, since the apparent y-intercept is below zero. We can see this shift more clearly if we replot this in terms of the number of delegates per electoral vote:

Note that the apparent purple point is actually an overlay of red and blue (Montana and Vermont).

Then, each state is given various bonuses based on when they hold their primary. For example, you get a larger bonus for holding your primary later in the season. And, for primaries held March 22 or later, you get a 15% bonus if you are part of a cluster of three or more neighboring states with primaries on the same day.

(Because in Democratic primaries, as in the Special Olympics, everyone is a winner, no state has an overall bonus of less than 15%.)

Delegates are also assigned to various jurisdictions that don’t actually get to vote in the presidential election, like Puerto Rico, and to “Democrats Abroad”, people living overseas, who would vote in the presidential election via absentee ballot in their home state.

How SHOULD delegates be allocated?

So, is this a sensible way to allocate delegates? It depends on the goal. The current system seems to be basically a hybrid of the electoral college system and a popular vote, with some additional features to reward party loyalty. It seems that the system aims to strike a balance among three competing goals:

Pragmatic considerations of electoral math. Elections are determined by the electoral college, so a system that mirrors the electoral college seems more likely to produce a candidate who can win.
A democratizing impulse. At the same time, there is a sense that the electoral college system is weird and not always fair. Skewing delegate weights towards population size, allowing proportional allocation of delegates within states, and allowing participation by groups that are normally excluded from the Presidential election all make the primary outcome a bit more like a nationwide popular vote.
Community Building. If you reward states that produce Democrats and get them to vote, as well as states that get in line and follow the rules, you presumably hope that this will lead to more of both.

However, if we take the pragmatic angle seriously, there is something missing from this calculation. In the current political environment, the outcome of a Presidential election depends primarily on how candidates perform in the swing states.

Barring a landslide, come November, the Democrats are going to win Washington DC and Massachusetts, and the Republicans are going to win Wyoming and Mississippi. If the goal is to nominate a candidate who can prevent a dystopian Trump presidency, Clinton’s win in Alabama and Sanders’s win in Vermont are irrelevant. The primary results we should pay attention to are those from states that could determine the election outcome. Fare more important are the close results in Iowa and Nevada, Sanders’s victory in Colorado, and Clinton’s in Virginia.

So, the other thing you could include in your allocation factor is a measure of “swinginess”. There are a lot of ways you could do this, but here’s a simple one: calculate the mean and standard error of the difference between Republican and Democratic vote percentages in each state over the past three elections. Assuming those values are Normally distributed, calculate the probability that the winner is different from what we expect from the mean. So, if the mean difference is exactly 0%, the probability would be 0.5. If the mean difference is 5%, and the standard deviation is also 5%, the probability would be about 0.17. If the mean difference is 30%, and the standard deviation is 5%, the probability is effectively zero.

Take this probability and multiply it by the number of electoral votes. The result is something like the number of electoral votes you can expect to get by doing well in that state.

What remains, then, is how to combine this factor with the other considerations. For example, if we use an Allocation Value that is 90% the existing formula, and 10% Swinginess, we get the following:

If we use these numbers to replot our graph of base delegates per electoral vote versus electoral votes, you can see how the states that have been close in the past get bumped up.

Those four highest points, from left to right, are Colorado, Virginia, Ohio, and Florida.

Is this fair? I’m not actually sure what that means in this context. Keep in mind that nearly 1/3 of the delegates exist either a) to directly express the will of the party elite, or b) to allow the national party to manipulate how and when the states hold their primaries.

A better question might be, would it work? Also, what other consequences might result? It seems intuitive that, for the Presidential election itself, a candidate’s ability to carry Ohio is more important than how much of a landslide they could rack up in California. On the other hand, winning by more (or losing by less) in non-competitive states could make a difference in down-ticket races. And if you discount the voters in solid-blue states too much, you risk alienating your base.

All in all, I suspect something along these lines would be an improvement. At a minimum, it might be a useful way to factor in “electability”, particularly in election years that are more likely to be decided by base turnout than by swaying independent voters.

Lost in Transcription

Monthly Archives: March 2016