Show me the data!
Header

After more than 6 months of waiting, my long overdue first paper has finally been published in Nature

Why do I say ‘long overdue’?
Doesn’t it usually take a long time for academic papers to get published?

Well… if you read it here [paywalled], you’ll see it’s only 600 words and 1 figure (at the absolute limit allowed by Nature’s editorial policy). It’s not really a full and lengthy contribution to science, just a simple “this previous paper is significantly wrong, and this is why”. I would have gone into greater depth of analysis had I been allowed to, but the strict word-limit rather prohibits this. I have no idea quite why it took so long to get accepted and published. I suspect Nature isn’t to blame. Politeness dictates that the original study authors can take a certain amount of time to reply to such letters, fair’s fair – but 6 months? Hmmm…

Anyway, some background and context:

My entire research thesis relies on re-analysing other people’s data. I have no access to specimens, I’m not a field palaeontologist, my research angle is ‘palaeoinformatics’ – large-scale re-analyses of hundreds if not thousands of palaeontological datasets. So naturally, when a new fossil specimen (reconstruction below) with novel phylogenetic data gets published in Nature I justifiably take a keen interest.

Artistic reconstruction of Diania cactiformis by Mingguang Chi as first published in Nature

Using the freely-available software program TNT, one can quickly and easily re-analyse the data matrix given in the Liu et al 2011 supplementary materials in a matter of seconds with any desktop computer. Any such reasonably informed parsimony analysis of this data (there are numerous parameters and settings one could specify), does NOT generate the consensus phylogeny they depict and furthermore the ‘real’ result generates cladograms in which the position of Diania cactiformis is significantly different.

It took several different re-analyses to be sure that I was onto something interesting. Frustratingly in phylogenetics, authors aren’t always as explicit in their methods sections as I’d like them to be. In this instance, it was not stated (as is true of most papers) what branch collapsing rules were followed. Is it safe to assume that they used the default setting? I think not!

So, once I was sure there was a problem, I emailed the corresponding author with my concerns. This was her reply in verbatim:

Hello, Ross, thank you for your attention, now I am very busy with writing an application (the deadline is March 10th), could you wait me for few days, I will show you all the phylogeny tree which I got, the methods, the rules and so on. Thank you very much!
All best regards!
Jianni

I never did get a follow-up email. [I assume because I submitted a formal reply to Nature which Jianni will have been notified of]. As a lowly grad student I’m sadly used to getting fobbed-off all the time…
Good thing I didn’t wait too long to formally reply either. It soon became apparent via Facebook that another research group intended to challenge this paper too. Well done to David Legg et al for their successful reply, also published in Nature.

Finally, I’d like to thank my supervisor Matthew Wills for helping me appropriately word my submission. I may have had the idea, and done the analyses, but it might not have been published without his excellent editorial input into the wording of the piece. We tried very hard to be polite and maintain the importance of the specimen, whilst necessarily pointing out the flaws of the analysis given.

I’ve been in Nature two times now in 2011! I didn’t write the first piece though. Can I make it a hat-trick? Time will tell…

As for the Liu et al. counter-reply which I’ve only just been allowed to see like the rest of the world – I’m surprised certain bits of it made it past peer-review. Their use of the PTP test is particularly intriguing and defies current scientific consensus on the validity of the usage of this test:

“Additionally, a significant value of the partitioning tail permutation (PTP) test (P = 0.01) suggests the presence of a clear phylogenetic signal in the morphological data, also strongly supporting the topology shown.”

First of all, I’ll generously assume they intended to refer to the permutation tail probability test not the “partitioning tail permutation” [sic] test. I don’t doubt the numerical result of the PTP test they present – it’s the inference they make from that result that I find illogical and unjustified given the numerous papers that have critically examined the PTP test – that this result somehow “strongly support[s] the topology shown” well; it doesn’t!!! One only has to look at the titles of numerous papers on the PTP test (although, please by all means read them!) to see that few if any can recommend it’s use for the purpose of supporting particular topologies:

  • Swofford et al (1996) The Topology-Dependent permutation test for monophyly does not test for monophyly.
  • Slowinski & Crother (1998) Is the PTP test useful? [Barely; only to determine if data has ‘signal’]
  • Peres-Neto & Marques (2000) When are random data not random, or is the PTP test useful? [Largely, no]
  • Harshman (2001) Does the T-PTP test tell us anything we want to know? [Largely, no]
  • Wilkinson, M. et al (2002) Type 1 error rates of the parsimony permutation tail probability test. [Points out errors in the Peres-Neto & Marques 2000 analysis, but still agrees that “the parsimony PTP cannot generally be assumed to guarantee well-supported phylogenetic hypotheses.”]

FYI the original papers describing the PTP test are Archie, 1989; Faith 1991; and Faith & Cranston 1991 [full citations and links given at the bottom]

There are other problems with the Liu et al reply which I hope others may be able to see for themselves but I’ll leave it at that for now. I perceive this statement for instance:

In this context, Mounce and Wills seem to have overlooked the potential significance of their reanalysis of our data.

to be a bit of a ‘cheap shot’ especially considering the extremely short word-limit enforced on our comment by Nature. Of course we would have loved to have described the implications of the reanalysis for each and every character – but this simply wasn’t relevant enough to the criticism we were presenting. How this was deemed relevant to the rebuttal of our valid and justified points in the Mounce & Wills comment I leave up to you to decide

I’d also like to say how much I admire the work of many of the scientists on the Liu et al paper. I think they produce some excellent work, and I’ve met Jason Dunlop in particular many times – he’s a nice guy and an excellent scientist. As a middle author I’m pretty confident he has little to do with the problems inherent in the original Liu et al Nature paper and the subsequent Liu et al counter-reply. My own objections to both papers are nothing personal – just an obsession with good science and logical reasoning!

Welcome to the world of academia…

    References & Links:

Archie, J. W. A randomization test for phylogenetic information in systematic data. Systematic Biology 38, 239-252 (1989). URL http://dx.doi.org/10.2307/2992285.

Callaway, E. Fossil data enter the web period : Nature news. Nature (2011). URL http://www.nature.com/news/2011/110411/full/472150a.html.

Coddington, J. & Scharff, N. Problems with zero-length branches. Cladistics 10, 415-423 (1994). URL http://dx.doi.org/10.1111/j.1096-0031.1994.tb00187.x.

Faith, D. P. Cladistic permutation tests for monophyly and nonmonophyly. Systematic Zoology 40, 366-375 (1991). URL http://dx.doi.org/10.2307/2992329.

Faith, D. P. & Cranston, P. S. Could a cladogram this short have arisen by chance alone?: On permutation tests for cladistic structure. Cladistics 7, 1-28 (1991). URL http://dx.doi.org/10.1111/j.1096-0031.1991.tb00020.x.

Harshman, J. Does the T-PTP test tell us anything we want to know? Systematic Biology 50 (2001). URL http://dx.doi.org/10.2307/3070842.

Legg, D. A. et al. Lobopodian phylogeny reanalysed. Nature 476, E1 (2011). URL http://dx.doi.org/10.1038/nature10267.

Liu, J., Steiner, M., Dunlop, J., Keupp, H., Shu, D., Ou, Q., Han, J., Zhang, Z., & Zhang, X. (2011). An armoured Cambrian lobopodian from China with arthropod-like appendages Nature, 470 (7335), 526-530 DOI: 10.1038/nature09704

Liu, J. et al. Liu et al. reply. Nature 476, E1 (2011). URL http://dx.doi.org/10.1038/nature10268.

Mounce, R., & Wills, M. (2011). Phylogenetic position of Diania challenged Nature, 476 (7359) DOI: 10.1038/nature10266

Peres-Neto, P. R. & Marques, F. When are random data not random, or is the PTP test useful? Cladistics 16, 420-424 (2000). URL http://dx.doi.org/10.1111/j.1096-0031.2000.tb00361.x.

Slowinski, J. B. & Crother, B. I. Is the PTP test useful? Cladistics 14, 297-302 (1998). URL http://dx.doi.org/10.1111/j.1096-0031.1998.tb00340.x.

Swofford, D. L., Thorne, J. L., Felsenstein, J. & Wiegmann, B. M. The Topology-Dependent permutation test for monophyly does not test for monophyly. Syst Biol 45, 575-579 (1996). URL http://dx.doi.org/10.1093/sysbio/45.4.575.

Wilkinson, M., Peres Neto, P. R., Foster, P. G. & Moncrieff, C. B. Type 1 error rates of the parsimony permutation tail probability test. Systematic Biology 51, 524-527 (2002). URL http://dx.doi.org/10.2307/3070887.

#RoadWarrior

May 26th, 2011 | Posted by rmounce in Conferences | phdchat | Travel - (Comments Off on #RoadWarrior)

I’ve been travelling rather a LOT in the last 6 weeks or so; Rhossili Bay, London, Edinburgh, Leicester, Barcelona, Cambridge. It’s so nice to be back in Bath for a bit again.

While the memories are still fresh I thought I’d write a short summary of what I’ve been doing -> so much to update on my CV!

    Wales

April 16 – 21 in Rhossili Bay helping my supervisor (Matthew Wills) teach undergrads about ecology, experimental design and real fieldwork. The weather was absolutely “lush” as they say in Wales. The students were well-behaved and did some excellent projects while they were there – staying up til past 3am in some instances on the last night to finish off their presentations!

The only downside was the lack of mobile phone signal. It’s stressful to be without internet for so long!

    Scotland

The very next week (April 26 – 28) I flew up to Edinburgh for a PaleoDB short course. Congratulations are sincerely owed to the organisers, especially Al McGowan, for taking the time to arrange such a beneficial event. PaleoDB is an *excellent* free-to-use resource but I do wish it was more Open. I made no hesitation in pointing this out during discussions. I have since applied to become a contributing member of the database (application pending).

    Leicester

After Edinburgh, there was the small matter of a conference to attend
Progressive Palaeontology 2011 was ably hosted at Leicester University. I gave a well-received talk on my research, sneaking-in some data sharing advocacy at the end:

Best of all, one of my labmates Anne O’Connor won the Best Poster prize – w00t!

    Spain

Next-up: a well deserved holiday in Barcelona sunshine. Beautiful beaches, bountiful cerveza, and amazing architecture. Lovely!

Inside the Sagrada Família

Inside the Sagrada Família

    Cambridge

No rest for the wicked… on the 19th and 20th I attended a short course at EBI-Hinxton entitled ‘Linking Open Data in Biology using Ontologies and Literature Mining‘. Had a superlative 3 course dinner in St Catherine’s college (Cambridge University) on the Thursday night, and a further meal in The Eagle the next night.

You might reasonably ask – why is a palaeontologist interested in ontologies and text mining? Well, it’s simple really – this field is crying-out for these techniques to be applied, and I hope I’ll be one of the one’s there first to utilize these powerful techniques on palaeontological data. The pickings could be rich, if only we had Open linked knowledge infrastructure in place… I shall no doubt blog more in future on this topic.

    London

Finally, tomorrow today I’ll be going to a London BioGeeks event. Really looking forward to finally meeting Mark Hahnel face to face for the first time. I really think his FigShare initiative is an excellent idea. I’ve supported it myself by adding a few bits of test data. Will no doubt add more in the future…

    Future Travel

Needless to say, all this travelling has completely wiped-out my £200 BBSRC Travel Grant. Have applied for more grants to fund my trips to Belfast, Berlin, Brazil and Las Vegas later-on this year!

*fingers crossed* – for richer or poorer, it’s going to be an exciting year!