Show me the data!

A Conference Abstract #OpenDraft

May 29th, 2011 | Posted by rmounce in Conferences

UPDATE 01/06/11 I’ve now submitted a modified version of this abstract. Many thanks to all who commented.

One of the many conferences I’m going to this year is a fairly big event. The abstract deadline also happens to be very soon too. So I thought I’d post a draft of my abstract here and see what peeps think.

Is my Latin correctly formed? What do you think? Here it is anyway…

Nullius in Calculo: On the Explicitness and Reproducibility of Cladistic Analyses

Author: Mounce, Ross C P

Abstract: The result of a cladistic analysis should be repeatable.
Yet I present here numerous examples of recent papers in which the results contained therein cannot be replicated, given only the content of the paper, supplementary materials and links. Barriers to study replication include (1) absence of requisite information, (2) typesetting errors, and even (3) author error. I argue that these problems, many of which are easily-spotted, should not be appearing in peer-review published papers with such regularity. I humbly suggest that reviewers and editors not only examine the words of papers, but also the underlying data and calculations: Nullius in Verba, Nullius in Calculo. Furthermore, I believe the reporting of phylogenetic analyses would greatly benefit from increased Standardization following community-agreed criteria c.f. MIAPA (Leebens-Mack et al, 2006), and data deposition in appropriate data archives specifically designed to accommodate phylogenetic data e.g. TreeBASE or MorphoBank. In addition to problems with reproducibility, I also detail problems with explicitness of method reporting. A detailed manual examination of over 300 recently published ILD tests provides evidence to suggest that method sections are rarely sufficiently explicit in their detail to exactly replicate the methods used to generate reported results. In particular I suggest that authors should be encouraged to explicitly state how ‘gaps’ are coded and treated, and which branch collapsing rules are followed in analyses. Different settings can and do generate different results, therefore all such important settings should be explicitly stated.

[end of abstract]

Some further comment:

Any feedback good/bad/indifferent would be much appreciated.
My supervisor has given it ‘the green light’ in principle, but obviously is keen for me to handle the topic delicately and sensitively

I’m not out to ‘name and shame’ individual errors with this – it’s the system that needs changing IMO, and I’ll do my damnedest to make that crystal clear when I give the talk.

I’d be pretty surprised if this abstract was rejected – I’ve got strong evidence including an accepted Nature paper demonstrating some of this. Would love to blog more about this but Nature’s embargo policy rather prevents/scares me from doing so! My last talk at a Systematics Association conference (below) went down a storm too, picking-up a special commendation from the judging panel for it’s “unscorable” uniqueness [I used a Prezi] so I can only hope this next one will be as successful.