[Taxacom] methodological plurality

Kipling (Kip) Will kipwill at berkeley.edu
Thu Nov 22 11:47:54 CST 2012


Mostly I tend to agree with Derek. Some fine points.

> Derek Sikes:
>
> However, if this approach was combined with the argument that only
> one method should ultimately be used (MP, ML, BI) then Parsimony
> would never be justified since it is really just a non-parametric
> version of a weird (overly parameterized) ML model that would never
> be chosen by AIC etc, leaving us with two options (ML & BI).


This is true and perhaps appropriate. But there are other very good
reasons one can use to justify accepting the “no-common process model”
of parsimony (weird and overly parameterized from a model view of the
world only). The complexity of morphological characters versus the
simplicity of sequence data seems a place that there remains real
traction for parsimony, but I would rather not degenerate the discussion
into a methods war.

> That being said, I always tell my students that they should know as much
> about their data as possible. One of the benefits of using multiple
> methods is that doing so, and knowing that each method has its own
> potential weaknesses, informs you about your data.
>
> For example, (and sorry, I know this horse keeps getting beaten) it is
> well known that if you have long branches mixed among short branches you
> might end up with a very different tree using MP than ML or BI due to
> long branch attraction. If you have such a dataset I would insist a
> student use multiple methods. The idea of course is akin to the
> sensitivity analysis that Kip described as being inadequate previously,
> and yes, it's not a full parameter-space sensitivity analysis, but it is
> a good, better-than-nothing, first pass at seeing if your dataset might
> pose problems for one method but not another. (And not to harp too much
> on MP - it is well known that ML and BI have their own weaknesses, but
> this knowledge can help identify problems and therefore help solve
> them). Any method can fail due to model misspecification.

I have seen rare papers that do this and I have no issue if it is done
for the purpose of detecting violations of assumptions and model
specification. I am not entirely convinced it is the best way to achieve
that goal. A Grant & Kluge paper (Data exploration in phylogenetic
inference: scientific, heuristic, or neither Cladistics. 2003) is a
useful starting point for teasing apart notions of support from those 
related to problem detection. Conflating finding no violations of a 
method's assumption and showing support for results might be a kernel 
issue here.

> If all methods yield the same answer this is not useless. It tells you
> that you probably don't have long branch attraction problems. If the
> branch support agrees among methods it tells you the Bayesian method is
> probably not suffering from a star-tree paradox scenario. If each method
> gives you different answers you must then dig deeper to figure out why.

But the answer to why might be because the underlying assumptions are 
different OR that data have issues. How do we know which and what do we do?

> Sometimes reviewers and readers really want to see these alternative
> answers - it helps them learn as well (rather than just say "different
> methods gave different answers but I am only presenting and discussing
> results from the best fitting model approach."

Might I suggest reviews take the data and analyze it in their own way f 
their world view differs? I guess not.

> I agree that one shouldn't use multiple approaches mindlessly as if by
> doing so one is "covering all the bases", but also argue that one
> shouldn't present only a single approach mindlessly ignoring results
> from other methods. I hope we can all agree that doing anything that can
> be considered 'mindless' in science is bad.

Amen.

-- 

Contact info:

Kipling W. Will
Associate Professor/Insect Systematist
Associate Director,Essig Museum of Entomology

send specimens to:
Essig Museum of Entomology
1101 Valley Life Sciences Building, #4780
University of California, Berkeley
Berkeley, CA 94720-4780

letter mail to:
130 Mulford Hall
ESPM Dept.- Organisms & Environment Div.
University of California
Berkeley, California 94720

fax 510-643-5438




More information about the Taxacom mailing list