Woman sitting in a library with books and a laptop.

3 major pitfalls of multimethod research

12 September 2023

The article at a glance

In this article Cambridge Judge Professor and editor of the Academy of Management Journal, Matthew Grimes, discusses how using more than one methodology in research can be useful but it can present major challenges for management scholars.

Category: Faculty news

Matthew Grimes.
Professor Matthew Grimes

The use of more than one methodology in a research project – known as ‘multimethod research’ – includes many advantages but there are important pitfalls that need to be avoided, as I and fellow editors of the ‘Academy of Management Journal write in a new editorial in the publication. 

Among benefits of multimethod research is a more complete understanding of organisations, which are themselves complex and multidimensional, and in fact a previous study showed that articles using this technique receive more citations than articles using a single research method. 

Yet more is not necessarily better, as multimethod research is accompanied by numerous challenges – and this research technique has in fact resulted in articles being rejected by journal reviewers. There are various reasons:

  • the combination of different methods can be redundant rather than elucidating
  • inappropriate methods are selected
  • the paper lacks alignment of theoretical and operational elements contained in the multiple methods

In preparing our editorial, AMJ editors reviewed all 238 articles published in the journal since 2020 in order to categorise multimethod articles into different types of papers. We identified various “archetypes” currently being produced by AMJ authors, with by far the most common (75%) using “triangulation” to offset the limitations of individual methodologies. Yet while this approach allows researchers to test the validity of their findings, we find that it often fails to deliver the benefits of mixed-method research.

What are the major pitfalls of multimethod research?

Here are the 3 major pitfalls that authors encounter in using multimethod research: 

1. Failing to explain how mixing methods helps

Failing to explain how mixing methods enables greater understanding or advances a particular theory; too often, authors cite the use of multimethod approaches as the main purpose of the paper, which is unlikely to impress journal reviewers. 

2. Poor methodological fit

A poor methodological fit between various studies included in an article given the current state of academic literature on the topic, including attempts to reinvent the wheel on a topic that is already well understood. 

3. Lack of complementarity

A lack of complementarity between the studies cited, which can often result in the end product being less than the constituent parts. This can occur when authors mix methodologies that in fact do pretty much the same thing – for example a survey and an experiment that test the same hypothesis. 

Avoiding challenges of multimethod research

To avoid these pitfalls, one route is to reduce the use of the ‘triangulation’ method of multimedia research and to instead try some of the less common approaches – what we identified as ‘test and explore’ (which tests an initial set of predictions and then gains a better understanding), ‘explore and test’ (which generates hypotheses followed by testing), and a full research cycle (in which methodologies are used to better understand naturally occurring phenomenon). 

We also recommend that authors who use a multimethod approach clearly explain early in their paper the reasons behind mixing methods and how this helps them significantly advance their theory relating to the topic at hand, and to ensure that different studies cited are appropriate given the current state of academic literature on that topic. 

Drawing comparisons with Hollywood

Another tip, drawn from Hollywood: develop a broad script before embarking on multimethod research, because such ‘storyboarding’ before collecting and analysing data helps to think through the full paper in advance and how a given research method will help advance the central plotline – ‘rising action’, ‘climax’ and ‘falling action’.  

To repeat, we are not urging authors to abandon multimethod research, as it holds tremendous potential in the right circumstances. But we do urge authors to channel that potential by thinking carefully about when multimethod research advances a paper’s underlying contribution to a given topic, and to find another approach if it does not.