Please let me present an excellent example of dodgy research, which in this instance is an online survey being touted as “public consultation”. I’m being a bit chicken by intentionally avoiding using an example from nursing/health, and using an example from an industry that I am not involved in*.  The example I have come across is by the Far North Queensland Ports Corporation Limited, trading as Ports North (a Queensland government-owned corporation); more info about Ports North here: www.portsnorth.com.au.

In Cairns consideration is being given to dredging the shipping channel to allow larger ships to access the port. However, because Cairns is synonymous with the Great Barrier Reef there are concerns about the impact of dredging on the living coral. This has been reported in the local newspaper (The Cairns Post), on the radio (ABC Far North Queensland) and via local university researchers (Coral CoE @ James Cook University). Ports North is preparing an Environmental Impact Statement (EIS) and are seeking community feedback as part of the EIS process. All of that sounds good, until we look at the questions.

Let’s take a step-by-step look at the online survey together (eight screenshots follow):

1. Survey Introduction

This is pretty good: it gives an explanation of the rationale for the survey, assures confidentiality, and is friendly and inviting: “Your feedback is important to us…” It is interesting to note the choice of words “expand the shipping channel” rather than “dredging the shipping channel”. Nevertheless, the survey is inviting community feedback to inform the environment impact statement, so it would be reasonable to expect there to be questions regarding this.

2. Awareness Question

The options in the drop-down menu are “Yes” and “No”. A simple question that again uses the word “expand” rather than “dredge”. For your consideration: what influence does choice of words/language make in surveys?

3. Support Question

The options in the drop-down menu are “Yes”, “No” and “Don’t Know”. Again, an interesting choice of words: I wonder if there would be a difference in responses if the question was changed from “Do you support this proposed community project?” to “Do you support dredging the shipping channel?”

4. The Push-Poll Question

Despite the previous question allowing for alternative responses, this question prompts the participant towards an affirmative response. An excellent example of push-polling.

5. The Abstract Question

We’re more than half way through the survey inviting community feedback to inform the environment impact statement. Still no mention of the environment in the survey, but at least this question does provide an opportunity for the participant to make mention of that, I guess. However, the question is a bit abstract – a random trawl for information that seems unlikely to yield much meaningful data to inform the environment impact statement.

6. Mailing List Question Number One

This question has nothing to do with the environment impact statement, but offers a “Yes” or “No” choice to joining the Ports North mailing list.

7. Mailing List Question Number Two

It does not matter what the response to the previous question is, the survey asks for contact details anyway.

8. The Demographic Question (Maybe)

I guess this question aims to capture whether the research participants are part of the Cairns community or not. It is the final in the survey.

Summary

Ports North have created an online survey that claims to seek community feedback as part of the process of preparing an environmental impact statement. However, none of the questions address environmental issues.

This is an excellent example of how to do dodgy research.

So What?

How does this relate to the meta4RN blog related to nursing/health? Well, there is an argument to be made that links health to the environment and economy, but that’s not the prompt for me. My reason for using this case study is twofold: [1] it relates to something happening in my backyard, and [2] because this survey reminded me of some of the surveys that I have been exposed to in my nursing career. There have quite a few examples of “research”, “evaluation” or “satisfaction” surveys that predetermine the answers by limiting/skewing the questions.

In a related tangent, there are plenty of examples of quackery that are passed-off as health research/interventions that can confuse us and our patients. It is fitting for nurses, midwives and other health professionals to be vigilant to signs of dodgy research, as described in the sci-ence.org comic “The Red Flags of Quackery v2.0”:

End

As always, comments are welcome.

Paul McNamara, 20th July 2014

P.S.

*Explanation re choosing an example to mock that is not directly related to health/nursing: I have a mortgage and don’t feel very secure. Call me “chicken”, but that’s they way it is at the moment. Cluck! Cluck!