Is the term ‘approach significance’ cheating?

For those of you who know me well, know this a topic that I am able to argue for hours and hours, however, in the interest of being a good scientist I shall look at this from both sides. But overall, I feel that approaching significance is a term that should not be used in scientific works.

According to Cumming (2010)* research is considered significant if p<.05, and seen as approaching significance if p<.10, however, he also pointed out that where possible exact p-values should be reported, as it gives the reader the ability to judge the significance of a result themselves. Other sources** agree with this and advise people writing up results that p-values that are greater than .05 but less than .10 should be discussed, and are seen as approaching significant, and an effect could be present.

I can completely understand this logic. I am a relativity logical person, and what is being suggested makes sense. Researchers spend a very very long time planning research and conducting research, and then once the analysis of the data comes through and your significance level is something like p=0.057, you would be very frustrated. Your results would be telling you that chance does play a part in results you have found, however a p value between .05 and .10 would suggest that chance is only a small part of this result.

Furthermore, there could be a number of reasons affecting the significance levels of your study, and maybe if one element was different then significance would be found. For example, your sample might not be truly representative of the population, and maybe if you had chosen a more representative population then significance may have been found. But this idea then brings in the ethics of manipulating the data and then adding participants to try and find an effect.

On the other hand, the term approaching significance suggests that there is a chance that the results found might not be as due to chance as the alpha level suggests. Now this means that people may conclude that an effect might possibly be there, creating a type one error, where an effect is found/thought to be there, when it is actually not. This is a big enough issues when talking about things that are significant, without bringing in approaching significance.

So, to conclude, I feel that yes using the term approaching significance is cheating, because the research is claiming an effect may be there if further investigation is undertaken, when the stats you have been give suggests otherwise!

* http://www.stat.auckland.ac.nz/~iase/publications/icots8/ICOTS8_8J4_CUMMING.pdf
** http://psych.hanover.edu/classes/psy220/resultdisc.htm

Advertisements

5 responses to “Is the term ‘approach significance’ cheating?

  1. I agree with what you have said, the term “approaching significance” should not be used. Where would you stop if you aloud .06 to be almost significant? If .06 was almost significant then 0.7 surely would as well, and so on. I understand that researchers may want to alter their findings due to publication bias, where non-significant results are rarely published. I know that some researcher have to work to a deadline to produce results. However, I do not see this as a valid excuse to word things differently (approaching significance) or altering the data you have just to find what you are looking for (keep adding more participants).

  2. Pingback: Last comments for my TA :) « psyalo

  3. Yeah I completely agree with what your saying (you have convinced me many a time with this rant so :]). I believe that approaching significance is mentioned it should only be in relation to the fact that significance was not found in this but for future research it may be an idea to do different control variables etc as other wise it is essentially just manipulating the data. This could just be done by the researcher to show that there work was not a waste of time and so need an ego boost or in the hope that further research will prove what they wanted initially but failed to achieve. If data is reported as approaching significance it is not representative to the public which will decrease their trust in research further especially if their basic stat knowledge does not know what to do with ‘approaching significance’.

  4. `yes completely agree with what your saying if thev alpha level was between o.5 and .10 it would be difficult to thell the exact significance of the data. i am glad that in such a varied subject area there is a stable enntity which identifies the significance of the result we produce. i find significant levels alot more helpfull than percentages in advertising which can be misconstrewed due to tha sample population; thus lessoning the strength of the percentage.
    x

  5. Pingback: final blog comments « psych31

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s