BiasinResearch.pdf

Biochemia Medica 2013;23(1):12–5 http://dx.doi.org/10.11613/BM.2013.003

12

Abstract

By writing scientifi c articles we communicate science among colleagues and peers. By doing this, it is our responsibility to adhere to some basic
principles like transparency and accuracy. Authors, journal editors and reviewers need to be concerned about the quality of the work submitted for
publication and ensure that only studies which have been designed, conducted and reported in a transparent way, honestly and without any de-
viation from the truth get to be published. Any such trend or deviation from the truth in data collection, analysis, interpretation and publication is
called bias. Bias in research can occur either intentionally or unintentionally. Bias causes false conclusions and is potentially misleading. Therefore, it
is immoral and unethical to conduct biased research. Every scientist should thus be aware of all potential sources of bias and undertake all possible
actions to reduce or minimize the deviation from the truth. This article describes some basic issues related to bias in research.
Key words: bias; sampling errors; research design

Received: 10 December, 2012 Accepted: 10 January, 2013

Bias in research

Ana-Maria Šimundić

University Department of Chemistry, University Hospital Center “Sestre Milosrdnice”, Zagreb, Croatia

*Corresponding author: [email protected]

Lessons in biostatistics

Introduction

Scientifi c papers are tools for communicating sci-
ence between colleagues and peers. Every re-
search needs to be designed, conducted and re-
ported in a transparent way, honestly and without
any deviation from the truth. Research which is not
compliant with those basic principles is mislead-
ing. Such studies create distorted impressions and
false conclusions and thus can cause wrong medi-
cal decisions, harm to the patient as well as sub-
stantial fi nancial losses. This article provides the in-
sight into the ways of recognizing sources of bias
and avoiding bias in research.

Defi nition of bias

Bias is any trend or deviation from the truth in data
collection, data analysis, interpretation and publi-
cation which can cause false conclusions. Bias can
occur either intentionally or unintentionally (1). In-
tention to introduce bias into someone’s research
is immoral. Nevertheless, considering the possible
consequences of a biased research, it is almost

equally irresponsible to conduct and publish a bi-
ased research unintentionally.

It is worth pointing out that every study has its
confounding variables and limitations. Confound-
ing eff ect cannot be completely avoided. Every
scientist should therefore be aware of all potential
sources of bias and undertake all possible actions
to reduce and minimize the deviation from the
truth. If deviation is still present, authors should
confess it in their articles by declaring the known
limitations of their work.

It is also the responsibility of editors and reviewers
to detect any potential bias. If such bias exists, it is
up to the editor to decide whether the bias has an
important eff ect on the study conclusions. If that
is the case, such articles need to be rejected for
publication, because its conclusions are not valid.

Bias in data collection

Population consists of all individuals with a charac-
teristic of interest. Since, studying a population is

http://dx.doi.org/10.11613/BM.2013.003 Biochemia Medica 2013;23(1):12–5

13

Simundic AM. Bias in research

quite often impossible due to the limited time and
money; we usually study a phenomenon of inter-
est in a representative sample. By doing this, we
hope that what we have learned from a sample
can be generalized to the entire population (2). To
be able to do so, a sample needs to be representa-
tive of the population. If this is not the case, con-
clusions will not be generalizable, i.e. the study will
not have the external validity.

So, sampling is a crucial step for every research.
While collecting data for research, there are nu-
merous ways by which researchers can introduce
bias in the study. If, for example, during patient re-
cruitment, some patients are less or more likely to
enter the study than others, such sample would
not be representative of the population in which
this research is done. In that case, these subjects
who are less likely to enter the study will be under-
represented and those who are more likely to en-
ter the study will be over-represented relative to
others in the general population, to which conclu-
sions of the study are to be applied to. This is what
we call a selection bias. To ensure that a sample is
representative of a population, sampling should
be random, i.e. every subject needs to have equal
probability to be included in the study. It should be
noted that sampling bias can also occur if sample is
too small to represent the target population (3).

For example, if the aim of the study is to assess the
average hsCRP (high sensitive C-reactive protein)
concentration in healthy population in Croatia, the
way to go would be to recruit healthy individuals
from a general population during their regular an-
nual health check up. On the other hand, a biased
study would be one which recruits only volunteer
blood donors because healthy blood donors are
usually individuals who feel themselves healthy
and who are not suff ering from any condition or
illness which might cause changes in hsCRP con-
centration. By recruiting only healthy blood do-
nors we might conclude that hsCRP is much lower
that it really is. This is a kind of sampling bias,
which we call a volunteer bias.

Another example for volunteer bias occurs by in-
viting colleagues from a laboratory or clinical de-
partment to participate in the study on some new

marker for anemia. It is very likely that such study
would preferentially include those participants
who might suspect to be anemic and are curious
to learn it from this new test. This way, anemic in-
dividuals might be over-represented. A research
would then be biased and it would not allow gen-
eralization of conclusions to the rest of the popu-
lation.

Generally speaking, whenever cross-sectional or
case control studies are done exclusively in hospi-
tal settings, there is a good chance that such study
will be biased. This is called admission bias. Bias
exists because the population studied does not re-
fl ect the general population.

Another example of sampling bias is the so called
survivor bias which usually occurs in cross-section-
al studies. If a study is aimed to assess the associa-
tion of altered KLK6 (human Kallikrein-6) expres-
sion with a 10 year incidence of Alzheimer’s dis-
ease, subjects who died before the study end
point might be missed from the study.

Misclassifi cation bias is a kind of sampling bias
which occurs when a disease of interest is poorly
defi ned, when there is no gold standard for diag-
nosis of the disease or when a disease might not
be easy detectable. This way some subjects are
falsely classifi ed as cases or controls whereas they
should have been in another group. Let us say that
a researcher wants to study the accuracy of a new
test for an early detection of the prostate cancer in
asymptomatic men. Due to absence of a reliable
test for the early prostate cancer detection, there
is a chance that some early prostate cancer cases
would go misclassifi ed as disease-free causing the
under- or over-estimation of the accuracy of this
new marker.

As a general rule, a research question needs to be
considered with much attention and all eff orts
should be made to ensure that a sample is as close-
ly matched to the population, as possible.

Bias in data analysis

A researcher can introduce bias in data analysis by
analyzing data in a way which gives preference to
the conclusions in favor of research hypothesis.

Biochemia Medica 2013;23(1):12–5 http://dx.doi.org/10.11613/BM.2013.003

14

Simundic AM. Bias in research

There are various opportunities by which bias can
be introduced during data analysis, such as by fab-
ricating, abusing or manipulating the data. Some
examples are:

reporting non-existing data from experiments •
which were never done (data fabrication);
eliminating data which do not support your hy-•
pothesis (outliers, or even whole subgroups);
using inappropriate statistical tests to test your •
data;
performing multiple testing (“fi shing for P”) by •
pair-wise comparisons (4), testing multiple end-
points and performing secondary or subgroup
analyses, which were not part of the original
plan in “to fi nd” statistically signifi cant
diff erence regardless to hypothesis.

For example, if the study aim is to show that one
biomarker is associated with another in a group of
patients, and this association does not prove sig-
nifi cant in a total cohort, researchers may start
“torturing the data” by trying to divide their data
into various subgroups until this association be-
comes statistically signifi cant. If this sub-classifi ca-
tion of a study population was not part of the orig-
inal research hypothesis, such behavior is consid-
ered data manipulation and is neither acceptable
nor ethical. Such studies quite often provide mean-
ingless conclusions such as:

CRP was statistically signifi cant in a subgroup of •
women under 37 years with cholesterol con-
centration > 6.2 mmol/L;
lactate concentration was negatively associated •
with albumin concentration in a subgroup of
male patients with a body mass index in the
lowest quartile and total leukocyte count be-
low 4.00 x 109/L.

Besides being biased, invalid and illogical, those
conclusions are also useless, since they cannot be
generalized to the entire population.

There is a very often quoted saying (attributed to
Ronald Coase, but unpublished to the best of my
knowledge), which says: “If you torture the data
long enough, it will confess to anything”. This ac-
tually means that there is a good chance that sta-
tistical signifi cance will be reached only by increas-

ing the number of hypotheses tested in the work.
The question is then: is this signifi cant diff erence
real or did it occur by pure chance?

Actually, it is well known that if 20 tests are per-
formed on the same data set, at least one Type 1
error (α) is to be expected. Therefore, the number
of hypotheses to be tested in a certain study needs
to determined in advance. If multiple hypotheses
are tested, correction for multiple testing should
be applied or study should be declared as explora-
tory.

Bias in data interpretation

By interpreting the results, one needs to make sure
that proper statistical tests were used, that results
were presented correctly and that data are inter-
preted only if there was a statistical signifi cance of
the observed relationship (5). Otherwise, there
may be some bias in a research.

However, wishful thinking is not rare in scientifi c
research. Some researchers tend to believe so
much in their original hypotheses that they tend
to neglect the original fi ndings and interpret them
in favor of their beliefs. Examples are:

discussing observed diff erences and associa-•
tions even if they are not statistically signifi cant
(the often used expression is “b line signif-
icance”);
discussing diff erences which are statistically •
signifi cant but are not clinically meaningful;
drawing conclusions about the causality, even •
if the study was not designed as an experiment;
drawing conclusions about the values outside •
the range of observed data (extrapolation);
overgeneralization of the study conclusions to •
the entire general population, even if a study
was confi ned to the population subset;
Type I (the expected eff ect is found signifi cant, •
when actually there is none) and type II (the ex-
pected eff ect is not found signifi cant, when it is
actually present) errors (6).

Even if this is done as an honest error or due to the
negligence, it is still considered a serious miscon-
duct.

http://dx.doi.org/10.11613/BM.2013.003 Biochemia Medica 2013;23(1):12–5

15

Simundic AM. Bias in research

Publication bias

Unfortunately, scientifi c journals are much more
likely to accept for publication a study which re-
ports some positive than a study with negative
fi ndings. Such behavior creates false impression in
the literature and may cause long-term conse-
quences to the entire scientifi c community. Also, if
negative results would not have so many diffi cul-
ties to get published, other scientists would not
unnecessarily waste their time and fi nancial re-
sources by re-running the same experiments.

Journal editors are the most responsible for this
phenomenon. Ideally, a study should have equal
opportunity to be published regardless of the na-
ture of its fi ndings, if designed in a proper way,
with valid scientifi c assumptions, well conducted
experiments and adequate data analysis, presen-
tation and conclusions. However, in reality, this is
not the case. To enable publication of studies re-
porting negative fi ndings, several journals have al-
ready been launched, such as Journal of Pharma-
ceutical Negative Results, Journal of Negative Re-
sults in Biomedicine, Journal of Interesting Nega-
tive Results and some other. The aim of such jour-
nals is to counterbalance the ever-increasing pres-
sure in the scientifi c literature to publish only posi-
tive results.

It is our policy at Biochemia Medica to give equal
consideration to submitted articles, regardless to
the nature of its fi ndings.

One sort of publication bias is the so called fund-
ing bias which occurs due to the prevailing number
of studies funded by the same company, related to
the same scientifi c question and supporting the
interests of the sponsoring company. It is abso-
lutely acceptable to receive funding from a com-
pany to perform a research, as long as the study is
run independently and not being infl uenced in
any way by the sponsoring company and as long
as the funding source is declared as a potential
confl ict of interest to the journal editors, reviewers
and readers.

It is the policy of our Journal to demand such dec-
laration from the authors during submission and
to publish this declaration in the published article
(7). By this we believe that scientifi c community is
given an opportunity to judge on the presence of
any potential bias in the published work.

Conclusion

There are many potential sources of bias in re-
search. Bias in research can cause distorted results
and wrong conclusions. Such studies can lead to
unnecessary costs, wrong clinical practice and
they can eventually cause some kind of harm to
the patient. It is therefore the responsibility of all
involved stakeholders in the scientifi c publishing
to ensure that only valid and unbiased research
conducted in a highly professional and competent
manner is published (8).

Potential confl ict of interest
None declared.

References
1. Gardenier JS, Resnik DB. The misuse of statistics: concepts,

tools, and a research agenda. Account Res 2002;9:65-74.
http://dx.doi.org/10.1080/08989620212968.

2. Hren D, Lukić KI. Types of studies, power of study and choi-
ce of test. Acta Med Croatica 2006;60 Suppl 1:47-62.

3. Holmes TH. Ten categories of statistical errors: a guide for
research in endocrinology and metabolism. Am J Physi-
ol Endocrinol Metab 2004;286:495-501. http://dx.doi.
org/10.1152/ajpendo.00484.2003.

4. Dawson-Saunders B, Trapp RG. Reading the medical lite-
rature. In: Basic&Clinical biostatistics. New York – Toronto:
Lange Medical Books/McGraw-Hill; 2004.

5. Simundic AM. Practical recommendations for statistical
analysis and data presentation in Biochemia Medica jour-
nal. Biochem Med 2012;22:15-23.

6. Ilakovac V. Statistical hypothesis testing and some pitfalls.
Biochem Med 2009;19:10-6.

7. Simundic AM. Biochemia Medica introduces the revised
policy on Statement of Confl ict of Interest. Biochem Med
2011;21:104-5.

8. Strasak AM. Statistical errors in medical research – a review
of common pitfalls. Swiss Med Wkly 2007;137:44-9.