Original Article
Evaluating the Impact of EBP Education:
Development of a Modified Fresno Test for
Acute Care Nursing
Margo A. Halm, PhD, RN, NEA-BC
Keywords
modified Fresno,
EBP education/
competencies,
acute care nursing,
novice-to-expert,
psychometrics
ABSTRACT
Background: Proficiency in evidence-based practice (EBP) is essential for relevant research find-
ings to be integrated into clinical care when congruent with patient preferences. Few valid and
reliable tools are available to evaluate the effectiveness of educational programs in advancing
EBP attitudes, knowledge, skills, or behaviors, and ongoing competency. The Fresno test is one
objective method to evaluate EBP knowledge and skills; however, the original and modified
versions were validated with family physicians, physical therapists, and speech and language
therapists.
Aims: To adapt the Modified Fresno-Acute Care Nursing test and develop a psychometrically
sound tool for use in academic and practice settings.
Methods: In Phase 1, modified Fresno (Tilson, 2010) items were adapted for acute care nursing.
In Phase 2, content validity was established with an expert panel. Content validity indices (I-CVI)
ranged from .75 to 1.0. Scale CVI was .95%. A cross-sectional convenience sample of acute care
nurses (n = 90) in novice, master, and expert cohorts completed the Modified Fresno-Acute Care
Nursing test administered electronically via SurveyMonkey.
Findings: Total scores were significantly different between training levels (p < .0001). Novice
nurses scored significantly lower than master or expert nurses, but differences were not found
between the latter cohorts. Total score reliability was acceptable: (interrater [ICC (2, 1)]) =
.88. Cronbach’s alpha was 0.70. Psychometric properties of most modified items were satis-
factory; however, six require further revision and testing to meet acceptable standards.
Linking Evidence to Action: The Modified Fresno-Acute Care Nursing test is a 14-item test for
objectively assessing EBP knowledge and skills of acute care nurses. While preliminary psycho-
metric properties for this new EBP knowledge measure for acute care nursing are promising,
further validation of some of the items and scoring rubric is needed.
INTRODUCTION
Over a decade ago, the Institute of Medicine (Institute of
Medicine [IOM], 2001) recognized evidence-based practice
EBP as a key solution to ensure care delivered has the high-
est clinical effectiveness known to science. To reach the IOM’s
(2007, p. ix) 2020 goal that “90% of clinical decisions will be
supported by accurate, timely and up-to-date clinical informa-
tion that reflects the best available evidence,” nurses need EBP
competencies to guarantee that relevant research findings are
integrated into clinical situations when congruent with patient
preferences (Melnyk, Gallagher-Ford, Long, Long, & Fineout-
Overholt, 2014).
BACKGROUND
A recent evidence synthesis reported 10 studies evaluating
the effectiveness of educational interventions in building EBP
attitudes, knowledge, skills, and behaviors of nurses (Halm,
2014). Interventions were primarily workshop or immersion
programs, but seminars, journal clubs, and EBP and research
councils were also evaluated via: (a) self-reported EBP attitude,
knowledge, and behavior (Chang et al., 2013; Dizon, Somers, &
Kumar, 2012; Edward & Mills, 2013; Leung, Trevana, & Waters,
2014); (b) PICO questions and activity diaries (Dizon et al.,
2012); (c) Edmonton Research Orientation (Gardner, Smyth,
Renison, Cann, & Vicary, 2012) and Clinical Effectiveness or
EBP Questionnaire (Sciarra, 2011; Toole, Stichler, Ecoff, &
Kath, 2013; White-Williams et al., 2013); and (d) interviews and
focus groups to identify qualitative themes about nurses’ expe-
rience in EBP programs (Balakas, Sparks, Steurer, & Bryant,
2013; Nesbitt, 2013; Wendler, Samuelson, Taft, & Eldridge,
2011). Varied measurement across studies limited estimation
of the effectiveness of EBP training (Dizon et al., 2012).
In a systematic review, Shaneyfelt et al. (2006) rec-
ommended valid and responsive methods to evaluate the
programmatic impact of EBP education and progression in
272 Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
CE
http://crossmark.crossref.org/dialog/?doi=10.1111%2Fwvn.12291&domain=pdf&date_stamp=2018-05-14
EBP competencies. As self-report is extremely biased (Lai
& Teng, 2011; Shaneyfelt et al., 2006); objective knowledge
tests that incorporate multiple-choice or short answers with
case-based decision-making like the Berlin Questionnaire
(Fritsche, Greenhalgh, Falck-Ytter, Neumayer, & Kunz, 2002)
or Fresno test were recommended to evaluate EBP knowledge
and skills (Shaneyfelt et al., 2006). The Fresno test, a valid and
reliable method to evaluate EBP knowledge and skills using
a standardized scoring rubric, has been validated with family
physicians (Ramos et al., 2003), physical therapy (Miller,
Cummings, & Tomlinson, 2013; Tilson, 2010), and speech
language (Spek, de Wolf, van Dijk, & Lucas, 2012).
SPECIFIC AIMS
As objective methods for assessing EBP knowledge and skills
of nurses are lacking, the specific aim of this study was to fill a
measurement gap by adapting the modified Fresno test (Tilson,
2010) for acute care nursing. Only with consistent use of psy-
chometrically sound methods can useful evidence be generated
about the effectiveness of various EBP teaching strategies—
new knowledge that can direct effective educational and pro-
fessional development programs for students and practicing
nurses. The specific research question was: Will an adapted
Fresno test discriminate EBP knowledge and skills between
novice, master, and expert acute care nurses?
METHODS
Research Design
A cross-sectional cohort design was used to replicate Tilson’s
(2010) modified Fresno test (Figure 1).
Phase I: Test adaptation. New scenarios on acute care nurs-
ing were developed for items #1–8 that remained unchanged.
Item #9 (clinical expertise) was retained despite removal due to
poor psychometric performance by Tilson (2010). Items #10–13
were modified for acute care although the EBP focus was un-
changed. Item #14 was modified to the best design for studying
the meaning of experience.
Phase 2: Content validity. Content validity was established
with a panel of four masters and doctorally prepared acute care
EBP experts from practice and academic settings. In round
one, panelists rated each item and rubric for clarity, impor-
tance, and comprehensiveness on a 5-point Likert scale. Pan-
elists provided feedback on whether items should be retained,
revised, dropped, or added (Polit & Beck, 2012). In round two,
items #10 (mathematical calculations for sensitivity, positive
predictive value) and #11 (relative and absolute risk reduction)
were replaced because the panel did not believe acute care
nurses would be expected to make these calculations without
a resource. These items were replaced (and reviewed) with
assessing tool reliability/validity and applying qualitative find-
ings. The scoring rubric (Figure S1) was modified to reflect item
alterations and ensure scoring consistency across subjects and
raters (Jonsson & Svingby, 2007). With a single overall score,
Figure 1. Study flowchart.
a passing score was defined as >50% of available points for in-
dividual items (Tilson, 2010). This passing score was set lower
than that defined as “mastery of material” (Ramos, Schafer, &
Tracz, 2003) to reduce the risk of a floor effect with novices.
A content validity index (I-CVI) was calculated for individ-
ual items by dividing the number of 4–5 ratings by the number
of experts. Mean (M) item ratings were 4.54 (clarity), 4.82 (im-
portance), and 4.75 (comprehensiveness). Only item 12 had an
I-CVI value <0.78 because the panel rated interpreting con-
fidence intervals lower on importance for acute care nurses.
The scale CVI of .95% was calculated by averaging I-CVIs,
exceeding acceptable standards of >.90 (Polit & Beck, 2007;
Table 1).
Phase 3: Validation of modified Fresno. After Institu-
tional Review Board exemption was obtained, invitations were
emailed to three cohorts: (a) novice nurses (less than 2 years of
Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
273
Original Article
Table 1. Modified Fresno Test Items (n = 90)
Scores
Item/EBPstepor
component Topic
Content
validity index
(I-CVI)
Possible
score
Passing
score
Novices
(n = 30)
M (SD)
Masters
(n = 30)
M (SD)
Experts
(n = 30)
M (SD) p value*
1 INQUIRE PICOquestion .92 0–24 >12 13.73 (7.37) 19.47 (3.71) 18.13 (4.55) .001 (N-M,N-E)
2ACQUIRE Sources 1.0 0–24 >12 15.03 (6.53) 20.33 (5.09) 17.53 (6.05) .004 (N-M)
3APPRAISE Treatment
design
1.0 0–24 >12 5.80 (6.77) 10.50 (6.90) 11.90 (5.87) .001 (N-M,N-E)
4ACQUIRE Search .92 0–24 >12 13.93 (5.06) 16.53 (4.69) 15.10 (4.69) .18
5APPRAISE Relevance .92 0–24 >12 7.47 (6.31) 9.77 (6.83) 12.03 (6.72) .03 (N-E)
6APPRAISE Validity .92 0–24 >12 7.30 (6.75) 10.67 (7.77) 10.23 (7.38) .16
7APPRAISE Significance 1.0 0–24 >12 3.40 (3.94) 9.97 (8.18) 7.70 (7.03) .001 (N-M,N-E)
8PATIENT
PREFERENCES
Patient
preference
1.0 0–16 >8 6.13 (4.36) 8.20 (5.59) 9.00 (4.95) .08
9CLINICAL
EXPERTISE
Clinical
expertise
1.0 0–8 >4 4.80 (3.04) 5.60 (2.49) 6.40 (2.49) .08
10APPLY Tools .92 0–12 >6 3.90 (4.18) 8.50 (3.35) 7.00 (4.12) .001 (N-M,N-E)
11APPLY Qualitative 1.0 0–16 >8 12.13 (4.75) 10.93 (5.35) 12.53 (6.19) .50
12APPRAISE Confidence
intervals
.75 0–4 >2 .13 (.73) .40 (1.22) 1.07 (1.80) .02 (N-E)
13APPRAISE Design
diagnosis
1.0 0–4 >2 .27 (1.01) .27 (1.01) .27 (1.01) 1.00
14APPRAISE Design
meaning
1.0 0–4 >2 2.13 (2.03) 3.73 (1.01) 3.87 (.73) .001 (N-M,N-E)
Total scores .95ScaleCVI 0–232 >116 96.17 (26.14) 134.87 (30.76) 132.77 (28.94) .001 (N-M,N-E)
*Scheffe post-hoc analysis: N = Novices;M = Masters; E = Experts.
experience after graduation from a bachelorette program) from
three U.S. Magnet hospitals; (b) master nurses (master’s pre-
pared) recruited via the National Association of Clinical Nurse
Specialists listserv; and (c) expert nurses (doctorally prepared)
recruited via the American Nurses Credentialing Corporation’s
Magnet program director’s listserv and faculty at Bethel Uni-
versity (St. Paul, MN, USA). Nurses in the expert cohort self-
affirmed their EBP expertise and teaching experience. Up to
1 hr (in one sitting) was allowed to complete the test with no
external resources; only notepaper and calculators were per-
mitted. Reminder e-mails were sent at 2 and 4 weeks. A $10
gift certificate incentive was offered upon completion. Some
participants did not answer all the items on the exam; these
participants were not included in the sample for each cohort.
Only participants who had a complete exam were included in
the analysis. Data were collected in 2015.
Two doctorally prepared nurses with expertise teaching EBP
served as raters after an orientation to the test items and scor-
ing rubric. Raters practiced scoring three pilot tests from the
three cohorts and resolved discrepancies that could threaten in-
terrater reliability (IRR; e.g., halo effect, leniency or stringency,
central tendency errors; Castorr et al., 1990; before scoring
commenced. A midway refresher session allowed raters to re-
view scores, reducing the threat of rater drift (Castorr et al.,
1990). Data were analyzed with SPSS Version 23.0 (IBM Corp.,
Armonk, NY, USA).
RESULTS
Descriptive Statistics
The total sample of 90 nurses included cohort (a) new grad-
uates (n = 30); (b) master’s prepared CNSs (n = 30); and
(c) doctorally prepared nurses (n = 30). Seventy-six percent
completed the test within 60 min (83% novices, 70% mas-
ters, 73% experts). Mean min for test completion were 56.43
Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
274
Development of a Modifi ed Fresno Test for Acute Care Nursing
Table 2. Psychometric Properties of Individual Items (n = 90)
%Passedbycohort
Item# Topic ICC IDI ITC
All
(n = 90)
Novices
(n = 30)
Masters
(n = 30)
Experts
(n = 30) χ2 p-value
1 PICOquestion .78 .43 .53 85.6 63.3 100.0 93.3 18.52 .0001
2 Sources .78 .35 .53 84.4 73.3 93.3 86.7 4.74 .09
3 Treatmentdesign .86 .61 .56 44.4 26.7 50.0 56.7 6.03 .05
4 Search .72 .26 .48 80.0 76.7 86.7 76.7 1.25 .54
5 Relevance .48 .65 .63 35.6 26.7 33.3 46.7 2.72 .26
6 Validity .47 .43 .50 32.2 20.0 43.3 33.3 3.76 .15
7 Significance .74 .52 .57 26.7 6.7 40.0 33.3 9.55 .01
8 Patient
preference
.55 .52 .39 52.2 36.7 50.0 70.0 6.77 .03
9 Clinical expertise .23 .22 .40 88.9 80.0 93.3 93.3 3.60 .17
10 Tools .76 .74 .68 68.9 40.0 90.0 76.7 18.77 <.0001
11 Qualitative .68 .17 .31 88.9 93.3 90.0 83.3 1.58 .46
12 Confidence
intervals
.90 .04 .12 13.3 3.3 10.0 26.7 7.50 .02
13 Designdiagnosis .61 .13 .12 6.7 6.7 6.7 6.7 .00 1.0000
14 Designmeaning .89 .35 .37 81.1 53.3 93.3 96.7 22.77 <.0001
Total score .88 N/A N/A .0001
(standard deviation [SD] 38.21) for novices; 57.20 (SD 42.54)
for masters; and 43.21 (SD 26.33) for experts.
Reliability Statistics
IRR was calculated using intraclass correlation coefficients
(ICC) for total score and individual items (Table 2). Total score
reliability was high at .88. Of the 14 items, 3 had excellent
reliability (>.80), 7 had moderate reliability (.60–.79), and 4
had questionable reliability (<.60). Items with questionable
IRR focused on relevance (#5), validity (#6), patient preference
(#8), and clinical expertise (#9). A Cronbach’s alpha coefficient
of .70 was obtained for internal consistency of the modified
exam.
Item discrimination index (IDI) was calculated for each item
by separating total scores into quartiles and subtracting the pro-
portion of nurses in the bottom quartile who passed that item
(>50% points per item was passing) from the proportion in the
top quartile who passed the same item. The 50% threshold has
been defined as “mastery of material” (Ramos et al., 2003) and
used in similar validation studies (Tilson, 2010). IDI ranges
from –1.0 to 1.0, representing the difference in passing rate
between nurses with high (top 25%) and low (bottom 25%)
overall scores. Eleven of the 14 items had acceptable IDIs >.2
(Table 2). Correlation between item and total score and cor-
rected item-total correlation (ITC) was assessed using Pearson
correlation coefficients. Twelve of the 14 items had acceptable
ITCs >.3 (Table 2). Low IDI and ITC items focused on con-
fidence intervals (#12) and design for diagnostic tests (#13).
Qualitative findings (#11) also had a low IDI.
Total Score Analysis
No floor or ceiling effect was apparent, indicating the test is ap-
plicable from novice to expert (Figure 2). As shown in Table 1,
total mean scores for novices (M 96.17, SD 26.14) revealed
that a passing score of 116 was not achieved in this cohort as
with the master (M 134.87, SD 30.76) and expert (M 132.71,
SD 28.94) cohorts. One-way analysis of variance (ANOVA)
demonstrated that overall mean scores were significantly dif-
ferent, F (2, 89) = 17.58, p < .0001, between cohorts. A post-
hoc Scheffe comparison showed novice total mean scores (M
96.17, SD 26.14) differed significantly from master (M 134.87,
SD 30.07, d = 1.36) and expert nurses (M 132.77, SD 28.94,
d = 1.33). Cohen’s d is an effect size measure that is used
to explain the standardized difference between two means,
Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
275
Original Article
Figure 2. Box plots for sum scores.
commonly reported with ANOVAs or t tests. There were no
significant differences between the master and expert cohorts.
Item Score Comparison
Post-hoc Scheffe analysis also revealed significant cohort dif-
ferences in eight items (Table 1). Novice nurses scored sig-
nificantly lower than master and expert nurses on PICO (#1),
sources (#2), treatment design (#3), relevance (#5), significance
(#7), tools (#10), confidence intervals (#12), and design mean-
ing (#14). On the other hand, the mean scores for four items
increased progressively across cohorts from novice to master,
and then from master to expert. These items were treatment
design (#3), relevance (#5), patient preference (#8), and con-
fidence intervals (#12). While not all items performed in this
manner, these items demonstrated mastery of EBP material
across cohorts.
Item Difficulty
Item difficulty (IDI) was calculated via the proportion of nurses
who achieved a passing score for each item (Table 2). Of the
14 items, none were easy (IDI > .8). Ten items (71%) were
moderate (IDI > .3), and 4 (29%) were difficult (IDI < .3;
Janda, 1998; Nunnally & Bernstein, 1994). In testing individual
items, all three cohorts scored below the passing cutoff for five
items: Treatment design (#3), validity (#6), significance (#7),
confidence intervals (#12), and diagnosis design (#13). Novice
and master nurses did not achieve a passing score for relevance
(#5), while novices did not pass patient preferences (#8) and
tools (#10).
Using chi-square analysis, seven items showed significant
differences in the proportion of passing scores between cohorts
(Table 2). Masters scored highest on PICO (#1), significance
(#7), and tools (#10). Experts performed best on treatment de-
sign (#3), design meaning (#14), patient preferences (#8), and
confidence intervals (#12).
In examining item discrimination based on the propor-
tion of nurses who passed the test (Table 2), some significant
items did not discriminate well between masters and experts:
(a) PICO (#1); (b) treatment design (#3); (c) significance (#7);
and (d) design meaning (#14). Items on sources (#3), search
(#4), relevance (#5), validity (#6), and expertise (#9) discrim-
inated on the IDI but did not assess unique EBP knowledge
and skills among the three cohorts (p > .05).
DISCUSSION
The Modified Fresno-Acute Care Nursing test is a 14-item test
for assessing EBP knowledge and skills. While the original
test assessed core principles of EBP steps, this replication val-
idated patient preferences and clinical expertise to fully assess
all EBP domains. The test has excellent content validity with
I-CVIs ranging from .75 to 1.0. Overall scale CVI was .95. In-
ternal consistency was acceptable at .70. Table 3 compares the
psychometric properties of the Modified Fresno-Acute Care
Nursing test with the original and modified tests.
Total scale reliability for the two independent raters was
excellent (.88). IRR for individual items was good to excellent
for 10 of 14 items (71%). One reason IRR may have been lower
for relevance (#5) and validity (#6) was the rubric complexity
that required raters to consider responses for both items when
scoring. Like Tilson (2010), IRR was less than desirable for pa-
tient preference (#8) and clinical expertise (#9). Some leniency
in scoring may have occurred with #8 when a nurse offered a
phrase that could elicit patient preferences, rather than stating
it as a question as specified in the rubric. As recommended by
Tilson (2010), clinical expertise should be retained as it covers
an essential EBP domain, but further revision and validation is
needed.
Item difficulty was moderate to high. Two items retained
from Tilson’s (2010) version had low IDI and ITC: Confidence
intervals (#12) and design for diagnosis (#13). These items were
difficult across cohorts and did not discriminate. Of the new
items, tools (#10) had acceptable psychometrics across ICC,
IDI, and ITC. The second qualitative item (#11) had accept-
able ICC and ITC but low IDI and did not discriminate across
cohorts. This finding may demonstrate that qualitative find-
ings have a rich tradition of emphasis across levels of nursing
education and practice.
While some items did not perform ideally, these items re-
main valuable to the larger research goal of developing an
objective and responsive method to evaluate EBP knowledge
and skills. Reasons for poor item performance may include
item characteristics, unknown sample characteristics, scoring
concerns, or a combination of these factors. Six items (#5, #6,
#9, #11, #12, and #13) need to be revised and retested before be-
ing removed. Although Tilson (2010) dropped clinical expertise
(#9), it covers an important EBP domain that other researchers
recognized as essential for measurement (Miller et al., 2013).
A range in item difficulty is best so that the high and low
range of ability can be evaluated. For item #12 (confidence
Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
276
Development of a Modifi ed Fresno Test for Acute Care Nursing
Table 3. Comparison of Reliability and Validity of Fresno Tests
Performance
Measure/acceptable
results
Original Fresno (Ramos
et al., 2003)
DutchadaptedFresno
(Speket al., 2012)
ModifiedFresno-physical
therapy (Tilson, 2010)
ModifiedFresno-AcuteCare
Nursing test (Halm, 2018
current study)
Population � Familyphysicians � Speech
language, clinical
epidemiology
students
� Physical therapy � Acute care nurses
Total score/# items � 212/12 � 212/12 � 224/13 � 232/14
Content validity
� ScaleCVI/>.90 � Not reported � .92 � Not reported � .95
Interrater reliability
� Interrater
correlation/
�
>.60
� Items: .72–.96
� Total score: .97
� Not reported
� Total score: .99
� Items: .41–.99
� Total score: .91
� Items: .23–.90
� Total score: .88
Internal reliability
� Cronbach’s/>.70
� Item-total
correlation
(ITCs)/>.30
� .88
� .47–.75 (items)
� .83
� .31–.76
� .78
� .20–.66
� .70
� .12–.68
Itemdiscrimination
� Item
discrimination
index (IDI)/>.20
� .41–.86; no items
hadweakor
negative
discrimination
� Not reported � .25–.68; no items
hadweakor
negative
discrimination
� .04–.74; 3 itemshad
weakdiscrimination
Construct validity
� Comparisonof
meancohort
scores
� Novice = 95.6+
� Expert = 147.5;
morepassedall
items (p < .05)
� Year 1 students
= 26.3*
� Year 2 students
= 69.3*
� Year 3 students
= 89.1*
� Masters students
= 154.2*
� Novice = 92.8
� Trained = 118.5
� Expert = 149.0++;
morepassed 11
items
(p < .03–.01)
� Novices = 96.17++
� Masters = 134.87;
morepassed3 items
(p < .01–.0001)
� Experts = 132.77;
morepassed4 items
(p < .01–.0001)
*p < .05; +p < .001; ++ p < .0001.
intervals), the IDI was low, most likely due to the low base
success rate; however, it did discriminate the high end of EBP
knowledge among cohorts. This item replaced a mathemati-
cal calculation and should be retained because of the growing
importance of understanding confidence intervals, although it
may need to be revised. Similarly, item #13 (design diagnosis)
was difficult. This item should be retained but reworded to in-
crease clarity that it is referring to selection and interpretation
of diagnostic tests.
Item #14 (design meaning) may have been too easy. This
item should be retained but reworded, so it is more difficult.
Since item #11 was labeled qualitative, it may have primed
nurses, and so item #14 (design meaning) should be moved
earlier in the test. Based on ITC performance, the rubric for
item #11 (qualitative) needs to be more difficult, requiring
more specific or unusually helpful or insightful advice to better
differentiate between a best possible (16 points) answer versus
a more limited (8 points) answer.
No floor or ceiling effects were evident, indicating that EBP
knowledge and skills, and not clinical experience, influenced
mean score differences (Tilson, 2010). Mastery of EBP material
was evident from novice to expert nurses on four items. The
Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
277
Original Article
Table 4. Uses of the Modified Fresno-Acute Care Nursing Test
Self-assessment Pre–post assessment
Academic settings 1. Students could use individual itemsand
scoring rubric asaguidewhen learningeach
EBPstep/component
2. Educators couldperiodically take the test
before andafter teachingEBPcourses to
identify areas for continual learning to
advance levels of EBPexpertise
1. Faculty could usepre–post scores to evaluate EBP
education in academicprograms (BSN,MSN,DNP,
PhD). Test scores could assist curriculum
design/redesign, andassessment of thequality/
rigor of course content, teaching styles, and
methods
2. Objective test scores could showhowstudent
outcomesare improving, data that canbeused for
accreditationpurposes
Acute care settings 1. Clinical/advancedpractice nurses canuse
individual itemsand scoring rubric as a
guide for learning eachEBP
step/component
2. Clinical nurses could take the test to assess
EBPstrengths andareas for improvement
before attendingEBPeducational activities
(Ramoset al., 2003)
1. Acute care educators and researchers could use
pre–post scores to evaluate EBPeducation for
clinical nurses
� Identifiedgapswould informneeds for
orientation/ongoing staff development
opportunities that advanceEBPcompetencies
2. Scores could be tracked tomonitor EBP
knowledge/skill progressionof nurses in attaining
higher levels of EBPcompetency. A 10%change is
meaningful in evaluating improvement in EBPskills
over time (McCluskey&Bishop, 2009)
� EBPknowledge/skills could beassessed for new
hires, existing nurses, aswell asmembers of
journal clubs, EBP/researchandpolicy/
procedure committees responsible for revising
policies/procedures/protocols/guidelines based
onbest available evidence
ability of the test to differentiate between novice nurses and
masters or experts was high but not across all three cohorts.
Historical threats to validity may be one explanation. As an
evolving concept, some nurses may not have had similar ex-
posure to EBP in doctoral education. Interestingly, acute care
nurses had longer times to completion (M 56.43, SD 38.21 for
novices; M 57.20, SD 42.54 for masters; M 43.21, SD 26.33
for experts) than those reported by Tilson (M 33.2, SD 8.7 for
novices; M 34.8, SD 10.0 for masters; M 40.5, SD 15.5 for ex-
perts). These differences may be due to the sample or changes
in the Fresno test.
EVIDENCE TO ACTION
The findings from this sample suggest EBP topics need re-
inforcement with acute care nurses in academic and practice
settings. Acute care nurses at all levels would benefit from
more education on appropriateness of designs for different
research questions, as well as assessment of validity, clinical
and statistical significance, and confidence intervals. Novice
nurses need more guidance in assessing patient preferences
and applicability of tools for practice. Both novice and master
nurses need more education on assessing study relevance. Ar-
eas for EBP education or reinstruction should align with the
national EBP competencies developed by Melnyk et al. (2014)
for clinical and advanced practice nurses. These competencies
provide the road map for expected levels of EBP in the clinical
setting.
Scores derived from the Modified Fresno-Acute Care Nurs-
ing test have many uses in both the academic and prac-
tice setting. As described in Table 4, the test and scoring
rubric can be used as self-study and assessment guides. While
test scores could be used in a pre–post fashion to docu-
ment the impact of educational programs in advancing EBP
knowledge and skills and competencies of acute care nurses,
the Modified Fresno-Acute Care Nursing test needs to un-
dergo further validation before such use occurs in practice or
academia.
LIMITATIONS
The first limitation is the lack of demographic information
for this small U.S. sample. Length of time since graduation
and years of EBP experience were not captured and may have
Worldviews on Evidence-Based Nursing, 2018; 15:4, 272–280.
© 2018 Sigma Theta Tau International
278
Development of a Modifi ed Fresno Test for Acute Care Nursing
influenced performance in the test. The sample of doctorally
prepared nurses who were recruited as EBP experts is a further
limitation because the test did not differentiate well between
experts and masters. Experts spent on average 13 min less time
to complete the test and thus, may not have thoroughly docu-
mented their EBP knowledge. The scores obtained in these
sample cohorts are not generalizable globally to acute care
nurses because the emphasis and amount of EBP education
may differ in general and across levels of nursing education in
developing or developed countries (Ciliska, 2005; Deng, 2015;
Holland & Magama, 2017).
Secondly, the scoring rubric is complex. Raters need EBP
experience and training to ensure reliable use of the rubric.
Pilot testing with opportunities to clarify scoring procedures is
essential for IRR. At least 10–15 min per test should be allocated
(Ramos et al., 2003; Tilson, 2010). This scoring time could be
a limitation if an educator or researcher desires an easy assess-
ment to evaluate …
Delivering a high-quality product at a reasonable price is not enough anymore.
That’s why we have developed 5 beneficial guarantees that will make your experience with our service enjoyable, easy, and safe.
You have to be 100% sure of the quality of your product to give a money-back guarantee. This describes us perfectly. Make sure that this guarantee is totally transparent.
Read moreEach paper is composed from scratch, according to your instructions. It is then checked by our plagiarism-detection software. There is no gap where plagiarism could squeeze in.
Read moreThanks to our free revisions, there is no way for you to be unsatisfied. We will work on your paper until you are completely happy with the result.
Read moreYour email is safe, as we store it according to international data protection rules. Your bank details are secure, as we use only reliable payment systems.
Read moreBy sending us your money, you buy the service we provide. Check out our terms and conditions if you prefer business talks to be laid out in official language.
Read more