Present: Abby
Crocker, Kairn Kelley, Amanda Kennedy, Ben Littenberg, Charlie MacLean, Betty
Rambur, Connie van Eeghen
1.
Start Up: A
virtual meeting for Amanda and Connie, thanks to Charlie’s technical expertise.
2.
Presentation: Betty
Rambur shared a concept paper as a precursor to a grant proposal intended for a
credentialing organization, American
Nurses Credentialing Center, which credentials NPs and hospitals for
Magnet status (FAHC is not a Magnet hospital; Dartmouth and Southwestern are). It is unusual for this organization to provide
research grants; this may be part of a strategy to demonstrate the value of
Magnet hospitals. One requirement of
Magnet hospitals is that nurses must be involved in research. This is very difficult for small
hospitals. The topic is about the unintended consequences
of measuring quality. The grant is
capped at $1.2m for 3 years, with no more than 30% indirects.
a. Three phases: development of perceptual measures of unintended
consequences, measurement by surveys across New England hospitals, and
development of nurse research skills.
i.
Survey population: 7,000
active nurses in Vermont; also will include the other states in the region
ii.
Phase 1: developing the
metrics (in Magnet hospitals) that identify unintended consequences; using
Magnet hospitals only may produce incomplete or biased metrics
iii.
Phase 2: measuring quality
usually results in a drop in quality measures at first, as participants start
seeing issues in quality previously ignored; these measure the harm that was
caused
iv.
Phase 3: develop research
capacity in nurses – how strongly should this section be emphasized? Talk to the grant administrator?
b. Budget considerations:
i.
Supports 10 interns, salaries and traveling,
interaction costs (i.e. maintaining the relationship with the interns
throughout this project)
ii.
Survey administration
c. Recommendations
i.
Write more simply, like the
elevator pitch that started this meeting
ii.
Are there more direct measures (involving primary data
collection or data base analyses) that can be proposed in the concept
paper? Something more than perceptual
surveys, e.g. Treatment with antibiotics – something that can be measured
objectively via EMR. There is a specific
nursing data set (NDQA?), many elements of which are proxies for staffing:
patient falls, pressure ulcers. Note:
accessing primary data in the non-Magnet hospitals could be difficult.
1. Keep
the survey as a metric
2. Consider
metrics connected to the nursing data set
3. Look
at the impact of 30-day mortality rate (due to delayed re-admission)
4. Hospital
acquired infection due to delayed discharge
5. More
immediate indications of harm
6. The
metrics may be the outcome of the development of the student interns
7. Consider
assigning each intern (or 2 interns) a metric for which a protocol is
written. All interns collect on all
metrics.
8. Create
a model that starts with latent factors (the perceptual data) and leads to the
harm metrics (using Reason’s Latent Factor Model in reverse)
iii.
Consider a systematic review as a first step in the
proposal, which can lead to semi-structured points for the focus group to
develop the issues
iv.
Consider requesting a lower level funding; have the
funder rethink the funding priorities
3.
Abby’s Update: Current status of her research project
a.
Three
possible articles, the first is focused on the predictors of neonatal abstinence
syndrome. Each column is a variable,
NAS+ babies, NAS- babies, p value.
Within those columns, Abby has identified the n, missing data, and
standard deviation. Discussion:
i.
State N
overall, and then n for each analysis, and don’t report the “missing data”
number
ii.
Univariate
variables and descriptors were used to create a predictive model. The table Abby developed shows where this
model came from. Variables with high p
values were removed from the model, one at a time. The final logistic regression model included
3 variables:
1.
Total
dose of methadone of mother per day at the time of delivery (only 100 moms were
on methadone) (lower is more protective)
2.
APGAR
score (higher is more protective)
3.
Head
circumference standardized for estimated gestational age (z score) (higher is
more protective)
iii.
This
model is not causal, just predictive.
The three variables are all immediately available at time of birth.
iv.
Consider
breaking the table down into several tables, to make interpretation easier for
the reader.
v.
Abby
also discovered that the babies with earlier birth years (from the beginning of
the study) were more likely to be NAS+.
This may be because reporting became more accurate as nurses got more
comfortable with this assessment. Birth
year doesn’t have predictive value.
vi.
Abby
also calculated the ROC (Receiver Operator Characteristic curve), which is
approximately 67% - so not a great predictive tool. Ben and Abby came up with a calibration
curve, which describes the logistic regression model: for each baby, there is a
predictive value of getting NAS (from 0 – 1).
Each baby gets a value on this scale.
The calibration curve assigns a percentage chance of getting NAS by
groups of scores. The predictive ability
of the curve should go up as the percentages go up. The confidence intervals are wide but the
slope is 45 degrees – so well calibrated, but not precise.
1.
Each
group (NAS+, NAS-, and all other babies) can be studied independently.
2.
Recursive
partition might be appropriate – per past heated discussions.
b.
Discussion
continued on the other tables and figures included in this publication and
segued into the beauty of APGAR scores. If
we use the ROC as a predictive tool, what is the action rule for an infant with
45% probability? This may not be
discriminative in a clinically relevant range.
a.
Nov 1: Kairn: final draft of article on IRR (no Connie,
Ben)
b.
Nov 8: Rodger & Connie: presentation on Strategies
on Implementing BHI
c.
Nov 15:
d.
Nov 22: THANKSGIVING… take the day off J
e.
Nov 29: Prema: draft grant application
f. Future
agenda to consider:
i.
Kairn – review of draft article on IRR
ii.
Ben: budgeting exercise for grant applications; NHANES
– lower female mortality for women taking birth control medications
iii.
Rodger: Mixed methods article; article on Behavior’s
Influence on Medical Conditions (unpublished); drug company funding. Also: discuss design for PCBH clinical and
cost research. Also: Prezi demo.
iv.
Amanda: presentation and interpretation of data in
articles
v.
Christina Cruz, 3rd year FM resident with
questionnaire for mild serotonin withdrawal syndrome
Posts
