Clinical Research Oriented Workshop (CROW) Meeting: Oct 24, 2012

Present: Abby Crocker, Kairn Kelley, Amanda Kennedy, Ben Littenberg, Charlie MacLean, Betty Rambur, Connie van Eeghen

1.                  Start Up: A virtual meeting for Amanda and Connie, thanks to Charlie’s technical expertise.     

2.                  Presentation: Betty Rambur shared a concept paper as a precursor to a grant proposal intended for a credentialing organization, American Nurses Credentialing Center, which credentials NPs and hospitals for Magnet status (FAHC is not a Magnet hospital; Dartmouth and Southwestern are).  It is unusual for this organization to provide research grants; this may be part of a strategy to demonstrate the value of Magnet hospitals.  One requirement of Magnet hospitals is that nurses must be involved in research.  This is very difficult for small hospitals.   The topic is about the unintended consequences of measuring quality.  The grant is capped at $1.2m for 3 years, with no more than 30% indirects.

a.       Three phases: development of perceptual measures of unintended consequences, measurement by surveys across New England hospitals, and development of nurse research skills. 

                                                  i.      Survey population: 7,000 active nurses in Vermont; also will include the other states in the region

                                                ii.      Phase 1: developing the metrics (in Magnet hospitals) that identify unintended consequences; using Magnet hospitals only may produce incomplete or biased metrics

                                              iii.      Phase 2: measuring quality usually results in a drop in quality measures at first, as participants start seeing issues in quality previously ignored; these measure the harm that was caused

                                              iv.      Phase 3: develop research capacity in nurses – how strongly should this section be emphasized?  Talk to the grant administrator?

b.      Budget considerations:

                                                  i.      Supports 10 interns, salaries and traveling, interaction costs (i.e. maintaining the relationship with the interns throughout this project)

                                                ii.      Survey administration

c.       Recommendations

                                                  i.      Write more simply, like the elevator pitch that started this meeting

                                                ii.      Are there more direct measures (involving primary data collection or data base analyses) that can be proposed in the concept paper?  Something more than perceptual surveys, e.g. Treatment with antibiotics – something that can be measured objectively via EMR.  There is a specific nursing data set (NDQA?), many elements of which are proxies for staffing: patient falls, pressure ulcers.  Note: accessing primary data in the non-Magnet hospitals could be difficult.

1.      Keep the survey as a metric

2.      Consider metrics connected to the nursing data set

3.      Look at the impact of 30-day mortality rate (due to delayed re-admission)

4.      Hospital acquired infection due to delayed discharge

5.      More immediate indications of harm

6.      The metrics may be the outcome of the development of the student interns

7.      Consider assigning each intern (or 2 interns) a metric for which a protocol is written.  All interns collect on all metrics.

8.      Create a model that starts with latent factors (the perceptual data) and leads to the harm metrics (using Reason’s Latent Factor Model in reverse)

                                              iii.      Consider a systematic review as a first step in the proposal, which can lead to semi-structured points for the focus group to develop the issues

                                              iv.      Consider requesting a lower level funding; have the funder rethink the funding priorities

3.                  Abby’s Update: Current status of her research project

a.       Three possible articles, the first is focused on the predictors of neonatal abstinence syndrome.  Each column is a variable, NAS+ babies, NAS- babies, p value.  Within those columns, Abby has identified the n, missing data, and standard deviation.  Discussion:

                                                  i.      State N overall, and then n for each analysis, and don’t report the “missing data” number

                                                ii.      Univariate variables and descriptors were used to create a predictive model.  The table Abby developed shows where this model came from.  Variables with high p values were removed from the model, one at a time.  The final logistic regression model included 3 variables:

1.      Total dose of methadone of mother per day at the time of delivery (only 100 moms were on methadone) (lower is more protective)

2.      APGAR score (higher is more protective)

3.      Head circumference standardized for estimated gestational age (z score) (higher is more protective)

                                              iii.      This model is not causal, just predictive.  The three variables are all immediately available at time of birth. 

                                              iv.      Consider breaking the table down into several tables, to make interpretation easier for the reader.

                                                v.      Abby also discovered that the babies with earlier birth years (from the beginning of the study) were more likely to be NAS+.  This may be because reporting became more accurate as nurses got more comfortable with this assessment.  Birth year doesn’t have predictive value. 

                                              vi.      Abby also calculated the ROC (Receiver Operator Characteristic curve), which is approximately 67% - so not a great predictive tool.  Ben and Abby came up with a calibration curve, which describes the logistic regression model: for each baby, there is a predictive value of getting NAS (from 0 – 1).  Each baby gets a value on this scale.  The calibration curve assigns a percentage chance of getting NAS by groups of scores.  The predictive ability of the curve should go up as the percentages go up.  The confidence intervals are wide but the slope is 45 degrees – so well calibrated, but not precise. 

1.      Each group (NAS+, NAS-, and all other babies) can be studied independently.

2.      Recursive partition might be appropriate – per past heated discussions.

b.      Discussion continued on the other tables and figures included in this publication and segued into the beauty of APGAR scores.  If we use the ROC as a predictive tool, what is the action rule for an infant with 45% probability?  This may not be discriminative in a clinically relevant range.

4.                  Next Workshop Meeting(s): Thursday, 2:00 p.m. – 3:30 p.m., at Given Courtyard Level 4. 

a.       Nov 1: Kairn: final draft of article on IRR (no Connie, Ben)

b.      Nov 8: Rodger & Connie: presentation on Strategies on Implementing BHI

c.       Nov 15:

d.      Nov 22: THANKSGIVING… take the day off J

e.       Nov 29: Prema: draft grant application

f.       Future agenda to consider:

                                                  i.      Kairn – review of draft article on IRR  

                                                ii.      Ben: budgeting exercise for grant applications; NHANES – lower female mortality for women taking birth control medications

                                              iii.      Rodger: Mixed methods article; article on Behavior’s Influence on Medical Conditions (unpublished); drug company funding.  Also: discuss design for PCBH clinical and cost research.  Also: Prezi demo. 

                                              iv.      Amanda: presentation and interpretation of data in articles

                                                v.      Christina Cruz, 3^rd year FM resident with questionnaire for mild serotonin withdrawal syndrome

Recorder: Connie van Eeghen