Monday, October 29, 2012

Clinical Research Oriented Workshop (CROW) Meeting: Oct 24, 2012



Present: Abby Crocker, Kairn Kelley, Amanda Kennedy, Ben Littenberg, Charlie MacLean, Betty Rambur, Connie van Eeghen

1.                  Start Up: A virtual meeting for Amanda and Connie, thanks to Charlie’s technical expertise.     

2.                  Presentation: Betty Rambur shared a concept paper as a precursor to a grant proposal intended for a credentialing organization, American Nurses Credentialing Center, which credentials NPs and hospitals for Magnet status (FAHC is not a Magnet hospital; Dartmouth and Southwestern are).  It is unusual for this organization to provide research grants; this may be part of a strategy to demonstrate the value of Magnet hospitals.  One requirement of Magnet hospitals is that nurses must be involved in research.  This is very difficult for small hospitals.   The topic is about the unintended consequences of measuring quality.  The grant is capped at $1.2m for 3 years, with no more than 30% indirects.
a.       Three phases: development of perceptual measures of unintended consequences, measurement by surveys across New England hospitals, and development of nurse research skills. 
                                                  i.      Survey population: 7,000 active nurses in Vermont; also will include the other states in the region
                                                ii.      Phase 1: developing the metrics (in Magnet hospitals) that identify unintended consequences; using Magnet hospitals only may produce incomplete or biased metrics
                                              iii.      Phase 2: measuring quality usually results in a drop in quality measures at first, as participants start seeing issues in quality previously ignored; these measure the harm that was caused
                                              iv.      Phase 3: develop research capacity in nurses – how strongly should this section be emphasized?  Talk to the grant administrator?
b.      Budget considerations:
                                                  i.      Supports 10 interns, salaries and traveling, interaction costs (i.e. maintaining the relationship with the interns throughout this project)
                                                ii.      Survey administration
c.       Recommendations
                                                  i.      Write more simply, like the elevator pitch that started this meeting
                                                ii.      Are there more direct measures (involving primary data collection or data base analyses) that can be proposed in the concept paper?  Something more than perceptual surveys, e.g. Treatment with antibiotics – something that can be measured objectively via EMR.  There is a specific nursing data set (NDQA?), many elements of which are proxies for staffing: patient falls, pressure ulcers.  Note: accessing primary data in the non-Magnet hospitals could be difficult.
1.      Keep the survey as a metric
2.      Consider metrics connected to the nursing data set
3.      Look at the impact of 30-day mortality rate (due to delayed re-admission)
4.      Hospital acquired infection due to delayed discharge
5.      More immediate indications of harm
6.      The metrics may be the outcome of the development of the student interns
7.      Consider assigning each intern (or 2 interns) a metric for which a protocol is written.  All interns collect on all metrics.
8.      Create a model that starts with latent factors (the perceptual data) and leads to the harm metrics (using Reason’s Latent Factor Model in reverse)
                                              iii.      Consider a systematic review as a first step in the proposal, which can lead to semi-structured points for the focus group to develop the issues
                                              iv.      Consider requesting a lower level funding; have the funder rethink the funding priorities

3.                  Abby’s Update: Current status of her research project
a.       Three possible articles, the first is focused on the predictors of neonatal abstinence syndrome.  Each column is a variable, NAS+ babies, NAS- babies, p value.  Within those columns, Abby has identified the n, missing data, and standard deviation.  Discussion:
                                                  i.      State N overall, and then n for each analysis, and don’t report the “missing data” number
                                                ii.      Univariate variables and descriptors were used to create a predictive model.  The table Abby developed shows where this model came from.  Variables with high p values were removed from the model, one at a time.  The final logistic regression model included 3 variables:
1.      Total dose of methadone of mother per day at the time of delivery (only 100 moms were on methadone) (lower is more protective)
2.      APGAR score (higher is more protective)
3.      Head circumference standardized for estimated gestational age (z score) (higher is more protective)
                                              iii.      This model is not causal, just predictive.  The three variables are all immediately available at time of birth. 
                                              iv.      Consider breaking the table down into several tables, to make interpretation easier for the reader.
                                                v.      Abby also discovered that the babies with earlier birth years (from the beginning of the study) were more likely to be NAS+.  This may be because reporting became more accurate as nurses got more comfortable with this assessment.  Birth year doesn’t have predictive value. 
                                              vi.      Abby also calculated the ROC (Receiver Operator Characteristic curve), which is approximately 67% - so not a great predictive tool.  Ben and Abby came up with a calibration curve, which describes the logistic regression model: for each baby, there is a predictive value of getting NAS (from 0 – 1).  Each baby gets a value on this scale.  The calibration curve assigns a percentage chance of getting NAS by groups of scores.  The predictive ability of the curve should go up as the percentages go up.  The confidence intervals are wide but the slope is 45 degrees – so well calibrated, but not precise. 
1.      Each group (NAS+, NAS-, and all other babies) can be studied independently.
2.      Recursive partition might be appropriate – per past heated discussions.
b.      Discussion continued on the other tables and figures included in this publication and segued into the beauty of APGAR scores.  If we use the ROC as a predictive tool, what is the action rule for an infant with 45% probability?  This may not be discriminative in a clinically relevant range.

4.                  Next Workshop Meeting(s): Thursday, 2:00 p.m. – 3:30 p.m., at Given Courtyard Level 4. 
a.       Nov 1: Kairn: final draft of article on IRR (no Connie, Ben)
b.      Nov 8: Rodger & Connie: presentation on Strategies on Implementing BHI
c.       Nov 15:
d.      Nov 22: THANKSGIVING… take the day off J
e.       Nov 29: Prema: draft grant application
f.       Future agenda to consider:
                                                  i.      Kairn – review of draft article on IRR  
                                                ii.      Ben: budgeting exercise for grant applications; NHANES – lower female mortality for women taking birth control medications
                                              iii.      Rodger: Mixed methods article; article on Behavior’s Influence on Medical Conditions (unpublished); drug company funding.  Also: discuss design for PCBH clinical and cost research.  Also: Prezi demo. 
                                              iv.      Amanda: presentation and interpretation of data in articles
                                                v.      Christina Cruz, 3rd year FM resident with questionnaire for mild serotonin withdrawal syndrome

Recorder: Connie van Eeghen

Tuesday, October 16, 2012

Clinical Research Oriented Workshop (CROW) Meeting: Oct 11, 2012



Present: Abby Crocker, Kairn Kelley, Rodger Kessler, Ben Littenberg, Prema Menon, Connie van Eeghen

1.                  Start Up: Here’s to IT helpers who need screen shots of “Locked.  You are not authorized to…” in order to believe that authorization is, in fact, denied.   

2.                  Presentation: Rodger & Connie – Aim Statement and Approach sections of R03 grant application for Toolkit for Primary Care Behavioral Health
a.       The group had an extensive and detailed discussion about how to present the design, which is essentially good, in ways that are easier to understand and better supported.  Key ideas:
                                                  i.      Eliminate redundancy and keep simplifying, starting with the Aims page.
                                                ii.      Conceptual diagrams should focus the reader on the key points for this, particular, application.
                                              iii.      Do not indulge in the luxury of stating in the narrative what is in a table, or vice versa. Choose one or the other; the page restrictions make this a necessity.
                                              iv.      Create new sections when logical: “Investigators” and “Intervention,” as examples.
                                                v.      Introduce the overall design in a few simple words. 
                                              vi.      Keep Aim 1 and 2 design elements distinct from each other; they have different kinds of subjects, data collection methods, analyses, etc.
                                            vii.      Every figure or table should have a well-defined question that it is answering and it should answer it as clearly and simply as possible.
                                          viii.      Simplify the analysis to two observations and, in limitations, explain why this is not a limitation.  Adjust for loss to follow up.
b.      Thanks for everyone’s input.  As usual, invaluable. 

3.                  Next Workshop Meeting(s): Thursday, 2:00 p.m. – 3:30 p.m., at Given Courtyard Level 4. 
a.       Oct 18: Christina Cruz, 3rd year FM resident with questionnaire for mild serotonin withdrawal syndrome
b.      Oct 25: Abby: update (Connie by phone)
c.       Nov 1: Kairn: final draft of article on IRR (no Connie)
d.      Nov 8:
e.       Nov 15:
f.       Nov 22: THANKSGIVING… take the day off J
g.      Nov 29: Prema: draft grant application
h.      Future agenda to consider:
                                                  i.      Kairn – review of draft article on IRR  
                                                ii.      Ben: budgeting exercise for grant applications; NHANES – lower female mortality for women taking birth control medications
                                              iii.      Rodger: Mixed methods article; article on Behavior’s Influence on Medical Conditions (unpublished); drug company funding.  Also: discuss design for PCBH clinical and cost research.  Also: Prezi demo. 
                                              iv.      Amanda: presentation and interpretation of data in articles

Recorder: Connie van Eeghen

Monday, October 8, 2012

Clinical Research Oriented Workshop (CROW) Meeting: Oct 4, 2012



Present: Abby Crocker, Kairn Kelley, Ben Littenberg, Charlie MacLean, Connie van Eeghen

1.                  Start Up:  Commentary on Atkins: The World is Fat.  Interesting, but not novel.

2.                  Presentation: Ben: NHANES III exploration
a.       NHANES III is a repeated, cross sectional survey of the non-institutionalized, non-military U.S. population.  The data sets and instructions on how to use them are on a CDC website.  Ben has merged several data sets from this survey, including adults, leptin, physical exam, and mortality (as recorded 18 years later) files which he shared with the group via STATA.  These data include some children and although some fields are missing, BMI (as an example) is present.
b.      There are 33,000 subjects with 3600 variables (some of which are administrative; some have been calculated by Ben).  Note that the use of “missing” as a value is field dependent and needs to be interpreted based on that field and, possibly, related fields.  Ben has calculated new variables to classify “missing” in some helpful ways.
c.       There is an extensive list of fields describing the population (demographics)
d.      There is a group of fields related to weights.  Although this population represents the majority of the adult US population, the sample was disproportionately weighted to some subgroups of the population.  The weights provide the multipliers to adjust for those differences in several ways: one multiplier is for a pseudo-stratum weight, one is for a pseudo-personal weight, and there are weights for specific surveys or groups of questions that were asked (e.g. allergy) and phases (years that the surveys were administered).  The weights can be applied directly to individual field values (age, weight of person, lab values, etc.) that, when summarized, reflect the underlying population.
e.       Ben opened a script file (.do) that provided examples of data field cleaning or generation for the use of queries related to cardiograms.  The weight is selected according to the portion of the population being studied and is used for calculations for “survey” commands.  A similar set of commands sets up the data for time analysis (e.g. Kaplan-Meier). 
f.       We practiced exploring the data set using this script file, finding an apparent relationship between “cutting back on meals” and “mortality.”  However, by adding other conceptually related variables (income class, age, sex) we found that mortality is better explained by age (oh, yeah) and income.  Good exercise, and helpful preparation to future, similar studies we are able to do.   

3.                  Next Workshop Meeting(s): Thursday, 2:00 p.m. – 3:30 p.m., at Given Courtyard Level 4. 
a.       Oct 11: Rodger and Connie: R03 resubmission draft
b.      Oct 18: Christina Cruz, 3rd year FM resident with questionnaire for mild serotonin
c.       Oct 25: Abby: update
d.      Nov 1: Kairn update?
e.       Future agenda to consider:
                                                  i.      Kairn – review of draft article on IRR  
                                                ii.      Ben: budgeting exercise for grant applications; NHANES – lower female mortality for women taking birth control medications
                                              iii.      Rodger: Mixed methods article; article on Behavior’s Influence on Medical Conditions (unpublished); drug company funding.  Also: discuss design for PCBH clinical and cost research.  Also: Prezi demo. 
                                              iv.      Amanda: presentation and interpretation of data in articles

Recorder: Connie van Eeghen