New publication by Kairn Kelley

Congratulations to PhD candidate Kairn Kelley MS whose research just appeared in the International Journal of Audiology.

Int J Audiol. 2015 Jun 22:1-7. [Epub ahead of print]

Inter-rater reliability for scoring children's dichotic words test responses.

Kelley KS1, Littenberg B.

Author information

Abstract

OBJECTIVE:

To determine whether rater agreement is randomly distributed or varies importantly with test-taker characteristics, test words, or rater experience with the dichotic words test (DWT).

DESIGN:

DWT was administered to 34 children in 1st-4th grade and responses scored by two raters. The proportion of rater agreement was calculated for each child and for each word. Correlates of inter-rater agreement were explored.

STUDY SAMPLE:

Two raters judged 6686 total responses from 34 children.

RESULTS:

Overall agreement between the two raters was 0.97. Test-taker scores ranged from 35%-91% (mean = 81%). Agreement was associated with score but not with test-taker age or sex. Test words spanned the full range of difficulty (pass proportion 0.06-1.00). Rater agreement was not randomly distributed among the words. Inter-rater agreement for test words ranged from 0.82-1.00 and was associated with pass proportion (Spearman's ρ = 0.28; p < 0.0001). However, there were words at all pass proportions with perfect or near-perfect agreement. Rater agreement improved from 0.94 on the first day of data collection to 0.98 on the fifth day (p = 0.026).

CONCLUSIONS:

Inter-rater reliability should be considered along with test item difficulty when developing speech audiometry materials, scoring protocols, and rater training.

Clinical Research Oriented Workshop (CROW) Meeting: June 18, 2015

Present:  Nancy Gell, Rodger Kessler, Ayodelle LeBruin, Ben Littenberg, Connie van Eeghen

Start Up: New journal to notice on the patient perspective that Nancy shared: Journal of Patient Centered Research and Reviews; volume 2 is now available.  Open access, online only. 

1.                  Discussion: Nancy Gell presented BeMobile:  Exit Interview Protocol for an exercise reminder program (via text messages) that continues after an oncology rehabilitation care period; data collected at 3 months (with efforts to collect data if they drop out).  Starting with a group that has completed only 4 weeks.

a.       Goals

                                                  i.      Intervention Design Feedback

                                                ii.      Intervention Final Assessment

                                              iii.      Publication, Fame & Glory

b.      Iron Triangle represents the ultimate tradeoff: Quality – Cost – Speed

c.       Outcome: Do you keep exercising (maintenance)

                                                  i.      Applies to both those that are still enrolled in BeMobile and those that are not (same questions for both)

                                                ii.      Consider different questionnaires: (they can’t be pooled, even if the questions are the same)

1.      Goal 1

2.      Goal 2

                                              iii.      Consider adding questions: barriers to exercise

                                              iv.      Consider removing all questions not directly supportive of the goals

2.                  Next Workshop Meeting(s): Thursdays, 1:00 a.m. – 2:00 p.m., at Given Courtyard South Level 4.   Remember: the first 15 minutes are for checking in with each other.  The schedule for future meetings will be re-evaluated in May.

a.       June 25: Cancel

b.      July 2: (no Marianne) Rodger: EHR build of algorithm for integrated treatment of diabetes

c.       July 9: Marianne: update

d.      July 17 (switch to Fridays):

Recorder: Connie van Eeghen

John Ware, inventor of the SF-36, to speak at UVM

John Ware, Jr. is the developer of perhaps the most important, most widely used, and most influential instrument in health services research. The SF-36 (and its offspring, the SF-12, SF-8 and others) are landmarks in the field of assessing health status and quality of life. It is very exciting that he is coming to speak on campus!

John Ware Jr., PhD
University of Massachusetts
 “Standardizing and Integrating Disease-specific and Generic Patient-reported Outcome Measures."
June 24th, 2015, 12:00-1:00 p.m.
Davis Auditorium, UVM Medical Center Campus

Sponsored by the Vermont Center on Behavior & Health.