Friday, September 12, 2014
Present: Kairn Kelley, Amanda Kennedy, Ben Littenberg, Connie van Eeghen
Start Up: Cats, dogs, and the work involved… when working on everything else! Kairn’s manuscript with Ben has been accepted – congratulations! A recent court decision by the U.S. Ninth Circuit Court of Appeals defined “auditory processing disorder” as an “other health impairment” under the Individuals with Disabilities Education Act. It would be nice to have evidence base for diagnosis and accommodations.
1. Discussion: Karin’s lit review update
a. Kairn has found the “Ear and Hearing” does publish systematic literature reviews
b. The purpose of this discussion is to identify the kind of meaningful findings this review will include
c. Kairn plans to evaluate different diagnostic dichotic listening tests to “support audiologists’ decisions about test selection and interpretation.”
i. There are 9 recorded tests (some of which have several versions)
ii. There is no coherent biologic model to support these tests
iii. Many tests that have been used to support common models aren’t commercially available
iv. The commercially available tests are not well validated for reliability (and other domains – see below) or by rigorous trials
d. The focus is comparing the tests and identifying whether any are supported for use
i. There is plenty of literature reviewing the process of auditory testing, but none that review the tests themselves
ii. The concept of reliability also has to be well explained, as do the other domains:
4. Value (not referenced in the literature – which should be so declared)
iii. Previous reviews reference correlation, which is not an adequate evaluation (this is for the Discussion section)
iv. Conclude with plan for next steps in research – of which the field is apparently wide open
e. The methods section has been written; it originally used a broad net and this can be more tightly defined
f. The objective of this review is to describe/summarize/identify/evaluate/summarize research studies that use any of these 9 tests and contain some evidence of reliability, accuracy, usefulness, and value. Ben: Summarize the literature on reliability, accuracy, etc. for 9 commercially available tests.
i. Eligibility criteria: Papers were eligible if they:
1. One of 9 tests
2. Children with normal hearing between ages 6 and 14 and are neurologically intact
3. “We reported any test… with a gold standard diagnosis.” (None meet this standard.) Must reference reliability or accuracy or influenced care (useful) or good value for money.
4. Or state as an exclusion: we will exclude any study that does not include reliability data or accuracy or usefulness or value
a. Will not include review articles
ii. Intervention: reported results of these tests and any evidence
1. Reliability: including correlation, Bland Altman plots, test-retest coefficients of variation, inter-rater reliability
2. Accuracy: Kairn originally thought that since there is no agreement on “auditory processing,” there are no studies meeting this requirement. Ben argued that the study is eligible for inclusion if it studies association with any reference test or gold standard.
a. The fact that the accuracy data that does exist only includes brain tumor cases is still a reason to discuss it and to identify the opportunity to study accuracy in children who are neurologically intact
3. Usefulness: TBD
4. Value: TBD
iii. Quality issues:
1. Sample size
g. Next steps:
i. Look at Kristi Johnson’s lit review on evaluating lymphedema diagnostic tools.
ii. Rewrite the methods section, focusing just on the steps related to selecting and analyzing the literature ONLY through the four elements of evaluation (nothing else). One result of this will be a simple count of the number of articles that included each of the four elements.
1. Stay close to the systematic review process
2. OK to collect notes as a narrative assessment (or a grading system) on the quality of the studies (but this is NOT the purpose of the study – do NOT lose focus; this review will not lead to a high level quantitative study of the literature)
3. Keep track of common evaluation criteria: age of study, size of sample, population of kids…
iii. Define Usefulness and Value in an unambiguous way.
iv. Sort the lit reviews based on the criteria for each of the four elements; document exactly where the evidence is that satisfies the criteria
h. Remember: do not point out every deficiency in the study. Stick to the focus: a story that convinces your colleagues that your evaluation is right – that no studies meet the criteria for evaluating auditory tests.
i. One study (Wilson) looked for connections between any of the test results and parent surveys for children referred to clinic
ii. It did not include reliability data (like test/retest)
iii. It did perform linear regression, but not accuracy data
iv. The article may be reported separately as included in the original filter, but excluded due to lack of test-evaluative data
2. Next Workshop Meeting(s): Thursdays, 11:30 a.m. – 12:45 p.m., at Given Courtyard South Level 4. Remember: the first 15 minutes are for checking in with each other.
a. Sept 18: Marianne: draft of IRB application
b. Sept 25: ???
Posted by Connie at 9/12/2014 10:12:00 AM