Roy, et al. 2002. J Speech Lang Hear Res.

Paper: N. Roy, B. Weinrich, S. D. Gray, K. Tanner, S. W. Toledo, H. Dove, K. Corbin-Lewis, and J. C. Stemple. Voice amplification versus vocal hygiene instruction for teachers with voice disorders: a treatment outcomes study. Journal of Speech, Language, & Hearing Research 45 (4):625-638, 2002.

Link to PubMed and abstract: PMID 12199394

First Commented on April 11, 2007

General Comments: This is an interesting example as this paper is cited often since it deals with teachers voice, etc. and it has already been challenged in the literature with a letter to the editor and then a response from the original authors:

J. P. Dworkin, G. G. Abkarian, R. J. Stachler, R. A. Culatta, and R. J. Meleca. Is voice amplification for teachers with dysphonia really beneficial? J.Speech Lang Hear.Res. 47 (2):353-357, 2004. author reply 358-65.

What I found interesting was that the only question I had with the paper had nothing to do with the letter to the editor and the response (I didn't think the Roy response was all that convincing anyway). Nevertheless, my own question is below.

First Comment: The authors use a voice indexing measure called the VHI (Voice Handicap Index) developed and discussed in the article below:

Barbara H. Jacobson, A. Johnson, Cynthia Grywalski, Alice K. Silbergleit, G. P. Jacobson, Michael S. Benninger, and C. W. Newman. The voice handicap index (VHI): development and validation. American Journal of Speech-Language Pathology 6 (3):66-70, 1997

In the Jacobson 1997 article, they develop the statistics by showing that a change of 18 points in the scale represent a significant change in the function of the voice. This is done by controlling repeated uses of the scale both within and across subjects. When Roy et al 2002 uses the VHI, they disregarded this completely. Roy et al discuss significant difference but they seemed to have miss used the VHI, thus potentially invalidating their entire conclusion set.

Final Comments: I would hope the Roy group addressed this since their study is often cited. This oversight could be severe.

Goding & Bierbaum, 1999. Otolaryngol.Head Neck Surg.

My first real post, and it may come with lots of updates, so please be patient.

Paper: G. S. Goding, Jr. and R. W. Bierbaum. Relationship of the posterior cricoarytenoid muscle to the posterior cricoid lamina. Otolaryngol.Head Neck Surg. 120 (4):493-498, 1999.

Link to PubMed and abstract: PMID 1018794

First Commented on April 10, 2007

Overall Feeling: Nice little paper which illustrate the general shape and direction of the Posterior Cricoarytenoid intrinsic muscle in the larynx (the main muscle which opens your vocal folds or vocal folds so you can breath).

First Comment: The authors never talk about error or uncertainty from the measurements. All they say is this:

The mucosa was dissected to expose the PCA and cricoid cartilage. Measurements of the vertical height of the PCL and the anterior cricoid arch were obtained with a caliper accurate to 0.001 in. (from Page 493, last paragraph)

...uh, so what! The real statement should include this line but with an additional line saying something like, "three researchers independently measuring the same thing resulted in an uncertainty of +/-0.1 inches". This is an easy catch that the reviewers missed. As a potential user of the data, I don't know how much to trust the exact numbers given. I need a confidence rating of some sort. From reading other works, I have seen uncertainty on the order of +/-0.3 cm in similar situations where measures are done by hand with a caliper. Assuming similar protocols, this would result in +/- 0.013 in. You may say back to me, "so what". But, it is not in the paper and it should be. Does it change the conclusions? No. Should it have been published without it? No. By missing such 'minor things' opens the door for publications where the conclusions will be dependent on such things.

Possibly more later...

After Peer Review, What is the purpose of this blog?

2007-04-10: So, what is the purpose of this blog?

Through my work in research, I spend quit a bit of time reading scientific publication which have been "peer reviewed". As I also participate in reviewing other scientist's work and articles, I know that I can miss things and, potentially, poor quality work may slip into scientific journals. The result is occasional miss statements that may or may not have impact on the conclusions of the published paper. Many of these do not have long term consequences but some do. Even the minor ones should be corrected as a new researcher may not have the experience to know what is correct or not. Therefore, my main purpose is to give me an outlet where I can post errors or miss statements (or just bad protocol) as I run across them.

Eventually, I believe that scientific journals will not be printed but will only be an organization that will peer review scientific papers. As a researcher, you would send an article to this 'journal' and they would review it. If it passed peer review, they would give you a 'peer reviewed accepted ID number' which you would place on the electronic document. That would be it. The next step would be to have a 'WIKI' or blog type interface where people can point out potential inconsistencies or confusion in an article.

UPDATE: 2007-04-11
For those not aware of how the peer review process works, I would suggest you read this. When I participate in a review, there are usually 2 other reviewers. It is likely that most errors or miscues that are allowed through the process are because of lack of time or effort on the reviewers part or lack of detailed expertise. I believe that most errors that make it through the process are inadvertent. Nevertheless, they should be pointed out. That is what this blog is for.

General peer review discussions are happening all over, check out this debate in Nature (i.e. the great and spacious scientific press release journal... real work is published in the regular journals-- stay tuned for an upcoming post about Nature's gaffe in voice research peer review).