2013年5月2日 星期四

Intraclass Correlation Coefficient (ICC) 與 Cronbach's alpha

學期初思索的問題之一(Questions in Mind)


概述


Intraclass Correlation Coefficient  (ICC) 是一個常被用來估計評估者間一致性的指標之一,主要是使用於以下的研究設計中:


                      評估者1    評估者2   評估者3  ....  評估者K

受試者1             5               4                5         ....        4

受試者 2            3               5                3         ....        2

受試者 3            3               2                1         ....        3

...                        ....             ....              ....       ....        .... 

受試者 J             5               4                 3        ....         5


其中:

1. 評估者可以被題目(ex:智力題目1、智力題目2)、時間點(ex: treatment前、treatment後)、場合等變項替換。因此ICC的測量重點為測驗分數橫跨不同的情況,分數穩定的情形。

2. 評估者的選取可以概分為三種情境 (Shrout & Flesis 1979):
  • Case 1: 對於每一位受試者,隨機選取k位評估者進行評估。每一位受試者接受相同數目(k)的評估者,但評估每一位受試者的評估者組合不同。

  • Case 2: 母群中隨機選取k位評估者,對每一位受試者進行評估。每一位受試者都接受相同的一組評估者。

  • Case 3: 母群中的k位評估者,對每一位受試者進行評估。

Case 2 與 Case 3的差異在於,Case 2中將評估者視為隨機效果(random effect),而在Case 3中將評估者視為固定效果(fixed effect)。

 

選擇合適的ICC公式

Shrout & Fleiss (1979) 以及McGraw & Wong (1996)總結出四個選擇ICC公式的原則:

  1. One-way or Two-way ANOVA:在Case 1的情況下使用One-way ANOVA;Case 2的情況下使用Two-way ANOVA。
  2. Random-effect or Mixed Effect:One-way ANOVA的情境下就是使用random effect;Case 2使用Two-way random effect;Case 3使用Two-way mixed effect (受試者random, 評估者fixed)。
  3. Agreement or Consistency:Agreement為想要了解測驗分數真實match的程度;Consistency為想要了解測驗分數的線性關係。
  4. Single or Mean measurement:計算ICC時使用的單位是每個評估者對於受試者的單一分數,抑或是評估者整體給予一個受試者的平均分數。


 使用SPSS 20進行計算

資料的排列行式如上面的表格,定義評估者變項為R1, R2,...Rk;受試者變項不需定義。


  1. Analyze-->Scale-->Reliability Analysis
  2. 將評估者變項丟入items的欄位中,並選擇Statistic進入新的小視窗
  3. 勾選Intraclass Correlation Coefficient,並選擇合適的ANOVA(ex:two-way random)以及想要探索的分數關係(ex: agreement)。
  4. 選擇Continue,並Run,報表上會自動出現Single以及Mean measurement的結果。


ICC與Cronbach's alpha

若是將評估者變項是為題目變項,則可將R1, R2,..,Rk視為item 1, item2, ..., item k。使用ICC假定為Two-way ANOVA、Consistency以及Mean measurement的結果會等於Cronbach's alpha。因此,可發現到Cronbach's alpha是ICC的一個特例。





參考文獻


Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 17, 30-46.

Partial Correlation, Semi-partial correlation, and beta coefficience

這是學期初開始自己想要思索的問題之一(Questions in mind)

Partial Correlation (淨相關)、Semi-partial correlation (半淨相關)以及Beta 係數(Coefficient)

以下的例子假設使用兩個獨變項(X1, X2)來預測一個依變項(Y)的情況。



  



有關Partial- 與Semi-Partial的數學式,可參考以下連結:

http://luna.cas.usf.edu/~mbrannic/files/regression/Partial.html


Beta的公式計算如下:

Beta 1 = [r (Y,X1) - {r(Y,X2)*r(X1,X2)}] - [1-r(X1,X2)^2]



以下直接以圖例進行說明,由於在Correlation相關數值的討論中,所有變項都經過標準化(除以各自的標準差),因此,以下所畫的每一個圓餅的面積都為1:


Partial Correlation


 Partial Correlation (X1, Y) = b/a+b

解釋:在Y的變異中,排除掉X2能夠解釋的部分(a+b)後,單獨被X1解釋(b)的比例




Semi-partial Correlation

 Semi-partial Correlation (X1, Y) = b/a+b+c+e = b/1 = b

解釋:在Y的變異(a+b+c+e=1)中,能夠被X1解釋(b)的比例。



Beta Coefficient

Beta (X1, Y) = b/ b+f

解釋:在X1的變異中,排除掉與X2的相關之後,能夠用來解釋Y的部分。


結論:
這三個指標,主要的目標都是放在該變項(ex: X1)能夠單獨解釋依變項(Y)的部分(b),唯一的不同在於立足的基準不同。


[Published Article] Convergent and Discriminant Validity of the WHOQOLBREF Using a Multitrait-Multimethod Approach

Convergent and Discriminant Validity of the WHOQOLBREF Using a Multitrait-Multimethod Approach

Yu-Yu Hsiao   Chia-Huei Wu   Grace Yao

Social Indicator Research (in press)

Abstract
 
The WHOQOL-BREF is a commonly used questionnaire in quality of life research. Previous research has shown that the four domains of the WHOQOL-BREF (physical health, psychological health, social relationships, and environmental health) are highly related. Whether these high correlations reflected the true relations across the domains or the influence of common method effect; however, is still unknown. This study examines the convergent and discriminant validity of the four domains of the WHOQOLBREF by using the multitrait-multimethod (MTMM) approach to control the method effect. Two different samples, with a total of 186 and 201 adults, respectively, were used in the present study. Each participant filled out the WHOQOL-BREF questionnaires using four different scaling methods (Likert-type scale, visual analogue scale, pie scale, and partner rating). The covariance matrix of the MTMM result was analyzed with confirmatory factor analysis modeling. Two models were applied, including the correlated traits-correlated uniqueness (CTCU) model and the uncorrelated traits-correlated uniqueness (UTCU) model. Results showed that the CTCU model fit the data better than did the UTCU model, suggesting that the variables tapping the four domains of the WHOQOL-BREF have excellent convergent validity; the four domains have moderate correlations, indicating that the four domains are related but not identical.
 
Keywords Multitrait-multimethod matrix, Quality of life, WHOQOL, Confirmatory factor analysis, Validity
Submitted: 03-13-2013
Accepted: 03-19-2013


(註記)
這篇文章從投稿到接受只花了六天,看起來真不可思議,但實際上背後也是歷經了許多波折。一開始這篇文章的target期刊是Quality of Life Research,但審了兩個多月後得到了reject的回應。之後又投了Educaitonal and Psychological Measurement和Journal of Happiness Studies,雖然備受煎熬但我們也從reviewers的建議中修改了文章的架構以及一些論述,同時也修改了實驗設計。最後的成品被SIR火速接受,這段日子的辛苦也算是有了代價。在寫這篇文章的過程中,我和共同作者們之間對於文章的架構以及討論的部分出現了部分分歧,但在討論的過程中讓我學習到如何在分歧中陳述自己的觀點,並嘗試完成對話的過程,這些都是在寫文章投稿之餘無價的收穫。


[Published Article] Is there a disparity in the hospital care received under a universal health insurance program in Taiwan?

Is there a disparity in the hospital care received under a universal health insurance program in Taiwan?

Yu-Yu Hsiao, Shou-Hsia Cheng

International Journal for Quality in Health Care (in press)

Abstract
Objective To analyze the disparity in hospital care among people of various socio-economic status (SES) under a universal health insurance scheme. Design A questionnaire survey mailed to discharged patients in October 2010. Setting This study included 183 large-scale hospitals in Taiwan. Participants A total of 3015 patients/caregivers completed the questionnaires, which yielded a response rate of 58%. Main Outcome Measures Three variables were included. The two access-to-care variables were admission route and accreditation level of the hospital in which the patient stayed. A structured questionnaire, the Patient-Reported Hospital Quality (PRHQ), was included to characterize patient’s experience of hospital stay. Results Patients with lower education were less likely to be admitted to a hospital according to a planned schedule, or to choose an academic medical center hospital. However, SES was not associated with the PRHQ scores. Furthermore, patients with unplanned admission were associated with lower PRHQ scores than those with planned admission to the hospital. Conclusions Under the universal health insurance system in Taiwan, lower education is associated with unplanned admission to a hospital, which might result in poorer perceived quality of care. Reducing unplanned admission is a challenge for health authorities in the future.
Keywords access to care, quality of care, hospital, socioeconomic status, Taiwan


Submitted: 11-22-2011
Revised: 02-20-2012
Accepted: 01-15-2013


(註記)
第一篇發表在英文期刊上,同時也是列為第一作者的文章。投稿到接受中間花了一年多的時間,但其實占了比較多時間的是在resubmitted到accepted的這段時間,長達一年。比較有趣的是,resubmitted之後我們等了半年多都沒等到回應,寫了幾封e-mail之後竟然才收到文章進行審查的e-mail。好在後來的過程還算順遂,沒有甚麼大問題就被接受了。

I am survived!!

這個學期在交出Dr. Thompson的Paper之後宣告結束了。

回想起這一個學期如同地獄般的schedule:每個星期20小時工作、15小時上課以及4小時的studygroup。還得利用時間寫Paper、寫作業、整理生活瑣事。

不知道自己是怎麼度過的,但總算是撐過去了。

雖然辛苦,但在學術上也有了不小的進展。除了感覺自己在統計的概念上有了顯著的進步外,實質上也發了兩篇Paper還拿到了台灣政府給的獎學金。

Bitter and sweet是我為這個學期下的註解。希望自己能夠一直進步下去。