Ocular Surface Disease Index Questionnaire (OSDI-Q), Unidimensionality, and Differential Item Functioning

Objective The OSDI-Q is a 12-item scale used instrument for DED prevalence, assess symptom severity, and impact on vision-related functioning. The research analysis investigates the precision, separation index and reliability of unidimensionality, and DEF, stratified by age and gender. The reliability analysis is to assess the internal consistency of the scale using Cronbach's α coefficient and McDonald's ω coefficient with item-rest correlations.

Method 634 respondents took part in the OSDI questionaire. Rasch analysis was conducted using precision, separation index, and reliability for items and person, item and person fit statistics, assessing unidimensionality stratified by age and gender, and the use of SPSS to conduct the reliability analysis of the scale using analytical coefficients.

Results The OSDI-Q items displayed robust reliability, with high PSI values of 7.63 and 0.98, respectively. Infit and outfit statistics were within acceptable ranges, explaining 39.90 % of variance. Differential item functioning was observed among the age groups involved. Additionally, the contrast in the principal component analysis of the unexplained variance exhibited an eigenvalue of 2.80 and a proportion of variance of 14%. DIF analysis showed significant differences for age groups of (16-26 years) and (27-40 years). The coefficients of Cronbach's and McDonald's are 0.85 for OSDI-Q demonstrated strong internal consistency.

Conclusions OSDI-Q displayed appropriate infit and outfits. DIF analysis results of OSDI-Q of age group (16-26 years) performed better in poor vision while age group (27-40 years) performed better in areas with low humidity. Furthermore, the results of DIF analysis in males group performed better in driving at night, while the female group showed better performance in the questinaire. The OSDI-Q items scale reliability data for Cronbach's scale show high internal consistency and reliability. This research can help the medical communities to begin a better surge for medically integrated research.

Dry eye disease (DED) is a disease of the tear and ocular surface, associated with hyperosmolarity of the tear film, leading to inflammation and amage of the ocular surface as frequently conveyed by discomfort in the eye, fatigue, and vision disturbance.^1,2 This so happens due to the climatic conditions that prevail in the location, hence it develops dryness in the eyes that gives rise to ocular systems as mentioned. The prevalence of DED ranges from 5.25% to 62.6%.^3-8Current studies show a rise in the prevalence of DED among the young adults and the adult population.^4-8 For instance, the prevalence of DED is high in Asian countries ranges from 20% to 52.4%, compare to Western countries, ranging from 7.6% to 16.2% ¹ These fluctuations are due to differences in geographical location, population, diagnostic techniques, race, and subject criteria. It is also to note that the difference of DED ranges in the regions could also be due to the genetic mutations among the families, which may have caused the degenerate occlusion of the disease that passes on from generation to generation.

Therefore, the presence of DED, several dry eye questionnaires were developed through OSDI questionnaire and McMonnies Dry Eye Questionnaire. ^1,9,10. It is specific to denote the severity of disease. However, it is used to differentiate subjects with dry eye from a normal population and does not grade severity or its effect on functions that are vision-related. Also, no formal reliability testing on this instrument and the questionnaire was published ^9,10,11 In contrast to OSDI severity, valid and reliable questionnaire was developed by the Outcomes Research Group at Allergan Inc (Irvine, Calif).^9,12

Differential Item Functioning is the 12-item Likert scale questionnaire designed to provide DED prevalence, assess DED symptom severity, and the impact on the vision-related functioning. The questionnaire is divided into three parts. The first five questions the frequency of the system. The next four questions are related to vision, the quality of life, and certainly the effect of the daily routine that induces the severity of the dry eyes. The last three questions pertain to the environmental triggers for a 1-week recall period. In this period, the volunteers of the research questionnaire are closely assessed in terms of their routine and the source that mainly induces the pain and severity of the eye. Hence, the severity score can range from 0 to 100 where 0–12 is normal, 13–22 is mild dry eye, 23–32 moderate dry eye, and 33–100 severe dry eye. Each answer is scored using a 5-point Likert scale indicating 0 as no symptoms and 4 as significant symptoms.⁹

The foundation of this research and the previous research studies have verified on the significant effect of DED on patient’s quality of life and the burden that it imposes on the healthcare systems worldwide.6,7 Current studies state that the prevalence of DED ha significantly risen in the age group of 17–29 years, complying to a range of 38% to 59%, ^4-8 This will add more burden on the healthcare systems and their financial investments. For instance, the annual cost of managing DED in the USA is $55.4 billion.¹³ Thus, OSDI must be culturally specific in order to assess the presence and absence of DED symptom’s frequency, reliability, and intensity. Hence, the first step to determine the self-reported health status of a particular instrument is to determine the reliability and the validity of the usage of that very instrument. These instruments can easily be used in densely populated areas and communities, where assessing the instruments’ reliability and validity is easily recognizable.¹⁴

The rationale of the study involves the incorporation of the Rasch analysis used in healthcare systems to delude and evaluate the research questionnaires and the score of the instrument ^15,16. The research questionnaire also provides the estimated latent trait amount, displayed by each subject, as also mentioned as the “person measure” and the amount of the trait necessary to respond in a certain way to each item, also corresponded as “item measure”¹⁷. Additionally, the research questionnaire provides the fittest statistics of the items listed in order to indicate whether the individual item is rightly contributing to the measurement of the latent trait or is it otherwise.^17,18,19

This research aims to evaluate the OSDI-Q measurement for accuracy, separation index, and reliability at the item and individual levels within the Saudi population, mainly aged between16 to 40. This age gap will assess the research item or instrument, while providing the fittest statistics to investigate the unidimensionality of the disease and the extent of severity. This research explores the potential Differential Item Functioning (DIF) across the ages and the gender difference within the set age range for the questionnaire. Hence, conducting the reliability and the validity analysis in this research is assessed by using the Cronbach's coefficient, McDonald's coefficient, and item-rest correlations. These correlations test have proved to have a great deal of importance in assessing the unidimentionality constraint on the severity of the case ensued.

A total of 634 participants aged 16–40 years who visited the two locations of the hospitals were selected by a systematic random sampling technique. The research aimed to use the inclusion and exclusion criteria to effectively meet the objectives and goals of the study. Hence, between March 2021 to March 2022, this cross-sectional study was conducted in order to utilize the efficiency of the OSDI-Q. the volunteers were randomly picked at both, the King Saud University Campus Clinic and Kingdom Hospital in Riyadh, Saudi Arabia.

The inclusion criteria were of the research study was the involvement of healthy individuals, non-contact lens wearers, participants with no history of systemic medication or ocular diseases, and those without refractive surgery²⁴. The OSDI-Q was conducted using the OSDI application softwarer. Allergan, Inc. OSDI questionnaire application was downloaded from the Apple store on the iPad for free. The questionnaire application allows you to export the questionnaire results to all participants aged 16–40 at two locations, where the participant’s doubts were addressed.

The inclusion criteria involve participants who have contracted the disease in some point in time and the severity of the disease has yet again become a core concern for them. This may also include participants who have a hereditary issue of dry eyes disease from their ancestors. Respondent who have weak eyes or who wear contact lenses, also those who are on systemic medication and ocular dysfunction were endorsed to be a part of the research study to rectify the issues and help researchers come up with a better understanding of the notion.

The sample calculation was performed using an online sample size software ²⁵ for better prevalence studies. Considering our population to be infinite, since the questionnaire was distributed to participants living in Riyadh, a 95% confidence level, a DED prevalence of 70% in computer users, and a study accuracy of 5%, a total of 323 participants was determined to be the necessary sample size for the implementation of the research study. Therefore, based on the sample size, this study required a minimum of 646 participants for both age groups. The response rate of the questionnaire was reported to be 98.14% since the participants aged 27–40 refused to participate in the study for reasons that are known to none. The sample size was carefully sought by the researchers to involve a wide range of people coming from diverse cultures and regions, so as to have a better understanding of the notion and where the issue arises from. This will also give the researchers a clear understanding of the disease and how to cure it at the source, hence knowing the root cause of the issue.

The OSDI-Q assesses the DED symptom severity, and DED severity impact on vision-related functioning it consists of 12 symptoms, evaluated on the basis of the frequency of the severity of the disease, its occurrence, and intensity. As mentioned in the introduction section. The system score can range from 0 to 100, where, 0–12 is normal, 13–22 mild dry eye, 23–32 moderate dry eye, and 33–100 severe dry eye. Each answer is scored using a 5-point scale where 0 indicates no symptoms and 4 indicates significant symptoms.⁸

This Research Ethics Committee approve the research study. All participants provided written informed consent and retained the right to withdraw from the study at any point. The research adhered to the principles set forth in the Declaration of Helsinki.

Software from SPSS Inc., Chicago, Illinois, USA, version 28.0 of the Statistical Package for Social Sciences, was used to analyze the research data. The data was presented usingvarious metrics, including means, standard deviations (SDs), and percentages. To evaluate the scale's internal consistency, reliability analysis was performed, utilizing both Cronbach's α coefficient and McDonald's ω coefficient, along with item-rest correlations, as applicable in a systematic operation. In order to assess the precision, separation index, and reliability for both items and individuals, item and person fit statistics, unidimensionality, and Differential Item Functioning (DIF) stratified by age and gender, Rasch analysis was carried out using Winsteps 5.6.1. Statistical significance was established at P<0.05 to gauge the level of association between the variables. The Rasch analysis was used to provide a framework model to help researchers develop and compare an empirical dataset to assess an instrument's properties of measurement. Precisely, the unidimensionality serves to quantify the parameters set in the questionnaire. The Cronbach’s coefficient was used to assess the internal consistency and the reliability of the instrument used in the subject matter.

The initial step involves assessing the separation index and reliability for both, the items and persons. When a person's skill can be differentiated across at least three strata, their person separation index (PSI) is 2. The cut-off in this investigation was 2 PSI, which is the minimal acknowledged degree of discriminating for an instrument to yield a valid measure, hence being consistent with the foundation of the study. Age groups in the research analysis can be distinguished using an instrument with a higher PSI ^15,20. Item separation provides an indication of how effectively a group of participants can differentiate among the items employed in the instrument. The item hierarchy is checked using item separation. In cases of low item separation, it implies that the sample size is not large enough to establish a clear hierarchy of item difficulties within the instrument. This hindrance causes a deflect in the research system and also implies the use of a larger sample size to assess an easier connotation and interpretation of the results. This situation typically arises when there are fewer than three distinguishable levels of item difficulties and the item reliability is below 0.9^15,20. When expressed as reliability, these statistics fall within the range of 0.0 to 1.0. A higher value indicates a robust separation and a greater measurement of the precision values. To put to test, Item fit mean square statistics were employed to evaluate the item and person fit statistics, where values within the 0.5 to 1.7 mean squares range are generally considered suitable. Using item fit mean square statistics, unidimensionality can be investigated to reserve a better understanding of the research statistics.²¹

Two key concepts to consider are unidimensionality and local independence. Unidimensionality helps suggest the researchers that only one-dimension results play a major impact on the responses to the items. Local independence, implies that once the trait level is taken into account, the response to one item is not influenced by the responses to the other test items. Rasch results were subjected to Principal Component (PC) analysis to further evaluate unidimensionality. The variance described by the first contrast should have an eigenvalue of 2, which is <2.²²

The use of the Differencing Index Functioning helps determine the differences between the age groups and gender to identify if the age groups or gender respond differently to a particular item or instrument used in the OSDI Questionnaire. The age groups used in the questionnaire were 16-26 years vs 27-40 years. Differencing Index Functioning was regarded as missing in the questionnaire if the magnitude of the response was less than 0.50 logits, substantial Differencing Index Functioning was regarded as values between 0.50 and 1.00 logits, and significant Differencing Index Functioning was regarded as values over 1.00 logits. The size CUMLOR (Cumulative Log Odd Ratio) indicates the effect size of the Differencing Index Functioning.²³

Reliability analysis on the questionnaire was strategized in order to evaluate the internal consistency of the Likert scale used in the research study and its evaluation. Cronbach's α coefficient, McDonald's ω coefficient, and item-rest correlations were computed to evaluate the scale's reliability and validity of the results and responses associated with it. The acceptable and the least minimum Cronbach α and McDonald's ω coefficient is 0.70 ²⁴. Cronbach's alpha coefficient helps measure the internal consistency and the reliability of the survey items. The researchers use this statistic to assist in making that determination on a uniform scale of 0-1. Cronbach's alpha coefficient assesses the degree of agreement in the research notion. However, coefficient omega, which is based on a one-factor model, is a measure that compensates the drawbacks of the alpha coefficient. The formulation of coefficient omega closely resembles the concept of dependability when a one-factor model that can roughly explain the covariance among the items. The assessment of the bottom bound of dependability is known as the alpha. Congeneric measurements are the only need for it to provide genuine dependability of the context of the research implications. On the other hand, dependability under tau-equivalence can be provided via the omega coefficient.

This Research Ethics Committee has approved the research study. All participants provided written informed consent and retained the right to withdraw from the study at any point. The research adhered to the principles set forth in the Declaration of Helsinki.

OSDI-Q PSI for items and persons results in a clear and concise perspective of the results of the research. A total of 634 participants were enrolled in the study after completing their questionnaire. The OSDI-Q PSI for items measurements was 7.63 with a reliability of 0.98. This shows high confidence of items measures, with different levels of difficulty in the instrument items. OSDI-Q PSI for persons was 1.64 with acceptable reliability of 0.73. The criteria of the person’s PSI does not meet the discrimination among participants, with 73% of confidence about the measures of person. Figure 1 illustrates that there is good distribution of OSDI-Q items, and the ability ranging from -2 logit and above +2 logit, and the histogram shows a wide range of participants ability to answer the items.

The OSDI-Q items with mean square infit and outfit statistics of between 0.87–1.47 mean-square and 0.85–1.39 means-square, respectively, are acceptable. The reliability precision 0.77 indicates that the ability to answer the items is good as illustrated in table 1. The variance explained of persons and items was 5.8435 (29.3%) and 2.1226 (10.6%), thus the total variance explained by the measures was 7.966 (39.9 < 40%). The total unexplained variance was 12.0 (60.1%). Analysis of the principal component analysis (PCA) show the eigen value units for unexplained variance in 1st to 5th contrasts were < 2 except 1st contrast. The unexplained variance in 1st contrast was 2.80 (14.0%). Yet, by examining the clusters of 1st contrast and Pearson correlation coefficient (r) show there were three clusters of 1st PCA contrast: item cluster 1-3 with r = 0.2701 < 0.5, item cluster 1-2 with r = 0.3153 < 0.5, and item cluster 2-3 with r = 0.3211 < 0.5, which indicated the unidirectionality of the items. Table 1

The results of analysis for OSDI-Q items based on age groups (16-26 years) and (27-40 years) show that the Differencing Index Functioning contrast was significant for poor vision (Rasch-Welch t-value = -4.49, p = 0.00001 < 0.05) and for areas with low humidity (Rasch-Welch t-value = 2.69, p = 0.007 < 0.05). While the Mantel-Haenszel test suggested a significant DIF effect at 5% level of significance for poor vision (Mantel-Haenszel Chi-square test = 13.4438, p = 0.0002 < 0.05) and for areas with low humidity (Mantel-Haenszel Chi-square test = 5.9734, p = 0.0145 < 0.05). The values of CUMLOR were negative for poor vision which indicated better performance for age group (16-26 years) while the values of CUMLOR were positive for areas with low humidity which showed better performance for age group (27-40 years) as shown in the Table 2.

The results of DIF analysis for OSDI-Q frequency items based on gender show that the Differencing Index Functioning contrast was significant for blureed vision (Rasch-Welch t-value = 3.59, p = 0.0004 < 0.05), for poor vision (Rasch-Welch t-value = 3.43, p = 0.0007 < 0.05), and for driving at night (Rasch-Welch t-value = -8.67, p = 0.0000 < 0.05). While the Mantel-Haenszel test suggested a significant DIF effect at 5% level of significance for blureed vision (Mantel-Haenszel Chi-square test = 0.77, p = 0.0001 < 0.05), for poor vision (Mantel-Haenszel Chi-square test = 5.9519, p = 0.0147 < 0.05), and for driving at night (Mantel-Haenszel Chi-square test = 45.4022, p = 0.0000 < 0.05). The values of CUMLOR were negative for driving at night which indicated better performance for male group, while the values of CUMLOR were positive for blurred vision and poor vision which showed better performance for female group as shown in the Table 3.

The OSDI-Q items scale has strong coefficient of Cronbach's and McDonald's coefficients i.e., 0.85, suggesting stability and consistency over 12 items. The scale demonstrated good internal consistency. These values suggest that the OSDI questionnaire is reliable in measuring ocular surface disease-related symptoms. Regarding the individual items, when each item was dropped from the scale, the item-rest correlations ranged from 0.376 to 0.657. This indicates that all items have a reasonable degree of association with the overall scale score. Additionally, the Cronbach's α coefficients for each item, when removed, ranged from 0.824 to 0.845, and the McDonald's ω coefficients ranged from 0.827 to 0.848. These values suggest that all items contribute positively to the internal consistency of the OSDI questionnaire, further affirming its reliability. This suggests that the OSDI questionnaire to be used assessing ocular surface disease-related symptoms and can be confidently used to measure the impact of such symptoms on individuals. Researchers and healthcare professionals can rely on the OSDI questionnaire to provide consistent and dependable assessments of ocular surface disease symptoms in their studies and patient evaluations. Table 4.

To accurately determine the presence or absence of a disease, research questionnaires need to be culturally diversified and tailored to meet requirements and goals. Hence, it can be used to evaluate its validity and reliability, especially in diverse populations and communities²⁵. This work examines the psychometric features of the OSDI-Q through the Rasch analysis when administered to a Saudi population in the targeted hospitals even though the OSDI-Q psychometric analysis has already been undertaken.

The OSDI-Q items, the PSI was 7.63 with a reliability of 0.98. This indicates that persons are different, and our measurements differentiate at more than two levels of ability among participants, and 80% of confidence limit of the person’s measurements. The persons separation index was 1.64 and reliability 0.73. The criteria of the person’s PSI does not meet the discrimination among participants, with 73% of confidence about the measures of person.

The OSDI-Q items the mean square infit and outfit statistics demonstrated acceptability, with values falling within the ranges of 0.87–1.47 and 0.85–1.39, respectively. The reliability precision was 0.77, indicating an acceptable ability to answer items. The histogram reports a wide range of participants' ability to answer items. The PCA show the eigen value units for unexplained variance in 1st to 5th contrasts were < 2 except 1st contrast. The unexplained variance in 1st contrast was 2.8009 (14.0%). Yet, by examining the clusters of 1st contrast and Pearson correlation coefficient (r) show there were three clusters of 1st PCA contrast: item cluster 1-3 with r = 0.2701 < 0.5, item cluster 1-2 with r = 0.3153 < 0.5, and item cluster 2-3 with r = 0.3211 < 0.5, which indicated the unidirectionality of the items.

In previous studies, assessment OSDI-Q psychometric characteristics involved the utilization of Rasch analysis, mean square error (MNSQ) statistics for infit and outfit, exploration of unidimensionality, and examination of differential item functioning based on gender and age.^25,26 It is worth noting that disparities in Rasch analysis findings may be attributed to the study design, analysis methods, sample size, characteristic of participants age and gender. The study was conducted on 241 patients who had a history of DED. The Rasch analysis shows OSDI-Q the most unidimensional of all items the PSI was 2.70 with a reliability of 0.88. All items Misfit was 7/12 (58%) and PCA 2.20, indicating compatibility with a unidimensional Rasch model.²⁵ Also, the OSDI was administered to female participants mean age 63 ± 8 years, post menopause. All 12 items had infit mean square statistics within the acceptable range of 0.7 to 1.3. PSI 2.16, which indicates that the OSDI can adequately discriminate between patients. PCA shows the first contrast was 2.6 i.e., 11.1% of the total variance), which is more than can be attributed to random data. The second contrast had an eigenvalue of 1.6, or 6.6% of the total variance. The instrument does not meet the standard of unidimensionality when tested using PCA of the model residuals.²⁶

In the current study, the results of Differential Index Functioning analysis of OSDI-Q items based on age groups show that the Differential Index Functioning contrast was significant for poor vision of people who live in areas with low humidity. The values of CUMLOR were negative for poor vision which indicated better performance for age group (16-26 years) while the values of CUMLOR were positive for areas with low humidity which showed better performance for age group (27-40 years). Furthermore, the results of DIF analysis based on gender show that the Differential Index Functioning contrast was significant for blurred vision, poor vision and people who drive at night. The values of CUMLOR were negative for driving at night which indicated better performance for male group, while the values of CUMLOR were positive for blurred vision and poor vision which showed better performance for female group. Till date, there is no Differential Index Analysis on OSDI-Q items conducted to exemplify.

The OSDI-Q items scale has the strongest internal consistency with Cronbach's and McDonald's coefficients of 0.85, suggesting the stability and consistency of the research questionnaire over 12 items. The scale demonstrated good internal consistency, reliability, and validity of the research notion of measuring ocular surface disease-related symptoms. All items of the research study have a reasonable degree of association with the overall scale score and contribute positively to the internal consistency of the OSDI questionnaire that further affirms its reliability. These finding suggests that the OSDI questionnaire is a reliable tool for assessing ocular surface disease-related symptoms and can be confidently to measure the impact of such symptoms on individuals. Researchers and healthcare professionals can rely on the OSDI questionnaire to provide consistent and dependable assessments of ocular surface disease symptoms in their studies and patient evaluations. The research finding is supported by previous, conjugated studies that reported the OSDI-Q items scale has a robust internal consistency with Cronbach’s ranging from 0.90 -0.92. ^9,14

OSDI-Q passed the Unidimensionality and the Differential Index Functioning test through all the goods displayed through appropriate infit and outfits. Differential Index Functioning analysis results of the OSDI-Q items show age group (16-26 years) which performed better in poor vision while age group (27-40 years) performed better in areas with low humidity. This notion can be further simplified in the sense of fractionating the respondents of the research by means of the region and their cultural diversity. Furthermore, the results of DIF analysis items in the male group performed better in driving at night, while the female group showed better performance in blurred vision and poor vision. The OSDI-Q items scale shows the reliability data for Cronbach's scale. It also shows high internal consistency and reliability. This scale can be used confidently by the academia, industry professionals, and the medical associations and communities to accurately depend on the measurements of the OSDI-Q frequency items and categories.

Accepted for Publication: December 8, 2023.

Published Online: 31-May-2024. doi:10.70706/jamaophthalmol.2024.7197

Open Access: This is an open access article distributed under the terms of the CC-BY-NC-ND License. © 2024JAMA Ophthalmology.

Corresponding Author: Haya Alfarhan, PhD, Optometry Department, Collage of Applied Medical sciences, King Saud University, Riyadh, Saudi Arabia (Halfarhan@ksu.edu.sa).