Adaptation of Internet Gaming Cognition Scale to Turkish: Reliability and Validity Study
Suleyman Cakiroglu, Vahdet Gormez
Istanbul Medeniyet University Goztepe Education and Research Hospital
Background: This study aims to investigate the validity and reliability of the Turkish version of Internet Gaming Cognition Scale.
Method: The sample of the study was created by reaching 327 adolescents and adults playing games through social media tools, gaming forums and platforms by using an online form created on the internet. Language equivalence study of the scale was conducted by two experts, one of whose native language was English and fluent in Turkish, and the other, vice versa. Reliability of the questionnaire was evaluated by the test-retest method and Cronbach’s alpha internal consistency analysis. Construct validity of the scale was examined by confirmatory factor analysis. SPSS and AMOS 26 software were used for statistical analyses.
Results: Cronbach’s alpha coefficient for internal consistency was 0.91. Item-total score correlations varied between 0.32 and 0.69 (p<0.01). Test-retest score averages did not differ for both the total and subgroup scores. Test-retest correlation was calculated as 0.849 (p<0.01). Confirmatory factor analyses for construct validity showed that factor loadings of items varied between 0.34 and 0.73.
Conclusion: This study concluded the validity and reliability of the Turkish version of Internet Gaming Cognition Scale.
Keywords: adolescent, gaming disorder, cognition, scale
Playing games offline or online has become a problem, and the frequency of which is increasing rapidly in all age and gender groups . Children and adolescents, in particular, are more sensitive to psychological problems that playing games excessively can cause . In the diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5), this problem was identified as Internet Gaming Disorders (IGD) for the first time, and it was defined as continuous and repetitive playing that causes clinically significant impairment or distress . Although it has not yet taken its place among diagnoses in DSM-5, it has been added as a problem “that needs further research” in the third part of DSM-5. In the 11th version of the International Classification of Diseases-ICD (ICD-11) manual, Gaming Disorder has entered the diagnostic system, and it has been accepted as a disorder . The ICD 11 defined the disorder as a situation with significant impairment in functioning by losing control on gaming, giving priority and creating time to play games to the extent of limiting daily life activities and continuing to play games despite negative results . With these developments, the need to understand the disorder more and to find out intercultural differences has become important.
The importance of cognitions about playing games is emphasized in the stage of defining gaming as a disorder by reaching a level that will negatively affect the individual’s life [5, 6]. These cognitions are based on misattributes of games and awards and irrational beliefs that the game meets the need for self-esteem . Identification of these irrational cognitions becomes very important in the treatment of the disorder . Identifying and reassessing distorted cognitions and showing that behavioural changes occur according to new evaluations and attributes are highly valuable in cognitive behavioural therapy (CBT). Shaping and justifying behavioural interventions with new cognitive attributes will be important for the effectiveness of the treatment .
King and Delfabbro clustered cognitive factors about gaming in four groups . These are: 1) beliefs about the value and tangibility of game reward, 2) maladaptive and inflexible rules about gaming behaviour (rules and bias), 3) over-reliance on gaming to meet self-esteem needs and 4) gaming as a method of gaining social acceptance [5, 6]. It has been stated that cognitions related to gaming can be evaluated in a spectrum ranging from normal cognition to dysfunctional beliefs. However, in addicted groups, it has been emphasized that in addition to inflexible rules and bias about gaming behaviour, main dominant belief is self-esteem, which is thought to be gained by playing games . By focusing on the four cognitive areas stated, Internet Gaming Cognition Scale (IGCS) was developed. The scale aims to differentiate specific cognitions that carry game-play behaviour to the extent of the disorder [5, 6, 9]. At the same time, the scale has managed to adapt the more general attributes to the behaviour of gaming [9, 11]. The model which assesses four cognitive areas has become quite important to develop a treatment model specific for the disorder. In addition, it is very important to show the cultural adaptation of the scale .
This adaptation study aims to investigate the psychometric properties of the Turkish version of Internet Gaming Cognition Scale in adolescent and adult age groups. Psychometric properties were found by evaluating the factor structure, internal consistency and test-retest reliability.
Data Collection Process: The sample of the study consisted of adolescents and adults engaged in online game-playing, and we reached them through social media tools (e.g. Facebook and twitter), gaming forums and platforms (e.g. twitch) by using an online form. The study was conducted in January 2020. The inclusion criterion was “being engaged in any activity related to a game played online and/or offline on a desktop/laptop computer, tablet, smartphone, game console or any other device”. A total of 354 individuals who met the inclusion criterion and agreed to participate in the study formed the sample of the study. Information text about the aim and process of the research was placed on the cover of the online form to be filled in by the participants. It was emphasized in the information text that the information provided by the participant would not be used outside the research and that the participant could leave the research at any time. A total of 27 participants were excluded from the study since they had more than 10% missing information in the scales they completed, and the analyses were performed with 327 individuals. It has been emphasized that a sample size of 300 and more individuals is enough for a factor structure to be measurable in scale adaptation studies . Ethics Committee approval was obtained from Istanbul Medeniyet University, Göztepe Training and Research Hospital (06.11.2020-2019/0448).
Sociodemographic Characteristics and Gaming Activity Information Form: This information form was created by the researchers to evaluate the socio-demographic characteristics (age, gender, educational status) and gaming habits (ways of accessing the internet, weekly hours of gaming, types of games played) of individuals included in the study.
Internet Gaming Cognition Scale: The scale was developed by King and Delfabbro (2016) in order to assess cognitions incompatible with gaming . The scale has 4 sub-dimensions and 24 items. The sub-dimensions are beliefs about game reward value and tangibility (4 items), maladaptive and inflexible rules about gaming behaviour (8 items), over-reliance on gaming to meet self-esteem needs (7 items) and gaming as a method of gaining social acceptance (5 items). In addition, under each dimension 17 sub-dimensions were defined in which sub-cognitive structures were identified. The scale has 3-Likert form which evaluates whether the participants agree with the question they are asked. The scale is scored as disagree (0 points), agree (1 point) and completely agree (2 points). According to the relevant instructions, a score of 1 and higher shows the presence of distorted cognition. The scores vary between 0 and 48 and high scores show higher irrational gaming cognitions. Cronbach alpha values of the scale were .87 for game reward value sub-dimension, .81 for maladaptive and inflexible rules sub-dimension, .90 for self-esteem sub-dimension and .85 for social acceptance sub-dimension [5, 6]. Total Cronbach alpha value of the scale reported in Chinese adaptation study was .91 .
Translation of the scale into Turkish
Language translation and cultural adaptation stages of the scale were organized according to the guide reported by World Health Organization . The scale was translated into Turkish from English by two independent translators, and the first draft form was made by the research team using the forms translated into Turkish. The first draft form was translated back into English by two independent translators. The original scale, the Turkish version of the scale and back-translated English versions were evaluated by the research team in terms of meaning and language, and the first form was created . In order to evaluate whether each item was sufficient or suitable to measure the specified behaviour in terms of content and quality, an expert evaluation form was created with Lawshe technique. With Lawshe (1975) technique, the ratings of expert opinions for each item were arranged as “Appropriate”, “Appropriate but should be edited” and “Should be omitted”. The form which was created was evaluated by a panel of 10 people experts in “Child and Adolescent Psychiatry”, “Assessment and Evaluation” and “Linguistics” in terms of its suitability with the Turkish language and culture . In line with the views of experts, necessary analyses were performed. With the analysis results, content validity rates and content validity index were determined, and it was decided whether each item should be in the scale. Next, a pilot study was conducted on a group of 30 people in order to evaluate whether the items were clear. The pilot study was conducted in the form of face-to-face interviews with young people aged 15-20 years, and the level of their understanding of each item was discussed. The participants evaluated the form as understandable and fluent and the present form of the scale was created.
The data in the study was analyzed by using the Statistical Package for the Social Sciences-IBM Software, version 24 and SPSS-IBM AMOS 22 programs. The participants’ sociodemographic characteristics were found by using descriptive statistics (number, percentage, average, standard deviation). Content validity of the scale was evaluated by using the content validity index. The quality of the items and the distinctiveness of item scales were analyzed with item analysis. Confirmatory factor analysis (CFA) with maximum probability estimation was performed to confirm the 4-factor structure of the original scale. Model fit was evaluated through model fit indices (X2, df, X2/df, CFI, IFI, RMSEA, RMR, SRMR). Whether the data were suitable for factor analysis was evaluated with the Kaiser-Meyer-Olkin (KMO) coefficient and Bartlett significance test. Internal consistency of the scale was calculated by using the Cronbach Alpha (α) coefficient. The invariance of the test over time was evaluated by using the test-retest method [17, 18]. The significance level was set as p <0.05 in all statistical tests.
327 individuals between the ages of 13 and 66 participated in the study. 43% of the participants were female and 57% were male. 85% of the participants stated that they used a smartphone as a gaming tool. 8 individuals (2,4%) reported their weekly gaming hour as 40 hours and more. The participants reported that they played crossword, strategy and sports games the most.
Content validity of the scale was evaluated by using the content validity index. Total content validity index obtained based on the views of 10 experts was 0.90. Content validity index of the scale items varied between 0.80 and 1.00. These values were well above the minimum value reported in the literature . As a result, it was found that the scale had a high level of content validity. The scale was tested on 30 individuals with a pilot application. The evaluation of the fluency and understandability of the items did not report any negative results.
The suitability of the relationship between sample size and variables for factor analysis was evaluated with KMO and Bartlett significance test. KMO coefficient was found as 0.93, while Bartlett significance test was found to be statistically significant (X2:3099.261, df:276, p<.001). Construct validity of the scale was tested with confirmatory factor analysis. Standardized factor loads of scale items were found to vary between 0.34 and 0.73. As a result of factor analysis, a modification was recommended for improvement in the goodness of fit indices. Taking the modification recommendations into consideration, the error variances between item 7 – item 21 and item 18 – item 19 were combined. As a result of this, fit indices were brought to an acceptable level. Confirmatory factor analysis Path diagram is shown in Figure 1, and the fit indices of the scale are shown in Table 1, respectively.
The quality of item scales was evaluated by using item analysis. According to item analysis results, corrected item-total correlation coefficient was found to be between 0.32 and 0.69 (Table 2).
Internal consistency of the scale was evaluated by using the Cronbach alpha coefficient. Cronbach alpha coefficient of the total scale and sub-dimensions are given in Table 3.
For the assessment of test-retest reliability, the scale was completed by 30 high school and university students aged 15 to 20 years for a second time after three weeks. No statistically significant difference was found between the total scores obtained from the first and second application (t:0.692; p:0.426). In addition, a positive, multidirectional and highly significant relationship was found between the total scores obtained (r:0.849; p<0.001).
Gaming disorder is gradually becoming more prevalent in all age groups . Treatment methods are needed to get rid of the harmful effects of the disorder . Understanding the various areas of incompatible cognitions related to internet games and the relationship of these with the disorder are very important for the treatment . Cognitions can be changed potentially in terms of habits of thinking about a situation and understanding these cognitions can facilitate the development of effective behavioural interventions to reduce the disorder [9, 20]. The present study aims to adapt Internet Gaming Cognition Scale into Turkish and to evaluate its psychometric characteristics. The results of the present study are very valuable for testing the validity of cognitive attributions in different cultures. The original factor structure of the scale defines four sub-dimensions . In the Chinese version, it has been stated that a three-factor structure has a better fit . Sub-dimensions of this three-factor version were classified as a perception of reward, perception of craving for gaming and reluctance to quit gaming. In addition, it was stated that the strongest relationship was in the sub-dimension of the perception of craving for gaming . The results of the present study showed that the model adapted to the four-factor original structure. The need for a new model in the Chinese version was stated as fit indices not giving enough results as a result of Confirmatory Factor Analysis. In addition, with this new model, it was emphasized that the model could expand the areas of intervention related to cognitive attributes . It is noteworthy that the dimensions of the Chinese version emphasize the results of the disorder rather than the cognitions. It is thought that the reluctance to quit gaming and craving for it emphasizes the cognitions about quitting gaming after gaming process reaches the stage of the disorder. Emphasis should be placed on the importance given to cognitions related to gaming, which becomes distorted as gaming progresses into becoming a disorder. Only in this way can the processes progressing to disorder be reversed.
Internal consistency of the scale was measured with Cronbach Alpha reliability coefficient. In the original scale, Cronbach Alpha reliability coefficient was reported as .87, .81 .90 and .85, respectively for each sub-dimension . Cronbach Alpha reliability coefficient of the Chinese version was reported as .91 . In the present study, Cronbach Alpha reliability coefficient of the Turkish version of the scale was found as .92 and as .67, .76, .81 and .81, respectively for sub-dimensions. In literature, the lower value of Cronbach Alpha coefficient is stated as 0.7, while the higher value is stated as 0.90 . This result shows that the internal consistency of the Turkish form has a good level.
In item-total correlation, values lower than .20 show that the items have poor understandability and that they should be omitted from the scale . According to the item analysis result, the corrected item-total correlation coefficient was found to be between 0.32 and 0.69. It is stated in the literature that these values should be between 0.20 and 0.80 . In this case, it was shown that the items were understood correctly, similar reactions were given by the participants, and corrected item-total correlation coefficient values were at very good levels.
Within the scope of the construct validity of the scale, Confirmatory Factor Analysis was conducted. It is stated that before factor analysis, Kaiser-Meyer-Oklin (KMO) and Barlett Sphericity tests should be examined to evaluate the suitability of the data set for factor analysis . In order to decide that the data set is suitable for factor analysis, KMO value should be higher than 0.60 and the result of the Bartlett test should be significant . For this scale, KMO coefficient was found as 0.93 and Bartlett significance test was found to be statistically significant (X2:3099.261, df:276, p<.001).
A modification was recommended for fit indices to reach an acceptable level as a result of the CFA conducted by adhering to the structure in the original version. The model is expected to show a better fit by providing the recommendations of modification . Figure 1 shows the CFA model and the modifications. Analyses results after each modification are as seen in Table 1. Fit indices according to CFA after modifications are as follows: X2/df:2.734, CFI: .855, IFI: .856, RMSEA: .073, RMR: .016, SRMR: .625. In addition, standardized factor loads of the scale items were found to vary between 0.34 and 0.73. CFA results were statistically significant, appropriate factor loads were shown, and the scale was found to have sufficient psychometric characteristics [17, 25].
The test-retest method was used to evaluate the reliability of the scale over time. In the study, no statistically significant difference was found between the total scores obtained from the scale applied with an interval of 2 weeks. In addition, a positive, multidirectional and highly significant relationship was found between the total scores obtained. The results show that the scale is suitable in terms of consistency and stability over time.
Conclusion and recommendations
The study has some limitations. First of all, the use of only online questionnaire prevents reaching all fractions (offline and game console). In addition, the fact that the data were collected only through self-assessment should be considered in terms of bias in answers.
In spite of all these potential limitations, the Internet Gaming Cognition Scale is a valid and reliable scale that can be used in Turkish society. The results support the use of Turkish version as a valid and reliable tool to show gaming cognitions among adolescents, young individuals and adults.
Conflicts of Interest
The authors have no conflicts of interest to declare.
Source of Funding
Authors did not utilize any funding from public or private facilities for performing this study.
 Király O, Griffiths MD, Demetrovics Z. Internet gaming disorder and the DSM-5: Conceptualization, debates, and controversies. Curr Addict Rep 2015;2(3):254-62.
 Witt EA, Massman AJ, Jackson LA. Trends in youth’s videogame playing, overall computer use, and communication technology use: The impact of self-esteem and the Big Five personality factors. Comput Hum Behav 2011;27(2):763-9.
 American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®): American Psychiatric Pub;2013.
 World Health Organization. 2019 [cited 2020 Feb 1]. Available from: https://www.who.int/features/qa/gaming-disorder/en/
 King DL, Delfabbro PH. Internet gaming disorder treatment: a review of definitions of diagnosis and treatment outcome. J Clin Psychol 2014;70(10):942-55.
 King DL, Delfabbro PH. The cognitive psychopathology of Internet gaming disorder in adolescence. J Abnorm Child Psychol 2016;44(8):1635-45.
 Billieux J, Potenza MN, Maurage P, Brevers D, Brand M, King DL. Cognitive factors associated with gaming disorder. In: Verdejo-Garcia A ed. Cognition and Addiction A Researcher’s Guide from Mechanisms Towards Interventions. Academic Press;2020. p. 221-30.
 King DL, Delfabbro PH. Internet gaming disorder treatment: a review of definitions of diagnosis and treatment outcome. J Clin Psychol 2014;70(10):942-55.
 Stevens MW, King DL, Dorstyn D, Delfabbro PH. Cognitive–behavioral therapy for Internet gaming disorder: A systematic review and meta‐analysis. Clin Psychol Psychother 2019;26(2):191-203.
 Marino C, Spada MM. Dysfunctional cognitions in online gaming and internet gaming disorder: A narrative review and new classification. Curr Addict Rep 2017;4(3):308-16.
 Forrest CJ, King DL, Delfabbro PH. The measurement of maladaptive cognitions underlying problematic video-game playing among adults. Comput Hum Behav 2016;55:399-405.
 Yu Y, Mo PK-h, Zhang J, Li J, Lau JT-f. Validation of the Chinese Version of the Revised Internet Gaming Cognition Scale among Adolescents in China: Maladaptive Cognitions as Potential Determinants of Internet Gaming Disorder. Int J Env Res Pub He 2020;17(1):290.
 Comrey AL, Lee HB. A first course in factor analysis. 2nd ed. Psychology press;2013.
 World Health Organization. 2009 [cited 2020 Feb 2]. Process of translation and adaptation of instruments. https://www.who.int/substance_abuse/research_tools/translation/en/
 Çapık C, Gözüm S, Aksayan S. Intercultural Scale Adaptation Stages, Language and Culture Adaptation:Updated Guideline. J Florence Nightingale Nurs 2018;26(3):199-210.
 Lawshe CH. A quantitative approach to content validity. Pers Psychol 1975;28(4):563-75.
 Büyüköztürk Ş. Sosyal Bilimler İçin Veri Analizi El Kitabı. 25th ed. Ankara: Pegem Akademi Publishing;2018.
 Koyuncu İ, Kılıç AF. The Use of Exploratory and Confirmatory Factor Analyses: A Document Analysis. Educ Sci 2019;44(198).
 Wilson FR, Pan W, Schumsky DA. Recalculation of the critical values for Lawshe’s content validity ratio. Meas Eval Couns Dev 2012;45(3):197-210.
 Forrest CJ, King DL, Delfabbro PH. Maladaptive cognitions predict changes in problematic gaming in highly-engaged adults: A 12-month longitudinal study. Addict Behav 2017;65:125-30.
 DeVellis R. Scale development:theory and applications. 4th ed. Los Angeles: Sage Publications;2017.
 Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use. 5th ed. Oxford University Press;2015.
 Tabachnick BG, Fidell LS, Ullman JB. Using multivariate statistics. 5th ed. Pearson Boston, MA; 2007.
 Meydan CH, Şeşen H. Structural equation modeling AMOS applications. 2nd ed. Detay Publishing;2011.
 Akyüz HE. Confirmatory Factor Analysis for Construct Validity: An Applied Study. Bitlis Eren Univ J Sci & Technol 2018;7(2):186-98.