Skin Picking Impact Scale: Reliability and Validity Study of Turkish Version
Julide Guler Kenara, Efruz Pirdogan Aydina, Ilknur Kıvanc Altunayb, Omer Akil Ozera, Kayıhan Oguz Karamustafalioglua
aUniversity of Health Sciences, Sisli Hamidiye Etfal Education And Research Hospital, Department of Psychiatry,Istanbul;
bUniversity of Health Sciences, Sisli Hamidiye Etfal Education And Research Hospital, Department of Dermatology Istanbul, Turkey
Background: In this study, we aimed to perform the validity and reliability study for Turkish version of Skin Picking Impact Scale (SPIS).
Methods: This study included 80 patients diagnosed with SPD according to DSM-5 diagnostic criteria. Patients were given Sociodemographic Data Form, Beck Depression Inventory (BDI), Beck Anxiety Inventory (BAI), Short Form-36 Quality of Life Scale (SF-36), Yale-Brown Obsessive Compulsive Scale Modified for Neurotic Excoriation (NE-YBOCS) and Turkish version of SPIS. Cronbach’s alpha coefficient and item-total correlation were examined for the reliability of the scale. Confirmatory factor analysis (CFA) was used to examine the factor structure of the scale and ROC (Receiver Operating Characteristics) was performed to the discriminatory validity analysis.
Results: Cronbach’s alpha internal consistency coefficient was determined to be 0.942. Item-total correlation coefficients of all questions ranged between 0.665 and 0.849. The one-factor structure showed a good model fit in CFA. Good correlations were observed between SPIS and number of sites of picking (rho=0.265, p=0.037), duration of skin picking per day (r=0.453, p<0.001), NE-YBOCS score (r=0[E1] .509, p<0.001), BDI (r=0.273, p=0.014), and total scores for SF-36 subscales (r= -0.262, p=0.019). [E2] SPIS was determined to be a good scale for distinguishing SPD patients and controls.
Conclusion: Our study shows that the ten-item version of SPIS, which consists of a single factor, can be used in a reliable and valid way for Turkish population.
Keywords: Skin picking disorder, scale, validity, reliability.
Skin picking is a body-focused repetitive behavior which may occur very commonly in humans. Skin picking behavior has been reported to be observed at high prevalence, reaching 46-90% [1–3] in university students and 17-63% in adults [4, 5]. Skin picking behavior can be classified as normal or pathological based on its frequency and intensity, and by considering if the behavior is of repetitive nature and if negative emotions or distress occur after the behavior . Picking behavior is of repetitive nature in some individuals and persists despite the suppression or reduction attempts; thus, a significant skin injury occurs resulting in marked distress or functional impairment. Such individuals meet diagnostic criteria for skin picking disorder (SPD).
SPD has been first included in diagnostic categories in DSM-5 . Epidemiological studies have reported that SPD affects about 1.2-5.4% of the population [1,4,5,8]. Patients spend most of their time during a day with skin picking and checking and concealing the site of picking (putting on make-up, applying bandage etc.) [9,10]. Patients often tend to be late for work or school because of the time they spend on skin picking. This disorder can also cause skin injury, ulceration, infection and serious sequelae , or even increased healthcare costs for the affected people . Injuries lead to shame and social isolation, and thus, the patients avoid most of the activities and spend most of their time at home with consequent negative effects on professional and academic life [9, 11]. One study reported lower psychosocial functioning and poor quality of life for patients with SPD compared to the healthy volunteers . A study in Israeli adults found increased disease severity, perceived stress, depressive and obsessive-compulsive symptoms and increased likelihood of diagnosis of SPD in first-degree relatives, and decreased psychosocial functioning in patients with SPD compared to the patients with subclinical skin picking disorder .
Keuthen et al. developed the Skin Picking Impact Scale (SPIS) to demonstrate the degree of psychosocial dysfunction caused by the disease severity in SPD and conducted a validity and reliability study. SPIS measures some emotional (e.g. “I feel embarrassed because of my skin picking”), social (e.g. “My relationships have suffered because of my skin picking”), and behavioral (e.g. “There are some things I can’t do because of my skin picking”) outcomes caused by skin picking. An increase in scale score indicates decreased psychosocial functioning . Scoring 7 and above on the scale indicates that the person has a self-destructive skin picking behavior. Functional impairment is a key criterion and the scale can be used as a determinant in differentiating between skin picking behavior and SPD. SPIS is a scale with good validity and reliability,  and is frequently used in treatment studies [15-17]. However, validity and reliability studies have not yet been conducted for Turkish version of SPIS. In this study, we aimed to perform the validity and reliability study for Turkish version of SPIS scale.
For the study, SPD patients diagnosed according to the DSM-5 diagnostic criteria were included among the patients who applied to the Dermatology and Psychiatry Outpatient Clinics and subsequently were referred to the Psychodermatology Outpatient Clinic.. [E3] Exclusion criteria were as follows: i) diagnosis of mental retardation or psychotic disorder; ii) current diagnosis or history of dementia; and iii) illiteracy. This study enrolled 80 SPD patients and 80 healthy controls without any psychiatric disorders. Participants were instructed on the purpose and design of the study, and the informed consents were obtained. The study was approved by the Ethics Committee of University of Health Sciences, Sisli Hamidiye Etfal Training and Research Hospital (Number: 1489).
Participants were given Sociodemographic Data Form, Beck Depression Inventory (BDI), Beck Anxiety Inventory (BAI), Short Form-36 Quality of Life Scale (SF-36), NE-YBOCS and Turkish version of SPIS scale. Other scales were chosen based on the original study to investigate correlations with the SPIS scale. Current comorbidities of patients were determined according to the Structured Clinical Interview for DSM-IV (SCID-I).
Skin Picking Impact Scale (SPIS): The scale was developed by Keuthen et al. from Trichotillomania Impact Scale (TIS) , a 28-item scale to evaluate the psychosocial outcomes of repetitive skin picking . This study concluded that the ten-item structure shows a high internal consistency following the psychometric analysis in 31 patients with self-destructive skin picking (provided that the patient meets the requirements for tissue loss due to repetitive skin picking and functional loss due to skin picking), and 78 university students with non-self-destructive skin picking. The self-report scale is 5-point Likert type. Items are scored from 0 to 5 with questions evaluating the period of last one week. Total score ranges between 0 and 50. The cut-off point is 7. Scores at or above the cut-off point differentiate the patients with self-destructive skin picking from those with milder skin picking. SPIS has a high internal consistency value (Cronbach’s alpha=0.93), and a good structural validity. SPIS scores of patients with self-destructive skin picking correlate with BDI (r=0.47, p<0.05), BAI (r=0.52, p<0.05), duration of skin picking (r=0.42, p<0.05), sense of satisfaction during skin picking (r=0.36, p<0.05), and shame following the skin picking (r=0.51, p<0.05), while no correlations are observed between SPIS scores of patients with non-self-destructive skin picking and above-listed measures. In an internet-based research, factor analysis was performed for SPIS in 650 patients with SPD. This analysis showed that the one-dimensional structure of the scale (eigenvalue=6.57) explained 62.02% of the variance and the item loadings were high, while the confirmatory factor analysis revealed the following values: χ2 = 60.28.86, df = 30, p < 0.001; RMSEA = 0.056; CFI = 1.0; NNFI = 0.99; thus, a good model fit was reported with these values. Cronbach’s alpha value of the scale was 0.94. SPIS was reported to be highly correlated with Sheehan Disability Scale (SDS) and Depression Anxiety Stress Scale (DASS) .
Sociodemographic Data Form: A data form was designed for study, which included questions about patients’ age, gender, educational status, employment status, family history of psychiatric disorders, age of onset, duration of disorder, sites of picking, time spent with skin picking per day, and frequency of skin picking.
The Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I): The interview was developed by First et al. for the current and lifetime prevalence of other psychiatric disorders . Its Turkish validity and reliability was prepared by Özkurkcugil et al. 
Beck Depression Inventory (BDI): A 21-item self-report scale was used to determine the presence and severity of depressive symptoms, for which the reliability and validity study for Turkish population was performed by Hisli .
Beck Anxiety Inventory (BAI): This is a self-report questionnaire designed to differentiate between anxiety symptoms and depression symptoms by questioning the common anxiety symptoms such as feeling angry, fear and fear of death using 21 items. It is a 4-point Likert-type scale. Reliability and validity study for Turkish population was performed by Ulusoy et al. .
Short Form-36 Quality of Life Scale (SF-36): This scale is used to measure the quality of life in patients with physical and psychiatric disorders, and in healthy subjects. It evaluates the eight dimensions of health including physical functioning, role limitations due to physical and emotional problems, social functioning, mental health, vitality (energy), pain and general health perception using 36 questions. The total score of eight subscales is calculated. Scores of subscales range between 0 and 100 with higher scores indicating good health status . Validity and reliability study of SF-36 was performed in Turkey .
Yale-Brown Obsessive Compulsive Scale Modified for Neurotic Excoriation (NE-YBOCS): This is a clinician-rated scale adapted from YBOCS scale by Arnold et al. to measure the severity of neurotic excoriation . It consists of 10 items with the first five items evaluating urge/impulse to skin picking, and the last five items evaluating skin picking behavior. Scoring is similar to that of YBOCS. Total score ranges between 0 and 40. Good reliability and validity of the scale were demonstrated in a treatment study .
For adaptation studies of SPIS scale, the permissions of the authors who developed the scale were obtained via e-mail. The scale was then translated into Turkish using translation-back translation method. Scale items were translated into Turkish by two psychiatrists with a good command of English and Turkish. Items translated into Turkish were then re-translated into English by a translator who is proficient in English. Afterwards, experts in the field performed an evaluation by comparing original and back-translated versions of the scale. All recommendations were reviewed, and the items of the Turkish version were edited.
SPSS 20.0 for Windows program was used for statistical analysis. Descriptive statistics were expressed as mean, standard deviation, minimum and maximum for numerical variables, and as number and percentage for categorical variables. Cronbach’s alpha coefficient and item-total correlation and Spearman-Brown reliability coefficients obtained by the split-half method were examined for the reliability of SPIS scale. A Cronbach’s alpha value of 0.70 and above indicates the good internal consistency of the scale. Data fit for factor analysis was examined by performing Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity. For data fit, KMO value should be higher than 0.60 and Chi-square value calculated by Bartlett’s test should be statistically significant . Confirmatory factor analysis (CFA) was used to examine the factor structure of the scale. . Model fit was evaluated via AMOS 22 program. The goodness of fit of the model tested in CFA was evaluated by several fit indices such as χ2 /sd, RMSEA, CFI, AGFI, GFI, NFI, and SRMR . For validity of the scale, Pearson’s correlation analysis was used in case of the normal distribution, and Spearman’s correlation analysis was used in case of the non-normal distribution. Student’s t test was performed to compare the SPIS score between the patients and healthy controls. ROC (Receiver Operating Characteristics) analysis was performed between the patients and healthy controls to determine the discriminate validity of the scale. Area under the curve (AUC), cut-off point, sensitivity and specificity values were presented. Statistical alpha significance level was considered to be p <0.05.
Sociodemographic of participants
Comparisons of sociodemographic data and SPIS score between patients and healthy controls were given in Table 1. The mean of age of patients was 32.7±14.9 and of the healthy controls was 25.4±8.7 and there was statistical difference (t=3.701, p<0.001). In terms of gender, 72.5% of the patients was female whereas 75% of controls was female and there was no statistical difference (χ²=0.129, p=0.719). The mean of education years of patient was 9.91±4.57, of the controls was 12.1±2.4 and there was statistical difference (t=-3.843, p<0.001). There was statistical difference between patients and controls in terms of marital status ( (χ²=0.682, p=0.033) while no statistical difference in terms of employment status (χ²=5.086, p=0.166).[E4] [E5]
Clinical characteristics of patients
Clinical characteristics, frequency of comorbidities and means of clinical scale of the patients were given in Table 2. The mean of age of onset was 21.4±15.1 years, the mean cited of picked was 2.6±1.3. 46.7 % of the patients (n=35) had family history of psychiatric disorder. 32.5 % of patients (n=26) had psychotropic treatments. The mean score of SPIS, NE-YBOCS, BDI, BAI, SF-36-Physical Health and SF-36-Mental Health were respectively 15.6±12.9, 19.7±6.2, 15.7±10.7, 17.5±14.1, 68.3±22.4 and 52.6±23.04. Current prevalence of onychophagia (30%), OCD (20%), OCD (10-36%), adjustment disorder (17.5%), generalized anxiety disorder (16.3%), anxiety disorder NOS (12.5%), somotoform disorder (12.5%), bipolar disorder (11.3%), major derpessive disorder (11.3%), trichotillomania (8.8%), special phobia (7.5%) panic disorder (6.3%) social phobia (5%), alcohol/substance addiction (3.8%), eating disorder (3.8%), dysthmia (2.5%), tic disorder (1.3%) and posttraumatic stress disorder (1.3%) was found in patient group according to the SCID-I[E6] . Sites of picking were back in 54.9% (n=43), face in 52.6% (n=42), fingers in 38.3% (n=31), trunk in 28.8% (n=23), legs in 26.3% (n=21), arms in 22.5% (n=18), scalp in 21.3% (n=17), genital area in 6.3% (n=5), endaural area in 3.8% (n=3), and foot in 3.8% of the patients. Picking frequencies per day were 4-6 times in 27.5% (n=22), 7-10 times in 23.8% (n=19), >20 times in 20% (n=16), 1-3 times in 17.5% (n=14), and 11-20 times in 11.3% (n=9) of the patients. Time spent with skin picking per day was 1-3 hours in 32.5% (n=26), 31-60 minutes in 32.5% (n=26), 16-30 minutes in 25% (n=20), 6-15 minutes in 7.5% (n=6), and >4 hours in 2.6% (n=2) of the patients.
KMO and Bartlett’s tests were used to evaluate the data fit for factor analysis. The analysis resulted in a KMO value of 0.848, and χ2 =503.69 and p<0.001 in Bartlett’s test, indicating suitability for factor analysis. [E7]
Confirmatory factor analysis (CFA): CFA was conducted for investigating factor structure and dimensionality of the SPIS. One-factor structure of the SPIS showing in orginal study was tested in CFA. Following values obtained in primary analysis indicated a poor model fit: χ2 =139.761, d.f=35, χ2 /sd=3.993 CFI=0.785, GFI= 0.753, AGFI=0.612, RMSEA=0.195, NFI= 0.737 and SRMR=0.082. Factor loadings were ranged between 0.613-0.814 and t values were ranged between 5.838-7.964 . If t values exceed 1.96, p value is at 0.005; t values exceed 2.56, then p value is at 0.001 . Error variances were ranged between 1.073-2.060 and t values were ranged between 5.198- 5.953 (p<0.001). Factor loadings were found sufficient and each item’s p value was at statistical significant . Hence, no item preferred to exclude from analysis. Thus, modification recommendations were considered. Based on these recommendations, errors in items 4 and 8 were associated with each other. Second model analysis resulted as unacceptable fit (χ2=97.957, d.f=34, χ2 /sd=2,881, CFI=0.869, GFI=0.805, AGFI=0.684, RMSEA=0.154, NFI= 0.816 and SRMR=0.072). And then errors in items 3 and 7 were associated with each other. Third analysis resulted as: χ2=75.027, d.f=33, χ2 /sd=2.274, CFI=0.914, GFI=0.850, AGFI=0.750, RMSEA=0.127, NFI= 0.859 and SRMR=0.061. Some fit indices were improved but RMSEA had not acceptable level. According to the modification indices, items 5 and 8 errors were associated and analysis repeated (χ2=58.722, d.f=32, χ2 /sd=1.838, CFI=0.945, GFI=0.883, AGFI=0.798, RMSEA=0.103, NFI= 0.890 and SRMR=0.057). RMSEA has not still been acceptable levels though the other fit indices were good. Based on modification suggestion, covariance was formed between items 6 and 9 errors . This resulted in following values: χ2=43.703, d.f=31, χ2 /sd=1.410, CFI=0.974, GFI=0.909, AGFI=0.838, RMSEA=0.072, NFI= 0.909 and SRMR=0.052. Thus, the fit indices reached acceptable levels. Loading values of one-factor structure ranged between 0.637 and 0.819 (p<0.001). Error variances were ranged between 0.870-2.090. And t values of unobserved variables changed between 3.763-6.019 (p<0.001)
Concurrent validity analyses: Correlations between SPIS and frequency of skin picking per day, duration of skin picking per day, number of sites of picking, NE-YBOCS severity, BDI, BAI and SF-36 subscale scores were examined to determine the validity of the scale. Spearman rank correlations between SPIS and the other scales was given in Table 3. Good correlations were observed between SPIS and number of sites of picking (r=0.265, p=0.037), duration of skin picking per day (r=0.453, p<0.001), NE-YBOCS score (r=0.509, p<0.001), BDI (r=0.273, p=0.014), and total scores for SF-36 subscales of physical health (r= -0.262, p=0.019) and mental health (r= -0.262, p=0.019), while poor correlations were observed with BAI (r=0.186, p=0.098) and frequency of skin picking per day (r=0.143, p=0.207) [E8]
Discriminatory validity analyses: The mean score of SPIS was 15.6±12.9 in patient group, 1.5±2.5 in healthy control group and there was statistical significant difference (t=9.586, p<0.001) (Table 1).
At the same time, ROC analysis was performed to evaluate whether it was a good criterion for distinguishing SPD patients and healthy control group. As a result of the analysis, AUC for SPIS was found to be 0.868 (95% CI = 0.810-0.927, p <0.001). >5 was accepted as a cut-off point for SPIS; its sensitivity was 72.5 % and specificity was 90 %.
Item-total correlation and Cronbach’s alpha coefficients were given in Table 4. Cronbach’s alpha internal consistency coefficient of SPIS was determined to be 0.942. Item-total correlation coefficients of all questions ranged between 0.665 and 0.849. Cronbach’s alpha coefficients if item deleted were ranged between 0.932-0.940. Spearman-Brown coefficient by split-half method was calculated as 0.935. The analysis revealed that the scale is highly reliable.
This study analyzed the validity and reliability of Turkish version of SPIS scale developed by Keuthen et al. . Overall, the scale consisting of ten items and one dimension was found to be valid and reliable after being translated into Turkish and examined for psychometric measures.
In our study,. values (χ2 =503.69, p<0.001) obtained in KMO and Bartlett’s tests suggested that our data was sufficient for factor analysis .). Factor analysis could not be performed due to the small sample size for SPIS in the original study of the scale . Snorrason et al. then performed the factor analysis for both the original and shortened versions of the scale in 650 patients with SPD in an internet-based research. Similar to our study, they reported that the one-dimensional factor structure (eigenvalue=6.57) explained 62.02% of the variance . However, a Brazilian study on the scale showed that the two-factor structure was valid and reliable .
We used CFA to test if the one-factor structure was confirmed. In the process of deciding the validity of the model established in CFA, goodness of fit indices, factor loadings, t values are taken into consideration . In our study, factor loadings were determined to be very good levels  and t values were found at statistical significant (p<0.001) while some items had high error variances. Even then, any items were not preferred to exclude from the model because of significance of t values. As the acceptable values could not be reached for CFI, GFI, AGFI, RMSEA, NFI and SRMR fit indices in primary CFA, and then modification recommendations were considered. In line with modification recommendations, errors in items 4 and 8, items 3 and 7, items 5 and 8 and items 6 and 9 were associated with each others. Last analysis showed a good model fit (χ2=43.703, d.f=31, χ2 /sd=1.410, CFI=0.974, GFI=0,909, AGFI=0.838, RMSEA=0.072, NFI= 0.909 and SRMR=0.052). As a result, even if the model with one- factor structure was demonstrated good fit and validity, the results need to be interpreted carefully caused by using too much modifications. The covariance values between the errors shown in the modification indices indicate the measurement error. It is thought that these measurement errors may be due to the characteristics of the respondents as well as the expressions of items . Errors of item 4 (“It takes me longer to go out because of my skin picking”) and 8 (“It takes me longer than others to get ready in the morning because of my skin picking”) which were associated with each other in our study are about behavior outcomes caused by skin picking. Errors of item 3 (“I hate the way I look because of my skin picking”) and 7 (“I feel unattractive because of my skin picking”) are about body perception or appearance caused by skin picking. These expressions of items as mentioned above may have been perceived in the same way by participants. Errors of item 5 (“I feel embarrassed because of my skin picking”) and 8 (“It takes me longer than others to get ready in the morning because of my skin picking”) and item 6 (“There are some things I can’t do because of my skin picking”) and 9 (“I don’t like people looking at me because of my skin picking”) which were associated with each others in our analysis, indicate social isolation, avoiding daily activities and shame caused by skin picking. Skin picking behavior lead to injuries, shame and subsequently checking and concealing the site of picking. Hereby, patients often tend to be late for work or school because of the time spending on skin picking or concealing behaviors. Moreover, patients avoid engaging intimial relationship or group activities or vacations for hiding skin injuries [9,11]. Expressions about embarrassment and behaviors related to avoidance may have been perceived as interweaving processes while patients was answering the questions. Our study population comprised patients who were referred from dermatology and psychiatry outpatient clinics. At this point, it is not wrong to say as an important part of patients have not awareness of their illness or tend to hide their problems. In addition, it is known that SPD patients are more impulsive  and have difficulties in recognizing and decribing emotions . Probably, the characteristics of the patients and similar aspects in the items expressions may have caused this confusion. So it is still important to consider the results carefully.[E9]
In line with our results, Snorrason et al. reported that the one-factor structure for SPIS showed a good model fit in CFA (χ2 = 60.28.86, df = 30, p < 0.001; RMSEA = 0.056; CFI = 1.0; NNFI = 0.99) . This study population included SPD patients from internet unlike our study which enrolled treatment seeking SPD patients. The difference in model fit indices between this study and ours may have been caused by this point. Our study found that the reliability of the one-factor scale was acceptable. Internal consistency coefficients of the factor were comparable to those in other studies in the field [6,14,30]. Reliability was considered to be sufficient based on the Spearman- Brown coefficient and Cronbach’s alpha level of >0.70 and item-total correlations of >0.30 .
Also within the scope of investigation on validity of the scale, SPIS was determined to be highly correlated with NE-YBOCS score and answers to questions “number of sites of picking” and “duration of picking per day” in Sociodemographic Data Form as well as BDI and total scores for SF-36 subscales of physical health and mental health. Other studies similarly reported that SPIS was highly correlated with DASS and SDS , BDI, BAI, emotions following the picking behavior , and CGI . In our study, score of SPIS was found different between SPD patients and healthy controls (p<0.001). AUC value of 0.868 was determined to be good in distinguishing between the patient and control groups . In the study of Keuthen et al. the cut-off point of the scale was reported to be 7 and above for differentiating between self-destructive skin picking and milder ones. The cut-off point has high sensitivity and specificity in differentiating between self-destructive and non-self-destructive skin picking . In our study, >5 was accepted as a cut-off point for distinguishing SPD patients from healthy controls; its sensitivity was 72.5 % and specificity was 90 %. Unfortunately, we could not perform to determine cut-off point for separating between severe skin picking and mild skin picking.
In conclusion, SPIS is a self-report scale which can rapidly provide insight on the degree of psychosocial dysfunction caused by skin picking behavior and the severity of disorder. Since this form of SPIS does not have to be applied by the clinician, it can be applied by dermatologists and psychologists consulted by patients with SPD, besides the psychiatrists. Our study evaluated the one-factor, ten-item version of SPIS by psychometric analysis and demonstrated that this version is valid and can be reliably used in our country. Future studies with a population-based sample can contribute the factor structure of the scale.
 Bohne A, Wilhelm S, Keuthen NJ, Baer L, Jenike MA. Skin Picking in German Students. Behav Modif. 2001;26(3):320–339. doi:10.1177/0145445502026003002
 Prochwicz K, Kałużna-Wielobób A, Kłosowska J. Skin picking in a non-clinical sample of young Polish adults. Prevalence and characteristics. Compr Psychiatry. 2016;71:77–85. doi:10.1016/j.comppsych.2016.08.013
 Keuthen NJ, Deckersbach T, Sabine Wilhelm M, Hale E, Fraim C, Baer L, et al. Repetitive Skin-Picking in a student population and comparison with a sample of self-injurious skin-pickers. Psychosomatics 2000;41(3): 210-5. doi: 10.1176/appi.psy.41.3.210
 Keuthen NJ, Koran LM, Aboujaoude E, Large MD, Serpe RT. The prevalence of pathologic skin picking in US adults. Compr Psychiatry. 2010;51(2):183–6. doi:10.1016/j.comppsych.2009.04.003
 Hayes SL, Storch EA, Berlanga L. An examination of the prevalence and severity in a community sample. J Anxiety Disord. 2009;23(3):314-9. doi:10.1016/j.janxdis.2009.01.008
 Keuthen N, Deckersbach T, Wilhelm S, Engelhard I, Forker A, O’sullivan RL, et al. The Skin Picking Impact Scale (SPIS): scale development and psychometric analyses. Psychosomatics 2001; 42(5): 397-403. doi: 10.1176/appi.psy.42.5.397
 American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th ed. Arlington: American Psychiatric Pub; 2013.
 Calikusu C, Kucukgoncu S, Tecer Ö, Bestepe E. Skin Picking in Turkish Students: Prevalence, characteristics, and gender differences. Behav Modif. 2016;36(1):49–66. doi:10.1177/0145445511420282
 Flessner CA, Woods DW. Phenomenological characteristics, social problems, and the economic impact associated with chronic skin picking. Behav Modif. 2006;30(6):944–63. doi:10.1177/0145445506294083
 Odlaug B, Grant JE. Clinical characteristics and medical complications of pathologic skin picking. Gen Hosp Psychiatry 2008; 30(1): 61-6. doi.org/10.1016/j.genhosppsych.2007.07.009
 Neziroglu F, Rabinowitz D, Breytman A, Jacofsky M. Skin picking phenomenology and severity comparison. Prim. Care Companion J. Clin. Psychiatry. 2008;10(4):306-12 doi: 10.4088/pcc.v10n0406
 Odlaug B, Kim S, Grant JE. Quality of life and clinical severity in pathological skin picking and trichotillomania. J Anxiety Disord. 2010; 24(8): 823-9. https://doi.org/10.1016/j.janxdis.2010.06.004
 Keuthen N, Curley E, Tung E, Ittah K, Qasem A, Murad S, et al. A comparison of clinical vs subclinical skin pickers in Israel. Ann Clin Psychiatry. 2016;28(2): 98-104.
 Snorrason I, Olafsson RP, Flessner CA, Keuthen NJ, Franklin ME, Woods DW. The skin picking impact scale: Factor structure, validity and development of a short version. Scand J Psychol. 2013;54(4):344–8. https://doi.org/10.1111/sjop.12057
 Aydin E, Kenar J, Altunay I, Kaymak D, Özer ÖA, Karamustafalioglu KO. Repetitive Transcranial Magnetic Stimulation in the Treatment of Skin Picking Disorder: An Exploratory Trial. J ECT. 2020; 36(1): 60-5. doi: 10.1097/YCT.0000000000000616
 Keuthen NJ, Jameson M, Loh R, Deckersbach T, Wilhelm S, Dougherty DD. Open-label escitalopram treatment for pathological skin picking. Int Clin Psychopharmacol. 2007; 22(5):268-74. doi: 10.1097/YIC.0b013e32809913b6
 Grant JE, Odlaug BL, Chamberlain SR, Kim SW. A double-blind, placebo-controlled trial of lamotrigine for pathologic skin picking: treatment efficacy and neurocognitive predictors of response. J Clin Psychopharmacol. 2010; 30(4): 396-403. doi:10.1097/JCP.0b013e3181e617a1
 O’Sullivan RL, Keuthen NJ, Ricciardi JN, Shera D, Savage CR, Borgmann AS, et al. The Trichotillomania Impact Scale, unpublished scale. 1994.
 First MB, Spitzer RL, Gibbon M WJ. Structured Clinical Interview for DSM-IV-TR Axis I Disorders (SCID). Research version, patient edition. New York, 2002.
 Özkurkcugil A, Aydemir Ö, Yıldız M. Adaptation and reliability study of Turkish structured clinical interview for DSM-IV Axis I Disorders. İlaç ve Tedavi Dergisi. 1999;12:233-6. [Turkish].
 Hisli N. A reliability and validity study of Beck depression inventory in a university student sample. Türk Psikoloji Dergisi – Turkish Journal of Psychology. 1989;7:3–13. [Turkish].
 Ulusoy M, Sahin NH, Erkmen H. Turkish version of the Beck anxiety inventory: psychometric properties. Journal of Cognitive Psychotherapy. 1998;12(2):163–72.
 Jenkinson C, Wright L, Coulter A. Criterion validity and reliability of the SF-36 in a population sample. Qual Life Res. 1994;3(1):7–12.
 Pinar R. Reliability and construct validity of the SF-36 in Turkish cancer patients. Qual Life Res. 2005;14(1):259–64.
 Arnold LM, Mutasim DF, Dwight MM, Lamerson CL, Morris EM, McElroy SL. An open clinical trial of fluvoxamine treatment of psychogenic excoriation. J Clin Psychopharmacol. 1999;19(1), 15-8.
 Grant JE, Odlaug BL, Kim SW. Lamotrigine Treatment of Pathologic Skin Picking. J Clin Psychiat. 2007;68(09):1384–91. doi: 10.4088/jcp.v68n0909
 Büyüköztürk Ş. Factor analysis: Basic concepts and using to development scale. Educational Administration:Theory and Practice – Kuram ve Uygulamada Egitim Yönetimi Dergisi. 2002; 8(4): 470-83. [Turkish].
 Schumacker RE, Lomax RG. A Beginner’s Guide to Structural Equation Modeling. 2nd ed. New Jersey, London: Psychology press; 2004.
 Harrington D. Confirmatory Factor Analysis. Madison Aveneu, New York:Oxford University Press;2009.
 Xavier ACM, Souza CMB de, Flores LHF, Prati C, Cassal C, Dreher CB. Improving skin picking diagnosis among Brazilians: validation of the Skin Picking Impact Scale and development of a photographic instrument. An Bras Dermatol. 2019;94(5):553–60. https://doi.org/10.1016/j.abd.2018.10.001
 Capik C. Use of confirmatory factor analysis in validity and reliability studies. Journal of Anatolia Nursing and Health Sciences. 2014;17(3):196-205.[Turkish]
 Yaslioglu MM. Factor analysis and validity in social sciences: Application of exploratory and confirmatory factor analyses. Istanbul University Journal of the School of Business. 2017;46: 74-85. [Turkish]
 Snorrason Í, Smári J, Ólafsson RP. Motor inhibition, reflection impulsivity, and trait impulsivity in pathological skin picking. Behav Ther. 2011; 42(3): 521-32.
 Roberts S, O’Connor K, Bélanger C.. Emotion regulation and other psychological models for body-focused repetitive behaviors. Clin Psychol Rev. 2013; 33(6): 745-62.
 Karakoç F, Dönmez L. Basic principles of scale development. Medical Education World. 2014; 40: 39-49.
 Fan J, Upadhye S, Worster A. Understanding receiver operating characteristic (ROC) curves. CJEM. 2006; 8(1): 19-20. https://doi.org/10.1017/S1481803500013336