Erin Gaylor week 8 EBP

A study published in Rheumatology International in 2013, investigated the efficacy of supervised exercise combined with transcutaneous electrical nerve stimulation in women with fibromyalgia.  A total of 66 female participants diagnosed with fibromyalgia according to the 1990 American College of Rheumatology criteria were admitted to the study at the outpatient clinic of Istanbul Physical Medicine and Rehabilitation Training Hospital, in Istanbul, Turkey.  

The participants were randomized by binary block randomization into two treatment groups with 30 participants each.  All participants participated in a 40 minute supervised combined exercise program for 3 days a week for 12 weeks in the rehabilitation hospital.  The median age in group 1 was 45.63 years old, and the median age in group 2 was 43.30 years old.   All participants in group 1 received TENS treatment once a day on weekdays for the first 3 weeks of the study, for a total of 15 treatments.  The TENS treatment sessions lasted for 30 minutes and were performed in the morning, whereas the exercise treatments occurred in the afternoon.  The same TENS apparatus was used on all participants and the electrodes were placed around the most painful areas such as the neck, shoulder, back and hip regions.  “Stimulation frequency was set at 80 Hz and intensity was set according to the patient’s tolerance” (Multu, 2013, p. 650).    All patients were evaluated at the baseline, at the 3^rd and then again at the 12th week.  The physician performing the measurements was blinded to the particpants’ groups. 

The measurements that were used were tender point count (TPC), myalgic pain score (MPS), the Fibromyalgia Impact Questionnaire, and the short form 36 health survey.  TPC consisted of 18 tender points that were measured by the same physician by palpation with the thumb.  The myalgic pain score evaluated the degree of tenderness at each tender point on a 4 point scale: 0 = no pain, 1 = mild pain, 2 = moderate pain with verbal exclamation and 3 = severe pain with withdrawal reaction.  “The Fibromyalgia Impact Questionnaire is a self-administered questionnaire that helps to evaluate physical function, work status and overall wellbeing in fibromyalgia patients” (p. 651).  It consists of 10 items and a total score that lies between 1 and 100.  “The short form 36 health survey is a generic test that measures quality of life through the perception of health by the patient” (Multu, 2013, p. 651).

The results of the study showed that TPC, MPS, FIQ, and SF-36 scores showed statistically significant improvement at the end of the 3^rd and 12^th weeks when compared with the baseline values at the beginning of the study.   There was no statistically significant difference between the scores of group 1 and group 2 with the exception of the MPS score at the end of week 3, which had improved more for group 1.  This result suggests that TENS provided temporary myalgic pain relief while administered during the first 3 weeks.   This supports the idea that TENS could be useful to encourage compliance with exercise programs, since many fibromyalgia patients develop a fear of movement due to anticipation of pain (Multu, 2013, p. 654).

Mutlu, B., Paker, N., Bugdayci, D., Tekdos, D., & Kesiktas, N. (2012). Efficacy of supervised        

exercise combined with transcutaneous electrical nerve stimulation in women with            fibromyalgia: A prospective controlled study. Rheumatology International, 33(3), 649-655. doi:10.1007/s00296-012-2390-8

Erin Gaylor week 7 EBP article

A study published in 2013 in Acta Psychiatrica Scandinavica examined the effects of both exercise and occupational therapy on 63 patients who had been diagnosed with either schizophrenia, schizoaffective, or schizophreniform disorder.  The 63 patients were randomly assigned to 2 hours of structured exercise or occupational therapy weekly for 6 months.  The assessments measuring mental and physical health were performed before and after the intervention.  Primary outcome measure for mental health change was measured by the Positive and Negative Syndrome Scale (PANSS) total score.  Secondary outcomes were measured by the Montgomery Asberg Depression Rating Scale (MADRS).  The Camberwell Assessment of Need (CAN) was used to measure the sum of met and unmet clinical and social needs.  Primary outcome for physical health was cardiorespiratory fitness (CRF) as assessed with a cycle ergo meter cardiopulmonary exercise test (CPET).  CRF was defined as peak work rate at the moment of exhaustion (Wpeak in Watts) and and highest oxygen uptake during the last 30 seconds of the test (VO2peak in ml/kg/min).  BMI, body fat percentage, blood pressure, triglyceride and cholesterol levels, and fasting plasma glucose levels were also measured.

The exercise group had 31 participants and met twice a week for an hour session for 6 months.  The sessions were supervised by a psychomotor therapist specialized in psychiatry.  The program included both muscle strengthening and cardiovascular exercises.  The muscle strength exercises consisted of 6 exercises per week: three times 10-15 repetitions maximum for biceps, triceps, abdominal, quadriceps, pectoral, and deltoid muscles.  The intensity of the exercises was increased gradually.

The occupational therapy group had 32 participants and also met twice a week for an hour session for 6 months.  The occupational therapy consisted of creative and recreational activities such as painting, reading, and computer activities.  The patients in this group were allowed a maximum of 60 minutes of moderate physical activity weekly outside the intervention.

The results for the compliant patients showed that both the exercise group and the OT group showed an improvement in PANSS total score.  The MADRS score of both groups also improved, but the exercise group showed significant improvements over the occupational therapy group.  CAN scores also improved for both groups, but most notably again for the exercise group. As expected, the exercise group also experienced greater improvements in cardiovascular fitness compared to the OT group.  For the other physical health measurements, the exercise group showed a trend reduction of fasting triglycerides.  This study provided evidence that both OT interventions focusing on muscle strengthening and cardiovascular exercise as well as creative and recreational activities can improve the mental and physical health of patients with schizophrenia. 

Scheewe, T. W., Backx, F. J., Takken, T., Jörg, F., Strater, A. C., Kroes, A. G., . . . Cahn, W. (2012). Exercise therapy improves mental and physical health in schizophrenia: A randomised controlled trial. Acta Psychiatrica Scandinavica Acta Psychiatr Scand, 127(6), 464-473. Retrieved February 21, 2015.

Y. Wooten Evidence Based Article Week 6

Azicnuda, E., Barba, C., Bivona, U., Ciurli, P., Formisano, R., Onder, G., Silvestro, D.      (2010).  Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of International Neuropsychological Society,  16: 360-368

       Impaired self-awareness (ISA) is very frequent in traumatic brain injury (TBI).  ISA is characterized by deficits in areas of executive functioning (EF). ISA affects  level of motivation for therapy as well as interfering with safe independent functioning.   The goal of this study is to confirm or clarify the correlation between EF and ISA in TBI patients. In particular, this study focuses on impaired metacognition: a patient’s inability to describe and/or recognize his or her deficits or monitor performance during tasks. Previous research is at best, ambiguous in terms of this correlation due to poor or overly heterogeneous sampling.  This study includes 55 severe TBI outpatients divided into two groups: patients with self- awareness (SA) and patient without SA.   Patients and a first degree relative are evaluated using the Patient Competency Scale, in which participants respond using a 5-point Likert scale (1=can’t do..5=can do with ease).  Patients are also given a battery of neuropsychology tests including the Wisconsin Card Sorting Test (WCST).  The WCST is considered to be the best and most reliable assessment of EF.  Results show a significant correlation between ISA and executive dysfunction.  Furthermore there is very a significant correlation between high percentage of perseverative errors and responses and low metacognitive SA.  This also suggests a strong correlation between cognitive inflexibility and ISA.

       

       The above results validate the importance of therapists, especially occupational therapists (OTs), having a good understanding of the behaviors associated with TBI ( disinhibition, perseveration, impulsiveness, negativity etc.).  These behaviors, mostly controlled by EF can often impede the rehabilitation process.  When treating TBI patients,  OTs need to have adequate skills in redirecting and motivation to help guide patients through a session.  OTs should also be skilled in using clinical judgement in selecting appropriate activities/ task that facilitate both cognitive skills and motor learning together while decreasing negative behaviors.

Article:

 Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury

Author: CIURLI, PAOLA; BIVONA, UMBERTO; BARBA, CARMEN; ONDER, GRAZIANO; SILVESTRO, DANIELA; AZICNUDA, EVA; RIGON, JESSICA; FORMISANO, RITA

ProQuest document link

Abstract:

Abstract

The aim of this study was to evaluate clinical, neuropsychological, and functional differences between severe traumatic brain injury (TBI) outpatients with good and/or heightened metacognitive self-awareness (SA) and those with impaired metacognitive SA, assessed by the Patient Competency Rating Scale (PCRS). Fifty-two outpatients were recruited from a neurorehabilitation hospital based on the following inclusion criteria: 1) age [greater than or equal to] 15 years; 2) diagnosis of severe TBI; 3) availability of neuroimaging data; 4) post-traumatic amnesia resolution; 5) provision of informed consent. Measures: A neuropsychological battery was used to evaluate attention, memory and executive functions. SA was assessed by the PCRS, which was administered to patients and close family members. Patients were divided into two groups representing those with and without SA. Patients with poor SA had more problems than those with good SA in some components of the executive system, as indicated by the high percentage of perseverative errors and responses they made on the Wisconsin Card Sorting Test. Moreover, a decrease in metacognitive SA correlated significantly with time to follow commands (TFC). This study suggests the importance of integrating an overall assessment of cognitive functions with a specific evaluation of SA to treat self-awareness and executive functions together during the rehabilitation process. (JINS, 2010, 16, 360-368.) [PUBLICATION ABSTRACT]

Links:Check for Full Text at CCBC

Full text:

(ProQuest: ... denotes non-US-ASCII text omitted.)
INTRODUCTION
Disorders of self-awareness (SA) are very frequent in traumatic brain injury (TBI) patients (Ben-Yishay et al., 1985; Bivona et al., 2008; Prigatano, Fordyce, Zeiner, Roueche, Pepping, & Wood, 1986; Sherer, Beargloff, Boake, High, & Levin, 1998). They can cause low motivation for rehabilitation (Malec & Moessner, 2001) and can interfere with safe and independent functioning (Flashman, Amador, & Mc Allister, 1998). Furthermore, these disorders can lead to poor outcome and difficulty in community integration (Trudel, Tyron, & Purdum, 1998) and employability (Sherer, Hart, & Nick, et al., 2003).
SA, which has been defined as the ability to recognize problems caused by damaged brain functions, has been divided into the following areas: intellectual awareness, which refers to patients' ability to describe their deficits or impaired functioning; emergent awareness, which regards patients' ability to recognize their difficulties as they are happening; and anticipatory awareness, which concerns patients' ability to predict when difficulties will arise because of their deficits (Crosson et al., 1989). SA has also been differentiated into a) metacognitive knowledge (or declarative knowledge) about one's abilities, which incorporates elements of intellectual awareness, and b) online monitoring of performance during tasks, which relates to emergent awareness and anticipatory awareness (O'Keeffe et al., 2007; Toglia & Kirk, 2000).
       A wide variety of methods have been adopted to assess the multi-faceted concept of SA of deficits and there is some controversy over the best approach to use (Fleming et al., 1996; Hart & Sherer, 2005; O'Keeffe et al., 2007). For example, different methods have been used to assess metacognitive SA: structured or semi-structured interviews, self-report questionnaires, spontaneous verbal reports of difficulties, and patients' judgment of their neuropsychological performance (Fischer, Gauggel, & Trexler, 2004; Fleming et al., 1996; Sbordone, Seyranian, & Ruff, 1998; Sohlberg, Mateer, Penkman, Glang, & Todis, 1998). The most commonly used method is to compare patients' ratings of injury-related difficulties with those of a clinician or family member (Fleming et al., 1996). This method is based on the assumption that the collateral reporter has a more accurate perception of the person's current abilities. A discrepancy in the direction of the patient reporting fewer problems is interpreted as reduced self-awareness (Pagulayan et al., 2007).
       The most widely used self-report scales for evaluating metacognitive SA are the Awareness Questionnaire (AQ) (Sherer et al., 1998) and the Patient Competency Rating Scale (PCRS) (Prigatano et al., 1986). The AQ is comprised of 17 items that evaluate patients' current functional abilities compared with their preinjury abilities. The items are rated on a Likert scale ranging from 1 (much worse) to 5 (much better). Scores vary from 17 to 85; a score of 51 indicates that the patient is functioning "about the same" as his/her preinjury level (Sherer, Hart, Nick, et al., 2003). The AQ also includes forms for patient self-ratings as well as family/significant other and clinician ratings. Reliability studies of the AQ have revealed internal consistencies (Cronbach's [alpha] = 0.88) for both patient and family ratings; however, test-retest reliability has not been reported (Sherer et al., 2003).
       The PCRS is more widely used than the AQ because it is easily interpreted, quick to administer, has excellent test-retest reliability (.85-.97) (Fleming, Strong, & Ashton, 1996; Prigatano and Altman, 1990) and has been validated in some cross-cultural studies (Hoofien and Sharoni, 2006; Prigatano, Bruna, Mataro, Munoz, Fernandez, & Junque, 1998; Watanabe, Shiel, Asami, Taki, & Tabuchi, 2000). It is a 30-item self-report questionnaire that requires patients and their relatives or clinicians to make an independent judgment of perceived degree of competency demonstrated in several behavioral, cognitive and emotional situations. The subject is required to use a 5-point Likert scale (ranging from 1, "Can't do", to 5, "Can do with ease"). Total PCRS scores range from 30 to 150, with higher scores indicating higher levels of competency. Comparing the PCRS patient ratings with those of a family member or clinician shows how realistic a patient is in evaluating his/her limitations (Borgaro & Prigatano, 2003).
The reliability figures reported by Prigatano and Altman (1990) for PCRS total scores were r = .97 for patients and r = .92 for relatives; significant (p < .05) test-retest correlations were reported for 27 (patient sample) and 28 (informants) of the 30 items (Leathem, Murphy, & Flett, 1998). Fleming, Strong, and Ashton (1998) reported acceptable 1-week test-retest reliability for patients with TBI using intra-class correlations (ICC r = .85). In the same study, internal consistency was strong for both patient ratings (Cronbach's [alpha] = .91, n = 55) and relative ratings of patients (Cronbach's [alpha] = .93, n = 50). Although the validity of the PCRS is based on the assumption that the relative and the clinician ratings are a true measure of competency, this has not been validated (Prigatano, 1996).
       Three different approaches have been used to score the PCRS. The first approach (Prigatano & Altman, 1990) is to compute three scores: a) the number of items in which the patient rating is higher than the relative rating (i.e., an index of the patient's underestimation of his disorder); b) the number of items in which the patient rating is the same as the relative rating (i.e., an index of the patient's good awareness of his disorder); and c) the number of items in which the patient rating is lower than the relative rating (i.e., an index of the patient's overestimation of his disorder). This first method allows separating patients into three groups based on their highest score.
       The second method (Prigatano et al., 1991) of scoring considers the difference between patient and relative ratings on specific items. If subtraction of the relative rating from the patient rating produces a positive number, the patient can be considered to overestimate his or her ability on that behavioral item. This method of scoring considers that patient and relative ratings may be at odds on the different items of the scale. For instance, Prigatano and Schacter (1991) used this method to demonstrate that patients tended to overestimate their abilities on items related to emotional and social behavior but tended to agree with relative ratings on items related to activities of daily living (Fleming et al., 1996).
Finally, in the third method (Fordyce et al., 1986; Prigatano et al., 1998; Roueche and Fordyce, 1987) the discrepancy between the total patient and relative rating scores is calculated to obtain an overall assessment of the patient's self-awareness. While higher positive patient-relative discrepancy scores (DS) are associated with more severe SA deficits, negative scores may indicate a patient's overestimation of his impairment (Cicerone, 1991; Prigatano & Altman, 1990).
       This method has several advantages: a) it allows separating patients into different SA groups based on the cutoff chosen; b) it allows obtaining an overall measure of the patient's SA; and c) it is easier to perform in a clinical rehabilitation setting than the other methods.
       Regardless of which scales and scoring methods are used many issues about SA impairment in TBI patients are still unclear (Bivona et al., 2008). For example, severity of acquired brain injury (ABI) correlated with measures of impaired self-awareness (ISA) in some studies (Leathem et al., 1998; Prigatano and Altman, 1990) but not in others (Anson & Ponsford, 2006; Bach & David, 2006; Bivona et al., 2008; Carton, & Robertson, 2007; O'Keeffe, Dockree, Moloney, Port, Willmott, & Charlton, 2002). This was probably due to methodological differences such as sampling (chronicity and etiology), testing instrument or classification of TBI severity using different indexes (i.e., Glasgow Coma Scale, Time to Follow Commands, and Post-traumatic Amnesia [PTA]). In any case, there is still a debate over how chronicity and severity of injury are related to SA (Dirette & Plaiser, 2007). Although in some studies (Hart, Seignourel, & Sherer, 2009; Ownsworth & Clare, 2006; Vanderploeg, Belanger, Duchnick, & Curtiss, 2007) chronicity correlated with increased levels of SA, in one large longitudinal study (Pagulayan, Temkin, Machamer, & Dikmen, 2007) no change in SA was found in the acute and postacute phases.
       Furthermore, the correlation between SA deficits and neuropsychological disorders is not clear (Allen & Ruff, 1990; Boake, Freeland, Ringholz, Nance, & Edwards, 1995). It is believed that executive functions (EF) have the greatest effect on degree of SA following brain damage (Hart, Whyte, Kim, & Vaccaro, 2005; Noè, Ferri, Caballero, Villadre, Sanchez, & Chirivella, 2005), and they are frequently impaired in TBI patients (Mattson & Levin, 1990; Stablum, Mogentale, & Umiltà, 1996). Nevertheless, even though many authors have tried to correlate executive dysfunction with ISA in TBI patients as yet no conclusive findings have been reported. Bach and David (2006), who investigated SA deficits with the PCRS, failed to demonstrate that EF disorders [explored by the Verbal Fluency (VF) Test and the Trail Making Test] are associated with reduced behavioral/social SA; however, their sample of ABI patients (TBI and stroke patients) was heterogeneous and they did not report severity and chronicity. More recently, Noè et al. (2005) evaluated EF by means of the Wisconsin Card Sorting Test (WCST), the Trail Making Test, and the VF Test and demonstrated a significant correlation between poor WCST performance and low SA (assessed by the PCRS). However, the study by Noè et al. was also based on a heterogeneous ABI population, so the extent to which it characterized a specific TBI population is not clear. Furthermore, both professionals and family members were indiscriminately enrolled (together) as significant others. Finally, Bivona et al. (2008) found a significant correlation between some components of the executive system (measured by WCST) and metacognitive awareness (assessed by AQ).
       Because EF are part of a very complex system, which includes behavioral, affective, motivational, and cognitive components (Apollonio et al., 2005), there is no comprehensive test for the executive system and many tools have been proposed to analyze its different aspects. However, it has been demonstrated that the WCST is an effective measure of multiple components such as abstract reasoning, problem solving, the ability to use response feedback information, cognitive flexibility, set-shifting and set-persistence capacity, concept identification, and hypothesis generation (Hanks, Rappaport, Millis, & Deshpande, 1999; Mukhopadhyay et al., 2008). In particular, it has been shown that perseverative responses on the WCST are an excellent measure of executive dysfunction (Johnstone, Hexum, & Ashkanazi, 1995) and that they correlate with SA impairment (Bivona et al., 2008; Noè et al., 2005).
The aim of this study was to replicate previous work examining SA in TBI and to address previous methodological limitations. The study used the best measure available, that is, the PCRS, to examine SA in a relatively homogenous sample of adults with severe TBI, using only first-degree relatives as informants. Like other studies in this area, the aim was to evaluate clinical, neuropsychological, and functional correlates of impaired metacognitive SA in severe TBI out-patients.

METHODS
Participants
       A total of 55 severe TBI outpatients were recruited from a consecutive series of 95 outpatients enrolled from May 2006 to October 2008 in the Post-Coma Unit of Santa Lucia Foundation, a neurorehabilitation hospital and research institute in Rome. The study was approved by the ethical committee of the Santa Lucia Foundation.
       The study participants were recruited from the overall sample of 95 outpatients who were evaluated in the above-mentioned time period based on the following inclusion and exclusion criteria: inclusion criteria: 1) age [greater than or equal to] 15 years; 2) diagnosis of severe TBI (GCS [delta] 8; Teasdale and Jennett, 1974); 3) availability of cerebral computed tomography or magnetic resonance imaging data; 4) PTA resolution [Levels of Cognitive Functioning Scale (LCF-S) score [greater than or equal to] 7 (Hagen, Malkmus, & Durham, 1972)]; 5) availability of informed consent; exclusion criteria: 1) aphasia [Token Test (De Renzi & Vignolo, 1962) score < or = 29]; 2) inability to undergo formal psychometric evaluation because of cognitive [Raven's Progressive Matrices (Basso, Capitani, & Laiacona, 1987) equivalent score < 1] and/or severe sensory-motor deficits; 3) previous history of drug and/or alcohol addiction, psychiatric disease and repeated TBI.
       Three patients had to be excluded after enrollment because they refused to complete the test battery (two because they had low tolerance of frustration and one because of excessive and unexpected fatigability).
       The final sample consisted of 52 patients, 44 males (85%) and 8 females (15%), with a mean age of 30.6 ± 11.1 years and a mean of 12.2 ± 2.9 years of education. The median interval in years from injury to date of assessment (chronicity) was .9 (Interquartile Range, IQR: .6/5.5). All patients had severe TBI with a median time to follow commands (TFC) of 20 days (IQR: 13/37) and a median PTA length of 60 days (IQR: 30/100). TFC was considered as the interval, in days, from coma onset until the patient was able to follow simple commands. PTA was calculated prospectively using the Galveston Orientation and Amnesia Test (GOAT) (Levin, O'Donnel, & Grossman, 1979) for 38 patients previously hospitalized in our rehabilitation unit; it was calculated retrospectively (based on information given by patients and family members) for the other 14 patients who had already recovered from PTA at the time of admission to our rehabilitation hospital. Based on reports that retrospective analysis is correlated ( r = .87) with prospective investigation of PTA (McMillan, Jongen, & Greenwood, 1996), the retrospective and prospective evaluations of PTA were considered equivalent.

Assessment
       A neuropsychologist (P.C., U.B., E.A., or D.S.) administered the following neuropsychological test battery to the patients in a quiet room in one or more sessions, depending on their fatigability: memory: Digit Span Test (forward and backward) (Orsini, 2003), Prose Memory Test (Novelli, Papagno, Capitani, Laiacona, Vallar, & Cappa, 1986); executive functioning: WCST (Heaton, Chelune, Talley, Kay, & Curtiss, 1993, 2000), Tower of London Test (ToL) (Krikorian, Bartok, & Gay, 1994), VF Test (Novelli et al., 1986); attention: Go-No Go Test of the Test for Attentional Performance (Zimmerman & Fimm, 1992). Moreover, the Neuropsychiatric Inventory (NPI) was administered to a relative to assess the patient's neuropsychiatric disorders.
       A neurologist (C.B.) made the functional assessment using the following scales: the Disability Rating Scale (Rappaport, Hall, Hopkins, Belleza, & Cope, 1982), the Glasgow Outcome Scale Extended (Wilson, Pettigrew, & Teasdale, 1998), and the LCF-S.
SA level was measured using the PCRS, which was translated into Italian by some researchers in our group (U.B., J.R., and R.F.). In line with a previous intercultural report (Prigatano et al., 1998), there were no specific difficulties regarding cultural adaptations.
       The PCRS was completed by both the patient and a family member. The neuropsychologist was present and assisted the patient in filling out the questionnaire. Only first-degree relatives were enrolled: 40 parents (77%), 10 partners (19%), and 2 children (4%) living with the patients or at least in daily contact with them. In accordance with most previous studies, the presence of SA deficits was measured by using the discrepancy between the self-rating and the family-member rating (Ownsworth, Fleming, Strong, Radel, Chan, & Clare, 2007; Pagulayan et al., 2007; Prigatano, 1996; Sherer et al., 1998; Walker, Blankenship, Ditty, & Lynch, 1987); a cutoff point was established for severity of SA deficits based on a discrepancy in the patient-relative cutoff score of ± 5 points (Prigatano et al., 1998). We chose close relatives rather than clinicians as raters because the former are in the best position to judge the patient's functional ability in daily life.

Statistical Analysis
       To assess unawareness, PCRS DS were computed by subtracting the relative rating from the patient self-rating. Pearson's correlation coefficients were calculated to study the correlation between PCRS DS and single variables.
Depending on the cutoff chosen, PCRS DS were categorized as follows: PCRS DS [greater than or equal to] 5 (index of poor SA) (Group A) and PCRS DS < 5 [index of good or (< -5) heightened SA (Group B)].
Differences between groups on the categorical parameters (based on the categorized PCRS DS) were tested using the Fisher exact test. Differences in continuous variables were assessed using analysis of variance comparisons for normally distributed parameters; alternatively, the Kruskal-Wallis test was adopted. Moreover, analysis of covariance was used to compare the adjusted means of the percentage of perseverative errors according to the PCRS DS groups by adjusting for TFC.
A p value below .05 (two-tailed) was considered statistically significant. All analyses were performed using SPSS for Windows, version 10.0.

RESULTS
       When all patients were considered, the PCRS DS ranged from -50 to +48. Group A (PCRS DS [greater than or equal to] 5) comprised 29 patients (56%) and Group B (PCRS DS < 5) 23 patients (44%). In particular, the latter group comprised 8 patients with good SA and 15 patients (DS < -5) with heightened SA. They were put into the same group because of potential limitations in power related to the small sample size.
Table 1 shows the distribution of demographic, clinical, functional, and neuropsychological data in terms of means, medians and IQR range for the entire sample.
Table 1.
Distribution of demographic, clinical, functional, and neuropsychological data
Note
TFC, Time to Follow Commands; PTA, Post-Traumatic Amnesia; DRS, Disability Rating Scale; LCF, Levels of Cognitive Functioning; GOS-E, Glasgow Outcome Scale Extended; WCST, Wisconsin Card Sorting Test; NPI: Neuropsychiatric Inventory.
As shown in Table 2, significant correlations were found between PCRS patient-relative DS, analyzed as a continuous variable, and TFC duration, WCST percentage of perseverative responses, and WCST percentage of perseverative errors.
Table 2.
Correlations of clinical and neuropsychological data with PCRS DS
Note
TFC, Time to Follow Commands; PTA, Post-Traumatic Amnesia; DRS, Disability Rating Scale; LCF, Levels of Cognitive Functioning; GOS-E, Glasgow Outcome Scale Extended; WCST, Wisconsin Card Sorting Test; NPI: Neuropsychiatric Inventory.
Table 3 shows the association between PCRS patient-relative DS subdivided into two groups based on a patient-relative DS cutoff of 5 and variables of interest. No statistically significant difference was found between groups for demographic, functional, or clinical variables. Regarding the neuropsychological variables, statistically significant differences were found between Group A (patients with poor SA) and Group B (patients with good or heightened SA) for the WCST percentage of perseverative errors (p = .04): Group A performed worse than Group B; a trend in the same direction was also found for the WCST percentage of perseverative responses (p = .08).
Table 3.
Association of demographic, clinical, and neuropsychological data with PCRS discrepancy score groups
Note
PTA, Post-Traumatic Amnesia; DRS, Disability Rating Scale; TFC, Time to Follow Commands; LCF, Levels of Cognitive Functioning; GOS-E, Glasgow Outcome Scale Extended; WCST, Wisconsin Card Sorting Test; NPI: Neuropsychiatric Inventory.
No significant difference was found between groups for any other neuropsychological variables or for any neuropsychiatric symptoms (assessed by the NPI).
To determine whether the relationship between unawareness and percentage of perseverative errors was due to the relationship both have with injury severity, adjusted means of percentage of perseverative errors, according to PCRS DS groups, were calculated by adjusting for TFC. In this analysis, participants in Group A had a borderline significantly higher percentage of perseverative errors than Group B (Group A mean 17.1, standard error 2.4 vs. Group B 10.2 standard error 2.7; p = .05).

DISCUSSION
       The main finding of this study is that patients with poor metacognitive SA have more problems than patients with good and/or heightened metacognitive SA in some components of the executive system. These include flexibility and the ability to inhibit a response, the ability to benefit from feedback, shifting of set and problem solving, as indicated by the high WCST percentage of perseverative responses and the WCST percentage of perseverative errors made by the unaware group of patients. Moreover, a decrease in metacognitive self-awareness correlates significantly with TFC.
These results were obtained using raw scores as a continuous variable. Furthermore, unlike previous reports (Anson & Ponsford, 2006; Bivona et al., 2008; Dirette and Plaisier 2007; Hart et al., 2009; Leathem et al., 1998), the correlation between the dysexecutive syndrome and ISA was confirmed also when patients were subdivided into different groups according to their level of SA (unaware vs. good-heightened SA).
       As affirmed previously (Johnstone et al., 1995), the WCST perseverative errors and/or responses can be considered an excellent measure of executive dysfunction and are among the most common data emerging from a complete neuropsychological assessment of TBI patients. Inability to inhibit a response is generally associated with impulsivity and low self-monitoring capacity. In TBI patients, unawareness might impair the ability to monitor cognitive rigidity. Therefore, these patients may be unable to inhibit a response, that is, to avoid perseverative errors.
       Considering the significant correlation between the above-mentioned WCST scores and low metacognitive SA levels, the lack of any relationship between the PCRS scores and the other measures of EF, such as the VF Test and the ToL scores, is worth noting. Perhaps it was due to the fact that the different tests used to assess EF measure different aspects of these functions (Bivona et al., 2008).
Our data are consistent with the results of other studies (Bivona et al., 2008; Noè et al., 2005) in which a significant correlation was found between these aspects of EF evaluated by the WCST and metacognitive SA. However, our study expands on the results of Noè et al. (2005) regarding the correlation between pathological number of categories and low SA. In fact, in the present study a correlation was found between low SA levels and percentage of perseverative responses and WCST percentage of perseverative errors, which strongly suggests a relationship between ISA and cognitive inflexibility.
Some methodological issues can be raised with regard to the present study and the study by Noè et al. While we enrolled only relatives as significant others, in the study by Noè et al., both professionals and family members were enrolled. In this case, the indiscriminate enrollment of professionals and relatives could be considered a methodological bias because their judgments are based on different criteria.
Indeed, family member ratings might be unreliable because of caregivers' high distress levels (Fleming et al., 1996) due to their long-term efforts to cope with the patient's post-trauma condition and the change in the quality of their relationship. Indeed, the latter tends to worsen over time due to the patient's lack of improvement (Bivona et al., 2008). Previous studies (Sherer, Hart, & Nick, 2003) have underlined that the patient-clinician discrepancy appears to be a more valid measure of SA impairment than the patient-family discrepancy. We do not agree with this. We believe that close relatives are in the best position to evaluate the outpatient's functional ability in daily life, as required by the PCRS.
       Furthermore, compared with the sample in the study by Noè et al. (2005), our sample was quite homogeneous. All of our subjects had suffered a severe TBI, all were outpatients who had recovered from PTA and were able to answer the questionnaires reliably. By contrast, in the study by Noè et al., the patients had sustained different types of acquired brain injury, TBI was not severe in all cases, and some of the patients were still in PTA.
       The present study confirms the finding of Bivona et al. (2008) of a correlation between ISA and executive dysfunction using a different tool, the PCRS, which is the most widely used scale to assess SA in TBI patients (Prigatano et al., 1998; Sherer et al., 2003). This tool is easily interpreted, quick to administer, and has excellent test-retest reliability (.85-.97) and internal consistency (Cronbach's [alpha] = .91-.95) compared with other quantitative or qualitative methods of evaluating self-awareness of deficits (Fleming et al., 1996; Prigatano & Altman, 1990). Previously, other authors (Hoofien, Gilboa, Vaakil, & Barak, 2004; Noè et al., 2005; O'Keffee et al., 2007) investigated SA by subdividing patients into groups based on different levels of awareness. However, they used different methodologies. For example, O'Keeffe et al. (2007) divided TBI patients into a high SA group and a low SA group based on a median split of a composite overall awareness score. This composite score included Z-scores from the metacognitive knowledge tests, online emergent awareness and online anticipatory awareness. Hoofien and coworkers (2004) also split TBI patients into three SA groups based on standard deviations of mean awareness scores. They calculated the difference between participants' self-evaluation of their cognitive disabilities in attention, memory and comprehension and their actual neuropsychological test scores in the same domains. We preferred to use the DS score to subdivide our TBI patients into different SA groups based on a cutoff point. In fact, this is the most commonly used method (Fleming et al., 1996; Fordyce & Roueche, 1986; Prigatano et al., 1998; Roueche and Fordyce, 1987) and is based on the assumption that the collateral reporter has the most accurate perception of the person's current abilities.
       A very interesting finding of our report is the fact that the percentage of aware patients (n = 8; 15%) was lower than the percentage of patients with heightened SA (n = 15; 29%), that is, than the patients with a DS score lower than -5 (who underestimated their competency). One reason for these findings could be the patients' chronic condition of having to face daily living difficulties. Indeed, this could have clinical implications for emotional distress, such as depression or anxiety (Jorge & Robinson, 2002; Kennedy, Livingston, Riddick, Marwitz, Kreutzer, & Zasler, 2005; Kreutzer, Seel, & Gourley, 2001; Rao & Lyketsos, 2000). It could be that neuropsychological factors contribute to unawareness and that heightened awareness is influenced by emotional factors. Nevertheless, we failed to find any correlation between neuropsychiatric symptoms (assessed by NPI) and self-awareness. It is also possible that in the heightened SA group relatives underestimated patients' difficulties on both the PCRS and the NPI so that patients' mood disorders were also neglected. In any case, further studies are needed to clarify this methodological issue.
       In line with previous reports (Bach & David, 2006; Bivona et al., 2008; Noè et al., 2005; O'Keeffe et al., 2007), we failed to find any correlation between unawareness and the other neuropsychological variables examined. Using a different methodology to assess SA, Hoofien et al. (2004) found a correlation between objective memory scores and subjective evaluations of that domain. However, these authors did not assess the executive system in their neuropsychological evaluation battery.
       Regarding clinical features, we failed to find any correlation between metacognitive SA and clinical features, such as chronicity and length of PTA (Bach & David, 2006; Bivona et al., 2008; Borgaro & Prigatano, 2002; Leathem et al., 1998; Noè et al. 2005; O'Keeffe et al., 2007; Port et al., 2002; Prigatano & Altman, 1990), except for TFC. This finding confirms that duration of unconsciousness (coma and/or vegetative state) may be an important prognostic factor in the neuropsychological outcome of patients with severe TBI (Formisano et al., 2005).
       One limitation of this study is the small sample size. In fact, studies based on larger samples are needed to make a more thorough investigation of the neuropsychological and clinical variables correlated with SA deficits. A larger sample of subjects could also be divided into three different groups (ISA, good SA, and heightened SA) so that not only the specific features of the ISA group could be evaluated but also the differences between the good and heightened SA groups.
       Further studies are also needed to make a better evaluation of the multidimensional system of SA (metacognitive and online monitoring of performance) according to the models outlined by various authors (Crosson et al., 1989; Toglia & Kirk, 2000) and suggested by O'Keeffe et al. (2007).
Nevertheless, considering the high percentage of self-awareness impairments (44%) found in our population of severe TBI patients, the significant correlation between some components of the executive system and metacognitive self-awareness strongly confirms the importance of integrating an overall assessment of cognitive functions with a specific evaluation of SA to treat self-awareness and executive functions together in the rehabilitation process.

ACKNOWLEDGMENTS
I have disclosed any and all financial or other relationships that could be interpreted as a conflict of interest affecting this manuscript. I confirm that we received no financial support to carry out this study.

References

REFERENCES
C.C. Allen, & R.M. Ruff (1990). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 4, 7-17.
K. Anson, & J. Ponsford (2006). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 20, 1-13.
I. Apollonio, M. Leone, V. Isella, F. Piamarta, T. Consoli, M.L. Villa, . (2005). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of the Neurological Sciences, 26, 108-116.
L.J. Bach, & A.S. David (2006). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Neuropsychological Rehabilitation, 16, 397-414.
A. Basso, E. Capitani, & M. Laiacona (1987). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Functional Neurology, 2, 189-194.
Y. Ben-Yishay, J. Rattok, P. Lakin, E.B. Piasetsky, B. Ross, S. Silver, . (1985). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Seminars in Neurology, 5, 252-259.
U. Bivona, P. Ciurli, C. Barba, G. Onder, E. Azicnuda, D. Silvestro, . (2008). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of the International Neuropsychological Society, 14, 862-868.
C. Boake, J.C. Freeland, G.M. Ringholz, M.L. Nance, & K.E. Edwards (1995). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 9, 273-283.
S. Borgaro, & G. Prigatano (2002). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 17, 526-534.
S. Borgaro, & G. Prigatano (2003). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 17, 847-853.
K.D. Cicerone (1991). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 6, 30-43.
C. Crosson, P.P. Barco, C. Velozo, M.M. Bolesta, P.V. Cooper, D. Werts, . (1989). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 4, 46-54.
E. De Renzi, & L.A. Vignolo (1962). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain, 85, 556-678.
D.K. Dirette, & B.R. Plaisier (2007). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 21, 1131-1136.
S. Fischer, S. Gauggel, & L.E. Trexler (2004). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 18, 547-562.
L.A. Flashman, X. Amador, & T.W. McAllister (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Seminars in Clinical Neuropsychiatry, 3, 201-210.
J.M. Fleming, J. Strong, & R. Ashton (1996). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury Brain Injury, 10, 1-15.
J.M. Fleming, J. Strong, & R. Ashton (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 13, 39-51.
D.J. Fordyce, & J.R. Roueche (1986). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Rehabilitation Psychology, 31, 217-229.
R. Formisano, U. Bivona, F. Penta, M. Giustini, M.G. Buzzi, P. Ciurli, . (2005). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Acta Neurochirurgica; 93, 201-205.
C. Hagen, D. Malkmus, & P. Durham (1972). Levels of cognitive functioning. Downey, CA: Rancho Los Amigos Hospital.
R.B. Hanks, L.J. Rappaport, S.R. Millis, & S.A. Deshpande (1999). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 80, 1030-1036.
T. Hart, P.J. Seignourel, & M. Sherer (2009). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Neuropsychological Rehabilitation, 19, 161-176.
T. Hart, & M. Sherer (2005). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 20, 285-286.
T. Hart, J. Whyte, J. Kim, & M. Vaccaro (2005). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 20, 333-347.
R.K. Heaton, G.J. Chelune, J.L. Talley, G.G. Kay, & G. Curtiss (1993). Wisconsin Card Sorting Test. Florida: Psychological Assessment Resources, Inc.
R.K. Heaton, G.J. Chelune, J.L. Talley, G.G. Kay, & G Curtiss. (2000).WCST: Wisconsin Card Sorting Test. Forma completa revisionata. Adattamento italiano a cura di Hardoy, M.C., Carta, M.G., Hardoy, M.J. e Cabras, P.L. in O.S. Organizzazioni Speciali (Ed) it. Firenze.
D. Hoofien, A. Gilboa, E. Vakil, & O. Barak (2004). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Clinical and Experimental Neuropsychology, 26, 278-290.
D. Hoofien, & L. Sharoni (2006). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Israel Journal of Psychiatry & Related Sciences, 43, 296-230.
B. Johnstone, C.L. Hexum, & G. Ashkanazi (1995). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 9, 377-384.
R. Jorge, & R.G. Robinson (2002). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Neurorehabilitation, 17, 311-324.
R.E. Kennedy, L. Livingston, A. Riddick, J.H. Marwitz, J.S. Kreutzer, & N.D. Zasler (2005). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 20, 512-526.
J.S. Kreutzer, R.T. Seel, & E. Gourley (2001).Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 15, 563-576.
R. Krikorian, J. Bartok, & N. Gay (1994). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Clinical and Experimental Neuropsychology, 16, 840-850.
J.M. Leathem, L.J. Murphy, & R.A. Flett (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Clinical and Experimental Neuropsychology, 20, 694-705.
H.S. Levin, V.M. O'Donnel, & R.G. Grossman (1979). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Nervous and Mental Disease, 167, 675-684.
J.F. Malec, & A.M. Moessner (2001). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Rehabilitation Psychology, 45, 227-241.
A.J. Mattson, & H.S. Levin (1990). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. The Journal of Nervous and Mental Disease, 178, 282-291.
T.M. McMillan, E.L. Jongen, & R.J. Greenwood (1996). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury Journal of Neurology, Neurosurgery, and Psychiatry, 60, 422-427.
P. Mukhopadhyay, A. Dutt, S.K. Das, A. Basu, A. Hazra, T. Dhibar, . (2008). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Progress in Brain Research, 168, 95-104.
E. Noè, J. Ferri, M.C. Caballero, R. Villadre, A. Sanchez, & J. Chirivella (2005). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Neurology, 252, 168-175.
G. Novelli, C. Papagno, E. Capitani, M. Laiacona, G. Vallar, & S.F. Cappa (1986). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Archivio di Psicologia, Neurologia e Psichiatria, 47, 477-506.
F. O'Keeffe, P. Dockree, P. Moloney, S. Carton, & I.H. Robertson (2007). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of the International Neuropsychological Society, 13, 38-49.
A. Orsini (2003). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Bollettino di Psicologia Applicata, 239, 73-77.
T. Ownsworth, & L. Clare (2006). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Clinical Psychology Review, 26, 783-795.
T. Ownsworth, J. Fleming, J. Strong, M. Radel, W. Chan, & L. Clare (2007). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Neuropsychological Rehabilitation, 17, 129-150.
K.F. Pagulayan, N.R. Temkin, J.E. Machamer, & S.S. Dikmen (2007). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of the International Neuropsychological Society, 13, 561-570.
A. Port, C. Willmott, & J. Charlton (2002). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 16, 277-289.
G.P. Prigatano (1996). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. The Clinical Neuropsychologist, 10, 191-201.
G.P. Prigatano, & I.M. Altman (1990). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 71, 1058-1064.
G.P. Prigatano, I.M. Altman, & K.P. O'Brien (1991). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. BNI Quarterly, 7, 27-33.
G.P. Prigatano, O. Bruna, M. Mataro, J.M. Munoz, S. Fernandez, & C. Junque (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Head Trauma Rehabilitation, 13, 29-38.
G.P. Prigatano, D.J. Fordyce, H.K. Zeiner, J.R. Roueche, M. Pepping, & B.C. Wood (1986). Neuropsychological rehabilitation after brain injury. Baltimore: Johns Hopkins University Press.
G.P. Prigatano, & D.L. Schacter (1991). Awareness of deficit after brain injury: Theoretical and clinical issues. New York: Oxford University Press.
V. Rao, & C. Lyketsos (2000). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Psychosomatics, 41, 95-103.
M. Rappaport, K.M. Hall, H.K. Hopkins, T. Belleza, & D.N. Cope (1982). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 63, 118-123.
J.R. Roueche, & D.J. Fordyce (1987). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Cognitive Rehabilitation, 1, 4-7.
R.J. Sbordone, G.D. Seyranian, & R.M. Ruff (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury Brain Injury, 12, 505-515.
M. Sherer, P. Bergloff, C. Boake, W. High Jr., & E. Levin (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 12, 63-68.
M. Sherer, T. Hart, & T.G. Nick (2003). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Brain Injury, 17, 25-37.
M. Sherer, T. Hart, T.G. Nick, J. Whyte, R.N. Thompson, & S.A. Yablon (2003). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 84, 168-176.
M.M. Sohlberg, C.A. Mateer, L. Penkman, A. Glang, & B. Todis (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury Journal of Head Trauma Rehabilitation, 13, 62-78.
F. Stablum, C. Mogentale, & C. Umiltà (1996). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Cortex, 32, 261-278.
G. Teasdale, & B. Jennett (1974). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Lancet, 13, 81-84.
J. Toglia, & U. Kirk (2000). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Neurorehabilitation 15, 57-70.
T.M. Trudel, W. Tyron, & C. Purdum (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Rehabilitation Psychology, 43, 276-281.
R.D. Vanderploeg, H.G. Belanger, J.D. Duchnick, & G. Curtiss (2007). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Rehabilitation Research and Development, 44, 937-950.
D.E. Walker, V. Blankenship, J.A. Ditty, & K.P. Lynch (1987). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Clinical Psychology, 43, 699-707.
Y. Watanabe, A. Shiel, T. Asami, K. Taki, & K. Tabuchi (2000). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Clinical Rehabilitation, 14, 172-177.
J.T. Wilson, L.E. Pettigrew, & G.M. Teasdale (1998). Metacognitive unawareness correlates with executive function impairment after severe traumatic brain injury. Journal of Neurotrauma, 15, 573-585.
P. Zimmerman, & B. Fimm (1992). Test Batterie zur Aufmerksamkeitsprüfung (TAP). Würselen: Psytest.