Improving Vocational Outcomes of Veterans With Psychiatric Disorders: Career Counseling & Development

AIM 1: Develop a career counseling and development treatment manual for Veterans with psychiatric disorders (i.e., Purposeful Pathways) along with a fidelity monitoring checklist based on veteran and provider needs assessment (n=16). Procedures. Manual development will be an iterative process that spans over Aim 1 and Aim 2 (Open Trial) and will include: 1) reviewing clinical procedures and interventions of the Vocational Evaluation Center (VEC) for career counseling and development in vocational rehabilitation, related treatment manuals (Opening Doors), and relevant career development theory; 2) conducting veteran and provider focus groups to gather perspectives on necessary treatment components; 3) developing a preliminary version of the manual through a two-day retreat attended by key CDA-2 mentors and consultants, and 4) revising the manual based on mentor, provider, and veteran feedback. A fidelity monitoring checklist will also be developed to monitor treatment adherence of clinicians implementing the Purposeful Pathways protocol. The fidelity monitoring checklist will be used in Aim 2 and Aim 3 of this study to monitor treatment adherence of the study clinicians. Needs Assessment Focus Groups. Two focus groups will be conducted with veterans (n=8), and two focus groups will be conducted with mental health providers (n=8) with knowledge in vocational rehabilitation practice (psychologists, social workers, and vocational rehabilitation specialists). The providers will be recruited from outside the Bedford VA to capture perspectives from clinicians who are not familiar with the Bedford VA Vocational Evaluation Center. Half of veteran participants will be recruited from currently enrolled TWE veterans. The other half of veteran participants will be individuals who previously participated in TWE and were unsuccessful in obtaining employment. Focus groups will be 90 minutes, and will be audio recorded for future transcription and analysis with NVIVO. Transcription will be completed by a VA approved contractor. A qualitative interview guide drawing upon Psychology of Working theory, protocol developed through the Vocational Evaluation Center (VEC) as well as the Opening Doors manual when conducting provider and veteran focus groups. Overall, the goal is to gain an unfiltered perspective on career development and factors related to successful vocational recovery, while also querying on aspects of existing practices An open-ended interview guide consisting of open-ended questions will focus on 1) barriers, potential solutions, and perceived needs to promote successful career outcomes among veterans with psychiatric disorders; 2) obtain feedback on the useful elements of Psychology of Working theory, as well as the VEC and Opening Doors protocols as it relates to their clinical experience/expertise; 3) explore issues related to augmenting VA vocational services with career counseling and development interventions. The purpose of the veteran focus groups are to gather personal accounts from people with lived experience on career development/vocational recovery, and to connect that to the best available research and clinical practice to inform treatment development. The qualitative interview guide will consist of open-ended questions to gather the following information: 1) barriers, potential solutions, and perceived needs to promote successful career outcomes among veterans with psychiatric disorders; 2) obtain feedback on the useful elements of Psychology of Working theory, as well as the VEC and Opening Doors protocols as it relates to their lived experiences; 3) gather information regarding preferred methods of implementation of adjunctive career counseling and development services. Sample. The eight provider participants will be recruited through emails to vocational providers and will be scheduled for one of two one hour focus group of providers only during regular work hours. These focus groups will be held via teleconference. Half of the eight veteran participants will be recruited from the Compensated Work Therapy TWE program, while half will be former TWE recipients who did not secure competitive employment. Veteran participants will be scheduled for a one of two one hour focus group of veterans only. All participants will be consented for participating in the interview and having their participation audio-recorded. Qualitative Analysis. A Hamilton Rapid Turn-around Analytic Approach will be used. This technique is suited to short-term projects using targeted interview guides. It improves on past rapid procedures through its systematic, rigorous approach, yielding results that are slightly less nuanced but more efficient. The PI will develop an analytic template organized by the topic areas from the semi-structured interview guiding qualitative interviews. The PI and two other researchers will apply the template to summarize two interviews to test the usability and relevance of the template to the transcripts. All three researchers will meet to establish consistency in how the template is used to summarize the transcripts. From here, all focus groups will be individually summarized (each focus group will be analyzed by two researchers to ensure reliability). PI will then transfer all the summary points into a master matrix (respondent x topic area) as an important step in drawing conclusions and deriving meaning from the transcripts. PI will then condense the summaries in the master matrix by identifying and organizing recurring themes. The other two researchers will review the condensed matrix to the original master matrix to confirm all central themes have been identified and to ensure consensus. Two-Day Retreat. Data gathered from the veteran and provider focus groups will be used to inform the design of the treatment protocol as well as the fidelity monitoring checklist. The retreat will be coordinated at the Bedford VA, and will be facilitated by Drs. Stevenson and Mueller and attended by mentors (Drs. Bakken, Blustein, Ellison, and Kelly), as well as consultants (Drs. Millner, Kenny, Duffy, Allan, and Autin). Drs. Stevenson and Mueller will facilitate the retreat beginning with an opening agenda and ground rules for facilitating dialogue and resolving conflicting viewpoints about focus group material, expert knowledge, and research evidence. Every theme will be presented and reviewed. Drs. Stevenson and Mueller will facilitate conversation about each theme and whether or not thematic content could be integrated into the treatment protocol considering the integrity of best available research and practice. In situations where consensus among content experts cannot be reached, the majority stance will determine inclusion or exclusion of content into the treatment manual. Content and Structure of the Proposed Intervention. Foremost, the intervention will largely be based upon the results of the needs assessment focus groups and qualitative analysis discussed above. Some of the proposed content is based on Psychology of Working Theory (PWT) and the corresponding Psychology of Working Counseling (PWC) model which identifies critical areas of intervention in vocationally-oriented treatment (e.g., assessment of work-related needs in terms of survival/power, social connection, and self-determination; increasing skills related to proactivity and adaptability to navigate career development processes; increasing awareness to environmental or other contextual factors influencing career choice and development; increasing sense of choice and agency over one's career development; and improving relational support systems to enhance vocational career development outcomes). In addition, the preliminary treatment goals and interventions for the Purposeful Pathways intervention take into account current VEC practices among veterans with psychiatric disabilities (e.g., exploring the impact of psychiatric and physical disability on one's work like and career choices) as well as the Opening Doors treatment manual (e.g., a person-environment fit model of career development that espouses a three step process of knowing oneself, knowing one's work-related options, and embarking on a career path that fits with self-knowledge). Anticipated length of treatment is eight weekly sessions which is informed by the average length of treatment veterans receive through the VEC. Sessions will be delivered to veterans through individual, face-to-face weekly meetings with a masters-level (or higher) trained counselor. Qualitative Inquiry of Initial Treatment Protocol. After developing an initial version of the Purposeful Pathways manual, qualitative interviews will be conducted among the eight mental health providers who participated in the first set of focus groups. A copy of the Purposeful Pathways manual will be given to each of these mental health providers. These reviewers will be asked to evaluate aims, structure, concerns, and other issues of acceptability of this intervention. After everyone independently reviews the manual, Dr. Mueller and I will set up two focus groups, which will include open-ended questions and general reactions to a short walk-through of the intervention. The purpose of these questions is to assess the perceived feasibility of the intervention (as discussed by Bowen and colleagues). Specifically, our interview guide will explore perceived acceptability, demand, implementation, and integration of Purposeful Pathways. Interviews will be audio recorded and will be transcribed by a VA approved contractor. These transcriptions will be analyzed using NVIVO. The same qualitative analytic approach previously discussed (Hamilton Rapid Turn-around) will be utilized for this qualitative inquiry. Any themes suggesting problems with acceptability, demand, implementation, and integration will be presented (via video conferencing) to the experts attending the two day manual development retreat. If consensus cannot be reached among the mentor and content expert about whether or not to make changes to the manual based on this qualitative inquiry, final decisions about treatment modifications will be based on the majority perspective. AIM 2: Pilot test the manual by treating a small number of Veterans (n=10) in an open trial. Sample/Recruitment/Screening. The sample will consist of 10, newly enrolling, Veterans in TWE. All Veterans who enter TWE will be informed about this voluntary research study. The study candidates will be invited to meet personally with a research assistant or the PI who will conduct the screening, thoroughly explain all procedures, and answer any questions before obtaining written informed consent on the consent form approved by Bedford VAMC Institutional Review Board (IRB). Any participant who is found to meet any exclusion criteria will not be eligible to participate in the study. The inclusion or exclusion of subjects will not be based on ethnic or racial characteristics. After screening and informed consent procedures are completed, the PI will begin meeting with the subject for weekly appointments following the Purposeful Pathways manual. Veterans participating in this phase of research will complete several surveys and questionnaires throughout the treatment. It is estimated that two Veterans per month will be recruited (total of 5 months to reach target sample of 10 subjects). During FY18, the Bedford VA enrolled 148 Veterans through TWE and 97.7% of these Veterans had a psychiatric diagnosis. Moreover, the average length of stay of Veterans in CWT programs was 419.6 days. Thus, recruitment of 10 subjects to participate in an eight-week treatment program, will be an achievable goal. Assessment Measures and Timeline. A variety of reliable and valid measures will be used in this phase to assess demographics, career development constructs, vocational behavior, employment functioning, working alliance, and client satisfaction. Qualitative Exit Interviews. At termination of treatment, participants will participate in a semi-structured, 30-minute exit interview with a research assistant who was not involved in the delivery of treatment. Questions for these qualitative exit interviews were derived from the items on the Client Satisfaction Questionnaire-8 (CSQ-8) - which will be completed by research participants at treatment termination - in order to provide more detailed information about the nature of participants experiences impacting satisfaction with treatment. The CSQ-8 measures satisfaction, perceived quality, and effectiveness of treatment. The following content areas will be assessed through these interviews (corresponding items from the CSQ-8 are listed but the items are not reproduced due to copyright): (1) general thoughts/experiences with treatment (CSQ-8 Question #1, 2, 3, 5); (2) perceived benefits of participating in treatment (CSQ-8 Questions #2, 3, 4, 6, 8); (3) specific changes and outcomes obtained because of participating in treatment (CSQ-8 Questions #6, 8); (4) perceived limitations of the treatment (CSQ-8 Questions #1, 4, 7, 8); (5) recommendations for enhancement/improvement of services (CSQ-8 Questions #1, 2, 5, 7, 8). We will also ask open-ended questions about usability of measures/assessments. Exit interviews will be audio recorded, for transcription and analysis. Data Management. Subject files will be kept in a locked cabinet with a locked office at the Bedford VA. Subjects will be given an ID number unrelated to any identifying information. An electronic, password protected file linking participants to their ID number will be kept on a VA computer on a secure, VA approved server. Data gathered through quantitative measures will be will be stored on a VA computer on a secure, VA approved server. All audiotaped interviews will be professionally transcribed by a VA approved contractor (see Budget Justification). These transcriptions will be stored on a VA computer on a secure, VA approved server. Data Analysis. The same qualitative analysis plan described in Aim 1 (the Hamilton Rapid Turn-around Analytic Approach) will be used for this aim. NVIVO will be used for analysis. Quantitative data gathered throughout the open trial will be used to triangulate findings from the qualitative exit interviews. A weaving approach will be used to integrate qualitative and quantitative findings through narratives on a theme-by-theme, or concept-by-concept basis. In this case, the themes/concepts will be based on the CSQ-8 (e.g., satisfaction with treatment, perceived quality of treatment and effectiveness of the treatment). We will assess for fit between the data looking for confirmation of results and possible expansion of findings. Patterns of concern, challenges, dissatisfaction, or improvements found in integrated data will be reviewed by the PI and mentors, possibly resulting in treatment modification prior to pilot RCT in Aim 3. Discordance in qualitative and quantitative results will also be reviewed by the PI and the primary mentor team, possibly resulting in modification to the assessment of client satisfaction (i.e., CSQ-8 or Qualitative Exit Interview questions) prior to pilot RCT in Aim 3. Whether or not to modify treatment will be based on fidelity to best available research and feasibility of implementing recommended changes. If consensus cannot be obtained among the mentor team, then the majority perspective will be used to make final decisions. Patterns in incomplete or missing data, reported fatigues, or issues with the measurement paradigm in this Aim of the study will be addressed before implementation during pilot RCT in Aim 3. AIM 3: Conduct a randomized controlled pilot study comparing Purposeful Pathways (n=25) to treatment as usual (n=25). Sample/Recruitment/Screening. The sample will consist of 50, newly enrolling, Veterans in TWE. Only veterans entering TWE will be recruited for this study, currently enrolled TWE or SE veterans will not be eligible. Recruitment, screening, and inclusionary/exclusionary criteria are the same as Aim 2 Study Procedures. Veterans who have completed screening and informed consent procedures will be randomized to one of two study arms: Purposeful Pathways (intervention) or treatment as usual (TAU; control). To ensure equal distribution across the intervention and control conditions, a permutated block randomization method will be utilized. Randomization will use computer-generated numbered lists; one for each condition (treatment and control). The lists will be generated through a block design where the block sizes are randomly selected among block sizes of either 4 or 6. There will be five rounds of each of the block sizes to randomize the 50 veterans into either the treatment or control condition. With this approach, every study arm assignment on the randomization list will consist of an even number of treatment or control assignments in random order. This approach has been discussed by previous researchers and not only guarantees balance in the two conditions, but also ensures that the investigator is not able to predict the treatment sequence as they will not know if the block size is 4 or 6. This procedure will result in 25 participants per condition which is in line with previous recommendations of a minimum of 15 subjects per condition. Subjects randomized to the treatment group will receive the full Purposeful Pathways treatment protocol, in addition to TWE (i.e., treatment as usual). Subjects randomized to the control group will receive treatment as usual (i.e., TWE). Subjects will be asked to attend a brief meeting with a researcher at a three-month follow-up. During this follow up meeting, participants will complete a variety of self-report questionnaires. Control Condition. Treatment as usual will serve as the control group for this aim of the study. Treatment as usual will consist of participation in Transitional Work Experience (TWE). Upon admission, veterans in TWE are quickly placed into a structured work setting. These time-limited work settings may be within the community, many are within the VA. CWT pays veterans through funds received by contracts from companies who contact TWE labor. There is minimal formal assessment or other vocational services prior to a veteran being placed into a work site. Work sites vary by TWE program. Common TWE placements at the VA include housekeeping, laundry, grounds maintenance, and food services. TWE program staff, which can range from peer support staff to masters trained vocational counselors, provide some on-site and off-site job coaching to help veterans maintain their TWE placement, in addition to some guidance on competitive job searching. TWE program staff tend to have (1.5 to 2 times larger caseloads that counselor in supported employment; approximately 25-40 veterans). Clinician Recruitment. The PI will recruit and train at least four master's-level trained clinicians, in the delivery of the CCD-V treatment protocol (two clinicians per 15 months). All clinicians will be employed at the Bedford VA. Treatment Adherence. There will be multiple fidelity checks to ensure treatment adherence among study clinicians. First, all Purposeful Pathways clinicians will participate in a didactic training introducing clinicians to the treatment protocol. Training of study clinicians is modeled after the VHA evidence-based practice (EBP) national training program. The national EBP training programs consist of an initial, three-day intensive didactic training. For this proposed study, Dr. Stevenson, as well as Drs. Mueller and Blustein, will lead a 9-hour workshop held over the course of three days (3 hours/day) for an initial didactic overview of the Purposeful Pathways protocol. Secondly, Purposeful Pathways clinicians will be required to attend weekly group consultation with the Dr. Stevenson (1 hour/week) for the first three months of providing treatment to address technical issues in the provision of the treatment. Thirdly, all Purposeful Pathways sessions will be video recorded, and these sessions will be reviewed and rated (on a variable time schedule) for treatment adherence by two staff researchers using the fidelity checklist developed in Phase One. Other researchers have noted that using a variable schedule to monitor treatment adherence among study clinicians can be an effective methodological approach to offset the time and resources needed to rate every-single session (in this proposed study there will be 25 Veterans assigned to the treatment condition, which consists of eight sessions, totaling 200 sessions). Once a session is rated, researchers will provide in-person, individual feedback to clinicians. Assessment Measures and Timeline. The same measurement paradigm described for Aim 2 (screening, first session, last session) will be used for Aim 3 with the addition of a three-month follow up. At the three-month follow up Veterans will complete the same battery of measures completed at their final session. Qualitative interviews will be conducted and will be conducted as described in Aim 2, with some additional (new) questions regarding acceptability of randomization. Data Management. The same data management system described for Aim 2 will be used for Aim 3. Data Analysis. Consistent with the recommended stage model for development of behavioral therapies, the aim of this Stage I Pilot RCT is to inform future treatment development, and research and evaluation plans. To have .80 power to detect differences at a two-tailed alpha level of .05, a medium effect would require N=125. This is beyond the scope of a CDA-2 award and is more appropriate for a Stage II trial. A total sample size of 50 is feasible and consistent with recommendations of 15 to 30 subjects per condition for stage I behavioral treatment development. Outcomes gathered from this study will provide feasibility and acceptability data, as well as preliminary efficacy. Primary analyses will involve examination of the feasibility and acceptability of a RCT of Purposeful Pathways. Feasibility will be verified by adequate 1) recruitment rates (i.e., 2-3 Veterans per month), 2) \>90% fidelity of clinician adherence to the treatment protocol, and 3) data retention (i.e., 70% of participants who attend at least 6 sessions of Purposeful Pathways. An acceptable level of attrition will be 30% in each condition. We will also examine the willingness of Veterans to be randomized with a goal of 80% of those approached who agree to randomization, the number of referrals from clinicians, ease of recruitment based on the average number of contacts needed to get consent and to schedule visits, the number of eligible participants from the pool of potential participants, and adherence and compliance to the treatment protocol. We will compare Veteran acceptability between groups by conducting an independent samples t-test on the CSQ-8. We will compare attrition rates (\<15% vs. \>15%) between the two groups using a chi square and logistic regression analysis. We will also obtain qualitative data about the feasibility and acceptability of the Purposeful Pathways treatment and randomization procedures to confirm and expand upon quantitative data. The Hamilton Rapid Turn-around analytic strategy (previously discussed in Aim 1 and 2) will be utilized for Aim 3 qualitative analysis. While efficacy is not our major analytic aim, given the preliminary nature of this study, we will examine differences in measures of vocational identity, vocational functioning, psychiatric functioning, work importance and confidence, career adaptability, job search behavior, decent work, and job need satisfaction to identify empirical targets for further refinement of the treatment and inform the design of the future Merit Review. Based on the intent-to-treat principle, all participants randomized to treatment will be included in analyses. We will run descriptive statistics as well as internal reliability (Cronbach's alpha) for all measures, Pearson's r to look at relationships between variables, and Cohen's D for any continuous measures and Cramer's v for any categorical measures to look at effect size. We will also look at individual trajectories of response to gain better insight into which participants do particularly well (or poorly) in the two interventions. Effect sizes and 95% confidence intervals will be used as only one element to estimate power and sample size to help plan for a Merit Review study.