FORGOTTEN PEOPLE Never-To-Be Forgotten (A Series): Lesson 20 – Members are Keenly Interested in Program Evaluation
on June 20th, 2016 at 5:19 pmIn 1964 I was interviewed for a job at Fountain House by Assistant Director Jim Schmidt. I responded to his question of what I was looking for in a job by saying that I was very interested in working with those suffering from mental illness in a community setting, not a hospital. Further, I wanted to be involved in program evaluation of outcomes of such community programs. I felt, however, that I needed to first get “my feet wet” in program before turning to research. That was certainly a presciently ironic phrase as I was hired to work in at a carwash with a group of members there on transitional employment.
It so happened that after three years in various program activities including two carwash locations, an evening hour group placement on the Ideal Toy Company assembly line and the small business project that a new direction opened up. The then Research Director moved on to a university position at the same time that an NIMH research grant period was coming to an end and a final report was needed. John Beard asked if I would be interested in helping him do the final data analysis and report preparation. This was a dream come true, however I had one request – that members would be needed to help in this effort. Beard immediately agreed and the result was the initiation of a new program unit. This then was how I first met Thorne, Jill, Maria, Kathy and a whole cadre of others who became interested in research. Photo by: J. Confino
Thorne had a graduate degree in Psychology and was an expert on the use of statistics. This expertise was brought to bear in analyzing the data collected for the NIMH study. That was his gift to Fountain House. In turn, we were able to help him return to a lucrative job as a programmer for a University-based research project.
Jill was an early participant in the newly created research unit, where she put her clerical skills to good use on the donated IBM keypunch and card sorting machines.
Kathy also was successful on several TE placements and obtained a full-time job at a law office library. While working, she completed Social Work school and with an MSW was able to secure a job at a Project Renewal residence.
Kathy also has been called on to provide peer review for several articles submitted to the Psychiatric Rehabilitation Journal. She served on the Fountain House Unusual Incidents Committee that reviewed consumer complaints.
One anecdote bears relating in reference to our success in conducting long-term follow-up. In collecting data for subjects in the original NIMH study, we obtained 9-year information on over 95% of the 352 Experimental and Control subjects. For the one person whom we were unable to locate, we scanned the original application form indicating his contact person as his father. On the application the father’s telephone number was at a public school in Queens. In calling the number we were informed that the father had retired from his position in the athletic department of the school and in checking the Queens telephone directory, no such person was listed.
Combining our brains, we surmised that the father, being a retired coach from Queens, might have just moved to Port St. Lucie, Florida, Spring time home of the NY Mets baseball team. We checked that area’s telephone directory and sure enough the father’s name was found. We called him and found that his son was doing well, living on Long Island, and he was pleased to give us his telephone number. We reached the son and were able to obtain the required information.
The research studies conducted at Fountain House documented three important findings: 1) psychiatric hospitalizations, while not eliminated by aggressive rehabilitation services, are greatly reduced and occur less frequently; 2) rehabilitation is a long-term process; and 3) members help obtaining more accurate and complete results,
John vividly illustrates these findings:
In the winter of 1968 I had a nervous breakdown and was hospitalized at Bellevue, the receiving hospital in New York City. I was subsequently transferred to a private hospital. Though the treatment I received there was far more humane, it fell short of what I needed to achieve a full recovery and to resume my college education immediately. In addition, at the time of discharge, no provision was made for any kind of follow-up care, such as therapy, halfway house or community based rehabilitation facility. I was told that there was nothing more the hospital could do for me, that I had to rely solely on my own resources.
Following a year of working and going to school part-time, I returned to college, but upon graduation I was faced with the same problem of what to do. For a year and a half, I drifted through a succession of jobs, lasting a few weeks or so, never making a real first start. Eventually I was rehospitalized. This time I received excellent treatment and was discharged in a reasonably short period. Most importantly, I was referred to Fountain House by a staff social worker, whose description of the employment program persuaded me to become a member. Over the next four years I participated in all aspects of the rehabilitation program; I did volunteer work in the prevocational day program, lived in a Fountain House apartment and worked on three jobs provided through the program of transitional employment…. Then I got my own full-time job as a paralegal with a small New York law firm, where I worked for a year. I (then was) employed by Fountain House.
.
Maria joined the Research Unit where she became active in helping with a number of the on-going research projects. She proved a valuable contributor in gathering follow-up data on a study documenting the replication of the TE model with some 100 programs located throughout the nation. This project was actually under the supervision of Horizon House in Philadelphia (Irving Rutman, Principle Investigator). The Final Report acknowledged her important role in the project.
The most basic lesson concerning research that I learned from these years of experience is that while most mental health research is directed at measuring pathology or the degree of its absence, there is as much to be gained by focusing on the degree of health or wellness that an individual is attaining. To determine what the level of this positive outcome is over time, we developed an instrument that recorded major changes in a person’s level of community adjustment. This instrument became known as COCA (Categories of Community Adjustment). COCA consists of the following 10 major identifiers or categories of community involvement:
1. Independent Employment
2. Transitional Employment
3. Daytime psychosocial rehabilitation involvement
4. Other rehabilitation or educational training program
5. Miscellaneous activities – vacation, family and home responsibilities, retirement
6. Physical illness, either hospital or homebound
7. Psychiatric hospitalization
8. In community but isolated and uninvolved in any formal activity
9. Deceased
10 No information, lost
Data collection consists of following individuals as they progress from a starting point, typically the day they start at Fountain House – thus being recorded in Category 3. Each time there is a change in category the date and category number are recorded. For purposes of validation, if after each 30-day period no change has occurred that date is also recorded.
If an individual occupies more than one category at a time the lower numbered category is to be recorded. Thus, for example, if one is independently employed and also attending Fountain House, the employment category is considered most descriptive of community status. Likewise, if one is psychiatrically hospitalized but also attending Fountain House during the day, Category 3 is selected as most descriptive.
One of the happier aspects of utilizing this instrument is that members are very much involved in data collection because among other things the information is both public and informative. Members can see that the results can be meaningful to them personally as well as significant to society in general. This can be seen in the table below, summarizing of data collected on 140 members over an 18-month period following their intake into Fountain House:
Adjustment Category at the End of 6, 12 and 18 Months
6 Months 12 Months 18 Months
Adjustment Category N % N % N %
1. Independent Employment 5 3.6 11 7.9 23 16.4
2. Transitional Employment 88 62.9 71 50.7 54 38.6
3. Fountain House 33 23.6 29 20.7 24 17.1
4. Other Rehab or Education 3 2.1 13 9.3 10 7.1
5. Miscellaneous 0 - 0 – 0 -
6. Physical Illness 0 – 0 – 1 0.7
7. Psychiatric Hospitalization 2 1.4 3 2.1 3 2.1
8. In Community – isolated 7 5.0 9 6.4 17 12.1
9. Deceased 1 0.7 2 1.4 2 1.4
10. Lost 1 0.7 2 1.4 6 4.3
Total 140 100.0 140 99.9 140 99.8
While these data were collected some 25 years ago (1981), they underscore the point that such information is of great interest to both members and the general mental health field. At the end of 18 months, 55% of members were employed either on independent jobs or transitional employment placements. An additional 7.1% were in school or other rehab programs, and only three members were in hospital. Of special interest from a research standpoint was the fact that through member contact we obtained 18-month outcome information on 136 of the 140 members (or 97.1% of the study population).
Readership comments are most cordially welcome at the URL below or by writing to
tandcassociates@gmail.com
We at TCA maintain editorial rights over feedback, discussions and comments.