Employment Retention and Advancement Study
Over the past decade America has modified its welfare system in an attempt to reduce the level of recipient dependence and increase the proportion of participants who are qualified for and take up productive jobs. Government agencies have created programs to assist participants in finding and keeping employment. Early efforts often focused on obtaining jobs, with less consideration of the wage and benefit levels and support system needed for a person to retain that employment. Finding a job was a first step, but keeping it and advancing to better-paying employment opportunities was a necessary second. While some welfare participants are able to succeed in their efforts, others need additional support in order to keep a job or move upward in a position already obtained.
The Manpower Demonstration Research Corporation (MDRC) was awarded a contract by the U.S. Department of Health and Human Services for the evaluation of several of these more intensive assistance programs. Named the Employment Retention and Advancement (ERA) evaluation, MDRC includes 15 assistance programs operating in 8 states from New York to California. HumRRO is responsible for locating and interviewing participants in these programs. Assistance participants are surveyed to gain information useful in determining each program's impact on employment retention and advancement. We assign those surveyed at random into experimental or control research groups, and contact them for interviews at the 12-month anniversary of that assignment.
The sample for each site is released in consecutive monthly cohorts. A site may have from 3 to 13 monthly cohorts. Data collection continues for each monthly cohort for 5 months after its random assignment anniversary month. For any monthly cohort, the interviewing effort begins at the 12-month anniversary of random assignment, e.g., data collection for the cohort comprised of sample members randomly assigned in October 2002 will begin in October 2003. During the first month of the effort, we place outgoing calls and make tracking efforts from the Telephone Center. We assign those cases not yet interviewed by this time for in-person tracking.
Trackers locate those individuals and facilitate their connection to the Telephone Center for interview completion. Trackers carry cellular telephones in the event the sample member has no telephone. Trackers have five months to complete each cohort, e.g., the October 2003 cohort will remain in the field through March 2004, at which time the cohort is closed to further effort.
As each monthly cohort is rolled from the Telephone Center to in-person trackers, a new monthly cohort is rolled into the Telephone Center. The assignment of each sample member in each cohort in each site along with all tracking records developed for that sample member are managed by the Master Database created at HumRRO. All interviews are conducted by computer-assisted telephone interview (CATI). The interview takes roughly 35 minutes to complete. A response rate of 80 percent is desired for most sites. HumRRO provides coded and cleaned data according to specifications approved by MDRC. Cumulative data files are provided every quarter.
Each site began its program based on its own schedule, but random assignment was carried out some time within a 2½ period from late 2001 to early 2004. All sites did not begin at the same time. Data collection across all sites will require 30 months with most sites requiring 9 months for completion. A total of nearly 10,000 sample members have been identified across the 15 sites.
The project began field implementation in January 2002. Data collection ended in July 2005. This has concluded for six of the 15 programs with an average overall response rate (total interviews / total sample) of 77 percent; all six sites hit their targeted response rate. Of those sought, 90 percent were found, 5% refused and an additional 5%, although not actively refusing, could not be interviewed before the field period expired. Active refusal rates varied by site, ranging from 3 percent to 9 percent.
