The power of wishful thinking revisited: the improbable growth of Healthy Families America

I have written before about the power of wishful thinking and how it causes people to disregard research and real-life results. In that earlier commentary, I discussed the successful promotion of a practice called race-blind removals based on data from an article by a scholar who now denies knowledge of their provenance, and which have been shown to be inaccurate. A program called Healthy Families America (HFA), which currently serves over 70,000 families per year according to its website, offers another example of the power of wishful thinking. This program has become the centerpiece of the nation’s oldest and largest charity dedicated to the prevention of child abuse, even though the program has failed to demonstrate its utility in preventing child maltreatment. This organization, now called Prevent Child Abuse America, launched HFA based on weak evidence that a program in Hawaii called Healthy Start Program (HSP) could prevent child maltreatment. The first experimental study of HSP found no impact on child maltreatment but did nothing to derail the launch of HFA. Studies of HFA programs around the country have found little evidence of reductions in child maltreatment, but the program has continued to grow and now serves more families than any other home visiting program. The story of HFA is a lesson in the power of wishful thinking and the failure of evidence (or lack thereof) to counteract it.

As told in a helpful history of home visiting, all modern programs can trace their origins to Henry Kempe, whose book, The Battered Child, brought about the recognition of child maltreatment as a national problem. To address child abuse, Kempe called for universal prevention through a network of home health visitors. Inspired by Kempe, modern home visiting began with Hawaii’s implementation of the Healthy Start Project (HSP) in 1975. As described in the 1999 evaluation by Duggan and colleagues, HSP was developed by the Hawaii Family Stress Center (HFSC) on the island of Oahu. It had two components: early identification (at the birthing hospital) of families with newborns at risk of child abuse and neglect and home visiting by trained paraprofessionals for those families classified as at-risk who agreed to participate. This initial program was never evaluated, but anecdotal information suggested it was successful in promoting effective parenting, and six similar programs were established on neighboring islands.

As described by Duggan et al., the Hawaii Legislature authorized a three-year pilot program focusing on one neighborhood in Oahu, which began in 1985. There was no control group in the pilot study, and the researchers used CPS reports and changes in family stress in participating families to measure program effectiveness. During the three-year pilot, there were few reports of physical abuse, neglect or imminent harm for program participants. Because evaluations of other home visiting programs had found much higher rates of reported maltreatment in comparison group families, these results were viewed as evidence that the program had a positive impact. According to Duggan and her co-authors, “The pilot study results might have been given too much weight, given the lack of a control group and the short period of follow-up for most families.” Nevertheless, the results of this unpublished study were enough evidence for the Legislature to expand HSP throughout Hawaii starting in 1989.

Home visiting in general was gathering steam in the 1980s and early 1990’s. In 1990, the U.S. General Accounting Office (GAO) issued a report promoting home visitation as a “promising early intervention strategy for at-risk families.” In its summary of research evidence, GAO focused mostly on health and developmental benefits for children, rather than maltreatment prevention. In 1991, the U.S. Advisory Board on Child Abuse and Neglect issued a report recommending a pilot of universal voluntary neonatal home visitation, stating that the efficacy of home visiting as a preventive measure was “already well-established.” The Board cited the results of a federally-funded demonstration begun 17 years earlier as well as the the nurse home visitation program started by David Olds in 1977. But HSP was not mentioned.

Despite the lack of a rigorous evaluation, the National Committee to Prevent Child Abuse (NCPCA, now called Prevent Child Abuse America) the nation’s “oldest and largest organization committed to preventing child abuse and neglect before it happens,” had become interested in using HSP as the nucleus of a national home visiting program. But first, NCPCA conducted a one-year randomized trial of HSP, as described by Duggan et al. The trial suffered from severe methodological limitations, including “less than ideal followup,” differential dropout rates in the program and control groups, the failure to blind interviewers to experimental or control status, and reliance on program staff rather than researchers to measure some outcomes. Nevertheless, the trial concluded that HSP reduced child maltreatment, and this apparently gave NCPCA the assurance it needed to invest in the model.

NCPCA launched Healthy Families America in 1992, with financial support from Ronald MacDonald House Charities, arranging visits to 22 states by Hawaii Family Stress Center Staff. The “theory of change,” or theoretical basis for the program, as quoted by Duggan et al, started with the targeting to all newborns and their parents, which allows for diversified service options determined by individual need. Also part of the theory was a commitment to change at the individual and community levels. Rather than impose a single service model, HFA contained a set of critical elements, which included the prenatal initiation of services and the assessment of all new parents. A network was launched to bring together researchers doing experimental and quasi-experimental studies of HFA programs.

Unlike NCPCA, The Hawaii Department of Health recognized the limitations of both the pilot study and the NCPCA study and initiated a more rigorous evaluation of HSP in 1994. This was a randomized controlled trial, with at-risk families identified at the hospital and randomly assigned to the experimental and control groups. In 1999 the results of the Evaluation of Hawaii’s Healthy Start Program were released as part of an issue of the Future of Children journal containing evaluations of six different home visiting models.  No overall positive program impact emerged after two years of service in terms of child maltreatment (according to maternal reports and child protective services reports). Early HFA evaluation results, published in the same issue, also failed to find effects on abuse and neglect in three randomized trials, which included the HSP evaluation discussed above and another Hawaii HSP study.

David Olds had had begun testing his Nurse Home Visiting Program in 1977 and already had long-term results on the program in Elmira, NY, as well as shorter-term results for a replication in Memphis, Tenn. That program, now known as Nurse Family Partnership, was very different from HFA. It was restricted to first-time teenage mothers and the home visitors were nurses rather than paraprofessionals. The nurses followed detailed protocols for each visit. The researchers found that among low-income unmarried women (but not other participants), the program reduced the rate of childhood injuries and ingestions of hazardous substances that could be associated with child abuse or neglect. Follow-up of the Elmira group when the children were 15 found that the nurse-visited mothers were significantly less likely to have at least one substantiated report of maltreatment. These results are particularly impressive because they overrode a tendency for nurse-visited families to be reported for maltreatment by their nurse visitors. The researchers concluded that the use of nurses, rather than paraprofessionals, was key to the success of the program. In their analysis of all six studies published in the volume, Deanna Gomby et al. concluded that while the HFA and HSP evaluations showed some change in maternal attitudes and self-reported behaviors related to abuse and neglect, only the Nurse Home Visiting Program showed impacts on abuse and neglect other than from self-reports.

Gomby and her co-authors also concluded that the results of the six home visiting evaluations were discouraging for those who had high hopes for home visiting for solving an array of problems. All the programs “struggled to enroll, engage and retain families.” Program benefits generally accrued to only a subset of enrolled families and were often quite modest. The authors explained the disappointing results by concluding that human behavior is hard to change, particularly when problems are community-wide. They recommended that “any new expansion of home visiting programs be reassessed in light of the findings presented in this journal issue” and stated that home visiting services are “best funded as part of a broad set of services for families and children.”

But the home visiting juggernaut was already in motion nationwide. And NCPCA had already made HFA its centerpiece program. Home visiting grew, and HFA grew with it. In 2010, Congress created the Maternal, Infant and Early Childhood Home Visiting Program (MIECHV), which was re-authorized in 2018 with funding of $400 million per year through FY 2022. According to the HFA website, HFA is the model most frequently implemented with MIECHV dollars. Home visiting programs can also receive funding through Medicaid, Title IVB and IVE of the Social Security Act, and many other funding sources.

The infusion of funding for HFA research by NCPCA initiative set in motion a multitude of research projects (both randomized trials and less rigorous studies) that continues to result in publications. Nevertheless, HFA research has yet to find solid evidence that these programs have an impact on child maltreatment: The California Evidence-Based Clearinghouse for Child Welfare (CEBC), the pre-eminent child welfare program clearinghouse, reviewed 19 research reports on HFA. It gave the program a rating of “4” on a scale of 1 to 5 for prevention of child abuse and neglect, meaning the evidence fails to demonstrate that the HFA has an effect on abuse and neglect. HFA did receive a rating of 1 for “child well-being,” based on its impacts on outcomes like physical health, child development, and school readiness. In contrast, Nurse Family Partnership was rated as “1,” “well-supported by the research evidence, for the prevention of child abuse and neglect, as well as for child well-being.

Despite the lack of evidence of its impact on maltreatment, HFA received a rating of “Well Supported” from the new clearinghouse established by the Family First Prevention Services Act (“Family First”) to determine whether a program can receive federal funding under Title IV-E of the Social Security Act. To get such a rating, the program must show improved outcomes based on at least two randomized trials or rigorous quasi-experimental studies. But these outcomes could be any sort of “important child and parent outcome,” (not just child abuse or neglect) and there is no standard for how to measure each outcome. Based on its review of all HFA studies that met their criteria for inclusion, the Clearinghouse found 23 favorable effects, 212 findings of no effect, and four unfavorable effects across 16 outcomes. This included five favorable effects on child safety based on parents’ self-reports of maltreatment, with no favorable effects on other measures of child safety. Self-report is generally frowned upon as a measure of child maltreatment, for obvious reasons. A positive impact of HFA might reflect that participants in HFA were more eager than control group members to provide the “right answer” to questions about maltreatment.

The “well-supported” rating from the Title IV-E clearinghouse opened up a new source of funding for HFA. Passage of Family First as Title VII of the Bipartisan Budget Act of 2018, allowed states to spend Title IV-E funds on programs on services to families with a child welfare in-home case. To take advantage of this new demand, HFA announced in September 2018 that families referred by the child welfare system were now able to enroll until 24 months of age. To serve these families, HFA introduced special child welfare protocols, with limited evidence that that the program was effective for parents who had already abused or neglected their children.* The program had already departed from its initial mission of screening all families with newborns in a geographic area. Even without the child welfare protocols, each program can choose its own admission criteria and there is no universal screening; potential participants are generally referred by health or child welfare agencies, who often can choose between several home visiting programs when referring a client.

Another part of HFA’s original theory of change was a “dual commitment to change at the individual and community levels.” As described by Daro and Harding in their 1999 evaluation of HSA, this meant that HFA “must move beyond direct efforts to help families and begin to serve as a catalyst for reshaping existing child welfare and health care efforts and improving coordination among other prevention and family support initiatives.” This vision has clearly gone by the wayside as HFA has become one choice in a menu of home visiting programs offered by local jurisdictions. Far from trying to enhance and coordinate available community offerings, HFA is busy trying to maximize its share of the pie through its public relations effort, exemplified by the self-promotional statements on its website.

It is disappointing that Prevent Child Abuse America (“Prevent Child Abuse,” formerly NCPCA), an organization that defines its mission as child abuse prevention, decided to fund HFA before it was proven to prevent child maltreatment and without apparently considering other approaches also being tested at the time. And it is concerning that the organization continued with this commitment even after the disappointing evaluations of 1999 might have led them to diversify their investment beyond HFA or even beyond home visiting or to focus more on advocacy rather than services. And finally, that Prevent Child Abuse continues to use charitable contributions made for the prevention of child abuse and neglect to fund a program that has not been proven after 40 years to accomplish this goal, raises serious ethical questions. Twenty-two of the 40 staff listed on the Prevent Child Abuse website have positions with Healthy Families America. Perhaps the charity has backed itself into a corner; it would be difficult to escape this commitment without serious repercussions.

Some federal administrators do not seem to be much more interested in evaluation results than Prevent Child Abuse. The legislation authorizing MCHIEV required a randomized controlled trial (RCT), which may provide useful information on the relative merits of these programs in addressing different outcomes. But strangely, HHS indicated in a response to a critique from the Straight Talk on Evidence Blog that it is not interested in a “horse race” between the models but rather is interested in assessing home visiting in general. This odd statement is an indicator of the kind of thinking that allowed Prevent Child Abuse to invest in HFA for 40 years despite the lack of evidence that it does “Prevent Child Abuse.”

The story of Healthy Families America is not an unusual one. My discussion of the Homebuilders program could also be called “the power of wishful thinking.” Such stories are all too frequent. They show us how wishful thinking can drive leaders to disregard research, especially after they have made a premature decision to commit to one program or course of action.

*One study of Healthy Families New York, published in 2018, looked at a subgroup of 104 mothers who already had a substantiated CPS report, and found a decrease in abuse and neglect among the mothers who were in the experimental group. However, the sample was small and was not planned in advance, so the authors recommend further testing home visiting programs as prevention of repeat maltreatment for child welfare-involved mothers.

One thought on “The power of wishful thinking revisited: the improbable growth of Healthy Families America

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s