I’m curious to get a few comments about something. Brad DeLong reposted a link today to a year-old study about the effect of paid family leave on the use of nursing homes. The intuition here is that seniors often have short stays in nursing homes, and they might instead recuperate at home if caregivers were able to take time off work to look after them. California passed a family leave bill in 2004, so the study looks at what happened to nursing home use compared to similar states. Here’s the basic result:
As the authors note, there’s no obvious break in the trendline after 2004:
Figure 1 provides no apparent suggestion of a downward shift in California’s trend line beginning in 2005, the first full year of PFL implementation; instead, it appears that in comparison-group states the post-2004 trends lie above the pre-2004 trends.
But wait! If you take a closer look, here’s what you get:
California is down 0.23 percentage points. The comparable states are down by only 0.09 percentage points. Once you do all the official math, it turns out that California showed a big decline relative to other similar states:
PFL [paid family leave] reduced the annual proportion of elders in nursing homes in California by 0.0065, about two-thirds of a percentage point. While this reduction may appear small in size, its relative magnitude is substantial when compared to baseline nursing home utilization levels. In 2003, the year prior to PFL implementation, 5.7 percent of California’s older adults resided in nursing homes. Thus, our estimated PFL effect implies a relative decline of over 11 percent in the proportion of elderly in nursing homes in California….Our estimate is statistically significant at conventional levels.
So here’s what I’m curious about. Even if we accept the paper’s conclusions about statistical significance and so forth, it’s still the case that it’s all based on a tiny absolute difference in a real-world trendline that’s inherently noisy. So should we take this result seriously? Or should we be skeptics unless we get similar results from other states?
