The Source of the Statistics and How They Were Analyzed
December 30, 2004
The Times used the Panel Study of Income Dynamics for its analysis of
income volatility among working families and for its examination of
specific income-threatening events.
The panel study has followed a nationally representative sample of
about 5,000 families and their offshoots for nearly 40 years and is the
most comprehensive publicly available income and earnings database in
the world. It is run by the University of Michigan and principally
underwritten by the National Science Foundation.
The families' identities are kept confidential.
The Times based its analysis of income-threatening events on techniques
developed during the 1980s by economists Richard V. Burkhauser of
Cornell University and Greg J. Duncan of Northwestern University.
Their methods were modified for the newspaper by economists Ann Huff
Stevens and Marianne E. Page at UC Davis. The paper employed a UC Davis
graduate student, Sami T. Kitmitto, to help generate the findings.
Stevens and Page supervised Kitmitto and advised the newspaper.
The Times chose to focus on seven types of events that people often
experience in the ordinary course of a work life. They were: divorce or
separation, a major decline in the work hours of a spouse, the death of
a spouse, the birth of a child, a major decline in the work hours of
the head of the household due to disability or retirement, a major
spell of unemployment for the head of the household, and a major
decline in the work hours of the head of the household because of
illness.
Of families who experienced one or several of these events during a
decade, the paper asked what fraction subsequently suffered a 50% or
greater decline in annual income.
In examining these events, the paper faced certain constraints. The
panel study was started during the 1960s, when most families had a
single male earner. As a result, it has only limited information about
the earnings, employment status and work hours of spouses during the
early years, and to this day collects much more data about the head of
the household than it does about any other family members.
Because of funding constraints, the study switched from annual data
collection to an every-other-year schedule after 1996. This required
the paper to examine events in two-year intervals in order to maintain
a consistent approach both before and after 1996.
In addition, the paper could not make distinctions it would have liked
to make between certain types of events - for example between drops in
the work hours of the head of a household due to disability and due to
retirement - because of the way that the panel study initially
collected the information.
In cases of a decline in the work hours of the household head because
of disability or retirement, as well as a decline in the work hours of
a spouse, The Times defined "major" drops to be at least 520 hours - or
the equivalent of three months of full-time work - in the course of a
year.
In cases of unemployment of the household head, as well as a decrease
in work hours of the head because of illness, "major" drops were
defined to be at least 320 hours - or the equivalent of two months of
full-time work - in the course of a year.
The newspaper took a number of steps to ensure that its examination was not distorted by data problems with the panel study.
For instance, it made certain adjustments to avoid double-counting
events such as divorce. It also eliminated a sub-sample of Latino
families that were added to the panel study in the early 1990s.
Independent researchers have concluded that the sub-sample was poorly
designed and that its inclusion could skew results.
In addition, the newspaper excluded the 2% of panel-study families with
the biggest upward swings in incomes and the 2% with the biggest
downward swings to ensure that those with extremely volatile situations
did not throw off the results.
The paper also sought to test whether the declines in income of 50% or
more truly threatened a families' living standards or - as with a drop
from, say, $400,000 to $200,000 - were big but of comparatively little
consequence. It did this by measuring the percentage of panel-study
families that not only experienced income declines of 50% or more in
each of the last three decades, but whose declines brought them to
within 150% of the government's official poverty line ($28,215 today
for a family of four).
The paper concluded that the percentage of families suffering this kind
of life-altering hit jumped sharply and in tandem with the overall
increase in families experiencing 50%-plus income declines.
To zero in on working families, The Times focused on men and women 25 to 64 years old whose households had some income.
The analysis looked at pretax income of all family members from all
sources, including workplace earnings, investments, public transfers
such as jobless benefits, food stamps and cash welfare, and private
transfers such as inheritances.
All amounts were adjusted for inflation and expressed in 2003 dollars.
When it came to gauguing income volatility, The Times used techniques
that were developed by economists Robert A. Moffitt of Johns Hopkins
University and Peter Gottschalk of Boston College. The Times also
consulted with Yale political scientist Jacob Hacker, who has conducted
his own analysis of income volatility among panel-study households and
has published results linking it to economic risk.
The newspaper employed two Johns Hopkins graduate students, Xiaoguo Hu
and Anubha Dhasmana, to help generate results. Moffitt guided their
efforts and advised the newspaper.
The Times analyzed the annual fluctuations of panel-study families'
income in five-year increments from 1970 through 2000. It asked how
each family's income made the journey from the beginning of a five-year
period to the end.
For example, for a family whose income rose $5,000 over a five-year
span, the paper examined whether the increase occurred in steady $1,000
annual increases or as a result of a big jump one year and a plunge the
next.
The Times' basic finding is that the income fluctuations that
individual families experience have grown substantially larger over the
last three decades, and especially the last 25 years.
The newspaper interpreted this rising volatility as evidence that
families are taking on more economic risk. In doing so, the paper took
a page from the financial markets, where the chief measure of a stock's
riskiness is how much its price bounces up and down relative to an
overall market measure such as the Standard & Poor's 500 Index.
The newspaper described its volatility results in terms of upper and
lower boundaries within which the annual income swings of two-thirds of
families in the panel study fell. It included only two-thirds in order
to avoid having its results distorted by a few extreme cases, such as
the movie star whose career dries up overnight or the hourly worker who
wins the lottery.
For a family to be included in its sample for any five-year period, The
Times required there be financial records for at least three of those
years. To avoid double-counting income in cases where families broke
up, the paper added such payments as alimony and child support to
households receiving them and subtracted the same amounts from those
paying.
As with its examination of income-threatening events, the newspaper ran
a series of tests to ensure that its volatility results were not the
product of data problems but reflected real changes in the economy.
For instanct, the paper eliminated the Latino sub-sample. It also ran
its numbers with and without adjustment for changes in the size of
families over time. It ran them with and without transfer payments,
which some researchers believe were poorly recorded for several years
during the early 1990s. And it ran them with and without any data for
1993, the year about which reserachers have the greatest concern.
In every case where the volatility results with some families cut out
from the sample were similar to results with those families included,
the paper added back the removed families. The aim was to make the
fewest possible adjustments to the database and to maintain the largest
possible sample.
