%Paper: ewp-em/9506004 %From: Eric Rasmusen %Date: Wed, 14 Jun 95 13:37:42 -0500 %Date (revised): Fri, 16 Jun 95 16:35:51 -0500 % December 5, 1992 % MInor style changes, may 13 1993, juyly 8 \documentstyle[12pt,epsf] {article} \parskip 10pt \begin{document} \parindent 24pt \titlepage \vspace*{12pt} \begin{center} \begin{large} {\bf Observed Choice and Optimism in Estimating the Effects of Government Policies }\\ \end{large} \vskip 20pt August 4, 1994 \\ \bigskip Eric Rasmusen\\ \vskip .7in {\it Abstract} \end{center} A policy will be used more heavily in a particular time and place where its marginal cost is lower. The analyst who treats times and places as identical will overestimate the policy's net benefit, especially for policy intensities greater than exist in his sample. In regression analysis, the problem can be solved by instrumental variables and a correction for heteroskedasticity. In an example using state-level data, the technique substantially increases the estimated responsiveness of the illegitimacy rate to transfer payments. \vskip .3in \begin{small} \noindent \hspace*{20pt} Indiana University School of Business, Rm. 456, 10th Street and Fee Lane, Bloomington, Indiana, 47405-1701. Office: (812) 855-9219. Fax: 812-855-3354. Internet: Erasmuse@indiana.edu.\\ JEL numbers and Keywords: C1, C3, C5, H3, I3. Estimation bias. Poverty. Political economy. Instrumental variables. % Draft: 7.4 (Draft 1.1, May 1990). % \footnote{xxx Footnotes starting with xxx are the author's notes to % himself.} I would like to thank Robert Barsky, Trudy Cameron, John Garen, Hashem Pesaran, Simon Potter, Sunil Sharma, Hal Varian, and seminar participants at Indiana University, the University of Michigan, the University of Rochester, and the Wharton School for comments. Much of this work was completed while the author was an Olin Fellow at the Center for the Study of the Economy and the State, University of Chicago, and on the faculty of UCLA's Anderson Graduate School of Management. \end{small} %---------------------------------------------------------------% %--------------------------------------------------------------- \newpage \begin{center} {\bf 1. Introduction.} \end{center} An important problem is how to judge the effect of a government policy by looking at data on its use and impact in various times and places. The task might be to estimate the effect of government transfers on poverty, of unemployment insurance on unemployment, or of the tax rate on tax revenue. Let the hypothesized relationship be $Impact = \beta Policy$, or \begin{equation} \label{e1} y = \beta x. \end{equation} The observed-choice problem, the subject of this article, is that very commonly $x = x(\beta)$, because the observed policies are not random. They are chosen in recognition of their costs and benefits in particular times and places, so $x$ depends on $\beta$, which differs across observations. If policies are used more where they are more effective on the margin, then both casual empiricism and estimates using ordinary least squares, are biased towards optimism about the effect of the policies. This is not like typical sources of bias such as omission of relevant variables, which can cause bias in either direction; rather, it is like measurement error with one regressor, which generates bias in a predictable direction. The mathematics of the observed-choice problem are relatively simple, relying on the theories of instrumental variables and random coefficients that are by now well-established in the econometrics literature, though perhaps not in exactly this combination. Nor is the idea that individuals make decisions based on costs and benefits new; this is the heart of economics. What this paper will contribute is the observation that if decisions are made by rational actors, then cross-section estimation of the effects of government policies will be biased, and biased systematically in favor of government activism. Section 2 will set up the estimation problem and the bias that results (subsection 2.1), show the sign of the bias (2.2), devise a consistent estimator (2.3), and discuss a different approach suggested by Garen (2.4). Section 3 will explain the problem more intuitively (3.1), distinguish it from other econometric problems (3.2), discuss related examples with discrete variables or nonlinearities (3.3), and compare the policymaking problem with the prediction problem (3.4). Section 4 will apply the analysis in a particular context, the effect of government transfer payments on illegitimacy. Section 5 concludes. %--------------------------------------------------------------- \bigskip \begin{center} {\bf 2. The Observed-Choice Problem} \end{center} \noindent {\bf 2.1. The Model} \noindent The analyst is trying to estimate relationship (\ref{e2}): \begin{equation} \label{e2} y = \beta x \; . \end{equation} Each of his $n$ observations consists of an impact level $y$ and a policy level $x$ for a particular time and place, subscripted $i$. The standard approach is to regress $y$ on $x$ in the belief that the true specification is \begin{equation} \label{e3} y_i = \beta x_i + \epsilon_i, \; \end{equation} where $\epsilon \sim (0, \sigma_\epsilon^2)$. As always in estimation, the analyst does not believe equation (\ref{e3}) to be more than an approximation. The true relationship is unlikely to be precisely linear, for example, but linearity is a good approximation when one does not know whether a convex, concave, or wavy function would be appropriate. Similarly, each time and place does not have exactly the same true coefficient, and a more accurate specification would be equation (\ref{e4}), in which the effect of the policy is different for each observation: \begin{equation} \label{e4} y_i = \beta_i x_i + \epsilon_i \; . \end{equation} Equation (\ref{e4}), however, is impossible to estimate, since it has $n$ parameters and there are only $n$ observations. Moreover, using approximation (\ref{e3}) might not be misleading, since, in the absence of other considerations, the regression of $y$ on $x$ does give an unbiased estimate of the average $\beta$. To see this, suppose that the true specification for $\beta_i$ in equation (\ref{e4}) is \begin{equation} \label{e5} \beta_i = \overline{\beta} + v_i, \end{equation} where $v \sim (0, \sigma_v^2)$and is independent of $\epsilon$. Using (\ref{e5}), equation (\ref{e4}) becomes \begin{equation} \label{e6} y_i = \overline{\beta} x_i + x_i v_i + \epsilon_i \; . \end{equation} The ordinary least squares (OLS) estimate of $\overline{\beta}$ is \begin{equation} \label{e7} \widehat{\beta}_{OLS} = \frac{\sum x_i y_i }{\sum x_i^2}, \end{equation} where $\sum$ will denote $\sum_{i=1}^n$ throughout the paper. If $v_i$ and $x_i$ are independent, the OLS estimate of $\overline{\beta}$ is unbiased, because % \footnote{xxx Unbiased, not just conssitent. See Maddala p. 151.} the expected value of expression (\ref{e7}) is \begin{equation} \label{e8} E \left( \frac{\sum x_i ( \overline{\beta} x_i + v_i x_i + \epsilon_i) }{\sum x_i^2} \right)\;, \end{equation} which equals \begin{equation} \label{e9} E \left( \overline{\beta} \frac{\sum x_i^2}{\sum x_i^2} \right) + E \left( \frac{\sum x_i^2 v_i}{\sum x_i^2} \right) + E \left( \frac{\sum x_i \epsilon_i }{\sum x_i^2} \right)\;. \end{equation} The first and last terms of (\ref{e9}) equal $ \overline{\beta}$ and 0, and the middle term equals 0 if $E (x_i^2 v_i) = 0$. Thus, if $x_i$ and $v_i$ are independent, OLS is unbiased. Despite the unbiasedness of $\widehat{\beta}_{OLS}$, heteroskedasticity does make OLS inefficient and biases the estimated standard errors. The variance of the error term for observation $i$ is $x_i^2 \sigma_u^2 + \sigma_\epsilon^2$, from equation (\ref{e6}), which varies depending on $x_i$. Although $ E (x_i v_i)=0$, observation $i$'s disturbance depends on the size of $x_i$. When $x_i$ is large, so is the disturbance, and observation $i$ ought to be weighted less heavily in the estimate. This ``varying-parameters'' heteroskedasticity is a well-known problem, and the estimate can be improved by weighted least squares as described below in Section 3.2.\footnote{For textbook discussions of varying-parameter models, see pp. 75-89 of Kennedy (1985) and pp. 390-393 of Maddala (1977).} A greater difficulty is that $v_i$ and $x_i$ are unlikely to be independent. After all, why is $x_i$ different from $x_j$? Policies are chosen for many different reasons, but benefits are always weighed against costs, and the variable $y$ that the econometrician is examining is very likely to be part of either the benefit or the cost. Suppose, for example, that $x$ is the level of cigarette taxation and $y$ is the amount of deadweight loss. Deadweight loss is a cost, and states where taxes create more deadweight loss will choose lower levels of taxation. Or suppose that $x$ is the level of cigarette taxation, and $y$, the amount of revenue raised, which depends on the potential for smuggling. Revenue is a benefit, and states where cigarette taxes raise more revenue may choose higher levels of taxation. The relevance of costs and benefits is robust to the details of why the policies are chosen. If the legislators aim to maximize social welfare, it is obvious that they will weigh costs and benefits. But even if their primary concern is to please special interest groups such as cigarette companies or the beneficiaries of state spending, the legislators will still consider the public costs and benefits if the general public has any political influence whatsoever (as Peltzman [1976] points out). It may well be that lobbying by cigarette companies makes every state set the tax too low from the viewpoint of social welfare, but states where the cost of the tax is low and the benefit is high will nonetheless have the highest taxes, because lobbyists would have to spend more there to obtain a given tax reduction. This logic says that $x_i$ depends on $\beta_i$, and on other factors which will be incorporated as an exogenous variable $w$, so a third equation, equation (\ref{e21}), is required to describe the complete system: \begin{equation} \label{e20} y_i = \beta_i x_i + \epsilon_i \; , \end{equation} \begin{equation} \label{e22} \beta_i = \overline{\beta} + v_i \; , \end{equation} and \begin{equation} \label{e21} x_i= \gamma_1 + \gamma_2 \beta_i + \gamma_3 w_i + u_i\;, \end{equation} \noindent where it will be assumed that: (i) $\gamma_1 + \gamma_2\overline{\beta} + \gamma_3 \sum w_i/N >0,$ (ii) $\overline{\beta}>0$, (iii) $w$ and $\overline{\beta}$ are nonstochastic, (iv) $\epsilon, u$ and $v$ are independent stochastic disturbances with mean zero and finite variance, and (v) $v$ has a symmetric distribution. Assumptions (i) and (ii) are normalizations. Assumption (i) says that the average value of $x$ is positive. Assumption (ii) says that the policy has a positive effect on the impact, whether the impact be desirable or not. Assumptions (iii) and (iv) establish what is exogenous. Assumption (v) says that the true coefficients are symmetrically distributed around their average of $\overline{\beta}$.\footnote{This assumption is used just following equation (\ref{e30}) below. The bias will exist regardless of whether there is skewness or not, but if $E v_i^3 \neq 0$, analysis of the sign of the bias becomes more complicated.} The system of equations (\ref{e20}) to (\ref{e21}) violates the assumptions of the OLS model in two ways, each harmless by themselves: random parameters and stochastic regressors. The simpler system consisting of (\ref{e20}) and (\ref{e22}) has random parameters, but OLS is still unbiased as an estimate of the expected value of the parameter. The simpler system consisting of (\ref{e20}) and (\ref{e21}) (in which case $\beta_i =\overline{\beta}$) has stochastic regressors, but OLS is also unbiased in that system. Like binary nerve gas, the two problems are harmless individually, but dangerous in combination. To see that the OLS estimate of $\overline{\beta}$ is biased, combine equations (\ref{e22}) and (\ref{e21}) to obtain \begin{equation} \label{e25} x_i= \gamma_1 + \gamma_2 \overline{\beta} + \gamma_2 v_i + \gamma_3 w_i + u_i \; . \end{equation} The critical middle term in the $\widehat{\beta}_{OLS}$ equation, (\ref{e9}), which for unbiasedness must equal zero in expectation, is \begin{equation} \label{e26} \frac{\sum x_i^2 v_i}{\sum x_i^2} \end{equation} or, using (\ref{e25}), \begin{equation} \label{e27} \frac{\sum (\gamma_1 + \gamma_2 \overline{\beta} + \gamma_2 v_i + \gamma_3 w_i + u_i)^2 v_i}{\sum x_i^2}. \end{equation} The summed quantity in the numerator can be written as \begin{equation} \label{e29} ([\gamma_1 + \gamma_2 \overline{\beta} + \gamma_3 w_i + u_i] + \gamma_2 v_i)^2 v_i \; , \end{equation} which equals \begin{equation} \label{e30} [\gamma_1 +\gamma_2 \overline{\beta} + \gamma_3 w_i + u_i]^2 v_i + 2[\gamma_1 + \gamma_2 \overline{\beta} + \gamma_3 w_i + u_i]\gamma_2 v_i^2 + \gamma_2^2 v_i^3, \end{equation} the expectation of which equals \begin{equation} \label{e31} 2\gamma_2[\gamma_1 + \gamma_2 \overline{\beta} + \gamma_3 w_i] \sigma^2_v, \end{equation} since ($E (v^3)=0$ by assumption (v), and $u$ and $v$ are independent. Expression (\ref{e31}) has the same sign as $\gamma_2[\gamma_1 + \gamma_2 \overline{\beta} + \gamma_3 w_i]$. Summed across the $n$ observations, this takes the same sign as $\gamma_2$, since the term in square brackets is positive by assumption (i). The parameter $\gamma_2$ represents how the marginal impact of the policy affects the policy level chosen. If the policy is used more where it is more effective, then $\gamma_2 >0$ if $y$ is a desirable impact and $\gamma_2 <0$ if $y$ is undesirable. Expression (\ref{e31}) takes the same sign as $\gamma_2$, so the conclusion would be that $\beta$ is overestimated if $y$ is desirable and underestimated if $y$ is undesirable. Whether $\gamma_2$ takes those signs is not obvious, however, and Section 2.2 is devoted to investigating it. %--------------------------------------------------------------- \bigskip \noindent {\bf 2.2. The Sign of $\gamma_2$: Is a Policy Used More Where it is More Effective?} Section 2.1 showed that the sign of the bias depends on the sign of $\gamma_2$ in equation (\ref{e21}), which is repeated here: $$ x_i= \gamma_1 + \gamma_2 \beta_i + \gamma_3 w_i + u_i\;. $$ What can be said about $\gamma_2$ in general, without knowing the particular application? Is the policy used more where it is more effective, so that $\gamma_2$ is positive where the impact is desirable and negative where it is undesirable? Let us use a general optimization problem to address the question. Consider one time and place $i$ (so we can drop the subscript $i$) where the policy $x$ has an impact $\beta_b x$ which produces a utility benefit of $B(\beta_b x)$, with $B'>0, B''\leq 0$; and an impact $\beta_c x$ which produces a utility cost of $C(\beta_c x)$, with $C'>0, C'' \geq 0$ (and either $C''>0$ or $B''>0$, to give the problem an interior solution). Assume the benefit and the cost to be separable, so the policymaker's problem is \begin{equation} \label{e1000} \stackrel{Max}{x} M(x) = B(\beta_b x)- C(\beta_c x). \end{equation} The first order condition is \begin{equation} \label{e1010} \frac{ \partial M}{\partial x} = \beta_b B' - \beta_c C'=0, \end{equation} and the second order condition is \begin{equation} \label{e1020} \frac{\partial^2M}{\partial x^2} = \beta_b^2 B'' - \beta_c^2 C'' < 0. \end{equation} \noindent The cross-partials are \begin{equation} \label{e1030} \frac{\partial^2M}{\partial x \partial \beta_b} = B' + \beta_b x B'' \;\; \end{equation} and \begin{equation} \label{e1040} \frac{ \partial^2M}{\partial x \partial \beta_c} = -C' - \beta_c x C'' <0. \end{equation} Because \begin{equation} \label{e1050} \begin{array}{lll} \frac{ d x}{d \beta_b} = - \frac{ \frac{ \partial^2M}{\partial x \partial \beta_b}}{\frac{ \partial^2M}{\partial x^2} } & & \frac{ d x}{d \beta_b} = (-) \frac{ (?) }{(-)}\\ \end{array} \end{equation} and \begin{equation} \label{e1060} \begin{array}{lll} \frac{ d x}{d \beta_c} = - \frac{ \frac{ \partial^2M}{\partial x \partial \beta_c}}{ \frac{ \partial^2M}{\partial x^2} } & & \frac{ d x}{d \beta_c} = (-) \frac{ (-) }{(-)}\\ \end{array} \end{equation} we can conclude that $ \frac{ d x}{d \beta_c} $ is always negative, but $\frac{ d x}{d \beta_b} $ might be positive. A less intense value of the policy is chosen when the cost parameter is big, but not necessarily when the benefit parameter is small. There are two implications for the bias of the OLS estimates in Section 2.1:\footnote{ It is interesting to note that the result on costs, and sometimes an benefits, leads to the same conclusion as the folk wisdom that estimation problems usually lead to coefficients that are too small.} (a) If $y$ is undesirable, a cost of the policy, then $\gamma_2 <0$ in equation (\ref{e21}). A bigger $\beta_c$ leads to a smaller $x$. Hence, in the original estimation problem, OLS underestimates $\overline{\beta}$ when the impact is undesirable. (b) If $y$ is desirable, a benefit of the policy, then $\gamma_2$ might be either positive or negative. If $B(\cdot)$ is close to linear, then $B''$ is small, expression (\ref{e1030}) is positive, and $\gamma_2 >0$: a bigger $\beta_b$ leads to a bigger $x$. If $B(\cdot)$ is heavily concave (i.e., the benefit $y$ has sharply diminishing marginal utility), then $B''$ is large and $\gamma_2 <0$. The more intuitive sign is $\gamma_2 >0$, which says that the policy is used more intensively where it is more effective, in which case OLS overestimates $\overline{\beta}$, the positive marginal impact. It is also possible, however, that the policy is used more intensively where it is less effective (the policymaker may wish to attain a threshold benefit, for example, which requires greater use of the policy if it is less effective). \bigskip It may be helpful to think of the policy $x$ as an expenditure, $PQ^d$, and the impact $\beta_b x$ as the quantity demanded, $Q^d$. Then $\frac{ x}{\beta_b x} = \frac{1}{\beta_b}$ is like the price of the good--- it is the expenditure divided by the quantity. When $P$ falls, $Q^d$ always rises. But for some goods, demand is elastic, and when $P$ falls, $PQ^d$ rises. For other goods, demand is inelastic, and $PQ$ falls. For goods with elastic demand, $\gamma_2 >0$, and for goods with inelastic demand, $\gamma_2 <0$. The direction of the bias of OLS thus depends on the elasticity of demand for the policy's benefits. In the original estimation problem, OLS will overestimate $\overline{\beta}$ if demand for the impact is elastic, and underestimate it if demand is inelastic. Yet another way to understand this is by realizing that the same problem comes up in trying to predict how factor choice changes with technical change. If the cost of labor goes up, one can confidently predict that a factory's use of labor will fall. If the effectiveness of labor goes up, one cannot predict whether the factory will use more labor or less. Ordinarily, we think it will use more, but that need not be the case. \bigskip \noindent {\bf 2.3. A Consistent Estimator for the Observed-Choice Problem} The observed-choice problem can be solved by using instrumental variables, even though it is not a conventional simultaneity problem. Begin with the system above: equations (\ref{e20}), (\ref{e22}), and (\ref{e21}). Equations (\ref{e20}) and (\ref{e22}) were combined to give (\ref{e25}), $$ x_i= \gamma_1 + \gamma_2 \overline{\beta} + \gamma_2 v_i + \gamma_3 w_i + u_i \; , $$ which can itself be rewritten as \begin{equation} \label{e31a} x_i= (\gamma_1 + \gamma_3 \overline{w} + \gamma_2 \overline{\beta}) + \gamma_2 v_i + \gamma_3 (w_i - \overline{w}) + u_i \; , \end{equation} where $\overline{w}$ is the sample mean of $w$. Using $(w_i - \overline{w})$ as an instrument for $x_i$, the instrumental variables estimator is \begin{equation} \label{e32} \widehat{\beta}_{IV} = \frac{\sum (w_i-\overline{w}) y_i}{\sum (w_i-\overline{w}) x_i}. \end{equation} Combining equations (\ref{e20}) and (\ref{e22}) yields $ y_i = \overline{\beta} x_i + v_i x_i + \epsilon_i$, which can be substituted into (\ref{e32}) to yield \begin{equation} \label{e34} \begin{array}{ll} plim\; (\widehat{\beta}_{IV}) & = plim \; \left(\frac{\sum (w_i-\overline{w}) (\overline{\beta} x_i + v_i x_i + \epsilon_i)}{\sum (w_i - \overline{w}) x_i} \right)\\ & \\ & = \overline{\beta} + plim \left(\frac{\sum (w_i-\overline{w}) v_i x_i}{\sum (w_i - \overline{w}) x_i} \right) + plim \left(\frac{\sum (w_i-\overline{w}) \epsilon_i)}{\sum (w_i - \overline{w}) x_i}\right).\\ \end{array} \end{equation} Substituting for $x_i$ from equation (\ref{e31a}) gives \begin{equation} \label{e34a} \begin{array}{ll} plim\; (\widehat{\beta}_{IV}) & = \overline{\beta} + plim \left(\frac{\sum (w_i-\overline{w}) v_i(\gamma_1 + \gamma_3 \overline{w} + \gamma_2 \overline{\beta})}{\sum (w_i - \overline{w}) x_i} \right) + plim \left(\frac{\sum (w_i-\overline{w}) v_i^2 \gamma_2 }{\sum (w_i - \overline{w}) x_i} \right) + plim \left(\frac{\sum (w_i-\overline{w})^2 v_i \gamma_3 }{\sum (w_i - \overline{w}) x_i} \right) + \\ & plim \left(\frac{\sum (w_i-\overline{w}) v_i u_i }{\sum (w_i - \overline{w}) x_i} \right) + plim \left(\frac{\sum (w_i-\overline{w}) \epsilon_i)}{\sum (w_i - \overline{w}) x_i}\right).\\ & \\ & = \overline{\beta}. \end{array} \end{equation} Thus, a consistent estimator can be obtained for $\overline{\beta}$ if an instrument, $(w - \overline{w})$, is available for $x$.\footnote{The constant is another suitable instrument for $x$ here, since $v$ has mean zero. If a constant is used as an instrument, then $w$ itself can be used, instead of $w- \overline{w}$. This problem differs from the standard instrumental variables problem, in which the difficulty is that $x$ is correlated with the disturbance $\epsilon$, so, since $\epsilon$ has mean zero, the instrument does not itself need to have mean zero. The special difficulty here is the $wv^2 \gamma_2$ term. Since $E v^2 \neq 0$, the instrument must have mean zero or the set of instruments must include a constant.} Heteroskedasticity is also a problem, because the error in equation (\ref{e24a}) is $ v_i x_i + \epsilon_i$, the variance of which, $x_i^2\sigma^2_v + \sigma^2_\epsilon$, is different for each observation. Weighted instrumental variables is appropriate, with weights $1/\sqrt{(x_i^2\sigma^2_v + \sigma_\epsilon^2)}$, which requires estimates of $\sigma^2_v$ and $\sigma^2_\epsilon$. One procedure to generate estimates of $\sigma^2_v$ and $\sigma^2_\epsilon$ is: \begin{enumerate} \item[(a)] Regress $x$ on $w$ and a constant to get fitted values $\widehat{x}$. \item[(b)] Regress $y$ on $\widehat{x}$ to get the estimated coefficient $\widehat{\beta}$. \item[(c)] Construct estimated errors $e_i = y_i - \widehat{\beta} x_i$. \item[(d)] Regress $e^2$ on $x^2$ and a constant. Let $\widehat{\sigma^2_\epsilon}$ be the estimate of the constant and $\widehat{\sigma^2_v}$ be the estimate of the coefficient. \end{enumerate} \bigskip \noindent {\bf 2.4. The Garen Technique} Garen (1984) solves a problem similar to the present one without using instrumental variables, though his procedure is equivalent to 2SLS in some examples (see Garen [1987]). Let us assume that $w$ is not a determinant of $x$, so no instrument is available. The system to be estimated is then: \begin{equation} \label{e42a} y_i = \overline{\beta} x_i + v_i x_i + \epsilon_i \; , \end{equation} and \begin{equation} \label{e42b} x_i= \gamma_1 + \gamma_2 \overline{\beta} + \gamma_2 v_i + u_i \; , \end{equation} Let us also assume that $u \equiv 0$, which will replace identification-by-instrument. The reason that OLS is biased in equation (\ref{e42a}) is that if $y$ is regressed on $x$, the regressor $x$ is correlated with the error term $vx$. This can be viewed as an omitted-variable problem, and including a consistent estimate of $vx$ as a separate regressor would eliminate the bias asymptotically. The analyst can estimate $v_i$ by $\widehat{v_i} = x_i - \overline{x} = \gamma_2 v_i$. This is biased unless $\gamma=1$, but that is unimportant, since the coefficient on $v_i x_i$ in equation (\ref{e42a}) is known to be unity and its regression estimate will be ignored anyway. The analyst can therefore regress $y$ on $x$ and $\widehat{v}x$ to obtain a consistent estimate of $\overline{\beta}$. This procedure cannot be used when $u$ does not equal zero---that is, when the policy is partly determined by factors unobserved by the analyst. In that case, $\widehat{v_i} = x_i - \overline{x} = \gamma_2 v_i+u_i$, which is correlated with $x_i$ because $x_i$ and $u_i$ are correlated. Because of the correlation with $x_i$, $\widehat{v_i} x_i$ is not a consistent estimator even of $\gamma_2 v_i x_i$, and a regression of $y$ on $x$ and $\widehat{v_i} x_i$ would not produce a consistent estimate of $\overline{\beta}$. Equation (\ref{e42a}) can be rewritten as \begin{equation} \label{e42c} \begin{array}{ll} y_i &= \overline{\beta} x_i + (\gamma_2 v_ix_i + u_ix_i) + ([1-\gamma_2] v_ix_i - u_ix_i) + \epsilon_i \;\\ & \\ &= \overline{\beta} x_i + \widehat{v_i} x_i + ([1-\gamma_2] v_ix_i - u_ix_i) + \epsilon_i \;.\\ \end{array} \end{equation} Thus, if $y$ were regressed on $x$ and $\widehat{v_i} x_i$, the regressor $x$ would be correlated with $u_i x_i$ in the error term, and the estimate of $\overline{\beta}$ would be biased. The bias disappears only if $u \equiv 0$. Hence the Garen technique, although it does not require an instrument for $x$, does require the analyst to have precise knowledge of the variables that determine $x$. \bigskip \begin{center} {\bf 3. Explanation, Examples, and Prediction} \end{center} \noindent {\bf 3.1 An Intuitive Explanation of the Observed-Choice Problem} The algebraic development of Section 2 makes it clear that OLS is biased when the observed-choice problem is present, but yields very little intuition as to why. Diagrams can make it considerably clearer, and can show why the sign of the bias is unambiguous when the impact is a cost but ambiguous when it is a benefit. Each of the diagrams in Figures 1 and 2 shows two localities, each with its own relationship between $x$ and $y$. These relationships, and the average of the two, are shown as rays through the origin. Localities 1 and 2 have slopes $\beta_1$ and $\beta_2$, and the average has slope $\overline{\beta} = (\beta_1+\beta_2)/2$. Policymakers 1 and 2 choose points on their respective rays. If they choose $x$ ignoring local conditions, $x_1$ and $x_2$ have the same expected value, and the expected average of the two observations is on the middle ray. This corresponds to OLS being unbiased. In Figure 1, $y$ is a benefit of $x$. In Figure 1a, the more effective a policy is in a locality, the {\it more} intensely it is used. $\gamma_2$ is positive, and a steeper slope makes a policymaker choose a higher level of $x$. Indiana, with a greater marginal benefit, chooses a higher policy level than Michigan, and $x_1> x_2$. If the econometrician draws a line through the origin to lie between the two observations and minimize the squared deviations, that line will have a slope {\it greater} than $\overline{\beta}$. OLS overestimates the marginal benefit. In Figure 1b, the more effective a policy is in a locality, the {\it less} intensely it is used. $\gamma_2$ is negative, and a steeper slope makes a policymaker choose a lower level of $x$. Ohio, with a greater marginal benefit, chooses a lower policy level than Nevada, and $x_1> x_2$. (Note, however, that $y_1 > y_2$; Ohio still ends up with a greater benefit than Nevada.) If the econometrician draws a line through the origin to lie between the two observations and minimize the squared deviations, that line will have a {\it negative} slope, contradicting theory. OLS underestimates the marginal benefit, and in fact gives an impossible result. \epsfysize=5in \epsffile{/Users/erasmuse/@Papers/Choice/Choice1.eps} In Figure 2, $y$ is a cost of $x$, and a steeper slope makes a policymaker choose a {\it lower} level of $x$: $\gamma_2$ is negative. Iowa, with a greater marginal cost, chooses a lower level than Wisconsin: $x_1< x_2$. If the econometrician draws a line through the origin to lie between the two observations and minimize the squared deviations, that line will have a slope {\it less} than $\overline{\beta}$. OLS underestimates the marginal cost. \epsfysize=3in \epsffile{/Users/erasmuse/@Papers/Choice/Choice2.eps} \noindent {\bf 3.2 Other Problems, to be Distinguished from the Observed-Choice Problem} The observed-choice problem is easily confused with other problems in estimation such as the mutual-cause problem, simultaneity, and the Lucas critique. It may be useful to distinguish it from them at this point. The {\it mutual cause problem} is present when variables $x$ and $y$ do not really have a causal relationship but are both caused by a third variable $w$ such that $x=x(w)$ and $y=y(w)$. If richer cities have better roads and fewer high-school dropouts, the correlation between good roads ($x$) and fewer dropouts ($y$) of is positive because of income ($w$). The quality of roads may be a good predictor of the dropout rate in equilibrium, but if the quality were changed arbitrarily the relationship would disappear. The result is an overestimate of the impact, whether it be a benefit or a cost. {\it Simultaneity} is present when not only does $y$ depend on $x$, but $x$ depends on $y$: $ y=y(x)$ and $x=x(y)$. Adding hospitals to a city reduces mortality, but a city with less mortality needs fewer hospitals. Simultaneity is not special to policy, and the bias can be either overestimation or underestimation, depending on the relationships between $x$ and $y$. The {\it Lucas critique} applies when the relation between $x$ and $y$ only lasts until the government tries to take advantage of it, because if $x$ changes, so does $\beta$: $\beta = \beta(x)$. Aggregate output only rises with the money supply if money supply growth is low, so any attempt to increase output by increasing the money supply fails. This problem, which is equivalent to nonlinearity in the relationship between $x$ and $y$, is special to policy, and it can cause either overestimation or underestimation, depending on how $\beta$ changes in response to $x$. The observed-choice problem is not the mutual cause problem, because $y$ does indeed depend directly on $x$. It is not simultaneity, because $x$ does not depend on $y$. And it is not the Lucas critique, because $\beta$ does not depend on $x$. It is most closely related to the ``selection bias'' or ``self-selection'' problems found in binary-choice models. The most obvious form of self-selection occurs when some individuals take actions that prevent them from appearing in the observed sample, but even if the individuals in the data set are chosen randomly and selection {\it per se} is not a problem, the values of independent variables that result from individual decisions might depend on unobserved heterogeneity (see Mundlak [1961], Heckman [1976, 1979], and Lee [1978]). The name ``selection bias'' is, in fact, misleading, since the problem exists even if the sample is the entire population. The observed-choice problem can be considered a form of the selection bias problem, because in both problems the level of the policy depends on other variables or disturbances in the model and OLS is biased. The standard selection bias model, however, does not involve varying coefficients or policy choices based on the effectiveness of the policy being analyzed. Example 4 in the next subsection may help to show the similarities and differences between the two problems.\footnote{ Garen (1984) has extended the standard selection-bias techniques to a context in which observations are sampled randomly from the population but the values of the regressors depend on the heterogeneity. He specifies the value of the regressors as being chosen from a large set of discrete choices, and the bias arises from an error term with a nonzero expectation that interacts with individual characteristics and the particular choice made by the individual. The solution he proposes is to estimate the nonzero expectation consistently and include it in the regression. The present paper approaches the problem more simply, using a regression model in which coefficients vary across observations and the policies depend on the coefficients, but a version of the Garen technique was discussed in Section 2.4.} \bigskip \noindent {\bf 3.3 Examples with Discrete Choice, Nonlinearities, and Selection Bias.} A selection of verbal examples may help to further reveal the intuition behind the observed-choice problem, to extend the implications to nonlinear estimation, and to distinguish it from the standard selection bias problem. In the following four examples, the policy takes just two levels, adoption or rejection. {\it Example 1: Hotel tax revenue, a desirable impact.} A state either has a low or a high hotel tax, trading off the increase in revenue against the harm to the hotel industry. In 25 states, the high hotel tax would raise \$100 in revenue per capita more than the low tax, and those states adopt the tax. In the other 25 states, the higher tax would so discourage business that the change in tax revenue per capita would be \$0. The analyst notices that the 25 states with the high tax have \$100 higher revenue per capita, a difference that is statistically significant. He therefore advises all states to impose high taxes, even though, in truth, the added benefit is zero. He has overestimated the benefit of increasing a policy's intensity. %\footnote{xxx With just two choices, I think I can predict that benefits will %be overestimated.} {\it Example 2: Welfare mothers, an undesirable impact.}\footnote{Section 4 contains an empirical version of Example 2.} Transfer payments to unwed mothers can be set at amount 2 or amount 3. In 25 states, the illegitimacy rate will be 200 or 300, depending on the transfer level, and those states set transfers equal to 2 (see Table 1). In 25 other states, the illegitimacy rate will be 200 regardless of the transfer level, and those states set transfers equal to 3. The analyst sees 25 states with transfers of 2 and illegitimacy of 200 and 25 with transfers of 3 and illegitimacy of 200. He concludes that transfers have no effect on illegitimacy, and he suggests that the low-transfer states can increase their transfers to 3 without any adverse effects. But doing so would in fact increase illegitimacy considerably, and the true average effect is an increase of 50 (= [25(100) + 25(0)]/50) in illegitimacy going from transfers of 2 to 3. He has underestimated the cost of increasing the intensity of a policy. \begin{center} \begin{tabular}{c|ccc|c} \multicolumn{5}{c}{ TABLE 1}\\ \multicolumn{5}{c}{ }\\ \multicolumn{5}{c}{ EXAMPLES 2 and 3}\\ \multicolumn{5}{c}{ }\\ \hline \hline \multicolumn{5}{c}{ } \\ \multicolumn{2}{c}{\underline{HIGH RESPONSE STATE}} & & \multicolumn{2}{c}{\underline{LOW RESPONSE STATE}}\\ Transfer & Illegitimacy & & Transfer & Illegitimacy\\ \hline & & & \\ {\bf 2} & {\bf 200} & &2 & 200\\ 3 & 300 & &{\bf 3} & {\bf 200}\\ 4 & 600 & &4 & 600\\ & & & \\ \hline \end{tabular}\\ \end{center} \bigskip {\it Example 3: The potential for bias is especially strong for policy intensities outside the sample range.} Add another transfer level to Example 2: amount 4, which would result in illegitimacy of 600. The low-transfer states keep their transfers at 2, and the high-transfer states stay at 3. The naive analyst advises that transfer levels can be increased to 4 in every state without any effect on illegitimacy. He is wrong; illegitimacy will rise everywhere. The value of policy is especially overestimated for intensities greater than exist in the sample. This last effect is not just the usual hazard of forecasting out of the observed sample range. The naive analyst may well admit that his predictions for transfers of 4 are outside of his sample range and less trustworthy because of possible nonlinearity in the effect of transfers on illegitimacy. But he will add that although possible nonlinearity reduces the reliability of the prediction, it could result with equal likelihood in either an overestimate or an underestimate of the effect. That is wrong. The very reason why the transfer level of 4 is not in his sample is that the effect is nonlinear in the particular direction unfavorable to the active policy. Nonlinearities outside the observed sample range could lead to either overestimation or underestimation. It could be that the policy is much {\it more} effective than we estimate in the range {\it lower} than we observe. Table 1 and Figure 3 illustrates the problems with extrapolation in either direction. Although the data in Figure 3 may represent the entire population of policy choices, it is not random; it is purposively chosen to be on the middle part of the benefit curve. \vspace*{24pt} \epsfysize=3in \epsffile{/Users/erasmuse/@Papers/Choice/Choice3.eps} Example 3 has some similarity to the Lucas Critique problem, because the marginal effectiveness of the policy depends on the policy level chosen. This dependence, however, would exist even if the policy levels were chosen randomly. What the observed-choice problem adds is the idea that the policies will be chosen so as to make the Lucas critique especially applicable. The Lucas critique says that {\it if} the variation in the data is too small, nonlinearities in the function being estimated are a big problem, where ``too small'' depends on the context. The observed-choice problem explains {\it why} the variation will be too small. \bigskip \noindent {\it Example 4. Job Training and Selection Bias. } The effect of job training programs is the paradigmatic problem for which economists have worried about selection bias (see Heckman \& Robb [1985], Heckman, Hotz \& Dabos [1987], or Lalonde [1986]). Suppose half of a group of unemployed people had wages of 100 in their previous jobs and half had wages of 120. They are offered training, but only the people with past wages of 120 accept the training, for some exogenous reason. The training makes no difference in productivity for either group. Afterwards, however, the trained people get jobs with wages of 120, and the untrained get wages of 100. If the naive analyst ignores the previous wages, he concludes that the training raised wages by 20 percent. Just as easily, the problem could have been that only the 100-wage people accepted training, in which case the bias would have been pessimistic rather than optimistic. In either case, techniques are available for correcting the problem.\footnote{An early article on this problem is Mundlak (1961), which notes that if good farm management, which is unobserved, has a positive additive effect on output and is correlated with use of some input, then the analyst will overestimate the effect of the input on output. For a simple exposition of this story, see pp. 204-207 of Varian (1992). } The observed-choice problem is different, because it arises out of heterogeneous effects of the training rather than heterogeneous initial wages. Suppose that all the unemployed had previous wages of 100, but half of them would get a benefit of 0 from the training, and half would get a benefit of 20. Those that would benefit from the training accept it. Afterwards, the trained workers have wages of 120 and the untrained workers have wages of 100. The inference that the training raised wages by 20 is correct, but the inference that the average effect of training across the entire population is 20 is incorrect; it is 10. In the observed-choice problem, unlike in the problem of heterogeneous initial wages, economics provides prior information on the direction of the bias.\footnote{More generally, in a continuous-variable version of this story, it could be that the workers with {\it lower} marginal benefit decide to get more training, because they need more hours of training to get the same improvement. Then the estimation bias goes in the opposite direction. But the direction of bias is unambiguous in a 0/1 model of training/no training. } %\footnote{xxx Wildasin 9?) suggested another twist on the seletin model: thre %is queueing up for admission to the training program,a dn the adminsraors let %in only the people who would benefit the most--- or, those who would leave %with the highest salaries.} %--------------------------------------------------------------- \bigskip \noindent {\bf 3.4 Prediction without Policymaking} The most important implication of the observed-choice problem is that OLS or the equivalent informal reasoning will lead the analyst to be too optimistic in recommending changes in policy because he will overestimate benefits and underestimate costs. Making predictions for policy recommendations, however, is different from making predictions in general, as has long been known.\footnote{See Haavelmo (1943), p. 278 of Hurwicz (1950) and p. 56 of Mundlak (1961).} Policy recommendations implicitly contain a kind of prediction answering the question: ``What will happen to $y_i$ if $x_i$ is changed by forces outside the model?'' A purer form of prediction asks: ``What will happen to $y_i$ if $x_i$ changes?'' These are two different things: ``What will happen after I change the policy'' might be different from ``What will happen after the policy changes?'' Recall the mutual-cause example in section 3.2 in which high-school dropouts and road quality are inversely correlated across cities. An OLS regression would mislead in making the policy recommendation that the roads be improved to reduce the dropout rate. But the OLS regression would correctly predict that a city with good roads is likely to have a low dropout rate. Likewise, simultaneity is a less dangerous problem for prediction than for policymaking. If a city has a large police force, then using the correlation between police and crime to predict a large amount of crime may be correct even though the causal link is that more police reduces crime. If the analyst wants reliable policy implications, he needs a theory of causation; if he just wants to predict, he can use correlation. Prediction given the observed-choice problem is more tortuous. OLS will underestimate the average impact on $y_i$ of a recommended increase in $x_i$ if $y$ is an undesirable impact, and instrumental variables estimates that impact correctly. But what if $x_i$ takes a large value for reasons internal to the model? If the analyst is asked to predict $y_i$ for a new observation $i$ that has a policy level of $x_i$, his answer should not be $\widehat{y} = \widehat{\beta}_{IV}x_i$, even though $\widehat{\beta}_{IV}$ is a consistent estimator of $\overline{\beta}$ and the true specification is $ y_i = \overline{\beta} x_i + x_i v_i + \epsilon_i$. A large value of $x_i$ is produced by a small value of $\beta_i = \overline{\beta} + v_i$ and therefore by a negative value of $v_i$. The IV estimator will overpredict $y_i$, because $E(y|x) \neq \overline{\beta} x$; instead, $E(y|x)= \overline{\beta} x + E(xv|x)$. The bias in prediction is the {\it opposite} of the bias in policy recommendation. But whether the bias for observation $i$ is positive or negative depends on the value of $x_i$. Although the bias is downwards when $x$ is large, it is {\it upwards} when $x$ is small. When $x_i$ is small, the marginal effect of policy is great, and $y_i$ is greater than predicted by the IV estimate. %\footnote{xxx On average will the prediction be correct (not %conditioning on x?} One could use Bayes Rule to estimate $E (\beta_i|x_i)= \int \frac{ f(x|\beta) f(\beta)}{f(x)}d\beta$, but this requires knowledge of the functional form of the distribution of $v$, since $\beta_i =\overline{\beta} + v_i$. \begin{center} \begin{tabular}{l|cccc} \multicolumn{5}{c}{ TABLE 2}\\ \multicolumn{5}{c}{ }\\ \multicolumn{5}{c}{ PREDICTION: HOTEL TAX REDUCTION}\\ \multicolumn{5}{c}{ }\\ \hline \hline Tax of new & True effect of & True revenue& Naive & Sophisticated\\ state & a high tax & & Prediction & Prediction\\ \hline & & & & \\ High & 100 & 100 & 100 & 50\\ Low & 0 & 0 & 0 & 0\\ & & & & \\ \hline \end{tabular}\\ \bigskip \end{center} Return to Example 1, the hotel tax. The naive analyst predicts that a state with a high hotel tax will have \$100 more in revenue, whereas the analyst who corrects for the observed-choice problem predicts \$50. The sophisticated analyst will do better in predicting the effect of increasing the tax in a state that currently has a low tax; he will predict \$50, the naive analyst will predict \$100, and the true increase will be \$0. For high-tax states, the sophisticated analyst predicts a \$50 from lowering the tax, the naive analyst \$100, and the truth is \$100, but over both kinds of states the sophisticated analyst will have lower mean squared error, as well as an unbiased estimate. In pure prediction, however, the naive analyst does better. Suppose that the problem is to predict the hotel tax revenue in a state outside the original sample, knowing only that the state has a high hotel tax. The naive prediction is that the new state's revenue will be \$100 higher than in low-tax states, and the ``sophisticated'' prediction is \$50. Since the reason the new state imposed a high tax was because it would raise revenue there, the true value is \$100, and the naive analysis yields the correct answer. The same would be true of a new state with a low hotel tax; the naive prediction that its revenue is \$100 below that of states with high taxes is correct, and the sophisticated prediction of \$50 is incorrect. The analyst must decide which kind of question he is answering. Instrumental variables is appropriate for answering questions about exogenous changes in policies, but not for answering questions about endogenous changes or for out-of-sample predictions. %--------------------------------------------------------------- \begin{center} {\bf 4. An Empirical Example: Illegitimacy and Aid to Families with Dependent Children } \end{center} As an empirical example, let us consider the problem of estimating the effect of welfare on illegitimacy. Simple economic rationality suggests that if transfer payments are made to individuals contingent on their being single mothers, the number of single mothers will increase. The only question is how much. A survey by Elwood \& Crane (1990) on the state of the black family suggests that the answer is very little. As Table 3 shows, the levels of transfer payments do not show any clear relation to the percentage of black children living with only a single parent, and we have no reason to believe that black women are less sensitive to monetary incentives than white women.\footnote{In fact, some evidence exists that black women are more sensitive to monetary incentives, not less. Kneisner, McElroy \& Wilcox (1988a) find that a greater correlation between poverty and illegitimacy for blacks than for whites, suggesting that a given monetary incentive might be more powerful for blacks simply because it is a larger proportion of total income. See also Kneisner, McElroy \& Wilcox (1988b), discussed below. } Since Aid For Dependent Children (AFDC) levels vary across states, cross-section estimates have also been made, both reduced-form and structural, but Elwood \& Bane tell us that ``In general, both methods reveal only weak to moderate effects of welfare'' (Elwood \& Bane, 1990, p. 74). A 1990 study by Darity \& Myers, for example, finds, using CPS data on individuals in different states, that the elasticity of female headship of black families with respect to welfare levels was just .075. This is a general finding from time-series and cross-sectional studies; in his {\it Journal of Economic Literature } survey, Moffit (1992, p. 31) says, ``The failure to find strong benefit effects is the most notable characteristic of this literature.'' At the same time, one longitudinal study, that of Kneisner, McElroy \& Wilcox (1988b), does find a significant effect of monetary incentives on illegitimacy: greater AFDC payments increases the number of women who become single mothers, especially for black women, although the size of the payment does not seem to affect how long they stay on AFDC. Thus, the general conclusion seems to be that the AFDC level in a state does not much affect the number of illegitimate births in that state, but apparently at the level of the individual, the AFDC level does affect the decision to become a welfare mother. \begin{center} \begin{tabular}{l|cccc} \multicolumn{5}{c}{TABLE 3}\\ \multicolumn{5}{c}{ }\\ \multicolumn{5}{c}{ TRANSFER PAYMENTS OVER TIME}\footnote{Table 3 is somewhat misleading. Housing and medical benefits are not included, and they increased substantially during the 1980's.}\\ \multicolumn{5}{c}{ }\\ \hline \hline & 1960 & 1970 & 1980 & 1988 \\ \hline AFDC and food stamp payment level & \$7,324 & \$9,900 & \$8,325 & \$7,741\\ \hspace*{6pt} (family of 4 with no income-- & & & &\\ \hspace*{6pt}1988 dollars CPI-U adjusted) & & & &\\ Percent of black children not & 33.0 & 41.5 & 57.8 & 61.4 \\ \hspace*{6pt}living with two parents& & & &\\ Estimated percent of black & 10.4 & 33.6 & 34.9 & 30.1 \\ \hspace*{6pt} children collecting AFDC& & & &\\ \hline \multicolumn{5}{l}{ Source: Table 3 of Elwood \& Bane (1990).}\\ \hline \end{tabular} \bigskip \end{center} The observed-choice problem applies to this situation and may help explain the discrepancy between the aggregate and the individual estimates. The observed-choice problem applies if the explanatory variable is a policy and the dependent variable is a cost. Although economists, with their occupational interests, tend to think of the disincentive AFDC provides to supplying labor, in the minds of the public, illegitimacy is viewed as one of the chief costs of AFDC, and it is reasonable to suppose that the marginal effect of AFDC differs across states for a variety of cultural and economic reasons that are difficult to pick up in aggregate regressions. One explanation for the time series evidence is that the social breakdown occurring in the 1960s and 1970s increased the marginal impact of AFDC on illegitimacy for any level of AFDC, shifting up the entire curve, so the government reduced the size of AFDC payments. Theory cannot predict whether the final effect of an increase in the marginal impact would be an increase or decrease in illegitimacy; here, it seems to have increased despite the cuts in AFDC. Similarly, the cross-sectional evidence might be the result of states in which AFDC would have a bigger effect on illegitimacy choosing lower levels of AFDC. It might be, for example, that the number of women of each age in a state is important to the effect of AFDC, and this is difficult to put into a state-by-state regression, with its limited degrees of freedom In longitudinal studies, on the other hand, more individual variables can be taken into account, and the observed-choice problem is diminished, which might explain the greater size and significance of the estimated coefficients.\footnote{Longitudinal studies are not immune from the observed-choice problem, but it is less likely to be severe. Suppose that individual Vermont women of given race, age, income, etc. respond more to AFDC than do Maine women. The Vermont legislature will choose a lower level of AFDC, other things equal, and the observed-choice problem is present. The advantage of individual data is that the analyst can at least adjust for race, age, and income, so if there exists a missing variable causing the problem, it must be something special to Vermonters {\it qua } Vermonters, not to Vermonters {\it qua} white, young, poor people. } In this section I will use state-level cross-sectional data to illustrate how instrumental variables with a heteroskedasticy correction might be used to improve our estimates of the effect of transfer payments on illegitimacy by researchers more familiar with welfare policy than myself.\footnote{A more thorough analysis would use data on counties or individuals, assemble price indices for each location, try nonlinear specifications, use more instruments, test overidentifying restrictions, test for whether the model should be fully simultaneous, etc. Most importantly, it would aggregate all the benefits of poverty, including medical benefits, housing benefits, and illegal income. In fact, Orr (1992) suggests an alternative explanation for the small effects of AFDC that have been discovered: that overall transfer payments show much less variance across states than do AFDC payments. Since the objective here is just to illustrate the observed-choice problem, such refinements are ignored. Certain of the model's simplifications would generally tend towards obtaining insignificant results and a small coefficient for AFDC. Adjusting for local prices, for example, would increase the real AFDC levels in the Southern states, which have high illegitimacy rates. Also, Nelson \& Startz (1990) find that when one variable is being instrumented using one instrument, the IV estimator has a central tendency in small samples that is biased in the direction of the OLS estimator---towards too small a coefficient, here. Nonetheless, given the small number of observations, the main substantive contribution of this analysis is simply to cast doubt on existing estimates that ignore the observed-choice problem.} Table 4 shows the complete dataset. The 1989 {\it Annual Statistical Supplement} of the {\it Social Security Bulletin} provides data on average monthly payments per recipient from the Aid to Families with Dependent Children (AFDC) program for each state plus the District of Columbia (a sample size of 51).\footnote{\label{f8}``AFDC'' is ``Aid to Families with Dependent Children, Amount of Payments, Monthly average per Recipient,'' for 1987, p. 342, p. xiii, 1990 {\it Statistical Abstract of the United States.}} This varies from state to state because the federal government does not pay for the entire amount, and gives states some flexibility in eligibility requirements, or even in whether they wish to participate at all.\footnote{ For details of the state and federal responsibilities in funding and eligibility criteria, see the {\it 1993 Green Book}, the annual report on entitlement programs of the Committee on Ways and Means, U.S. House of Representatives, which contains additional data on maximum possible benefits per family, state shares of the payments, payments over time, etc. } The 1990 {\it Statistical Abstract of the United States} provides data on the illegitimacy rate, as well as on the average disposable personal income per capita in the state, the percentage of urbanization, and the percentage of the population that is black. It will be assumed that these are the relevant exogenous variables.\footnote{\label{f9} ``Illegitimacy'' is ``1987 births to unmarried women, percent,'' p. xiii, 1990 {\it Statistical Abstract of the United States}. ``Income'' is ``Disposable personal income per capita, 1988,'' p. xviii. ``Urbanization'' is ``Resident population in metro areas, 1988, percent,'' p. xii. ``Dukakis vote'' is calculated from ``1988 percent for leading party,'' p. 246. ``South'' takes the value of 1 if the state is southern under the {\it Statistical Abstract's} definition, and 0 otherwise. ``Black'' is the 1990 percentage, p. 26. Estimation was done using the matrix operations in {\it Mathematica} (Champaign, Illinois: Wolfram Research).} A simple regression of illegitimacy on AFDC and a constant yields the following relationship: \begin{equation} \label{e100} \begin{array}{lll } Illegitimacy &= 26.91 &{\bf -0.034* AFDC}, \\ & (3.05) & {\bf (0.026) } \end{array} \end{equation} (standard errors in parentheses) with $R^2=.03$. Equation (\ref{e100}) implies that high AFDC payments reduce the illegitimacy rate, but this is, of course, misleading because the simple regression leaves out important variables. Regression (\ref{e101}) more appropriately controls for a variety of things which might affect the illegitimacy rate: \begin{equation} \label{e101} \begin{array}{lll ll} Illegitimacy &= 15.74 &+ {\bf 0.016* AFDC} & -0.00011* Income &+0.024* Urbanization \\ & (3.65) & {\bf (0.021)} & (0.00042) & (0.033)\\ & &&&\\ & - 1.60* South & + 0.56*Black, & &\\ & (1.71) & (0.06) & &\\ \end{array} \end{equation} with $R^2=0.79$. Equation (\ref{e101}) would leave us with the conclusion that AFDC payments have almost no effect on the illegitimacy rate. Nor, surprisingly, do any of the other variables except race have large or significant coefficients. The coefficients are small enough, in fact, that one might doubt whether increasing the size of the dataset would change the conclusions: the variables are insignificant not because of large standard errors, but because of small coefficients. If the theory of this paper is correct, the problem with equation (\ref{e101}) is not lack of data, but that the coefficient on AFDC, $\beta_{AFDC}$, is properly a cause of the level of AFDC. For purposes of estimation, some identifying instrument is needed to replace AFDC, although fortunately a complete model of political decisionmaking is not required. The instrument used here is the percentage of the state's vote in the 1988 presidential election that went to Michael Dukakis, which is correlated with a state's liberalism and hence with its tendency to prefer higher levels of AFDC. This is a suitable instrument if (i) liberals tend to value the net benefits of AFDC more highly than do conservatives, (ii) the presence of Dukakis voters, conditioning on the other variables in the model, is not a direct cause of illegitimacy, and (iii) the presence of Dukakis voters is not a direct result of the current rate of illegitimacy. The decisionmaking model does need to be separable in $\beta_{AFDC}$ and the instrument: \begin{equation} \label{e103} AFDC = \gamma_1 f(\beta_{AFDC}) + \gamma_2 g(Dukakis \; vote) + u. \end{equation} Equation (\ref{e103}) is the equivalent of equation (\ref{e21}) in the theoretical part of the article. Even if the functions $f$ and $g$ were known, equation (\ref{e103}) could not be estimated, since $\beta_{AFDC}$ is unknown. But equation (\ref{e103}) does not have to be estimated to use instrumental variables. Instead, the other exogenous variables in (\ref{e102}) plus the vote for Dukakis can be used as instruments for $AFDC$. If $Z$ is the 51-by-6 matrix $$ Z= (Constant, Dukakis \;Vote, Income, Urbanization, South, Black), $$ and $$ X= (Constant, AFDC, Income, Urbanization, South, Black), $$ then using the instrumental variables estimator $(Z'X)^{-1}Z'y$, the estimates become \begin{equation} \label{e102} \begin{array}{lll ll} Illegitimacy &= 18.43& + {\bf 0.19 * AFDC} & - 0.0023* Income & + 0.091*Urbanization \\ & (6.03)& {\bf (0.096) } & (0.0013) &(0.064)\\ & & & & \\ & + 3.32* South & + 0.65*Black.& &\\ & (3.72) & (.11) & &\\ \end{array} \end{equation} In regression (\ref{e102}), the signs on the variables match intuition and theory. AFDC causes more illegitimacy, and higher incomes reduce it. Not all variables are statistically significant, but the standard errors are at least smaller than the coefficients. {}From this regression, one might hope that a larger sample size would bring all the variables into significance.\footnote{The biggest outlier for three variables--- the illegitimacy rate, percentage of blacks, and vote for Dukakis--- is the District of Columbia. When D.C. is excluded, the coefficient on AFDC in equation (\ref{e102}) is .18 (with standard error .11) instead of .096.} The theory of this paper instructs us to take an additional step: heteroskedasticity is still present, so weighted least squares should be used. Following the procedure suggested in Section 2.3 generates the following equation, where $\widehat{s^2}$ is the variance of the residuals from regression (\ref{e104}):\footnote{The standard errors are not presented here to test whether the regression coefficients are different from zero. The theory says that $\sigma_\epsilon^2$ and $\sigma^2_v$ are positive; the only question is how to best estimate the magnitudes.} \begin{equation} \label{e104a} \begin{array}{lll } \widehat{s^2} = &20.43& + .00065*AFDC^2 \\ & (11.27) &(.00065)\\ \end{array} \end{equation} From equation (\ref{e104a}), $\widehat{\sigma^2_\epsilon} = 20.43$ and $\widehat{\sigma^2_v} =.00065$. This suggests that if the $\beta_{AFDC}$ coefficients are normally distributed, about two-thirds of the states' coefficients lie within an interval of length $.51 (=2\cdot \sqrt{.00065}$).\footnote{The fact that the rounded standard error equals the rounded coefficient is coincidence. Since the estimated average coefficient is on the order of .21, the size of the interval implies that assuming normality and ignoring our prior beliefs that the coefficient on $AFDC^2$ is positive is probably a mistake. The need to incorporate prior information is even clearer if the regression is run without the outliers of DC and Utah, in which case the estimated coefficient on $AFDC^2$ is negative: $-.00011$, with a standard error of .00030.} Having estimated an equation determining the size of the error for an individual observation, it is now possible to use weighted least squares or GLS to re-estimate the main equation. If we define $\Omega$ to be a diagonal matrix with $20.43 + .00065*AFDC_i^2$ on diagonal $i$, then the GLS-IV estimator is $\widehat{\beta} = (X'Z(Z'\Omega Z)^{-1} Z'X)^{-1}X'Z(Z'\Omega Z)^{-1}y$, with standard errors being the square roots of the diagonals of $[(y-X \widehat{\beta})'\Omega^{-1}(y-X \widehat{\beta})/(51-6)][X'Z(Z'\Omega Z)^{-1} Z'X)^{-1}$, which yields \begin{equation} \label{e105} \begin{array}{lll ll} Illegitimacy = &18.62&{\bf + 0.21* AFDC}& -0.0024* Income & +0.094*Urbanization\\ & (6.44) &{\bf (0.10)} & (0.0014) & (0.070)\\ & & & & \\ & + 3.19* South & + 0.64*Black,& &\\ & (3.68) & (0.11)& &\\ \end{array} \end{equation} The estimates stay roughly the same as with unweighted instrumental variables (though note that AFDC does pass the boundary of significance at the 5 percent level, since $t(45)=2.014$).\footnote{The small size of the heteroskedasticity may make one wonder whether the observed choice problem is the true problem in this example. The observed choice problem grows worse with increasing heterogeneity in the $\beta_i$, and so does the amount of heteroskedasticity. It is not follow, however, that the observed-choice problem is trivial if heteroskedasticity is small, because the observed-choice problem also depends on how the states react to the differences in the $\beta_i$. If $\beta_i$ is almost the same in every state, but states react very strongly to $\beta_i$ in choosing their levels of $x_i$, then the observed-choice problem can still be severe.} Equation (\ref{e105}) says that if the average monthly AFDC payment in a randomly chosen state rises by 10 dollars, our best estimate of the increase in the illegitimacy rate is 2.1 percent. For the average state, this would be an increase in the AFDC payment of 8.1 percent producing an 8.6 percent increase in the illegitimacy percentage, an elasticity of 1.06. The coefficient on AFDC is both economically and statistically significant.\footnote{Do recall the caveat earlier: this analysis ignores other welfare benefits such as food stamps, medicaid, and housing subsidies. If they are correlated state by state with AFDC, then what looks like the impact of a 10-dollar, 8.6 percent increase in AFDC is actually the effect of a more-than-ten-dollars, 8.6 percent increase in total welfare benefits. Thus, increasing AFDC by itself might not have such a large impact, though welfare policy as a whole would. If, on the other hand, AFDC and other benefits are negatively correlated, the method here underestimates the effect of additional welfare income. } Regarding the other variables: if the state's per-capita income rises 1000 dollars, illegitimacy falls 2.4 percent; if urbanization increases 10 percent, illegitimacy rises .94 percent; if the state is in the South, the illegitimacy rate is 3.19 percent higher, and if the black percentage rises 10 percent, the illegitimacy rate rises 6.4 percent. Notice the contrast with the initial multiple regression using OLS, equation (\ref{e101}). The sign has changed on $South$, all coefficients except the constant and $Black$ have at least doubled, those on $Urbanization$ and $South$ have more than doubled, and those on $AFDC$ and $Income$ are more than ten times their initial size. The estimated elasticity of illegitimacy with respect to AFDC for the average state rises from .08 to 1.06. Adjusting for the observed-choice problem clearly has a large effect. As a final, perhaps tangential, point, the regression of $AFDC$ on the other variables may be of interest. Since the theory assumes that $\beta_{AFDC}$ is one of the variables that explains $AFDC$, and since illegitimacy is endogenous, this regression is misspecified and biased, but it gives some idea of the important correlations: \begin{equation} \label{e104} \begin{array}{lll ll} AFDC &= -63.08& -0.50*Illegitimacy & +0.012* Income & -0.37*Urbanization \\ & (32.40)& (1.13) & (0.002) &(0.22)\\ & & & & \\ & -19.21 * South & -0.71*Black,& + {\bf 1.51(Dukakis\;Vote)}&\\ & (11.37) & (0.69) & {\bf (0.63)} &\\ \end{array} \end{equation} with $R^2=0.69$. The high $R^2$ indicates that the fit of the instrumental variables is quite good (and it only falls to .68 if $Illegitimacy$ is dropped). The coefficient on $Dukakis\;Vote$ is large and positive with a small standard error, indicating that there is a strong correlation between AFDC and the vote for Dukakis even after conditioning on the other variables. The negative sign and high standard error on $Illegitimacy$ gives some comfort that the instrumental variables regression results are not due to instrumenting for an endogenous AFDC, instead of through a negative correlation with $\beta_{AFDC}$. One might wonder whether the political strength of parents would cause a positive link between illegitimacy and AFDC, but, in fact, the conditional correlation is negative. If it had been positive, and the most important effect is that illegitimacy causes AFDC, then the instrumental variables estimator here would still be consistent, but one would expect that instrumental variables would produce {\it smaller} coefficients than OLS, not larger, because under OLS some of the apparent effect of AFDC on illegitimacy would really be due to the positive correlation between the political power of the parents and AFDC. \pagebreak \vspace*{-1in} \thispagestyle{empty} \begin{footnotesize} \begin{tabular}{|l|lll lll r|} \hline \hline State & Illegitimacy & AFDC & Income & Urban- & Black & Dukakis &Unexplained Illeg.\\ & & & & ization & & vote & (from (\ref{e105})) \\ & (\%) & (\$/month) & (\$/year) & (\%) & (\%) & (\%) & (\%)\\ \hline Maine & 19.8 & 125 & 12,955 & 36.1 & 0.3 & 44.7 & 2.8 \\ New Hampshire & 14.7 & 140 & 17,049 & 56.3 & 0.6 & 37.6 & 2.3 \\ Vermont & 18.0 & 159 & 12,941 & 23.2 & 0.4 & 48.9 & -4.9 \\ Massachusetts & 20.9 & 187 & 17,456 & 90.6 & 4.8 & 53.2 & -6.2 \\ Rhode Island & 21.8 & 156 & 14,636 & 92.6 & 3.8 & 55.6 & -5.2 \\ Connecticut & 23.5 & 166 & \framebox{19,096 } & 92.6 & 8.2 & 48.0 & 2.3 \\ \hline New York & 29.7 & 166 & 16,036 & 91.2 & 16.1 & 51.6 & -3.8 \\ New Jersey & 23.5 & 119 & 18,615 & \framebox{100} & 14.4 & 43.8 & 6.2 \\ Pennsylvania & 25.3 & 111 & 14,072 & 84.8 & 9.4 & 50.7 & 3.4 \\ \hline Ohio & 24.9 & 102 & 13,326 & 78.9 & 11.0 & 45.0 & 2.6 \\ Indiana & 22.0 & 84 & 12,834 & 68.1 & 8.4 & 40.2 & 4.9 \\ Illinois & 28.1 & 101 & 15,150 & 82.5 & 16.1 & 49.3 & 6.7 \\ Michigan & 20.4 & 156 & 14,094 & 79.9 & 14.6 & 46.4 & \framebox{-14.0 } \\ Wisconsin & 20.7 & 160 & 13,296 & 66.5 & 4.8 & 51.4 & -8.5 \\ \hline Minnesota & 17.1 & 171 & 14,037 & 66.6 & 1.6 & 52.9 & \framebox{-11.0 } \\ Iowa & 16.2 & 124 & 12,475 & 43.4 & 1.9 & 54.7 & -3.5 \\ Missouri & 23.7 & 87 & 13,340 & 66.0 & 10.8 & 48.2 & 5.9 \\ North Dakota & 13.9 & 125 & 11,388 & 38.4 & 0.5 & 44.0 & -7.2 \\ South Dakota & 19.4 & 94 & 11,611 & 29.1 & 0.3 & 47.2 & 6.2 \\ Nebraska & 16.8 & 108 & 12,773 & 47.6 & 3.4 & 39.8 & -0.2 \\ Kansas & 17.2 & 110 & 13,235 & 53.4 & 5.8 & 44.2 & -1.2 \\ \hline Delaware & 27.7 & 99 & 14,654 & 65.9 & 18.9 & 44.1 & 2.1 \\ Maryland & 31.5 & 115 & 16,397 & 92.9 & 26.1 & 48.9 & -0.4 \\ DC & \framebox{59.7 } & 124 & 17,464 & \framebox{100} & \framebox{68.6} & \framebox{82.6} & 0.5 \\ Virginia & 22.8 & 97 & 15,050 & 72.2 & 19.0 & 40.3 & -2.1 \\ West Virginia & 21.1 & 80 & 10,306 & 36.5 & 2.9 & 52.2 & 2.1 \\ North Carolina & 24.9 & 92 & 12,259 & 55.4 & 22.1 & 42.0 & -6.0 \\ South Carolina & 29.0 & 66 & 11,102 & 60.5 & 30.1 & 38.5 & -5.0 \\ Georgia & 28.0 & 83 & 12,886 & 64.8 & 26.9 & 40.2 & -3.5 \\ Florida & 27.5 & 84 & 14,338 & 90.8 & 14.2 & 39.1 & 5.0 \\ \hline Kentucky & 20.7 & 72 & 11,081 & 46.1 & 7.5 & 44.5 & 1.4 \\ Tennessee & 26.3 & 54 & 12,212 & 67.1 & 16.3 & 42.1 & 5.7 \\ Alabama & 26.8 & \framebox{39} & 11,040 & 67.5 & 25.6 & 40.8 & 0.5 \\ Mississippi & 35.1 & \framebox{39} & \framebox{9612} & 30.5 & 35.6 & 40.1 & 2.4 \\ \hline Arkansas & 24.6 & 63 & 10,670 & 39.7 & 15.9 & 43.6 & 1.3 \\ Louisiana & 31.9 & 55 & 10,890 & 69.2 & 30.6 & 45.7 & -1.4 \\ Oklahoma & 20.7 & 96 & 10,875 & 58.8 & 6.8 & 42.1 & -4.8 \\ Texas & 19.0 & 56 & 12,777 & 81.3 & 11.9 & 44.0 & 0.9 \\ \hline Montana & 19.4 & 120 & 11,264 & 24.2 & \framebox{0.2} & 47.9 & 0.5 \\ Idaho & 13.0 & 95 & 11,190 & \framebox{20.0} & 0.4 & 37.9 & -0.6 \\ Wyoming & 15.8 & 117 & 11,667 & 29.2 & 0.8 & 39.5 & -2.3 \\ Colorado & 18.9 & 109 & 14,110 & 81.7 & 3.9 & 46.9 & 1.3 \\ New Mexico & 29.6 & 82 & 10,752 & 48.9 & 1.7 & 48.1 & \framebox{14.0} \\ Arizona & 27.2 & 92 & 13,017 & 76.4 & 2.7 & 40.0 & \framebox{12.0} \\ Utah & \framebox{11.1} & 116 & 10,564 & 77.4 & 0.7 & \framebox{33.8} & \framebox{-14.0} \\ Nevada & 16.4 & 86 & 14,799 & 82.6 & 6.9 & 41.1 & 3.2 \\ \hline Washington & 20.8 & 157 & 14,508 & 81.6 & 2.4 & 50.0 & -4.8 \\ Oregon & 22.4 & 123 & 12,776 & 67.7 & 1.6 & 51.3 & 1.5 \\ California & 27.2 & 191 & 16,035 & 95.7 & 8.2 & 48.9 & -6.8 \\ Alaska & 22.0 & \framebox{226} & 16,357 & 41.7 & 3.4 & 40.4 & \framebox{-10.0} \\ Hawaii & 21.3 & 134 & 14,374 & 76.3 & 1.8 & 54.3 & 1.1 \\ \hline United States & 24.5 & 124 & 14107 & 77.1 & 12.4 & 46.6 & 0\\ \hline \hline \multicolumn{8}{c}{ }\\ \multicolumn{8}{c}{ }\\ \multicolumn{8}{c}{\bf Table 4: The Data and the Regression Residuals}\\ \multicolumn{8}{c}{(Extreme values are boxed. Sources and definitions are in footnotes \ref{f8} and \ref{f9}.)}\\ \end{tabular} \end{footnotesize} %--------------------------------------------------------------- \bigskip \begin{center} {\bf 5. Concluding Remarks} \end{center} When the independent variable in an econometric problem is the result of a policy decision and the dependent variable is a cost or benefit of that decision, the OLS estimate will have a tendency to overestimate the net benefit of the policy. This will happen if the decisionmakers are rational, even if the dependent variable is not their main concern, and the coefficients vary across observations, two conditions which are harmless separately but dangerous when present in combination. The observed-choice problem applies to a variety of policies. Whether the analyst wishes to estimate the effects of unemployment insurance, transfer payments, police protection, or speed limits, he should worry about the source of the variation in policies across space and time. If the variation arises from factors unrelated to the main effect being analyzed, OLS is unbiased, but if it arises from differences in the marginal cost or benefit of the policy, bias is introduced. If every decisionmaker is optimizing, then in equilibrium there is no net benefit from changing any policy, but an outside observer, seeing differences in policies correlated with differences in total benefits, may be fooled into thinking that change would help. Even if the variation in policies does not arise from differences in the coefficient, there may still be an observed-choice problem for any extrapolation beyond the observed data. If the coefficient changes with the level of policy---that is, if the policy has a nonlinear effect---then policymakers will avoid policy ranges for which the marginal costs are high or the marginal benefits low. The absence of a policy from the data provides information about its effect. The observed-choice problem provides a reason why social experiments are useful. In one experiment, described by Woodbury \& Spiegelman (1987), unemployed people in Illinois were selected randomly and offered a \$500 bonus if they accepted a job within 11 weeks and held it for at least 4 months. The most obvious reason for such an experiment is that existing variation in policies was insufficient: no state offered such a policy, so its effect could not be measured. A second reason is that the experiment controlled for state-specific effects. A third reason is the observed-choice problem: if Illinois adopted such bonuses as a general policy, instead of being chosen for an experiment, one might conclude that Illinois adopted the policy because it was especially effective there. Experiments that assign policies randomly eliminate this problem. They are, on the hand, costly and full of practical difficulties, as Heckman et al. (1987) point out, so the clever econometrician may, in the end, still be more cost-effective than the clever experimentalist. When policies differ, one should ask why. For the economist, as for the Freudian, nothing happens by accident. If policies depend on their potential impacts, then naive estimates of those impacts are biased. This will ordinarily be the case, since costs and benefits, not random whims, are the motivations behind policy. Therefore, not only must one construct a model of how $x$ determines $y$; one must think about whether $\beta_i$ determines $x_i$. If it does, then the uncorrected estimates should only be used as upper bounds on policy effectiveness, or instrumental variables should be used to correct the estimates. This can make an important difference in problems such as estimating the effect of AFDC on illegitimacy. %--------------------------------------------------------------- \newpage \bigskip \begin{center} {\bf References} \end{center} Committee on Ways and Means, U.S. House of Representatives (1993) {\it Overview of Entitlement Programs: 1993 Green Book}. Washington: U.S. Government Printing Office, 1993. Darity, William and Samuel Myers (1990) ``Impacts of Violent Crime on Black Family Structure,'' {\it Contemporary Policy Issues} (October 1990), 8: 15-29. Department of Commerce (1990) {\it Statistical Abstract of the United States}, Washington: Superintendent of Documents, U.S. Government Printing Office. Ellwood, David \& Jonathan Crane (1990) ``Family Change Among Black Americans: What Do We Know?'' {\it Journal of Economic Perspectives}, 4: 65-84 (Fall 1990). Garen, John (1984) ``The Returns to Schooling: A Selectivity Bias Approach with a Continuous Choice Variable,'' {\it Econometrica}, 52 (September 1984) pp. 1199-1218. Garen, John (1987) ``Relationships among Estimators of Triangular Econometric Models,'' {\it Economics Letters}, 25, 39-41. Haavelmo, Trygve (1943) ``The Statistical Implications of a System of Simultaneous Equations,'' {\it Econometrica}, 2:1-12. January 1943. Heckman, James (1976) ``The Common Structure of Statistical Models of Truncation, Sample Selection and Limited Dependent Variables and a Simple Estimator for Such Models,'' {\it Annals of Economic and Social Measurement}, 5: 475-492. Heckman, James (1979) ``Sample Selection Bias as a Specification Error,'' {\it Econometrica}, 47: 153-161, January 1979. Heckman, James, V. Joseph Hotz \& Marcelo Dabos (1987), ``Do We Need Experimental Data to Evaluate the Impact of Manpower Training on Earnings,'' {\it Evaluation Review}, 11: 395-427. Heckman, James, \& Richard Robb (1985), ``Alternative Methods for Evaluating the Impact of Interventions: An Overview,'' {\it Journal of Econometrics}, 30: 239-267. Hurwicz, Leonid (1950) ``Prediction and Least Squares,'' in Tjalling Koopmans, ed., {\it Statistical Inference in Dynamic Economic Models}, New York: John Wiley and Sons, 1950. Kennedy, Peter (1985) {\it A Guide to Econometrics}, Second Edition, Oxford: Basil Blackwell Ltd, 1985. Kneisner, Thomas, Marjorie McElroy, and Steven Wilcox (1988a) ``Getting into Poverty Without a Husband, and Getting Out, With or Without,'' {\it AEA Papers and Proceedings}, May 1988, 78: 86-90. Kneisner, Thomas, Marjorie McElroy, and Steven Wilcox (1988b) ``Individuals and Families in Transition: Understanding Change Through Longitudinal Data,'' Papers Presented at the Social Science Research Council in Annapolis, Maryland, March 16-18, 1988, U.S. Dept. of Commerce, Bureau of the Census. Lalonde, Robert (1986) ``Evaluating the Econometric Evaluations of Training Programs with Experimental Data,'' {\it American Economic Review}, 76: 604-620. Lee, Lung-Fei (1978) ``Unionism and Wage Rates: A Simultaneous Equation Model with Qualitative and Limited Dependent Variables,'' {\it International Economic Review}, 19: 415-433. Lucas, Robert E. (1976) ``Econometric Policy Evaluation: A Critique,'' {\it Journal of Monetary Economics}, 1976 Special Supplement on the Phillips Curve, pp. 19-46. Maddala, G. (1977) {\it Econometrics}, New York: McGraw-Hill, Inc., 1977. Moffit, Robert, (1992) ``Incentive Effects of the U.S. Welfare System: a Review,'' {\it Journal of Economic Literature}, March 1992, 30: 1-61. Mundlak, Y. (1961) ``Empirical Production Functions Free of Management Bias,'' {\it Journal of Farm Economics} 443 (February 1961), pp. 44-56. Nelson, Charles and Richard Startz (1990), ``Some Further Results on the Exact Small Sample Properties of the Instrumental Variables Estimator,'' {\it Econometrica}, 58: 967-876, July 1990. Orr, Lloyd (1992) ``Cross-Section Multiple Program Variance in Welfare Benefits,'' working paper, Indiana University Department of Economics, June 1992. Peltzman, Sam (1976) ``Toward a More General Theory of Regulation,'' {\it Journal of Law and Economics} 19 (August 1976), pp. 211-40. Social Security Administration (1989) {\it Annual Statistical Supplement} to the {\it Social Security Bulletin}, Washington: Superintendent of Documents, U.S. Government Printing Office. Varian, Hal (1992) {\it Microeconomic Analysis, Third Edition}. New York: W.W. Norton \& Company. Woodbury, Stephen \& Robert Spiegelman (1987) ``Bonuses to Workers and Employers to Reduce Unemployment: Randomized Trials in Illinois,'' {\it American Economic Review}, September 1987, pp. 513-530. %--------------------------------------------------------------- \end{document}