The rice yield and real agricultural wage in Bangladesh increased by 3.8% and 2.1% per annum respectively from 2000 to 2010. Over the same period, the share of hired labor in agriculture decreased from 19.4% to 15.5%. A focus of this paper is to understand if the observed changes in wages and hired labor are in part due to agricultural productivity growth as reflected in increasing rice yield. To estimate the effects of agricultural productivity, we take advantage of an Upazila (subdistrict)-level panel data set from Bangladesh and exploit variations in rainfall across Upazilas and over time. We find that a positive rainfall shock has a significant positive effect on crop yields, wages, per capita household expenditure and labor supplied to market activities (including own farming). The effect on hired labor is, in contrast, negative and statistically significant. In a standard neoclassical model, higher agricultural productivity affects wages and hired labor through labor demand; a rightward (leftward) shift increases (reduces) both wages and the amount of hired labor. The finding of a negative hired labor response to agricultural productivity growth with a higher wage thus appears puzzling. We develop a model where heterogeneity in labor supply response due to differences in productivity in home goods production can lead to a decline in hired labor when agricultural productivity increases, even though the equilibrium wage increases. Since the poor in rural areas depend disproportionately on wage labor, a decline in hired labor may be interpreted by some as evidence of adverse effects on poverty and inequality. The theoretical analysis, however, shows that the poor benefit from agricultural productivity growth even when the labor supply responses result in a decline in hired labor.