Agricultural labor productivity (ALP) remains low in China. The unique land tenure system and fast-growing land rental market in China provides a new perspective in understanding the ALP problem. This study investigates the impact of participation in the land rental market on operator households’ agricultural labor productivity in rural China. Using national representative farm-level data, propensity matching score (PSM), and mediation analysis, we find that farm households’ ALP has improved about 43% after renting in farmland. The mediation analysis reveals that farm size in cultivation, family farm labor input, and capital services input are mediating variables through which land rental affects ALP. The indirect effect of land rental on labor productivity is mainly through the single-variable mediation path, namely farm size, and family farm labor input. Additionally, the two-variable mediation path is through farm size and capital services input, farm size, and family farm labor input.