Decent employment in agriculture is part of the general narrative about working conditions. It is an unquestionable objective but its position high in the international agenda contributes to sideline major structural issues faced by many agricultures around the world. This is particularly the case in sub-Saharan Africa and Asia, which represent about 90% of global agricultural workers, and where agriculture still plays a major role in employment and the economy. Different trajectories of structural transformation and rates of demographic growth result in different employment challenges which are central for possible improvement of work conditions in agriculture. However, the existing narrative about agricultural development remains shaped by the modernization paradigm based on technical progress and the centrality of the productive optimum. Its progressive adoption around the world has resulted in major productivity increases which deeply impacted agriculture through farm differentiation and concentration followed by a massive exit of farm workers. This process of change results today in major environmental and social sustainability challenges which prevent the replication of previous structural transformation pathways. Yet, the current policy architecture supporting agriculture promoted by international agencies and relayed in national policies continue to focus on the same modernization recipes. Framed by a market-led agenda, they are based on increased productivity and improved access to markets, and focus on a better provision of public goods. They target the “agripreneurs” who are able to adapt to the competitive economic environment. Attention is progressively paid to environmental challenges which results in a “smart-modernization” agenda based on sustainable intensification, while alternative approaches like agro-ecology remain limited to local experiences. Mainstream policies prevent addressing the continuing structural issues faced by many developing countries, as well as global sustainability issues. In that context, improving working conditions in agriculture is part of a necessary global approach about the development of the sector, where the multiple roles that agriculture plays beyond the production of food and feed must be acknowledged and discussed. It implies a reinvestment in strategic thinking which has to be supported by new research about the labor content in agriculture and the agri-food sector, and by a consideration of existing local dynamics in order to identify adequate development models for agriculture.