Southern Africa is likely to be heavily affected by a changing climate and the brunt will have to be shouldered by smallholder farmers in rural areas. Long-term experiments on climate-smart sustainable intensification practices offer the opportunity to evaluate and assess the potential impact of a more variable climate on crop productivity. Here, we used meta-analytic and meta-regression approaches to assess the response of different Conservation Agriculture (CA) systems across experiments as compared to conventional practices (CP) of varying experimental duration, established in trial locations of Malawi, Mozambique, Zambia, and Zimbabwe under an increasingly variable climate. We assessed how different agro-environmental yield response moderators such as type of crop diversification and amount of rainfall affect maize yield responses. Smallholder farmers, often living below the poverty line, are primarily concerned about short-term gains from agriculture systems accepting loss in longer-term sustainability. We therefore aim to identify cropping systems that may provide both short-term gains and longer-term sustainability. Results show that: a) long-term trends in yield performance are a result of many factors; b) the greatest yield gains between the best performing CA and least performing treatments at each location ranged between + 34% and + 117%; c) the greatest yields were found in direct seeded rotation systems; d) type of crop diversification and type of crop used in the diversification strategy affect yield response, with rotations involving legumes being more responsive than any practice without diversification; e) CA systems gains increase with time of practice as compared to CP and these responses are more pronounced under low to moderate rainfall, and in well drained soils. We therefore conclude that crop yield response under CA is determined by many yield defining agro-environmental factors and benefits of CA become more apparent with time.