Soil fertility is key to sustainable intensification of agriculture and food security in sub-Saharan Africa. However, when soil nutrients are not adequately managed, smallholder farming practices slowly erode soils to almost inert systems. This case study contributes to the understanding of such failures in marginal areas. We integrate agronomic and social sciences approaches to explore links between smallholder households’ farming knowledge and soil fertility in an ethnopedological perspective. We interview 280 smallholder households in two areas of the Ethiopian highlands, while collecting measures of 11 soil parameters at their main field. By analyzing soil compositions at tested households, we identify a novel measure of soil management ability, which provides an effective empirical characterization of the soil managing capacity of a household. Regression analysis is used to evaluate the effects of household knowledge on the soil management ability derived from laboratory analysis. Results highlight the complexity of knowledge transmission in low-input remote areas. We are able to disentangle a home learning and a social learning dimension of the household knowledge and appraise how they can result in virtuous and vicious cycles of soil management ability. We show that higher soil management ability is associated with farmers relying to a great extent on farming knowledge acquired within the household, as a result of practices slowly elaborated over the years. Conversely, lower soil management ability is linked to households valuing substantially farming knowledge acquired through neighbors and social gatherings. The present study is the first to formulate the concept of soil management ability and to investigate the effects of the presence and the types of farming knowledge on the soil management ability of smallholder farmers in remote areas. We show that farming knowledge has a primary role on soil fertility and we advise its consideration in agricultural development policies.