On-chip polarization-sensitive photodetectors offer unique opportunities for next-generation ultra-compact polarimeters. So far, mainstream approaches have relied on the anisotropic absorption of natural materials or artificial structures. However, such a model is inherently restricted by correlation between the polarization ratio (PR) and diattenuation, leading to small PR values (1 < PR < 10). Here, we report nanoantenna-mediated semimetal photodetectors, which enable configurable polarity transition by exploiting the vectorial and non-local photoresponse in semimetals. By tuning the orientation of nanoantennas, PR values vary from positive (unipolar regime) to negative (bipolar regime), covering all possible numbers (1 → ∞/−∞ → −1). In particular, the PR values at the polarity-transition point could approach infinity. Such a polarity transition hereby transcends the conventional PR–diattenuation relationship. Furthermore, our device allows the subtle measurement of polarization-angle perturbation down to 0.02° Hz−1/2 in the mid-infrared range. Our findings highlight the potential of semimetals as a promising material platform for miniaturized polarimetry.