Cement-based composites with thermoelectric effect have important application prospects in alleviating urban heat island effect, low-energy snow and ice melting on winter roads, and low-cost scale energy conversion and harvesting. Great progress has been made in the study of the thermoelectric properties of cement-based composites enhanced with carbon materials, and the addition of carbon materials increases the electrical conductivity of cement-based composites, but easily decreases the Seebeck coefficient. Reduced graphene oxide (RGO) by Cl2/HNO3 pretreatment increased the electrical conductivity of cement-based composites while also improving the Seebeck coefficient of cement-based composites, which was attributed to the increased content of RGO defects, which increased the effective mass of carriers, thus enhancing the Seebeck effect of cement-based composites and improving the thermoelectric properties of cement-based composites. When the Cl2 pretreatment RGO content was 5.0wt%, the electrical conductivity of the cement-based composites was 1.54 S/cm, the Seebeck coefficient was −56.61 μV/°C and the ZTmax value of 1.08 × 10−4 was achieved, with average output power and thermoelectric conversion efficiency of 2.91 × 10−2 mW/m2 and 3.79 × 10−6. The cement-based composites prepared by Cl2 pretreatment of RGO achieved the highest ZT at the same carbon material content.