Recent data suggest a suboptimal antibody response to COVID-19 vaccination in patients with hematological malignancies. Neutralizing antibodies (NAbs) against SARS-CoV-2 were evaluated in 276 patients with plasma cell neoplasms after vaccination with either the BNT162b2 or the AZD1222 vaccine, on days 1 (before the first vaccine shot), 22, and 50. Patients with MM (n = 213), SMM (n = 38), and MGUS (n = 25) and 226 healthy controls were enrolled in the study (NCT04743388). Vaccination with either two doses of the BNT162b2 or one dose of the AZD1222 vaccine leads to lower production of NAbs in patients with MM compared with controls both on day 22 and on day 50 (p < 0.001 for all comparisons). Furthermore, MM patients showed an inferior NAb response compared with MGUS on day 22 (p = 0.009) and on day 50 (p = 0.003). Importantly, active treatment with either anti-CD38 monoclonal antibodies (Mabs) or belantamab mafodotin and lymphopenia at the time of vaccination were independent prognostic factors for suboptimal antibody response following vaccination. In conclusion, MM patients have low humoral response following SARS-CoV-2 vaccination, especially under treatment with anti-CD38 or belamaf. This underlines the need for timely vaccination, possibly during a treatment-free period, and for continuous vigilance on infection control measures in non-responders.