The TRAPPIST-1 system contains seven roughly Earth-sized planets locked in a multiresonant orbital configuration1,2, which has enabled precise measurements of the planets’ masses and constrained their compositions3. Here we use the system’s fragile orbital structure to place robust upper limits on the planets’ bombardment histories. We use N-body simulations to show how perturbations from additional objects can break the multiresonant configuration by either triggering dynamical instability or simply removing the planets from resonance. The planets cannot have interacted with more than ~5% of one Earth mass (M⊕) in planetesimals—or a single rogue planet more massive than Earth’s Moon—without disrupting their resonant orbital structure. This implies an upper limit of 10−4 M⊕ to 10−2 M⊕ of late accretion on each planet since the dispersal of the system’s gaseous disk. This is comparable to (or less than) the late accretion on Earth after the Moon-forming impact4,5, and demonstrates that the growth of the TRAPPIST-1 planets was complete in just a few million years, roughly an order of magnitude faster than that of the Earth6,7. Our results imply that any large water reservoirs on the TRAPPIST-1 planets must have been incorporated during their formation in the gaseous disk.