A mathematical framework is developed for the analysis of causal fermion systems in the infinite-dimensional setting. It is shown that the regular spacetime point operators form a Banach manifold endowed with a canonical Fréchet-smooth Riemannian metric. The so-called expedient differential calculus is introduced with the purpose of treating derivatives of functions on Banach spaces which are differentiable only in certain directions. A chain rule is proven for Hölder continuous functions which are differentiable on expedient subspaces. These results are made applicable to causal fermion systems by proving that the causal Lagrangian is Hölder continuous. Moreover, Hölder continuity is analyzed for the integrated causal Lagrangian.