Solid electrolyte interphase (SEI) is an electrically insulating and ionically conductive passivation layer which is formed on the electrode surface through electrolyte decomposition. SEI is crucial to battery performance because it plays a vital role to determine the Coulombic efficiency, cycle life, capacity, and safety. Given the intricated formation mechanisms and the complicated structures and compositions of SEI, the in-depth understanding of SEI is still challenging. This review is dedicated to critical discussion on recent advances in understanding the formation mechanisms of SEI. The important factors, including electrolyte components, temperature, areal current, and electrode materials, that affect the formation, morphology, structure, composition, and properties of SEI layers are discussed. In situ/operando characterization techniques used to look into the surface morphology, electrochemical performance, chemical composition, structure, and mechanical properties of SEI layers are emphasized. The recent progress of the state-of-the-art cryogenic electron microscopy aimed at atomistic visualization of SEI is highlighted. Multi-scale theoretical simulations employed to study the thermodynamic and kinetic properties of SEI are also discussed. In addition, the SEIs formed on various anodes using solid-state electrolytes are also presented. Finally, the outstanding challenges and future directions in understanding SEI are presented. This review is envisioned to offer new insights into rationally designing the SEI layers for the development of next-generation high-performance rechargeable batteries.