Na1/2Bi1/2TiO3-based materials exhibit potential for applications in high-power ultrasonics. The composites of Na1/2Bi1/2TiO3-yBaTiO3 (NBTyBT; y denotes mole %) with ZnO inclusions were demonstrated to stabilize a ferroelectric equilibrium that led to enhanced thermal depolarization temperature (Td) and increased mechanical quality factor (Qm). This work addresses the influence of the matrix phase by investigating two limiting choices based on symmetry (tetragonal/rhombohedral) and polar (relaxor/ferroelectric) nature. While the composites constituting the tetragonal NBT9BT (non-ergodic relaxor) matrix phase exhibit improved Td, the critical temperatures in the composites with rhombohedral NBT3BT (displaying spontaneous ferroelectric order) exhibit only marginal changes. Further, NBT3BT composites feature a 45% increase in Qm, while the corresponding increase is roughly three-fold for the NBT9BT composites. A 3-D Finite Element Method is used to simulate the electric field gradient at the matrix/inclusion interface, with the effective field distribution estimated to be higher than the applied field for highly conducting ZnO inclusions. The electrical properties indicate that, while the deviatoric stress at the matrix/inclusion interface stabilizes the ferroelectric equilibrium for the relaxor matrix phase, the stresses disrupt the long-range order for the ferroelectric matrix phase. These results establish the volume-limit of the second phase to stabilize a ferroelectric equilibrium, in addition to substantiating the role of residual stress evidenced by changes in the polar nature. Finally, a comparison of the composites with different NBTyBT phases is presented, with NBT6BT:ZnO composites demonstrating an optimal increase in both Td and Qm.