Electro-optic modulation is a key function in optical data communication and possible future optical compute engines. The performance of modulators intricately depends on the interaction between the actively modulated material and the propagating waveguide mode. While a variety of high-performance modulators have been demonstrated, no comprehensive picture of what factors are most responsible for high performance has emerged so far. Here we report the first systematic and comprehensive analytical and computational investigation for high-performance compact on-chip electro-optic modulators by considering emerging active materials, model considerations and cavity feedback at the nanoscale. We discover that the delicate interplay between the material characteristics and the optical mode properties plays a key role in defining the modulator performance. Based on physical tradeoffs between index modulation, loss, optical confinement factors and slow-light effects, we find that there exist combinations of bias, material and optical mode that yield efficient phase or amplitude modulation with acceptable insertion loss. Furthermore, we show how material properties in the epsilon near zero regime enable reduction of length by as much as by 15 times. Lastly, we introduce and apply a cavity-based electro-optic modulator figure of merit, Δλ/Δα, relating obtainable resonance tuning via phase shifting relative to the incurred losses due to the fundamental Kramers-Kronig relations suggesting optimized device operating regions with optimized modulation-to-loss tradeoffs. This work paves the way for a holistic design rule of electro-optic modulators for high-density on-chip integration.