In this article, we review strong light-matter coupling at the interface of materials science, quantum chemistry, and quantum photonics. The control of light and heat at thermodynamic limits enables exciting new opportunities for the rapidly converging fields of polaritonic chemistry and quantum optics at the atomic scale from a theoretical and computational perspective. Our review follows remarkable experimental demonstrations that now routinely achieve the strong coupling limit of light and matter. In polaritonic chemistry, many molecules couple collectively to a single-photon mode, whereas, in the field of nanoplasmonics, strong coupling can be achieved at the single-molecule limit. Theoretical approaches to address these experiments, however, are more recent and come from a spectrum of fields merging new developments in quantum chemistry and quantum electrodynamics alike. We review these latest developments and highlight the common features between these two different limits, maintaining a focus on the theoretical tools used to analyze these two classes of systems. Finally, we present a new perspective on the need for and steps toward merging, formally and computationally, two of the most prominent and Nobel Prize-winning theories in physics and chemistry: quantum electrodynamics and electronic structure (density functional) theory. We present a case for how a fully quantum description of light and matter that treats electrons, photons, and phonons on the same quantized footing will unravel new quantum effects in cavity-controlled chemical dynamics, optomechanics, nanophotonics, and the many other fields that use electrons, photons, and phonons.