Magneto-optical spectroscopy based on the transverse magneto-optical Kerr effect (TMOKE) is a sensitive method for investigating magnetically-ordered media. Previous studies were limited to the weak coupling regime where the spectral width of optical transitions considerably exceeded the Zeeman splitting in magnetic field. Here, we investigate experimentally and theoretically the transverse Kerr effect in the vicinity of comparatively narrow optical resonances in confined quantum systems. For experimental demonstration we studied the ground-state exciton resonance in a (Cd,Mn)Te diluted magnetic semiconductor quantum well, for which the strong exchange interaction with magnetic ions leads to giant Zeeman splitting of exciton spin states. For low magnetic fields in the weak coupling regime, the Kerr effect magnitude grows linearly with increasing Zeeman splitting showing a dispersive S-shaped spectrum, which remains almost unchanged in this range. For large magnetic fields in the strong coupling regime, the magnitude saturates, whereas the spectrum becomes strongly modified by the appearance of two separate peaks. TMOKE is sensitive not only to the sample surface but can also be used to probe in detail the confined electronic states in buried nanostructures if their capping layer is sufficiently transparent.