In this study, we introduce the concept of “modal radar” as a novel structural damage analysis methodology for HFR (high-frame-rate)-video-based modal testing that can accurately quantify both the degrees and positions of structural damage. Our modal radar method consists of four parts: (1) vibration displacement measurement using an HFR video camera, (2) stochastic subspace identification to estimate modal parameters, (3) improved local Fisher discriminant analysis for supervised learning based on modal parameters, and (4) feature space normalization for damage quantification. Based on our modal radar method, numerical simulations were performed for beam-shaped cantilevers with different weights as structural damage, and the positions and degrees of the weights were accurately estimated in the normalized feature space, even when the objects to be observed were not involved in the sample set for supervised learning. These tendencies were also confirmed in HFR-video-based modal testing of actual steel cantilever beams with different weights, which were excited by human finger tapping. These experimental results demonstrated the effectiveness of our concept of “modal radar” for accurate structural damage quantification.