This article details a new, free resource for continuous video assessment named YouDemo. The tool enables real time rating of uploaded YouTube videos for use in science, technology, engineering, and mathematics (STEM) education and beyond. The authors discuss trends of preservice science teachers’ assessments of self- and peer-created videos using the tool. The trends were identified from over 900 assessments of 170 videos, with over 131 unique users. Included in this data set is a 2-year study focusing on 27 preservice science teachers (from a 5-year study of 76 total science preservice teachers) and their use of the tool. The authors collected both quantitative (numerical scores) and qualitative data (open-ended questions) from the 27 participants. Findings show that (a) rating two metrics had a non-zero bias between the two metrics; (b) preservice teachers found continuous video rating beneficial in enabling video assessment, promoting critical thinking, and increasing engagement; and (c) preservice teacher’s self-assessment was uncorrelated with their peers’ assessment. Additionally, the elements to enable skill improvement were met, including (a) a well defined task, (b) a challenging task, (c) immediate feedback, (d) error correction, and (e) practice. Implications include improvement in preservice teacher reflection and discussions, especially related to STEM content and pedagogy.