Students can use the program to dissect video clips and transcripts to make taking notes and understanding easier with VideoSticker.
The pandemic has significantly increased the trend of using video as a popular educational tool. Video is a storytelling format that has numerous advantages over text or lecture-based training, particularly for teaching linear processes and physical tasks, but it also has several disadvantages. Finding previously watched stuff to review might be challenging. And taking notes is particularly difficult and complicated.
According to Hari Subramonyam, a research professor at the Stanford Graduate School of Education and a junior faculty fellow of the Stanford Institute for Human-Centered Artificial Intelligence, “the bottom line is viewers find it challenging to extract and synthesize significant information from video.”
Subramonyam and two collaborators, Yining Cao from the University of California, San Diego, and Eytan Adar from the University of Michigan, have created VideoSticker, a note-taking program especially for video-based learning, by utilizing computer vision and natural language processing tools.
According to Subramonyam, taking notes while watching a video currently involves manually grabbing screenshots, clipping and cropping visual elements, reading transcripts, and typing notes by hand while switching between the video player and a note-taking app.
The strong visual and textual note-taking program VideoSticker, however, integrates video, audio, transcripts, and note-taking. Even better, VideoSticker can automatically recognize and remove objects from the video and add them to the note-taking area using AI. Then, VideoSticker searches through the transcripts using methods similar to those used in chatbots and popular voice recognition to extract important words and align it with the pertinent images in the note-taking section.
All in all, it gives pupils a significant advantage over manual note-taking and enables them to concentrate on the key information to increase understanding and retention.
According to Subramonyam, “Video is most frequently linked with entertainment, and that’s a totally different experience from an instructional environment for knowledge gathering.” We’re asking students in the TikTok era to watch hour-long videos and learn important ideas more and more often. They simply don’t now have the necessary software tools for that. The void is filled by VideoSticker.
Concept to Practice
According to research, when students are actively creating mental models of an idea, they learn most effectively. According to Subramonyam, these models can take several shapes, including concept maps, diagrams, and timeline visualizations.
In essence, VideoSticker transforms the passive experience of watching a video into an active one where students learn by modifying the visuals and text and adding their own explanations and comments to those parts to reinforce comprehension and retention.
According to Yining Cao, adaptable and interactive representations for learners are crucial for supporting note-taking and comprehension.
Screenshot of the videosticker platform demonstrating the app’s note-taking functionality
Figure: The user interface for VideoSticker, including the video window and controls in the top left corner, the searchable transcript in the bottom left corner, and the full-featured note-taking area in the right corner. (Courtesy of Stanford University’s Hari Subramonyam)
“Cool,” said the students.
The researchers conducted preliminary user testing with a cohort of 10 graduate and undergraduate students to assess the viability of the VideoSticker technique. Each person finished a 75–90 minute note-taking session that includes two readings of difficult scientific material on cell mitosis—one for first viewing and the second for annotation. Then they shared what they had learned.
Each participant remarked on how easy it was to switch between their notes and the video content in VideoSticker. It’s nice to be able to control and detach things in the video viewing panel. Therefore, I can observe how it changes within the framework of the video,” says one participant. The way that VideoSticker extracted visual items and lined them up with the text transcript was also praised by users.
Subramonyam asserts, “In general, I think VideoSticker is a terrific illustration of how AI supports the learning context to balance manual note-taking with the experience of learning that is so crucial to actual comprehension. A technology like this will be essential as video usage in the classroom increases, as it will undoubtedly do in the upcoming years.
The team intends to work with educators in the future to test VideoStickers in authentic educational settings and provide the technology for purchase.