What has happened to video in the smartphone age? Over the course of the 20th century, the combination of image and sound in video became ever more refined, and today the medium is, in a sense, relearning how to function without sound. Not completely silently, of course. Dialogue, sound design, and music still play an enormous role when we create a video. However, modern editors increasingly have to assume that someone who comes across a video on Instagram, TikTok, Facebook, or in any other feed may be sitting on a bus, be in the office, be lying next to a sleeping partner, or simply not want to wear headphones. The result is a small cultural about-face: video is, in a sense, making a comeback as text.
And this is not limited to social media. A 2023 YouGov survey found that 63% of Americans under 30 prefer to watch TV with subtitles, even when they understand the language; among the overall population, the figure was 38%. Respondents cited, among other reasons, better comprehension, difficulty with accents, and noisy environments. The same trend can be seen in Great Britain: in a separate survey by the market research company YouGov, 61% of 18- to 24-year-olds said they prefer subtitles when watching films or television in their native language, compared with 28% of all adults in the country. Subtitles have existed for a long time, but it has now become completely normal to switch them on while watching – although they used to serve mainly to make foreign-language films easier to understand or to ensure accessibility.
Silent video is not simply video with the volume off
Interestingly, reading subtitles cannot completely replace sound. A recent study published in PLOS ONE in 2024 examined exactly this, using eye tracking as well as measurements of comprehension, cognitive load, immersion, and enjoyment among 168 viewers. For the participants who watched the videos without sound, cognitive load was higher, while their scores for comprehension, immersion, and enjoyment were lower than for the participants who watched the same videos with sound. Their gaze also relied more heavily on the subtitles.
This suddenly makes the problem of video editing look quite different. If viewers simply get an ordinary video without sound and with a few automatically generated subtitles at the bottom of the screen, they can technically understand the content, but they have to do significantly more mental work to do so. An edit optimized for silent viewing must therefore transfer part of the communicative function of the soundtrack to the image.
Research clearly shows that subtitles can be helpful. In an analysis of more than 100 empirical studies, cognitive scientist Morton Ann Gernsbacher concluded that subtitles can improve comprehension, attention, and memory. The benefits were particularly pronounced for deaf and hard-of-hearing viewers, people watching content in a language other than their native one, and children and adults who are learning to read. However, the effect was not limited to these groups. Ofcom also reports that many people without hearing loss use subtitles – particularly younger viewers.
Modern editors therefore now have to deal with two groups: people who need subtitles to have access to video content at all, and those who have simply become accustomed to reading videos.
When subtitles became part of the picture
Social platforms are at least partly responsible for this. Facebook noticed relatively early that automatically played sound in the newsfeed was problematic. In a study published in 2016, 80% of users reacted negatively when a video ad unexpectedly started playing on their mobile device, and 41% of the videos tested by Facebook made practically no sense without sound. These figures are now quite old and should not be understood as representative of today's Facebook users. However, they show how the design of user interfaces forced video producers to think visually about information that had previously been conveyed through sound.
TikTok went one step further. Experts who study subtitles there found that subtitles are increasingly being used not just as transcription but as a means of visual design: positioning, emphasis, and other stylistic elements can, for example, identify speakers or reinforce certain meanings. In other words: subtitles were able to break away from their traditional place at the bottom of the screen.
This is where ordinary video editing software takes on an unexpectedly important role. Anyone using Clideo, for example, can generate subtitles automatically, edit text and timing, adjust the appearance, embed the subtitles directly into the video, and export a separate subtitle file in SRT format. On a feature list, none of this sounds particularly revolutionary. Culturally speaking, however, such tools enable almost anyone to additionally convert spoken information into readable information.
The key word here is: restructure. It is not just about automatic transcription. A subtitle that only appears after the crucial visual information, or a subtitle that contains an entire sentence and is hardly manageable to take in on a small smartphone display, may be correct in content and still work poorly. In the field of automatic subtitling, the arbitrary splitting of spoken language into individual lines is criticized in particular, because it affects how easily a text can be grasped. Editing subtitles therefore means editing rhythm just as much as language.
The edit has to take over part of the sound
Once sound can no longer be taken for granted, other editing decisions also become more important. Reaction shots may need to stay on screen a little longer so that emotions are understandable even without laughter or audible shock. A change of location that would normally be explained through dialogue may need an on-screen text. A product demonstration may need visual instructions instead of a voice explaining the next step. Facial expressions, framing, B-roll, graphics, and the visual hierarchy of text begin to carry information that would otherwise be taken in through the ear.
A tool like Clideo can take care of the mechanical side – generating subtitles, correcting the timing, styling text, and embedding it in the finished video. However, the editor still has to decide what actually needs to be shown. Moreover, there is an important difference between trendy text elements on the screen and subtitles that serve accessibility. The former can summarize or highlight certain statements. Proper subtitles must accurately reproduce dialogue and relevant sounds. Text burned permanently into the image gives the creator control over how it looks everywhere the video is played. A separate subtitle track, on the other hand, gives the viewer more control, provided the platform in question supports it.
This difference is important, because a video that works without sound is not automatically accessible. Giant animated words bouncing around a person's face can work superbly for a social media clip and at the same time be a poor substitute for precise and easily readable subtitles.
The bigger change, then, is not that sound has become unimportant. The results of the PLOS ONE study actually point to the opposite: viewers generally understand and enjoy subtitled content better when sound is still available. What has changed is rather the basic assumption behind the edit. An editor can no longer take it for granted that every viewer will hear the soundtrack.
Modern videos therefore increasingly have to be able to exist in two forms at once: once with the volume turned up and once almost like a moving page that is read. Good video editing does not choose one of the two variants. It ensures that the story still works even when one of them drops away.
