WALLS OF TL;DR

VFR (variable framerate) is a pretty tricky concept. Not only is it hard to understand, it’s hard to work with and hard to really wrap your mind around, as evidenced by for example the VFR thread on the AnimeSuki forums, or this other recent thread which is a good demonstration of some of the most common confusions surrounding the concept of VFR.

This post will mainly be an attempt to explain how to deal with VFR material when it’s stored in a container that does not support VFR (such as AVI) or working with VFR material in non-VFR-aware programs (such as Avisynth). Why, I don’t know. Maybe it’s because trying to explain a concept to someone else is one of the best ways of gaining a better understanding of it yourself. Maybe because I think it’ll be useful. Maybe I just felt like writing something. Oh well.

WARNING: Huge wall of technobabble ahead. No trolling or dumb attempts at jokes in this post.

In my experience, the hardest thing to understand for people new to the VFR concept (myself included) is that the term “framerate” is just an illusion, and that the Avisynth function AssumeFPS() doesn’t do what you think it does. This has to do with the fact that people tend to see the frames of the video, their playback speed and the associated synchronization to the audio as one big black box, while in reality they are entirely different concepts that must be thought of as separate entities. Since I’m an Avisynth freak, I’ll give some examples, demonstrating the differences between ChangeFPS() and AssumeFPS(). Assuming a CFR source (no surprises so far):

avisource("x.avi") # say 25fps for simplicity's sake
changefps(50) # clip now has twice as many frames and is 50fps, but audio is still synched
assumefps(25) # clip still has twice as many frames as the original, but is now played at half speed compared to original
changefps(12.5) # clip now has the same amount of frames as the original, still plays at half speed
assumefps(25) # clip is now the same as the original again

See what I’m trying to get at here? ChangeFPS() modifies the AMOUNT of frames by duplicating or removing frames, while AssumeFPS() just changes the speed of playback. Or to be more precise, ChangeFPS() changes frames, AssumeFPS() changes timestamps.

Grasping this fundamental difference between frames and timestamps is quite possibly the most important thing to understand about VFR. Which brings us to the concept of VFRaC (VFR assumed to be CFR), as jfs named it. In a CFR clip, all frames have the same amount of time between them. In VFR clip, they do not; some frames are displayed for longer periods of time than others. Now the tricky thing about loading all the frames of a VFR clip into a program that assumes that everything is CFR is that all the timestamps get erased and overwritten with ones corresponding to a CFR framerate, resulting in the VFR assumed to be CFR situation. Now, since you (presumably) have all the timestamps stored in a separate file, this isn’t a big deal since you can do any kind of processing that doesn’t touch the amount of frames or change the frame ordering around and then add the timestamps back when you convert the end result back into a container that does support VFR.

The problems start when you try to do something in your non-VFR-aware program that depends on the audio being in sync, like karaoke, other subtitles, or just the seemingly simple act of cutting the video and the audio in the same place. Since the frame timestamps aren’t correct, the audio won’t sync (remember: the audio is separate from the video). There are ways around this, of course. Some VFR-aware programs can help you in some ways; Aegisub for example will, given a VFRaC clip and a timecodes file, convert a subtitles file so that all the timestamps matches the VFRaC clip so you can use the subtitles in a non-VFR-aware program and still have them match up to the audio after you’ve added the proper timestamps back in with your muxing program. Some non-VFR-aware programs (i.e. Avisynth) are so versatile that you can partially work around the issue within the limits of the program itself. Say for example that you wanted to cut out a small part of a VFR MKV, apply some Avisynth filters to both the audio and the video and then mux it to a new MKV, preserving its VFR-ness. This is possible with only Avisynth and a text editor, since Avisynth supports both loading all the frames of a VFR clip (converting it to VFRaC) as well as doing a conversion to “real” CFR by duplicating or removing frames. First you’d have to load the VFRaC clip and find the start and end frames of the cut, then load a converted-to-CFR variant of the clip (using, for example, DirectShowSource(convertfps=true)) and find the same start and end frames (the frame numbers would, of course, be different). Then, to cut out the video and the corresponding audio section you’d do something like this:

vid = avisource("vfrac.avi").trim(start,end) # video only
aud = directshowsource("vfr.mkv", convertfps=true, fps=23.976).trim(start2,end2) # audio and video, in sync
audiodub(vid,aud)
# do the rest of the filtering here

Then you’d need to edit the timecodes file so it only included the segment you just cut out.
Note that with this approach, it’s quite possible that the audio and the video would have different lengths, so you’d need to make sure that your encoding program wouldn’t do anything funny with regards to that.

In some programs however it’s just not possible to work around the problem. In that case the best approach is most likely to just process the audio and the video separately. Or bite the bullet and just convert to CFR. If you were doing something like a VFR AMV, you’d more or less need to write the timecodes file yourself… One of these days I’ll create an AMV with just Avisynth and Aegisub just to show the world how it Should Be Done.

And with that, I’ll conclude this post with a hope that you learned something.