Today on the Fluffy Channel: magic tricks with the Matroska container. Namely, editions, file linking and ordered chapters. (Warning: very large amounts of techobabble walls of text ahead.)
These are easily some of the most (if not the most) obfuscated features of the format; a few people know how to use them (and it has been used in one or a few releases) but absolutely noone has written any simple guides to it (hence this blog post). Which is a pity, because it’s an immensely powerful feature. To start off, why would you want to hack XML files and play with obscure hexadecimal segment ID’s? Well, with the above features you can do things like:
- Make any combination of audio/video/subtitle tracks into one choice in the track selection menu (useful to change between dub/original+sub in one click)
- Create chapters-only Matroska files that when played display a virtual timeline pulling video/audio/subtitles from other Matroska files
- A practical implementation of the above might mean that you encode the OP of a TV series only once and then use chapters to get it inserted into the timeline of each episode
- Let the viewer choose between multiple “angles” or versions of the video (said versions may or may not be in the same file)
- Arbitrarily link MKV files or parts of MKV files so that they appear to the viewer as a whole
- And so on and so forth.
For this exercise, you will need: a) the latest version of mkvtoolnix, b) the latest version of Haali’s Matroska splitter (or just latest CCCP), c) your favorite text editor (never leave home without it) – preferably one with XML syntax highlighting, and d) a brain (in good working condition, fully awake). Let’s boldly go, etc.
File (or Segment) Linking and Segment UID’s
All of these concepts tie into each others somewhat, but linking is probably the most simple to understand, as well as the simplest to accomplish. Any combination of Matroska files with the same set of tracks and the same codecs can be linked so that when any one of them is opened, a virtual file containing the entire timeline of linked files is created and displayed to the viewer. The most common usage of this is to encode a full-length movie into one 1.4 GiB file and then split it into two 700 MiB parts that, when one of them is played, behave as if it contained the entire movie in one file. This simple use of the feature is very easy to accomplish; in mkvmerge GUI (henceforth known as mmg) just hit the Global tab and enable splitting in whatever form you want and make sure to tick the box “link files”.
If you look further down in the same tab you’ll notice that there’s an option for “File/segment linking”. This feature lets you link the muxed file to any two other files (with the same set of tracks and codecs used). Anyway the option asks you for something called a Segment UID. What is that, where do we find it and how do we use it?
First of all, I have to let you in on a dirty secret. File linking isn’t actually file linking at all, since the linking isn’t done by files or by filenames. It’s actually segment linking, and it’s done by segment Unique Identifiers or “segment UID’s” (SUID’s). Most Matroska files contain only one segment, but there’s absolutely nothing that stops you from having more than one segment per file. When the splitter looks for linked files, it just scans all Matroska files in the same folder until it finds the linked segment UID; hence you can rename the files all you want and the linking will still work. On the other hand you can’t remux the files because that changes the UID.
So how do we know what segment UID we should link to? The answer is, we use mkvinfo on the file we want to link to. You can use the GUI or the CLI, what you’re looking for is something like this:
Segment UID: 0xa2 0xa6 0xa2 0xa7 0x45 0x22 0x6b 0x85 0x85 0xd4 0x75 0x5e 0x6c 0xf4 0x67 0x60
You then copypaste that string of hexadecimal digits (everything after the colon) in the boxes in mmg to do the linking.
If you had a TV series that always had the opening theme as the first thing of each episode, this means you could encode the OP as a separate file, chop it off from all the actual episodes, and then use linking to link in the OP as “previous segment” to all the episode files. Which would mean that to a user with the proper splitter all the episodes would appear to have the OP encoded in, but in reality it’s actually only encoded once and stored in a separate file. Nifty, you think? You haven’t seen a tenth of it yet. Read on…
Chapters, Black Magic Style: Ordered Chapters
Some of you may have used the simple (OGM-based) chapters format before, but that won’t suffice for what we want to do here, so let’s take a look at an example of a Matroska XML chapters file: chapters-example.xml
This section and the following ones will involve a lot of hacking of these in a text editor; you can either create them yourself by hand or you can use
mkvextract chapters "file.mkv" > chapters.xml to extract them from existing MKV files (you can then use that as a template for further hacking).
Anyway, the first thing you’ll notice is that the big tag enveloping all the chapter entries (<ChapterAtom>) is the <EditionEntry> one. If you want multiple editions in the same chapter file, you will need more than one of these (we’ll get to this later). The rest is fairly straightforward, you have an unique chapter ID (automatically generated, but you can edit it yourself too), a start time (i.e. where your video player seeks when you click “go to this chapter”), and some display stuff (what will show up in the splitter menu). You’ll also notice that you can hide chapters so they don’t show up in the menu. Personally I’ve never used that particular feature.
You can also look at the XML in the mmg Chapter Editor to get a nice tree-view of all the tags. It also prevents you from writing invalid syntax, and creates chapter UID’s for you automagically so using it may be a good idea, but I’m far from sure it supports all the interesting features we’re going to use (it doesn’t support ordered chapters or segment linking at all for example) so I’m going to stick with my trusted text editor.
Update: Harukalover mentions that avimux GUI has a surprisingly good editor for this.
So what’s so interesting about this? It’s just chapters, isn’t it? Well, there are some tags you can add yourself to make things more interesting, the most important being
<EditionFlagOrdered>1</EditionFlagOrdered>. This turns on heavy magic mode and turns our plain old chapters into ordered chapters, which are quite different. Normal chapters are just seek points; ordered chapters are a description of a virtual timeline. Each ordered chapter has both a start time and an end time. To help you understand this concept, let’s do something silly with ordered chapters. Say that we have a 30 second clip that we want to loop 3 times, but to save space we only want to encode it once. Ordered chapters to the rescue! orderedchapters-example.xml
Take a look at the example. First off, ordering is turned on by setting the edition flag. Then, the section of the file given by
ChapterTimeEnd is played once per
ChapterAtom (note that both ChapterTimeStart and ChapterTimeEnd are mandatory). Note that the two last chapters are hidden so they won’t show up as seek points in the splitter. See the virtual timeline? When this is played, the viewer will just see one 1:30 long clip. Mux it for yourself and see; any input file will do, it just has to be longer than 30 seconds or weird things may happen (also make sure to tick “no chapters” in mmg).
To reiterate the virtual timeline stuff (this is important): normally when you play a file it plays all the encoded stuff within, from start to end (the “real timeline”. With ordered chapters on the other hand… instead of the splitter displaying the real timeline to the viewer, a virtual timeline is constructed from the real one. What is played when is determined by the chapter start and end points (which refer to the real timeline). To the viewer, it appears as if the virtual timeline is the real one, but internally the splitter seeks to the given points in the real timeline to construct the virtual one.
(You may want to read the above paragraph more than once.)
Ordered Chapters and Segment Linking
Now that we understand the virtual timeline, how about this linking stuff from other files thing we’ve been hearing about? Well, that can be done too and it isn’t too hard. Ordered chapters are not limited to creating the timeline just from the current file, you can also specify that a given chapter should use a certain part of another file, specified by its segment UID. It’s not hard, really. Say that in the example before we wanted the middle of the three iterations to be taken from another file (presumably containing something else). orderedchapters-example2.xml
See what I did there? Just add
<ChapterSegmentUID format="hex"> followed by the SUID you want to pull in things from (0x before each hex digit optional) and blammo, you have linked in stuff from another file. Not specifying a ChapterSegmentUID means that the current segment will be used.
You can mux this yourself to any file and see what happens too, you just need to replace the SUID with one pointing to another of your MKV’s in the same folder. Just make sure that both of them have same number of tracks in the same order and that they all are using the same codecs. Also make sure to tick “no chapters” in mmg or weird things can happen.
Editions and You
In Matroska, an edition is a set of chapters that is presented to the viewer; you could define it as a “timeline”. Most Matroska files (and indeed most if not all other multimedia files) only has one edition. However, Matroska isn’t limited to just one edition per file, you can have as many as you want. By itself it doesn’t have any immediately obvious uses other than the fact that you could abuse it to have several sets of chapter names in different languages, but if you combine it with ordered chapters things become very interesting. Among other things, it means you can have several virtual timelines co-existing in the same file; the viewer can (if he has a proper splitter (i.e. Haali’s)) choose between them as he wishes.
Update: Haali pointed out to me that you can actually have multi-language chapters without editions; by default the language matching the system’s default language will be displayed.
Ever wanted to have two versions of an opening theme and letting the viewer choose between them seamlessly? With editions you can. Encode both versions of the opening and the rest of the episode or what-have-you to one file (order doesn’t matter). Now create two ordered chapters files, one for each timeline (one for op1 + main episode, one for op2 + main episode). Then you combine them using your trusted text editor by adding another
<EditionEntry>, like so: orderedchapters+editions-example.xml
Note that there are now two
<EditionEntry> tags, each with its own set of chapters and hence each with its own timeline. When played, regardless of which edition is chosen (it will default to the first, see the
<EditionFlagDefault> flag) will appear to be a clip that is 4 minutes long, the first 30 seconds being identical, the next 60 differing and the last 180 being identical again. If you look at the XML, you will notice that the first edition/timeline will play the source file like this: [chapter point] 30s [chapter point] 60s [chapter point] JUMP to 02:30, 180s.
The second edition on the other hand looks like this: [chapter point] 30s [chapter point] JUMP to 01:30, 60s [chapter point] 180s.
As usual you can test-mux my sample XML with any MKV longer than 5 minutes (remember to tick “no chapters”) and see what happens.
For added confusion and/or awesomeness you can of course combine editions with segment linking in the ordered chapters, so you can have one (or both) of the openings in a separate file(s). This may actually be a good idea to avoid making things difficult for people without proper splitters (such as MPlayer users on *nix for example). For bonus points, combine ordered editions with unordered ones.
Sky’s the Limit: Advanced Haxing etc.
You might have noticed that in the previous example that the editions showed up as Edition 1 and Edition 2. Not too descriptive, is it? Well, you can name them too, but it isn’t so easy as to just adding something to the chapters XML file. Instead you have to add a tags file as well. This is XML too, and it would look something like this for the previous example: tags-example.xml
Just set the
EditionUID in the
Targets tag to the UID of the edition you want to change the name of, and then set the
String to the name of it. Note that the
Name tag is actually the name of the tag itself (must be TITLE to make anything show up) and not what will actually show up in the splitter menu.
Another potentially interesting thing you can do with ordered chapters is to create small chapters-only files that when played show an entire TV series as one single long virtual file. Just create a 5 or 10 frames long blank video with the same settings that you’re using for the rest of the series, with corresponding audio and subtitle tracks, create ordered chapters file with segment linking, mux, and blammo 50 KiB file that when played display as a 5 hour long TV series. Of course you can include editions in it as well for added awesomeness.
Yet another opportunity: tracksets. This is actually not included in the Matroska specs, it’s just a feature of Haali’s splitter. By abusing the tags file you can set and/or disable/enable combinations of tracks when a certain edition is selected. Hello, multi-language stuff! To use, add a tag with the
Name set to TRACKSETEX, and the
String value set to:
EditionUID VideotrackUID AudiotrackUID SubtitletrackUID Three-letter_language_code Optional_name
Target is necessary. Setting any of the track UID’s to “.” (a dot, full stop, whatever; without quotes) means “don’t change”, setting the subtitle track UID to “x” (lowercase letter X; again without quotes) means “disable subtitles”. When the edition given by the EditionUID is activated, the given set of tracks will be chosen. The track UID’s can be found with mkvinfo. If you’re too lazy for that you can also just specify track numbers if you say # before the number.
Example for the same sample as before: tags-trackset-example.xml
Note that since you can’t remux a file without changing the SUID, it might be worth to know that there’s a feature in mmg’s chapter editor that allows you to write chapters straight into an MKV file without remuxing it; it can be found under the Chapter Editor menu -> Save to Matroska file. Don’t do it to a file that already has chapters or you may get weird issues though.
Update: This is slightly incorrect; Haali’s GDSMux (comes with the splitter package and with the CCCP) lets you specify the SUID yourself, so if you use that you can remux while keeping it.
Caveats, Assorted Traps and Various Interesting Things to Note
- Make sure all your various UID’s really are U or you’ll run into trouble. (They must at the very least be unique to the set of files/segments you’re working with.)
- Never try to add ordered chapters to a file that already has chapters (ordered or not). Always tick “no chapters” when remuxing, just to be safe!
- Remember that the virtual timeline created by ordered chapters replaces the ordinary timeline; if there is a part of the source video that is not covered between your ChapterTimeStart/End’s, that part will never be displayed to the user at all (might be useful for hiding easter eggs in your releases).
- On a related note, remember that all ChapterTimeStart/End’s refer to timestamps of the underlying segment. If the underlying segment is VFR, this means you must take the timestamps after the timecodes have been applied! One method of doing this is using Aegisub with timecodes loaded, another is to demux the v2 timecodes and amuse yourself by counting frames and matching them to line numbers.
- If a subtitle line spans an ordered chapters segment switch point, there will probably be artefacts and/or other problems. So don’t do that.
- Likewise, there may be audio artefacts during such switch points under certain circumstances, so try placing the switch points where the audio is either silent or has a lot of noise going on.
- The requirement that the linked segments have the same sets of tracks and codecs used has a few caveats; similar settings has to be used as well. To be precise, the CodecPrivate header has to be the same (update: this may not be so strict after all, YMMV). Usually if something is encoded with the same settings, it will be, but your mileage may vary. See Haali’s codecs list for more information.
- Related to the above, if you want to have different editions with or without certain audio and/or subtitle tracks, remember that you can abuse silent audio tracks and/or subtitle tracks without any actual lines to satisfy the track number/codec requirement.
- Still related: for the commonly used subtitle formats S_TEXT/ASS and S_TEXT/SSA, the entire [v4 Styles] section is stored in the CodecPrivate header, so if you use ordered chapters and softsubs, all styles you use have to be present (and identical) in all subtitle tracks.
- Since the timelines of all ordered chapters are taken directly from the underlying source timeline, to get subtitles to synch you need to make sure they synch with the “real” timeline.
- If you’re using x264 to encode, you can force certain frames to be I-frames even if they normally wouldn’t be by manipulating the stats file, which may be interesting for handling virtual timelines efficiently. (Hint: if you seek to somewhere that isn’t an I-frame, for the seek to take effect you need to decode everything from the last I-frame to the sought frame, and virtual timelines are all internally constructed by seeking. Translation of the hint: you might get “hiccups” when going over chapter switch points if the chapter doesn’t start on a keyframe.)
- You can also do the same with XviD if you are so inclined, by using zones.
- While the video resolution of all the included segments has to be the same, the DAR (display aspect ratio) does not necessarily have to be. This has interesting implications for stuff that changes AR somewhere in the middle. (Player support for changing AR mid-stream may vary a lot, test thoroughly before using.)
The author would like to thank the following people and/or entities (in alphabetical order):
Without them, this guide probably wouldn’t be here.
Or, OK, well, if two or three doom9 forum posts by Haali and some random #darkhold logs count as references then I guess…
See http://forum.doom9.org/showthread.php?t=99489 for the thing that started it all.
See the mkvmerge documentation for information on the XML chapter and tags file formats.
See Haali’s page for the TRACKSET/TRACKSETEX details.