I made a video of some old games running on my new laptop. It may be a 1986 IBM PC Convertible 5140, with 640kB of RAM, an 8088 processor and two floppy drives, but it great for gaming on! For certain values of ‘great’.
There’s plenty to talk about in the machine, which I may get to as some future date, but I’m not going to. Instead I’m going to talk about making the video, because this was my first experience with a video project which involved serious editing and production, and oh boy.
It all started with a microphone. Or rather, the absence of one.
A while back, just after I got the 5140, I discovered it had a CGA graphics adapter and composite out, so I had to try running the classic 8088mph graphics demo from 2015; this achieves the impossible and managed to persuade a 1981 CGA graphics card, capable of graphics in four colours, to display 30fps animations in 1024 colours. By cheating.
The 5140 has some hardware quirks and it didn’t run very well. They were interesting enough that I ended up videoing it, using my cell phone as a camera. It worked, but wasn’t very satisfactory — while cell phones are pretty good cameras, they have terrible microphones, and the result was tinny and at times you can hear me breathing, which nobody wants.
I needed a microphone for videoconferencing anyway, and this pushed me over the edge into ordering one of the cheap but surprisingly good condenser microphones that’s coming out of China now: a Vxmba NW800 (also branded as Neewer, Toner, etc). Then, I came across a cheap format converter in a junk shop which would let me display composite video on a modern HDMI-capable LCD. I had all the pieces to make an epic video! So I had to make an epic video. Or at least a video.
Not long after I had a studio set up. By which I mean my dining room.
⇡Cell phones vs. microphones
So, first thing to do? Plug the microphone into the phone.
Enter the wonderful world of cell phone headset jacks. The earphone socket on the top of a phone allows connecting a standard set of stereo earphones, but they also carry an extra connector for mono microphone input — this is intended to let you plug a earphone/microphone headset in for hands-free use. The trouble is, there are two different competing standards, and you need to know which one your phone has. Apple and most Android devices use the CTIA standard, while Nokia tends to use the OMTP standard.
But I figured out the pinout, and soldered one up. Didn’t work. The phone would simply refuse to acknowledge there was an external microphone hooked up, and as there’s no UI to tell you this and it instead it silently falls back to the internal microphone, this was thoroughly frustrating to figure out.
Enter the wonderful world of microphone impedances.
Android expects the external microphone to have a resistance of about 1kΩ; pressing the volume controls on the headset shorts out the microphone with small resistors. The phone measures the resistance and from this detects which button is pressed. A very low resistance indicates that the headset doesn’t have a microphone (because the plug lacks a microphone connector, and the microphone ends up shorted to ground); a very high resistance is interpreted as not having anything plugged in.
My microphone has a balanced XLR output, which after conversion to 3.5mm jack ends up as a high impendance. This was why the phone didn’t think anything was plugged in.
The bodgy solution was simple: a resistor shorting the microphone to ground. This let the phone detect the microphone, and the resistor was high enough that the microphone’s power supply could still modulate the signal. Admittedly, I did end up with this abomination for letting me connect the microphone and a monitoring headset into the phone…
(Note: I’ve unscrewed the shroud to the microphone connector so you can see the resistor. The TRRS plug was ripped out of the cable for an old headset.)
…but it worked.
⇡Cell phones vs video
So after checking the sound levels (pointlessly — Android doesn’t have a microphone gain control, so I wouldn’t have been able to do adjust anything anyway), I spent a day recording. So, so much footage. Then I had to throw it all away.
The problem here was that, to my total surprise, cell phones don’t take fixed frame rate video. Ask one to record at 30fps, and what you get will be approximately 30fps. Each frame comes along at an irregular interval and is uniquely timestamped. The exact interval can vary hugely depending on the conditions. Some of my footage had sections lower than 20fps (and I’d told the phone to record at 60fps).
This is apparently known as variable-frame-rate video, or VFR, and all phones do it, both Android and iPhones (presumably because they use similar camera hardware).
It’s problematic, firstly because 20fps is awful and unusable, but also because none of the software I could find would edit it. It seems that most video editors assume you’re using video from a real video camera at a locked frame rate, and Weird Things™ would happen if you give it VFR video — my editor of choice is Blender, which is particularly problematic (if you configure it for a 30fps project, and import a video clip, it plays the video clip back at 30fps regardless of the video clip’s own timing! Which means that essentially it ends up playing at a variable speed, totally out of sync with the audio).
(I suspect this is a problem which only affects amateurs — professionals will use real video cameras, not cell phones, which will record at a rock solid 30fps (or 29.97, or 59.94, or 60). So there’s no real incentive to add support for editing VFR video. Which is a shame, as the forums are full of confused people wondering why their editing project isn’t working.)
I believe that phones do this because cell phone cameras record video by taking a series of still images, and the camera module doesn’t let you accurately control the exposure; the camera module just runs the exposure until the picture’s done, however long that takes.
It’s possible to convert VFR to constant frame rate — CFR — but it’s lossy. Transcoding 20fps to 30fps will involve repeating every third frame, resulting in jerky video, which I didn’t want.
I tried various approaches, including recording at roughly 120fps and then transcoding down to 60 or 30; by using such a high frame rate I thought I could guarantee that the resulting video was smooth. (This actually worked, but used appalling amounts of disk and the picture quality was poor.) I could not find a good solution.
The poor solution I eventually did find was to more or less randomly find a combination of settings that produced 30fps-ish results, then locking the exposure to prevent the camera from adjusting things too much. It wasn’t close enough to 30fps to reliably edit, but it was close enough to transcode to my desired 29.97fps without too much quality drop:
ffmpeg -y -i $in \ -acodec copy -vcodec libx264 \ -pix_fmt yuvj420p -preset veryfast -tune fastdecode \ -crf 20 -r 29.97 -flags +ildct+ilme -x264opts keyint=1:tff=1 \ $out
⇡Expect filming to take longer than you expect
So I had to reshoot everything to fix the frame rate issue. And then I had to reshoot again, and again, and…
I had an overall script which allowed me to break down the shots into various groups. These consisted of:
opening sequence (including changing monitors)
showing the games start up on the external monitor, from a distance
showing the game footage itself on the external monitor, in closeup
showing the game footage itself on the internal screen, in closeup
intertitles (in extreme closeup)
Changing monitors required powering off the machine, so I tried to do all the external monitor filming, then all the internal monitor filming, and so on. Of course this meant that I had to move the camera, and reproducing the exact camera angle and exposure settings to reproduce a shot was almost impossible, so I every reshoot involved rerecording all the footage. (You’ll notice that I forgot to reshoot some of the intertitles, and the exposure doesn’t match.)
Filming LCD monitors is hard. Really hard. The colours show up weird. There’s flicker. They’re glarey — they pick up every stray light in reflection; you’ll never see it with the naked eye, but the camera will pick up every blobby smear. On multiple occasions I had to reshoot sequences because I’d forgotten to turn off a light behind the camera and there were reflections. I had to tinker with the colour balance to make the colours on the LCD look right (which is why my dining room looks orange).
The internal screen was especially problematic, as it was very low contrast, angled badly, and the primitive backlight had an absurd amount of flicker. (After spending ages with the antibanding feature on the phone I eventually discovered that the flicker went away if I set the phone to 50Hz antibanding. Which is interesting, as CGA refreshes at 60Hz. I wonder if the mono screen works differently? Of course, then I had to reshoot.)
Every reshoot I had to work through the games, loading them one at a time, playing them for a bit, and then moving on to the next one. Overkill took seven minutes to load from floppy. Seriously. I have the video to prove it. Then I had to play through the level (at a third speed; the 5140 is very slow). Eventually I got too good and had to deliberately play badly to produce interesting footage. Prince of Persia and Commander Keen at least had demos, so I just had to sit and wait while they played themselves…
I did also want to include the Secret of Monkey Island demo, but for some reason it wouldn’t render in colour, and the text was impossible to read and it was unplayably slow anyway.
So I finally had a set of video clips, transcoded into solid 29.94fps, ready for editing. Now, to edit them!
I used Blender. Its video editor, the VSE, is pretty nice. It’s rock solid, easy to use once you’ve climbed a short way up Blender’s cliff-like learning curve, and with a nice set of features. Importing a video clip, slicing it up, trimming and rearranging segments etc, all happen via a couple of key presses. I did not have to learn the entire user interface, which I’ve tried before. It’s got a decent set of transitions, mostly automatic — I tried some wipes but eventually stuck with crossfades due to simplicity (select source, select destination, press SHIFT+A, select Effect -> Crossfade from the menu, done). Every clip has tunable parameters for things like audio volume, colour saturation, speed etc, and every numeric parameter can be animated via keyframes; the UI isn’t precisely intuitive, but it can be made to work.
There’s a Text effect which lets you add arbitrary text to the image, which I used for the subtitles. It’s pretty crude and won’t let you change the font, and it doesn’t have support for dimming the background behind the text. The result wasn’t particularly satisfactory and some of the subtitles are a bit low contrast.
I did encounter some minor bugs, mostly just misrendering of audio and video in the preview window. But on the whole I’d recommend it.
Actually choosing the clips to show was by far the most interesting part of the entire process. I edited down ruthlessly, removing any and all dead air. The setup montage went through several iterations (and a reshoot) before I ended up with something I was happy with. It took a while to realise that I needed to film the printer attachment sequence twice, once from each angle… and then a while longer before I remembered to use the same combination of hands both times! The opening and closing sequences actually had a rough script, which I actually wrote down.
The music tracks were added as simple sound clips. Blender will show the volume envelope of an audio clip, which is invaluable for figuring out where the silent spots are; some of the music is cut and spliced together. Segueing from the Elite theme to the real recording was amazingly straightforward — even the tuning matched!
I’m not entirely sure the subtitle commentary works. Subtitle timing is tough. The BBC recommands 0.33 seconds per word, but I couldn’t be bothered to count and measure, so I don’t think the interval’s particularly comfortable. We’ll see. I think if I do this again I need to go to more effort to make them readable, like masking out part of the image and placing them there. The font’s ugly, but as I said above, it’s not configurable.
Once editing was complete, I had to render the video. Rendering directly to a
.mkv file took over an hour. I had to do this repeatedly, for reasons
described below, and once I pressed the wrong button and aborted the render 45
In fact, Blender recommends you don’t try to render directly to video —
instead they prefer you to render the video to discrete
.jpg frames, the
audio to a standalone
.flac file, and then encode with ffmpeg as an
additional step. This is a lot more fiddly but considerably faster, and also
it allows you to resume a render if you, e.g., accidentally press a button
and abort the render. To pick an example completely at random. (It also
allows rerendering segments of the video by setting the start and stop
frames, which I didn’t need to do, but can save a lot of time.)
ffmpeg -f image2 -framerate 29.97 \ -i '$baked_video_frames/%04d.jpg' -i $baked_audio \ -c:v libx264 -preset slow -profile:v high -crf 18 -coder 1 \ -pix_fmt yuv420p -movflags +faststart -g 30 -bf 2 -c:a aac \ -b:a 384k -profile:a aac_low $out.mkv
The resulting video file was 2GB.
⇡Uploading to Youtube
At this point I thought I had finished, and that I could send it to Youtube and be done. Sadly, no.
The upload worked fine, and was quite quick, but then… and experienced Youtubers will probably be seeing this coming… I got a Content ID match on some of the music I was doing.
What this meant was that Youtube had flagged some of my music as being owned by a third party (via a scummy royalty collection agency). This was not in fact showstopping; I could have gone ahead and uploaded it, but the third party would be able to control monetisation on my video, which essentially meant that they could insert adverts.
Let me rewind a bit and explain the music situation.
I wanted to use The Blue Danube because of the Elite segment. That particular piece has been used for Elite since the Commodore 64 version, where it would play when you enabled the docking computer, as a homage to the film 2001. So I wanted to transition from the bleepy title track to the real recording, and then climax with the crash at the end. The Youtube Audio Library has a beautiful clean copy [FIXME insert link], free for use. But I needed music for the rest of the video too, and it needed to match, so I went to musopen to get some public domain Strauss recordings.
It was one of these recordings which Youtube was flagging.
I started out pretty angry, assuming that this was a scummy collection agency claiming ownership of public domain music, but then I investigated in order to figure out the provenance of the track. It seems musopen got it from the European Archive Foundation, now defunct; investigating the EAF’s website via Internet Archive revealed that it was actually a public upload site, and they didn’t appear to do any checking of the license of any of the music people uploaded. So even though musopen claimed the track was available under a Creative Commons license, as far as I could tell this was completely unfounded: the ownership claim could have been entirely valid. The original uploader could have simply lied about the license.
So, even though I could have gone ahead and used the track if I were willing to cede monetisation rights to a scummy collection agency, I decided to switch to different music anyway, where I could prove the provenance.
Sadly public domain Strauss recordings are few and far between. I eventually settled for recordings off some old 78rpm shellac records from the 1940s; Internet Archive has a huge stock of them [FIXME insert link] and the quality’s not bad (as well as having a nice retro crackle that I think fits well with my retro computer).
These are technically under copyright, as public domain effective starts at 1923, but almost certainly nobody will care (at least, they’re not producing Content ID matches). And if there is a claim on them, I do at least know the provenance.
(The most annoying part of this is that 78s can only store about three minutes, so I had to reedit the entire Elite sequence to match the music…)
What would I do differently if I did this again?
beg, borrow or steal a real video camera. Cell phones are good, but are just too much hassle. (I haven’t mentioned that recording video on my Nexus 5X uses so much power that even with a fast charger plugged in, the battery would still go flat. Also, the phone got red hot.)
make a checklist and go through it before recording. (Background lights turned off? Microphone in position? Power plugged in?)
budget twice as long as you expect (even after taking into account that it’ll take twice as long as you expect).
make marks for reproducable camera angles.
unless there’s a rock solid paper trail for the license, avoid royalty free music sites like the plague — they simply can’t be trusted.