⇡The bad news
I have an FPV drone: a Hubsan H107D. It’s a great little thing. It’s only got two problems. Firstly: I’m really bad at flying it; and secondly, if I tell it to record video the files are hopelessly corrupt and unreadable. I had to write a program to fix them.
So, the Hubsan 107D drone. I bought this several years ago, and it mostly sat on a shelf (which is unjust, because it’s really good, and is still really good even years later).
The only downside is that the video is recorded on the handset and is pretty grainy. If you don’t care about FPV and instead want higher-quality on-board recording, look at the H107C, which is half the price and flies the same. Be careful — the H107D+ is a completely different machine. The H107L and H107P are also apparently different, but I know nothing about them.
I took it out for a walk recently, and when I got back and looked at the SD card, I discovered to my horror that it had lost about 2/3 of the video. (Not that there was anything interesting on it other than endless shots of trees in extreme closeup, approaching rapidly.) The files were so badly mangled that most programs wouldn’t even recognise them as being AVIs.
This is what ffmpeg said about one of them:
[avi @ 0x55dc7ed593e0] Something went wrong during header parsing, tag LI has size 1070163, I will ignore it and try to continue anyway. [mjpeg @ 0x55dc7ed5ab40] EOI missing, emulating Truncating packet of size 625017215 to 47092736 Input #0, avi, from '005.AVI': Duration: 218:28:05.87, start: 0.000000, bitrate: 0 kb/s Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 720x240, 60 fps, 60 tbr, 60 tbn, 60 tbc Stream #0:1: Audio: pcm_alaw ( / 0x0006), 8000 Hz, 1 channels, s16, 64 kb/s
It found two frames, one of which was this.
Yes, well. I could tell there was more in the file, because I could see it if I looked at a hex editor. But I couldn’t find any programs, not one, which would read it.
⇡The good news
It turns out that the H107D saves video in a really simple format (which it then mildly corrupts, loses parts of, and then attempts to save onto a filesystem which is also corrupted). MJPEG is essentially a stream of JPEG frames, one after the other.
A look at the JPEG file specification shows that JPEG files are streams of labelled chunks, and they start with a particular kind of chunk (the SOI) and end with another kind (the EOI). Looking at the AVI file with a hex editor… yup, there they are.
00010F30 35 A7 24 93 72 B5 AC D6 AE DB AB 6E B6 F5 FB CF 5.$.r......n.... 00010F40 FF D9 8B 3F 18 41 00 00 30 30 64 63 7C 40 00 00 ...?.A..00dc|@.. 00010F50 FF D8 FF E0 00 15 9F 66 30 9E 00 00 00 00 00 00 .......f0....... 00010F60 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
FF D9 ends one frame, then there’s some AVI stuff, and then the
next frame starts with the
So it should be possible to write a program that will look through the file and extract everything which looks like a JPEG file, right? I’ll get some junk but if there are any intact frames in there, it should find them, right?
The answer is yes. This works fine.
(Probably only of use if you know your way around a C compiler, I’m afraid.)
It was bodged together in a blazing hurry and is certainly not bug free. But it worked well enough to recover a reasonable amount of video from the corrupted files. So, that’s a success.
⇡The problem behind the solution
But it didn’t find all the video, which made me sad; I was particularly looking for the clip which culminated in me taking this photo:
…because it looked really weird on the controller screen.
The H107D controller apparently loses video if you turn it off or if the radio connection to the drone goes down or it’s Tuesday; this will most likely be because it forgets to update things like the file size. So it was entirely plausible that the lost video could be on the card somewhere — just not in a file.
Luckily, jpegfinder can be run against the entire SD card! (It wasn’t luck. I wrote it that way.)
The short version is — it didn’t work. jpegfinder was fine, and recovered about nine minutes of video of random crashes, including some from when I flew it years ago and then deleted the video (I obviously used the same card), but it would just crash about 20% of the way through the card.
This wasn’t a bug in my code. It was a bug in the card.
[ 4731.570892] sd 8:0:0:1: [sdg] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 4731.570896] sd 8:0:0:1: [sdg] tag#0 Sense Key : Medium Error [current] [ 4731.570898] sd 8:0:0:1: [sdg] tag#0 Add. Sense: Unrecovered read error [ 4731.570901] sd 8:0:0:1: [sdg] tag#0 CDB: Read(10) 28 00 00 26 b8 80 00 00 08 00 [ 4731.570903] print_req_error: critical medium error, dev sdg, sector 2537600
The card claimed to be an 8GB card; but everything above a certain point (which varies) is inaccessible.
Trying to identify the card with hdparm says this:
/dev/sdg: SG_IO: bad/missing sense data, sb: 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ATA device, with non-removable media Model Number: ÿÿUÀÊûÉñÿÿVÊûÉñÿÿV@ÊûÉñÿÿVÊûÉñÿÿVÀÊûÉñ Serial Number: ÉñÿÿTÀÊûÉñÿÿUÊûÉñÿÿ Firmware Revision: ÿÿUÊûÉñ Media Serial Num: _ÊûÉñÿÿ_@ÊûÉñÿÿ_ÊûÉñÿÿ_ÀÊûÉñÿÿ`ÊûÉñÿÿ Media Manufacturer: `@ÊûÉñÿÿ`ÊûÉñÿÿ`ÀÊû Transport: 0xc9f1
Which looks… fine?
Fake SD cards are a big thing — if you buy suspiciously cheap cards off places like eBay, you’ll frequently find that a 32GB device will only store 8GB of data, because it’s really an 8GB chip with ‘32’ printed on the outside.
But I’m not sure this is one. Multiple runs tend to show the cutoff point at different places. Once it started erroring out at about the 1MB point.
I suspect it’s just old and faulty. SD cards do wear out with time. (But I’ll keep playing with it in the mean time, because it’s interesting.)
The one thing for sure is: my video is lost forever, so I’ll have to go and take more. Maybe I’ll use a card which I know works this time.