The bad news

I have an FPV drone: a Hubsan H107D. It’s a great little thing. It’s only got two problems. Firstly: I’m really bad at flying it; and secondly, if I tell it to record video the files are hopelessly corrupt and unreadable. I had to write a program to fix them.

So, the Hubsan 107D drone. I bought this several years ago, and it mostly sat on a shelf (which is unjust, because it’s really good, and is still really good even years later).

[Publicity shot of drone]

The only downside is that the video is recorded on the handset and is pretty grainy. If you don’t care about FPV and instead want higher-quality on-board recording, look at the H107C, which is half the price and flies the same. Be careful — the H107D+ is a completely different machine. The H107L and H107P are also apparently different, but I know nothing about them.

I took it out for a walk recently, and when I got back and looked at the SD card, I discovered to my horror that it had lost about 2/3 of the video. (Not that there was anything interesting on it other than endless shots of trees in extreme closeup, approaching rapidly.) The files were so badly mangled that most programs wouldn’t even recognise them as being AVIs.

This is what ffmpeg said about one of them:

[avi @ 0x55dc7ed593e0] Something went wrong during header parsing, tag [0][0]LI has size 1070163, I will ignore it and try to continue anyway.
[mjpeg @ 0x55dc7ed5ab40] EOI missing, emulating
Truncating packet of size 625017215 to 47092736
Input #0, avi, from '005.AVI':
  Duration: 218:28:05.87, start: 0.000000, bitrate: 0 kb/s
    Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 720x240, 60 fps, 60 tbr, 60 tbn, 60 tbc
    Stream #0:1: Audio: pcm_alaw ([6][0][0][0] / 0x0006), 8000 Hz, 1 channels, s16, 64 kb/s

It found two frames, one of which was this.

[Mangled frame of video]

Yes, well. I could tell there was more in the file, because I could see it if I looked at a hex editor. But I couldn’t find any programs, not one, which would read it.

The good news

It turns out that the H107D saves video in a really simple format (which it then mildly corrupts, loses parts of, and then attempts to save onto a filesystem which is also corrupted). MJPEG is essentially a stream of JPEG frames, one after the other.

A look at the JPEG file specification shows that JPEG files are streams of labelled chunks, and they start with a particular kind of chunk (the SOI) and end with another kind (the EOI). Looking at the AVI file with a hex editor… yup, there they are.

00010F30   35 A7 24 93  72 B5 AC D6  AE DB AB 6E  B6 F5 FB CF  5.$.r......n....
00010F40   FF D9 8B 3F  18 41 00 00  30 30 64 63  7C 40 00 00  ...?.A..00dc|@..
00010F50   FF D8 FF E0  00 15 9F 66  30 9E 00 00  00 00 00 00  .......f0.......
00010F60   00 00 00 00  00 00 00 00  00 00 00 00  00 00 00 00  ................

FF D9 ends one frame, then there’s some AVI stuff, and then the next frame starts with the FF D8.

So it should be possible to write a program that will look through the file and extract everything which looks like a JPEG file, right? I’ll get some junk but if there are any intact frames in there, it should find them, right?

The answer is yes. This works fine.

The solution

Here!

Jpegfinder's github repository

Send me pull requests!

(Probably only of use if you know your way around a C compiler, I’m afraid.)

It was bodged together in a blazing hurry and is certainly not bug free. But it worked well enough to recover a reasonable amount of video from the corrupted files. So, that’s a success.

The problem behind the solution

But it didn’t find all the video, which made me sad; I was particularly looking for the clip which culminated in me taking this photo:

[Not a publicity shot of drone]

…because it looked really weird on the controller screen.

The H107D controller apparently loses video if you turn it off or if the radio connection to the drone goes down or it’s Tuesday; this will most likely be because it forgets to update things like the file size. So it was entirely plausible that the lost video could be on the card somewhere — just not in a file.

Luckily, jpegfinder can be run against the entire SD card! (It wasn’t luck. I wrote it that way.)

The short version is — it didn’t work. jpegfinder was fine, and recovered about nine minutes of video of random crashes, including some from when I flew it years ago and then deleted the video (I obviously used the same card), but it would just crash about 20% of the way through the card.

This wasn’t a bug in my code. It was a bug in the card.

[ 4731.570892] sd 8:0:0:1: [sdg] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 4731.570896] sd 8:0:0:1: [sdg] tag#0 Sense Key : Medium Error [current] 
[ 4731.570898] sd 8:0:0:1: [sdg] tag#0 Add. Sense: Unrecovered read error
[ 4731.570901] sd 8:0:0:1: [sdg] tag#0 CDB: Read(10) 28 00 00 26 b8 80 00 00 08 00
[ 4731.570903] print_req_error: critical medium error, dev sdg, sector 2537600

The card claimed to be an 8GB card; but everything above a certain point (which varies) is inaccessible.

Trying to identify the card with hdparm says this:

/dev/sdg:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

ATA device, with non-removable media
	Model Number:       ÿÿUÀÊûÉñÿÿVÊûÉñÿÿV@ÊûÉñÿÿVÊûÉñÿÿVÀÊûÉñ
	Serial Number:      ÉñÿÿTÀÊûÉñÿÿUÊûÉñÿÿ
	Firmware Revision:  ÿÿUÊûÉñ
	Media Serial Num:   _ÊûÉñÿÿ_@ÊûÉñÿÿ_ÊûÉñÿÿ_ÀÊûÉñÿÿ`ÊûÉñÿÿ
	Media Manufacturer: `@ÊûÉñÿÿ`ÊûÉñÿÿ`ÀÊû
	Transport:          0xc9f1

Which looks… fine?

Fake SD cards are a big thing — if you buy suspiciously cheap cards off places like eBay, you’ll frequently find that a 32GB device will only store 8GB of data, because it’s really an 8GB chip with ‘32’ printed on the outside.

But I’m not sure this is one. Multiple runs tend to show the cutoff point at different places. Once it started erroring out at about the 1MB point.

I suspect it’s just old and faulty. SD cards do wear out with time. (But I’ll keep playing with it in the mean time, because it’s interesting.)

The one thing for sure is: my video is lost forever, so I’ll have to go and take more. Maybe I’ll use a card which I know works this time.