How I tuned the dirt removal filter, part 2

HOW THE DE-SPOT FILTER WORKS
=====================

The De-Spot filter uses several parameters to identify what is dirt and what is not. In general it works by looking for groups of pixels on an individual frame that don't exist on the preceding, or subsequent, frame. Here are the primary settings (or, at least, the only ones I used!):

p1 = sensitivity to detect new spots
p2 = sensitivity to find size/edges of spots
mthres = sensitivity to identifying dirt spots vs non-dirt spots in-motion
minpts = minimum spot-size
pwidth = maximum spot width
pheight = maximum spot height

The three 'sensitivity' settings are actually kinda misnamed as higher settings don't necessarily mean More Sensitive. P1 and P2 are the minimum 'difference' to trigger a spot detection, so higher values make it LESS likely that they'll detect a spot (or, in the case of P2, detect all of it). MTHRES is the amount of 'motion' required for a spot to be ignored, i.e., low values make the filter skip a lot of spots! The default setting for MTHRES is very cautious, 16, and misses a lot of dirt.

FYI - The function for detecting motion is called Motion Estimation (ME) and is the most important element of the filter!


KEEPING NOTES
============================

I eventually realized that I needed to keep detailed notes about specific spots/problems for specific Despot settings, and added the following section (all commented-out) to my AVS script:

------------
# FRAME # - NOTES
# 11 - hair standing-up from back of head @ mthres=43 p=24,8
# 15 - tear, requires mthres>=43 @ p=16,12
# 24 - hair 'aimed' at baby's head
# 36 - fingerprint next to (same size as) baby's head, requires p=64,8
# at p=64,6 it's only half removed, at p=64,4 it's totally missed
# 40 - round dirt
# 55 - ME error on ear @ mthres=43 p=36,12 or 56,12

# ME NOTES
# mthresh >= 20 to catch dirt at frame 77
# mthresh >= 25 to catch dirt at frame 81
# pwidth >= 120 to catch dirt at frame 112
# Motion-Estimation errors at frames 158, 179, 200, 221
# minpts >= 80 fixes half the error at frame 158
# minpts >= 110 fixes error at frame 158, 221
# minpts >= 140 fixes error at frame 200; still has error at 179
------------

TUNING THE FILTER
===============================

Here's the 'general' procedure I followed:

1. First I observed what kind of results I obtained from the default settings in De-Spot. The settings for p1, p2, and mthres are relatively insensitive, i.e., they miss a lot of obvious dirt.
2. Starting with MTHRES (detect motion) I increased the value until it detected all the legitimate dirt. At this point, however, it was also making a lot of mistakes, e.g. detecting eyes and ears as 'dirt'.
3. Within that opening scene (the dirtiest, btw) I varied the P1 and P2 settings to see if I could eliminate any of the false positives.
4. I increased the maximum allowed dirt-size, pwidth and pheight, so that the filter wouldn't ignore the bigger clods of dirt.
5. Finally I increased the minimum dirt-size, minpts, to try and eliminate false-positives on eyes/ears while still detecting actual dirt (which was usually quite large).

Rinse, Repeat! Actually, I spent most of my time varying MTHRES and P1/P2.
_______________

Here's an example of the ME error on the eye of a bouncing baby (my aunt, btw)


FINAL OUTPUT
=========================

I quickly realized that I couldn't fix everything in the whole video with just one group of settings. So I used VirtualDub to chop-up the original video into scenes, or even sub-scenes, and created custom AVS scripts for each. I also took the opportunity to adjust the brightness and contrast on several of the scenes, to try and make them easier to see.

Finally, for each scene, I output it to the FFDSHOW video codec set to MJPEG '100%' quality. I was then able to load all the clips together in Nero Vision and create a much nicer DVD than I originally received.