Update on Canon MP620 and Ubuntu 9.10

To be perfectly honest, I'm seriously considering dumping Ubuntu. Much as I love it, I just KEEP HAVING stupid problems -- USB devices working/then not working, and now my printer failed again.

Fortunately, after JUST a couple hours of "having fun" with my Canon linux drivers I've managed to get everything working again. I've previously (Dec 2008) blogged on this and this is now an update,
http://technicaltony.blogspot.com/2008/12/installing-canon-mp620-on-ubuntu.html
__________

Ever since my 9.04 > 9.10 upgrade I'd lost the ability to print from the rear-tray (as have many others apparently). What a ridiculous problem to be having.

Here was the solution:

1. download and install the latest Printer Definition file (PPD)
http://sourceforge.net/project/showfiles.php?group_id=210977
Download and extract 'ppdMP620-630en-1.5.tar.gz'

2. Follow the included README instructions and copy the PPD file to the system's PPD folder,
> sudo cp canonmp620-630en.ppd /usr/share/ppd/

3. Copy the CONF file to the system's Canon library,
> sudo cp cifmp610.conf /usr/lib/bjlib/

4. download and install the latest "IJ Common" DEB,
http://support-sg.canon-asia.com/contents/SG/EN/0100159004.html
> sudo dpkg -i cnijfilter-common_3.00-1_i386.deb

*** Don't forget there are other DEBs required as explained in my original post ...

5. Delete the existing printer object from System > Administration > Printing

6. Reboot! Not sure if that's totally necessary but it works. This ensures that the new drivers are loaded, and the printer should be auto-detected at login.

If you now check the Properties of the recreated printer it should say:
Make and Model: Canon MP620-MP630

WARNING! INCOMPATIBLE MEDIA TYPES!

Again, this is probably something that would never happen under Windows...

The final step -- for printing photos, at least -- is to select Media Type: Glossy Photo Paper. If you try the other options, e.g. "Photo Paper Pro", the computer will say it's processing and the printer will momentarily say it's receiving and then.... Nothing. The print job will be unceremoniously canceled without any warning or error.
__________

Obama's next State Of The Union

I just had a quick thought:
President Obama's next State Of The Union speech is going to be crazy popular!

At least, it will be VERY widely watched, probably the most in history. I'm thinking of the domestic audience but this will probably be true internationally, too.

Think of all the drama surrounding him now:
- The country has been teetering on the verge of a new Great Depression
- He's on the verge of 'reform' of the byzantine American health insurance system
- The Right Wing absolutely viscerally hates him
- And he's widely acknowledged, even by his detractors, as a great speaker.

I'll have to get out the popcorn, and my score card! Now, when is this all scheduled? His first State Of The Union speech was in February 2009, just after he'd taken office, so he hadn't had a chance to really dig in yet.
___________

Using AHCI to install XP on new computers

I have a retail copy of Windows XP Home (Service Pack 2) which I use to run my home-made Home Theatre PC (HTPC). With my latest build I ran into a new problem -- it wouldn't install! Instead I kept getting a BSOD. Eventually I realized that the new motherboard probably wasn't recognized by the old (circa 2005?) OS. Fortunately, there was a solution, though it took me a while to make it work.

I knew that my new motherboard AND my new SATA drives were configured for the newer 'AHCI' drive-mode. The existing SATA drivers on my XP-SP2 disc didn't support AHCI. So my first thought was to use the F6 boot-option to include the new drivers. But despite attempts to load various versions of the 'Intel Storage Matrix Manager' drivers it continued to either hang or BSOD.

Ultimately the solution was to slipstream the drivers into the CD image. While I was at it, I also added the SP3 update. (Actually, the first thing I tried was slipstreaming SP3 in the hopes that it included the AHCI support; it didn't.) I already knew how to slipstream SP3 but had no idea how to added the Intel drivers. Fortunately, this web page explained how to use a nifty utility called 'nLite" which automated it.
www.howtogeek.com/howto/windows/resolving-setup-did-not-find-any-hard-disk-drives-during-windows-xp-installation/

So, in summary, what did not work:
- using WinXP Service Pack 3
- using F6 with Intel drivers on a floppy

What did work:
- using nLite to slipstream both SP3 and the Intel drivers onto a new CD
___________________

Bad news is 'good' for prenatal DS test?

Last June 2008 I mentioned a new prenatal test for Down Syndrome that could potentially eliminate new DS births. Now, the newspaper says that the company, Sequenom, probably faked the research tests.
http://www3.signonsandiego.com/stories/2009/sep/30/sense-anxiety-sends-sequenom-stock-down-39

The CEO, CFO, and two VPs of Research have all been fired, and the company stock has dropped by nearly half. They still insist the test works but admit that they need to start from scratch in proving it. So that's at least another few years before it ever comes to market (if at all).

I'm not sure how to feel about this, still. Yes, I think it's a good idea to have such a test. But I also know that over 90% of expectant parents choose to abort if/when they learn that their unborn child has DS. Hopefully, continuing research will find a way to significantly offset the negative effects of DS and then there won't be so much concern about it. Wouldn't it be wonderful if DS could simply be considered a treatable condition, like diabetes?
_________________________

Turning the defamatory tables on Glenn Beck

Glenn Beck is the latest right-wing demagogue to attack the 'left' and has been particularly nasty in comparing Barack Obama with Adolf Hitler, as well as other assorted absurdities. As with most of these right-wing attacks, it doesn't matter if there's any truth to the attack -- it's merely mud-slinging. Now, someone has turned the tables on him by using the Internet to sling mud back at him.

My new favorite web-site is glennbeckrapedandmurderedayounggirlin1990.com. No one seriously thinks he ever did any such thing but the site is starting to turn up in web searches for his name, potentially casting a defamatory pall over him. Frankly, I think he deserves it. Live by the sword, die by the sword.

A further irony is that his success is entirely dependent on his abuse of the First Amendment, to protect his hate speech. Now this web-site is practicing a similar form of hate speech but he's suing them, claiming that it's not protected.

Here's a short discussion of the case from Political Irony.com
http://politicalirony.com/2009/09/08/did-glenn-beck-murder-httpglennbeckrapedandmurderedayounggirlin1990-com/
_________________

TV schedule for UCSD Thornton hospital

For some reason I can't seem to find a valid TV schedule for the hospital here. I've been staying here with my mom for the past week and I'm starting to watch more and more TV, so I'd like to know what's on. Currently, I'm stuck just channel surfing.

I've tried doing some Internet searches but the best I've been able to come up with are some old (but still cached) hits for a "erctv" page on the campus' UCSD.edu web-site, now gone. Here's the channel listing I found, and which seems to be correct:

CH NAME	CH NAME
07 NBC	37 ESPN
08 CBS	38 ESPN2
09 IND	39 TLC
10 ABC	40 HBO Family
11 PBS	41 HBO West
12 XEWT (Tijuana)	43 HBO
13 HDH Info	44 MTVU
18 SRTV	45 Comedy Central
19 All College TV	46 TBS
20 ERC TV	47 WGN
21 CW	48 Cartoon Network
22 UCSD TV	49 BET
23 FOX	50 MTV2
24 History	51 VH1
25 Univision	52 Food Network
26 MyNetworkTV (primetime only)	53 Travel Channel
27 FX	54 E!
28 Lifetime	55 Spike
29 Nickelodeon	56 AMC
30 MTV	58 SciFi
31 CNN	59 Discovery
32 Headline News	60 Fox Sports
33 TNT	61 Current TV
34 Fox News	62 USA
35 BBC	63 Sundance
36 A&E

_____________

Cleaning up the Obama-rhea of the No We Can't crowd

We need a new 'meme' to describe the rabid anti-Obama crowd. I got the idea from reading Roger Ebert's excellent blog entry regarding memes and the anti healthcare reform crowd,
http://blogs.suntimes.com/ebert/2009/08/death_panels_an_excellent_phra.html

The general idea is that certain phrases are so perfectly self-encapsulated as to gain a life of their own, independent of whether or not they actually mean anything. The latest example is "Death Panels" which will probably live on and define the argument forever. I really think we need to counter it with a comparable meme to describe and undermine the wackos who are pushing "Panels". And it probably goes without saying that it needs to be a meme with the name "Obama" as part of it. Hmm...

Clearly, there is a die-hard group of people who believe that President Obama is the anti-christ and who vehemently and reflexively oppose anything he supports. I suspect there's unacknowledged racism at work here but I don't know if there's any point trying to pin that on them. Still, my first idea for a meme to describe them is "The Obama Lynch Mob". Or, if that's too blunt, maybe call them "The Obama Mob"

Alternatively, maybe we should label the reaction rather than the people? Maybe "Obama Vomit" or "Obama (Gag) Reflex" ? Even better: "Obama-rhea" oooh yeah, that's juicy! (eww) On second thought, though, diarrhea is something that's justified, as if his policies are so unhealthy as to be automatically rejected by the Body Politic (gotta love this analogy though :- )

Obama-spite. Whiners. Gotta think of another analogy... Something tantamount to children and temper tantrums? How about "Anti-Hope Tantrum"? Or, the "No We Can't" crowd? Since these people frequently claim that Obama is part of 'secret' conspiracy maybe they should be diagnosed with "Obama-noia".
_____________

End The Federal Budget Shellgame!

Here's a modest proposal:
Forbid federal agencies from 'purchasing' each others' debts!

The BIG BIG BIG one is the so-called Social Security "Lockbox". All those trillions of dollars of 'savings' have really just been used to prop-up our deficit spending, through the SSA 'purchasing' Treasury bills. Instead, I propose they be required to only purchase home loans. In this way, the "lockbox" would actually own tangible assets and would also provide a steady market for home mortgages.
____________

Another great parody - about 'public' health insurance

I know, I know, this is turning into a lazy-man's blog but this Salon.com parody is just so spot on,
http://www.salon.com/opinion/feature/2009/06/30/police/

Obama has said, if he had the chance to do it all from scratch he'd setup a single-payer system of healthcare. So why is he caving-in to the entrenched private insurers who are doing such a terrible job? Would anyone seriously question the need for universal -- publicly paid-for -- police service?

Here's something unique I can add to the discussion: Fire protection used to be privately funded, but everyone now agrees that it is better as a public service. How is medicine any different?

UPDATE: I wanted to make sure I wasn't blowing smoke about the private fire depts. I found some other recent articles making the same comparison,
http://open.salon.com/blog/bill_beck/2009/06/21/better_burnt_than_red
http://theamericanscene.com/2007/10/26/private-firefighters

Great parody of anti-Gay Marriage 'arguments'

Modern Day Snake Oil

My mother is suffering from cancer, unfortunately, and now this otherwise rational and intelligent woman is succumbing to the modern-day equivalent of snake-oil salesmen, hoping to find a cure in the field of "Alternative Medicine." Today my mom went to a 'doctor' who sold her $500 in dietary supplements that -- apparently -- are all going to cure her cancer! I'm glad she invited me along for her 'appointment' as I got a first-hand look at how the scam works.

CON MEN ARE ALL ALIKE

First of all, the guy showed-up in practically a full surgical outfit even though we were only meeting in a small strip-mall office suite. Very 'professional' looking! He then proceeded to talk-up a STORM full of medical and biological terminology. It was impossible to consider or evaluate any of his statements because he JUST KEPT TALKING. Regardless of the forum, I can tell when I'm being manipulated in this way. I started-out wanting to like him (even as I knew I wasn't likely going to agree with him) and, in fact, he seemed very earnest and honest, like he really believed his own schtick. But after TWO HOURS I had to bite my tongue lest I threaten to report him for fraud.

Note -- He was VERY careful not to say that any of these treatments would cure her cancer! So there was nothing I could report him for. (dangle...) In this way he was obviously very practiced at his sales pitch. He also gave several anecdotes of people he had cured -- but without mentioning if they were also receiving traditional cancer treatments.

I'm not actually saying he was a con man. He may very well have believed all that crap he spouted. But if that's the case he needs to realize that the high-pressure sales pitch is only going to work on gullible or overly-polite people, while alienating everyone else.

TRY PULLING *MY* FINGER, DUDE!

Since my mom has an ACTUAL VERIFIABLE ILLNESS, I had assumed we were merely there to discuss and recommend possible treatments. But, no, he also gave my mom an on-the-spot 'examination'. His entire diagnostic technique consisted of -- and I kid you not, you can't make this stuff up -- PULLING HER FINGER. How unscientific is this?? Oh, let me count the ways! He would ask my mom's "body" if it "wanted" a particular supplement, then pull her finger, and based on the resistance he claimed to feel, would tell my mom if it was a good treatment or not. Even assuming he believed all this (which I doubt) my mom could vary her response for both conscious and unconscious reasons, and he could vary the strength of his 'test' pull in the same way. As I watched this charade I started imagining tricks I could play on him, such as swapping his 'test vials' so that they were all the same -- hence impossible to use for a valid test. I knew that his 'results' would not change if I did.

The worst part for me, actually, was when I thought of a dastardly joke we could play on him, and then I had trouble not laughing. Since the 'doctor' insisting on pulling her finger, what if I could've planted a gag Fart Maker? Every time he pulled your finger --- pffft. "Sorry!" I wonder how long he could continue under those conditions :-)

MORE TRICKS OF THE TRADE

I noticed that he did a good job of 'eliminating' several 'unnecessary' treatments. In other words, he made a good show of not trying to sell her every possible pill. I also noticed that he correctly 'diagnosed' all the various ailments my mom had already mentioned to him. Kinda reminded me of TV fake John Edwards' use of undercover agents to interview the crowd before each show, to find-out what secret they were hoping he could divine.

For each ailment he would quickly explain a highly technical reason why this or that biochemical was 'off' and how this or that supplement would fix it. Of course, the human body is far more complicated than that. As my cousin once told me, "If your body needs more zinc, you can't just eat more of it. Your digestive tract will simply 'pass' most of it." In the same vein (ha ha) I have recently read several articles about once-promising drug treatments which, for reasons unknown, failed to work outside the lab. And that's for legitimate pharmaceuticals, not fly-by-night 'homeopathic' remedies that make no actual medical claims.

DOCTOR QUACK'S BACKGROUND

I had to wonder if this guy was even a legitimate doctor. His wall was festooned with certificates and degrees, and so I set about to make a study of them. Here's a quick summary:

1974 - BS from a christian college in Texas
1976 - MD from Baylor Medical in Houston
1977 - license in Oregon
1980-1 - certificate of appreciation for "Management" of a Texas hospital or something
1983 - license in Wyoming
2003 - license in CA from "Medical Board of CA"

There were various others which I've forgotten, but the one that really got my attention was the 2nd. Is it possible to get a medical degree in just 2 years? When I got home afterwards I googled "Baylor" as "Medical Board of CA", just to make sure they were real. I also looked him up on both the CA and TX medical boards' web-sites, and everything looked legit (and there were no actions against him).

WHY I'M NOT WORRIED ABOUT HER (MUCH...)

So maybe he really does believe all that crap he spouted? I hope so. Fortunately, my mom also has a team of doctors at UCSD who claim to be open to alternative medicine -- but still want to review her supplements, just "to make sure none will react" with her ACTUAL treatment. According to doctor snake-oil, "none of these treatments interact with each other." In other words, they're probably all placebos! Wasn't there such an expose a few years ago about the "supplements" industry?

P.S. I can't entirely blame the cancer for her loss of reason. She was already under the thrall of her "bio-energetisist" whom I've learned she pays several thousand dollars a year for monthly treatments. But at least he also performs a legitimate service -- physical therapy -- as part of his fee.

P.P.S. Despite my harsh negative view of this 'doctor', I'm not doctrinarily opposed to Alternative Medicine. I just want it to prove it works! Is that so much to ask? Maybe the foods we eat, and the plastics we use, really do cause cancer through some as-yet-unknown mechanism. I have no doubt that today's "modern" medicine will seem like medieval torture in the near future. But that doesn't excuse all the unscientific unverifiable unBELIEVABLE claims of all those supplements my mom just paid top-dollar for. (dangle dangle...)

WSJ bags on home ownership as an 'investment'

Even the supposed cheerleaders of the 'mainstream' economic-view are now starting to dish on homeownership. They point out that homes-as-investment have only returned at most 2.2% (after accounting for inflation) over the past decade.
http://finance.yahoo.com/real-estate/article/107122/Is-Your-Home-a-Good-Investment?

I'm just sayin' ....

How I tuned the dirt removal filter, part 2

HOW THE DE-SPOT FILTER WORKS
=====================

The De-Spot filter uses several parameters to identify what is dirt and what is not. In general it works by looking for groups of pixels on an individual frame that don't exist on the preceding, or subsequent, frame. Here are the primary settings (or, at least, the only ones I used!):

p1 = sensitivity to detect new spots
p2 = sensitivity to find size/edges of spots
mthres = sensitivity to identifying dirt spots vs non-dirt spots in-motion
minpts = minimum spot-size
pwidth = maximum spot width
pheight = maximum spot height

The three 'sensitivity' settings are actually kinda misnamed as higher settings don't necessarily mean More Sensitive. P1 and P2 are the minimum 'difference' to trigger a spot detection, so higher values make it LESS likely that they'll detect a spot (or, in the case of P2, detect all of it). MTHRES is the amount of 'motion' required for a spot to be ignored, i.e., low values make the filter skip a lot of spots! The default setting for MTHRES is very cautious, 16, and misses a lot of dirt.

FYI - The function for detecting motion is called Motion Estimation (ME) and is the most important element of the filter!


KEEPING NOTES
============================

I eventually realized that I needed to keep detailed notes about specific spots/problems for specific Despot settings, and added the following section (all commented-out) to my AVS script:

------------
# FRAME # - NOTES
# 11 - hair standing-up from back of head @ mthres=43 p=24,8
# 15 - tear, requires mthres>=43 @ p=16,12
# 24 - hair 'aimed' at baby's head
# 36 - fingerprint next to (same size as) baby's head, requires p=64,8
# at p=64,6 it's only half removed, at p=64,4 it's totally missed
# 40 - round dirt
# 55 - ME error on ear @ mthres=43 p=36,12 or 56,12

# ME NOTES
# mthresh >= 20 to catch dirt at frame 77
# mthresh >= 25 to catch dirt at frame 81
# pwidth >= 120 to catch dirt at frame 112
# Motion-Estimation errors at frames 158, 179, 200, 221
# minpts >= 80 fixes half the error at frame 158
# minpts >= 110 fixes error at frame 158, 221
# minpts >= 140 fixes error at frame 200; still has error at 179
------------

TUNING THE FILTER
===============================

Here's the 'general' procedure I followed:

1. First I observed what kind of results I obtained from the default settings in De-Spot. The settings for p1, p2, and mthres are relatively insensitive, i.e., they miss a lot of obvious dirt.
2. Starting with MTHRES (detect motion) I increased the value until it detected all the legitimate dirt. At this point, however, it was also making a lot of mistakes, e.g. detecting eyes and ears as 'dirt'.
3. Within that opening scene (the dirtiest, btw) I varied the P1 and P2 settings to see if I could eliminate any of the false positives.
4. I increased the maximum allowed dirt-size, pwidth and pheight, so that the filter wouldn't ignore the bigger clods of dirt.
5. Finally I increased the minimum dirt-size, minpts, to try and eliminate false-positives on eyes/ears while still detecting actual dirt (which was usually quite large).

Rinse, Repeat! Actually, I spent most of my time varying MTHRES and P1/P2.
_______________

Here's an example of the ME error on the eye of a bouncing baby (my aunt, btw)


FINAL OUTPUT
=========================

I quickly realized that I couldn't fix everything in the whole video with just one group of settings. So I used VirtualDub to chop-up the original video into scenes, or even sub-scenes, and created custom AVS scripts for each. I also took the opportunity to adjust the brightness and contrast on several of the scenes, to try and make them easier to see.

Finally, for each scene, I output it to the FFDSHOW video codec set to MJPEG '100%' quality. I was then able to load all the clips together in Nero Vision and create a much nicer DVD than I originally received.

How to remove dirt from old film transfers, part 1

BACKGROUND
=====================

I just finished a very exciting project: My mom located a really old family film from when she was a child back during the Great Depression! The film is absolutely one-of-a-kind and had become sort of a legend to me, as I'd heard that it existed but no one knew what had happened to it. Well, my mom found it several months ago and I secretly had it transferred to digital, in preparation for Mother's Day. This was not my first foray into the realm of digital film transfers and I wanted to do the best possible transfer of this priceless recording. And because it was on 16mm film I was hopeful that it would have significantly better picture quality than the less-old 8mm films I'd previously converted.

Now, there are lots of shops offering film-to-DVD transfer. But I've spent literally years searching for someone who could perform film-to-BluRay transfers, i.e., to capture the full picture quality, in high-definition. I finally located this shop:

Video Conversion Experts
1010 W Chandler Heights Rd, Ste 2
Chandler, AZ 85248

www.videoconversionexperts.com
(800) 575-6202

For my little 11-minute film it cost $200 to transfer it to digital (at high-resolution) and create both a Bluray and DVD version. If I'd known all the work I was going to put into fixing the file I could've saved myself about half the cost and just requested the raw files; this is what ended-up happening anyways.

(Actually, I just checked and they charge MORE for just giving you the raw files. Apparently they're charging you for the resulting picture quality rather than the amount of work required on their part.)


THE ORIGINAL TRANSFER
=====================

Alas, the film showed all the classic signs of old age. First of all, whoever owned the camera (which was surely a luxury back in the 1930's) didn't do a very good job with the settings. Most of the scenes were either over-exposed or under-exposed. And one segment of Christmas 1936 was out of focus for the entire time. Even worse, the film did not have the high-resolution of detail that I'd hoped for.

Most upsetting, however, was the amount of dirt on the film, especially in the first 30 seconds or so. I'd assumed that the film-transfer machines were capable of detecting and 'deleting' these problems. Instead, removing this dirt from the image was what inspired me to fix the entire film.

Actually, I wanted to fix the transfer and the DVD, as the version they sent me was accidentally recorded in "2-hr" mode (only 5 Mbps, or half the max) and with noticeable distortion. Since it was such a short piece of film there was no reason not to have recorded it at maximum quality, 10 Mbps or "1-hr" mode in their parlance. They also billed me unnecessarily for audio transfer (there was none). Rather than ask for a refund I asked them to just send me the original raw high-definition files. They described them as "HD-AVI" and so I was curious to see what they meant by that. Basically, their film-transfer machine outputs 1920x1080i HDV (M-JPEG). I did some quick tests and found that the compression was set at 85%, a reasonable setting but one that I ignored during my subsequent editing.


THE SOFTWARE I USED
=====================

• VirtualDub 1.8.8
  www.virtualdub.org
• AviSynth 2.5.6
  www.avisynth.org
• De-Spot filter for Avisynth
  avisynth.org.ru/despot/despot.html
• FFDshow
  www.videohelp.com/tools/ffdshow

VirtualDub is a general-purpose video editor and can be used as-is without any installation. AviSynth is a frame-server which has to be installed, and then the De-Spot filter operates as a a DLL which has to be copied into AviSynth's 'Plugin' sub-folder. Finally, I installed FFDshow and set it's MJPEG filter to record at 100% quality (effectively loss-less).

Note - All of these programs are open-source software, i.e., they cost nothing to use!

The one commercial program I used was Nero Vision (from the Nero Ultra Suite) to create a new DVD with menu and soundtrack.


THE GENERAL PROCEDURE
=====================

The first thing I did was identify what format everything was recorded-in. I then used VirtualDub to demux the original AVI (separate the audio and video streams). Even with 85% quality/compression the resulting video file was 6 GB (for 11 minutes!). Finally, I setup an initial AVS script, both to trim the black segments on left/right (the film had an aspect ratio of 4:3 whereas the 'high definition' transfer was done at 16:9 widescreen) and also to start experimenting with the Despot settings.

Here's what my AVS script looked like for the first segment:
(subsequent segments were similar with only slight adjustments to the Despot parameters)

------------
# REMOVE DIRT FROM 1ST SCENE (FRAME 1-64)

AVIsource("16mm_HDAVI.avi")

# SelectEven() to reduce frame-rate by half
SelectEven()
# Crop(Left-Pos,Top-Pos,Right-Size,Bottom-Size)
Crop(250,0,1440,0)

# run 'despot' plugin for old film
LoadPlugin("c:\program files\avisynth 2.5\plugins\despot.dll")
ConvertToYV12()

## Default settings for p1, p2, and mthres
#DeSpot(p1=24, p2=12, mthres=16, sign=1, minpts=10, interlaced=false)
# p1=sensitivity for detecting new dirt spots
# p2=sensitivity for finding the edges of dirt spots
# mthres=sensitivity for differentiating dirt spots from spots in-motion
# include 'show=1' for diagnostic mode

DeSpot(p1=64, p2=8, pwidth=540, pheight=300, minpts=80, mthres=43, show=1)
ConvertToRGB24
------------

I was then able to open the .AVS file directly in Virtualdub and immediately see the effect. Note the use of 'show=1' in the Despot parameters -- this produced a very cool diagnostic effect! Instead of showing the final result, the diagnostic mode shows you a colored-in version where you can see which parts of the image look like 'dirt' but are being left-alone because of Motion Estimation (shown in gray) versus which parts will actually be removed (shown in purple).


One question I had was about removing the duplicate frames. The original file was 1080i 29.97 fps and every other frame was a duplicate (which interfered with detecting dirt on individual frames!). I used the 'SelectEven' command in my script and this removed the duplicates while also reducing the frame-rate to 14.985. Most importantly, it didn't seem to reduce the resolution. I had read that M-JPEG was recorded in interlaced format and so I was afraid that SelectEven might result in a 1920x540 image; it didn't.

Whew! This is by far my longest blog entry to-date. So I think I'll take a break, make this a two-parter. In my next blog entry I'll detail the method I used to tune the de-spot filter. It was as much an art as a science!

There is no Social Security Trust Fund

Here's a quick follow-up to my previous blog re non-existent Social Security Trust Fund. A recent Atlantic Monthly article pointed-out that the "trust fund" actually consists of nothing more than a filing cabinet full of (worthless) Treasury Bonds.

Which led me to this 2005 story in USA Today,
http://www.usatoday.com/news/washington/2005-04-05-trust-fund_x.htm

I understand that the government faces a difficult situation:
How do you 'bank' or otherwise invest the excess income (which is really just a loan) from Social Security? 'Investing' in the government's own bonds, however, seems misleading -- if not illegal or even treasonous!

I wish there were a way of forcing the federal government to:

1. Stop buying Treasury bonds
2. Remove the SS overpayments from the federal budget, i.e., so they weren't being used to paper-over deficit spending

Maybe they could use the SS money to buy gold or oil reserves, or something else that would represent a long-term (>20 years) investment?

Tirade against Federal Budget and Bank Bail-outs

I'm so tired of hearing all this pointless drivel about the economy and the banks and new Stimulus bill. The basic fact that never seems to get stated is this:
The banks lost our money.

So what do we do about it? So far it looks like the only politically expedient solution is to 'print' more money, devaluing our remaining money in the process. But there are other solutions and there's also more back-story to how we got here.

The really important issue that's being ignored is the so-called -- and NON EXISTENT -- Social Security Trust Fund. For decades now the government has been using the excess income from social security withholding to paper-over our deficit spending. Now that money is going to come due and IT'S GONE. Printing more money could 'fix' this problem, too, but only if we stop cost of living adjustments (COLA) and thereby screw all the people who are expecting SS to support them.

Frankly, part of me isn't sympathetic. I mean, all the baby boomers who are about to get screwed by SS are the same people who've been benefiting from artificially low government deficits -- yes, LOW, because otherwise the deficit would have swelled long ago, and taxes would have had to have been raised. So why should I now have to pay all those 'back taxes' ?

Back to the first issue re banks, why isn't anyone talking about a 'windfall profits' tax on real-estate speculators? These are the people who sold houses at the top of the housing bubble (in 2005/06) but did not turn around and reinvest those funds in another house. This is where all "the bank's" money went! It should be easy enough to find these people based on their tax returns. I don't mean to be punitive, in fact, I congratulate them on their financial acumen (and luck). But isn't this the very definition of "windfall profits", i.e., profit at the unreasonable expense of others?

Since the FDIC is totally gonna run out of money I say we recover some of that lost money and directly funnel it back to the original 'owners' via FDIC bailouts of personal savings.
.

Why Renting Beats Buying A House

Did I write this while I was asleep? It sounds exactly like something I would say :-)
http://realestate.yahoo.com/promo/5-reasons-renting-still-beats-buying.html

Basic premise: Home prices never change!
When accounted for inflation and expenses the (financial) value of owning a home is a wash. So treating it like an investment is foolish. For hundreds of years home prices have been a constant-function of the cost of owning them, no more, no less. And the housing bubble of the past 8 years was totally predictable, and has yet to finish deflating.

The author also makes note of this Atlantic article which envisions less future-need for home ownership,
http://www.theatlantic.com/doc/200903/meltdown-geography
.

Circuit City Liquidated at Full Price, amazing!

My hat goes off to the company that liquidated the old Circuit City stores. I've been dropping by every couple of weeks, waiting to find a deal on any of several pieces of equipment. And when I heard that this was the last week I thought My Time had finally come. I'd even prepared for this by signing-up for a new VISA card, one that included extended warranty coverage -- i.e., I could get an extended warranty regardless of where I bought something.

But I just went by my local Circuit City and almost everything is already gone! And the few remaining pieces are still full price?! By "full" I mean typical street price, not the fictitious "suggested retail" price. And half the remaining stuff is demo units, too. Who's buying this stuff? Must be P.T. Barnum's friends....
.

More fuel for anti-Mortgage Deduction fire

Apparently I'm not the only person who's still worked-up about this. Yesterday's NY Times had an essay calling for the gradual elimination of the mortgage deduction,
http://economix.blogs.nytimes.com/2009/02/24/killing-or-maiming-a-sacred-cow-home-mortgage-deductions/
.

The Mortgage Interest Deduction has to go!

I have been waiting for years, literally, to buy a house. My wife and I even made offers on a couple of houses back in 2003 (we were outbid, no surprise) before deciding "this market is a bubble" and waiting it out. Now, we're house hunting again and I've been consumed with the idea that prices got oh so high in part because of the home-mortgage interest deduction. In general, I think the deduction has been 'priced-in' to the cost of houses so it doesn't really save anyone money. But when I tried searching for the history of the deduction I found that the issue was much more complicated than just that.

A great source of information is this 2006 NY Times article,
http://www.nytimes.com/2006/03/05/magazine/305deduction.1.html?pagewanted=print

Basically, there has been an allowed deduction for 'interest' since the beginning of time (or, at least, the introduction of the American income tax). Until 1986 you could even deduct your credit card interest. Surprisingly, the mortgage deduction is only used by 1/3rd of homeowners? The presumption is that most people don't itemize their deductions, hence they can't claim their interest payments.

Much as I disagreed with him on most issues, I gave kudos to GW Bush for pushing 'tax simplification'. The panel he empaneled even proposed doing-away with the deduction but GW refused to support it, knowing what a political hot-potato it was.

In 2005 there was a widely discussed Slate.com editorial supporting the idea,
http://www.slate.com/id/2130017/
Clearly, there's already been a lot of discussion about the why's so I'll just let those links speak for themselves.

The one thing I wanted to add to the discussion was this suggestion:
Gradually phase-out the deduction at a rate linked to inflation.

In other words, existing homeowners would continue to receive the same deduction they originally expected (and no doubt factored into their purchasing decision). But the deduction would go away as the time-cost of their money also reduced their mortgage cost. So these existing homeowners wouldn't 'get ahead' like they'd expected. The whole point of this (beside 'tax simplification') is that new buyers like myself would get better prices on houses.
.

Finally, a dual-cpu HD video player

The new DivX v7 is the first video player (that I know of) to support dual-core CPUs. This is a major development as all previous software was unable to coordinate both halves of a now-typical dual-core. So you had to have a really fast CPU to play high-definition videos especially if they were encoded in the new h.264 format. Now, with DivX 7 you can manage it with a much slower dual-core, i.e., with older machines.

Unfortunately, this is only available on Windows (not Linux).

I've done some quick testing and so far the program lives-up to it's claims! I was able to play a 1080p h.264 clip on a dual-core AMD Turion 1.7 GHz, which is about a 3 year-old model. With all previous programs this processor could only play it at half-speed. Now, it can just barely play it at full-speed, and only if the video is full-screen (not windowed) and without anything else displayed on screen such as the mouse pointer.

One problem, however, is that it does not appear to include any deinterlacing. So even though it's fast enough to play 1080i AVCHD clips they look terrible because of the interlacing artifacts.
.

Further info on playing AVCHD

I've had a chance to test this on several computers now and can make the following clarifications and improvements to the suggested procedure.

NOTES

• You need at least a ~2.4 GHz processor. It doesn't playback correctly on my 1.9 GHz whereas it plays-back with spare cpu cycles on my 2.7 GHz AMD X2, as well as on a 2.7 GHz Intel Core2.
• The existing 0.9.4 version (in the standard repository) plays AVCHD just as good as the newer "1.0~git" nightly. So there's no need to go through uninstalling, adding the 'nightly' repository etc. But you may still need to reset the configuration files.
• The UI on the 1.0 version has a lot of bugs, so it's preferable to use 0.9.4 in that regard.
  

TWEAKS
There are 3 changes you should make in the configuration:

1. in Tools > Preferences, under Instances, enable "Allow only one instance".
2. Click on Show Settings:All (radio button in bottom-left). Under Input/Codecs > Other > FFmpeg,
   set "Skip loop filter for h.264" = ALL
3. under Video > Filters, on the right-side screen, scroll down the page to the second listing (for Video Output Filter Modules) and enable "Deinterlacing video filter"

That last change re deinterlacing will cause VLC to default to "Discard" mode which uses the least CPU. If you have CPU cycles to spare, of course, you could then go down into the Filters > Deinterlace settings and select a better mode.
.