Figured it out. I was using WinFF wrong. I needed to select "HTML5" in the "convert to" box and "H264sd" in the "preset" box. I had been using "MPEG4" in the "convert to" box -- which did puzzle me since I had an mp4 to start -- so after a few more tries I have it working in the FF and Chrome versions I listed above and IE9.
Great plugin. Thanks.