On March 11, 2007, Daylight Savings Time starts earlier than last year. Congress moved the date with a simple vote. Now you and I have to do all the dirty work to make our computerized world fall into place. Are you ready?

When I was 15 and enthused about ham radio, I bought from Allied Radio one of those 24-hour wall clocks and set it for Greenwich Mean Time. Yeah, I was that geeky. What that clock showed me is that all the fuss about time zones and daylight saving time is completely unnecessary. Why couldn’t the whole world run off one time and change their local school and work times to whatever they wanted? Does it really matter if they’re going to work at 0800 in London and we’re going to work at 1800 in San Diego? Instead, governments and corporations spend enormous resources tracking zone boundaries and daylight saving periods. DST serves only as a way to change the numbers on the clock, and is more meaningless the closer you get to the equator where sunrise and sunset times don’t change much. If safety and conservation are important at the upper latitudes, why don’t we just change the starting times of the school or work day to suit the daylight available?

In 1973, they messed with daylight saving time briefly because oil supplies were squeezed. They wanted more daylight in the afternoon so that we didn’t run so many lights at work. But what I remember is that we were suddenly getting up in utter dark, running the heat and lights at home an hour earlier than before. The problem was that at the 45th parallel, there was only so much daylight. At the time I worked at a small daytime only Class IV AM station in Oregon, and during December and January I would sometimes work weekends from the 8:30 AM sign-on to the 5:30 PM sign-off. According to Wikipedia, studies of the effects of DST have variously concluded savings of energy and traffic deaths as well as loss of savings to economic efficiency and increased energy peak loads at sunrise.

On March 11 this year, we’re changing from the usual early April changeover to DST to March 11. The United States change is part of the Energy Policy Act of 2005. Again the justification is to synchronize the workday to the hours of daylight available so that we are less dependent on Middle East oil.

The international ramifications are interesting. Canada decided to adopt the U.S. plan to move DST; Mexico did not. For the first time that I can remember, Tijuana will be in a different time zone from San Diego from March 11 until April 1. My station, XETV Fox 6, will be in the uncomfortable position of advertising programming times to its majority U.S. audience in Pacific Daylight Time, while the country of license observes Pacific Standard Time.

You may have work to do in preparation for March 11 this year. If you run Windows XP or Vista on the internet, you should be able to download their normal maintenance update to change the automatic DST switch date. If you run a machine isolated from the net, you’ll have to schedule a download or just change the time zone of the workstation in question.

It turns out that the Dallas Semiconductor real time clock (RTC) chips that control time in many computers have hard-coded DST changeover dates so that can’t be overcome except by manually changing the time or GMT offset.

You can program Macintosh computers to change automatically on the correct dates by editing the /etc/localtime file. Instructions can be found here. Linux.com has a clock howto with DST information.

If you depend on Java runtime applications, see Sun’s DST FAQ. It appears that if you accept the Java RT update being pushed to your workstations this month, you should be okay.

You’d do well to look around for time-dependent systems that may need your intervention. Some vendors are offering software upgrades to automate the GMT offset at the new time. Triveni, for example, is asking $500 to upgrade its PSIP management software that includes revised DST changeover dates. If you have a collection of similarly needy boxes around your broadcast facility and you can’t get away with simply changing the GMT offset yourself, this time change is going to cost you.

A little over two years ago, Fox 6 photojournalists were showing up at our service shop with odd, seemingly unrelated problems with their camcorders. The symptoms involved anything from servos to color encoding to erratic controls. We learned that the solutions, however, were almost always the same–change out a group of surface-mounted electrolytic capacitors or replace the entire circuit board. The DVCPRO camera or studio deck always went away working again until another, unrelated, failure occurred. Repair veterans like Greg Capalbo of GRC Electronics were warning us that the failed capacitor scenario was an epidemic, especially with Panasonic gear, but also to a lesser extent with Sony and other brands. And Panasonic was not supporting out-of-warranty fixes with either parts or labor.

As we saw more electronics come in with bad capacitors, it became increasingly clear that the future duties of our ENG specialist Ladd Prier might involve an endless, daunting series of tracing erratic component behavior to a circuit, then buying new caps, awaiting their arrival, then swapping them out and hoping the bad behavior would disappear. We visualized the coming stack of broken equipment and unhappy users. There had to be a better way.

Ladd and I come from a time when a technician took pride in tracing an obvious equipment failure to one or more components directly responsible. When an RCA TR-3 2″ videotape player headwheel wouldn’t spin up, I would get the four-inch thick manual, an extender, and Tektronix 545 oscilloscope and through a series of calculated guesses, eventually find the open or shorted germanium transistor or diode, then replace it and check a couple of waveforms before pronouncing the monster ready for duty.

Fast forward 25 years. Today’s equipment contains little undocumented black boxes. When the servo fails, you change out the servo board. But what if all boards fail on all news equipment? We were looking at a potential of essentially having to replace part-by-part all of our five-year-old Panasonic equipment.

A Better Way

On a whim, I called our temporary agency, TLC Staffing, and asked if they had any registered rework specialists. In manufacturing, these are the people who take circuit boards with errors on them, either by design or by manufacturing defect, and change out components as ordered to resurrect those boards. {mosimage}They are skilled at performing Lilliputian surface-mount technology soldering, and with incredibly tedious repetition.

TLC sent us Vu Le, who appeared businesslike, cheerful, and ready for action. Over the next ten weeks, he replaced nearly every surface-mounted capacitor in every Panasonic DVCPRO deck, both from the field and from inside the plant. He changed up to 400 caps a day for a total of nearly 20,000. Total cost: nearly $16,000 including parts. Every VTR and camcorder finished worked like new, especially when heads were changed at the same time. There were no more mysterious misbehaviors, a fraction of the downtime, and Ladd was now free to take care of the more normal maladies of ENG.

Months after Vu went on to other work, we noticed more equipment failures that seemed as though they could be capacitor related. Modules in our Wheatstone TV-80 mixer started having problems with bandpass and open circuits. Our digital Hitachi studio cameras had assorted color problems.

This time we didn’t hesitate. We hired temp Daniel Yanez to swap out all the electrolytics in those devices, though less careful to estimate the extent of the project. Now some 10 weeks later, Daniel is still slaving away, module after module. He began filling a three-gallon plastic jar with the tiny devices, and now it’s nearly a third full.

Obviously, if we knew the more about the exact nature of the failures, we could be more selective about the replacement. Were some of the caps higher grade? Which values really fail? If temperature is a factor, couldn’t we perform an IR scan to find the hotspots? Since we didn’t have a good composite history of the mixer and cameras across their user base, we didn’t have the knowledge to pick and choose what devices to replace, so we replaced them all. The basis for doing so was 100% empirical and 0% scientific.

Epilogue

Industry experts continue studying the science behind the SMT chip capacitor failures. It surprises me to learn that they blame not just chip chemistry, but physical conditions during manufacturing and board assembly. It appears as though bad things are happening to good companies with respect to their products not living as long as expected.

We bought replacement capacitors that were labeled as having a long expected life, but there’s no assurance that we’ll even double their life. In fact, we’ve replaced the camcorders in a surprise blast of capital funding. I expect the 7-year-old standard definition cameras and analog audio mixer can’t be far behind.

Daniel is looking for his next gig. Got bad stuff?

Comments

Courtesy CGC Communicator

The capacitor saga dates back to about 1998 and the wide use of surface mounted electrolytics. Those parts have a time and temperature rating. The ones supplied in most gear do not have the most generous ratings. While we deal with Sony, Panasonic, JVC, etc., their original board assembly may all be
done at one outside vendor. Thus, plenty of gear was built with marginal capacitors.  I first ran into this with Panasonic DVC-Pro recorders.  One audio card that was troublesome had 50 of the beasts on it.

The best way to test for bad capacitors is with a Capacitor Wizard that uses a high frequency to measure reactance. It works fine testing caps with no need to unsolder.

Roy Trumbull, roy547 (at) msn.com

The Panasonic DVC-Pros are the first and worst detected.

We tried Panasonic replacement kits and they didn’t fix all symptoms. I decided not to test the capacitors, figuring that we would have too many iterations of failure/remove/disassemble/test/replace/reassemble/return. Our rework guy could replace 400 caps per day, so I just said to heck with testing –replace them all!

It’s been my experience that measuring in-circuit reveals the problems in only one point in time and doesn’t address future cap failures. In our case, we haven’t had any symptoms in our DVCPRO decks since the mass replacement of capacitors.

Gary Stigall

URLs FOR ELECTROLYTIC PROBLEMS

Scroll down to the Great Capacitor Scare:
http://catless.ncl.ac.uk/Risks/22.73.html#subj9

This article discusses acidic leakage damaging circuit boards in mobile radios:
http://www.repeater-builder.com/motorola/spectra/spectra-caps.html

While this article is oriented towards PC board mounted caps, the cause is the same – bad electrolyte:
http://en.wikipedia.org/wiki/Capacitor_plague

And more links yet:
http://www.ttiinc.com/object/ME_Zogbi_20050919.html
http://www.badcaps.net/
http://www.theinquirer.net/default.aspx?article=24596 (re Dell)

All in all, it looks like bad electrolyte is the root cause.
In some cases it causes a reduction in capacitance value (which
changes circuit performance), in others it blows the tops off the
caps, in others the electrolyte eats its way out of the cap and
eats up the board.

Mike Morris, WA6ILQ