Sunday, June 12, 2022

Progress Update & Troubleshooting Power Driver Issues

I originally wrote this update back in September 2021, but I got sidetracked and it never got posted.  I'm posting it now, as a precursor to the follow-up posts that are coming soon.


Chameleon Pinball Engine Software Updates

Chameleon Pinball Engine software updates were more extensive than I expected, as I was in the middle of a massive rewrite 4 years ago when I took a break from the project.  It has taken a lot of effort to figure out what changes still had to be made.

This has actual been a source of disappointment for me.  In my head, I was remembering being much further along in my coding.  I had certainly brainstormed a lot of features, and even begun coding a few of them, but upon code examination I'm finding little of it complete.

A lot of the missing functionality has been in the playfield configuration section, which means I have to write new code to handle all the new features I've been planning for years.

The good news is that I've improved my coding skills over the past five years, and I'm a much better programmer than when I started the Chameleon Pinball Engine all those years ago.  The new features are coming along quite easily.


Another Chameleon Power Driver Flaw

Four years ago, in my runup to the 2017 Southern-Fried Gaming Expo, I was testing my Chameleon Power Driver v4.0 with a Magna-save electro-magnet, and it was working fantastically... until it wasn't.  I thought my new design was immune to Transient Voltage Spikes (TVS), as if somehow the laws of physics ceased to apply to my design.  Physics proved me wrong, and the power driver was easily fried when hundreds of volts rushed in from the collapsing magnetic field.

Click through for the rest of the story.


I determined that my improved design merely isolated the host controller from the power surge, but not the power driver itself.  With this newfound realization, I promptly soldered on some TVS diodes to apply the exact same fix that I used on the v3 power driver that's been in use in Modern Firepower for 7 years now.

So imagine my surprise last week as I was doing my initial solenoid tests when a working board stopped working in the middle of the test.  What happened?  Based on the symptoms, it was a classical case of a TVS fried board, so why hadn't the TVS diode protected the board?

Was something different compared to Modern Firepower?

My Power Driver v4 design was heavily based upon the design I used for Modern Firepower, so I expected similar performance.  The main differences were that it had 8 drivers instead of 16, the layout was much more dense, and an extra set of PNP + NPN transistors allowed it to be controlled with 5v signals instead of solenoid voltage level signals.  These are actually pretty small changes, as each individual power circuit is essential unchanged.

So I popped the hood of Modern Firepower to take a look at the power driver, which has worked flawlessly for 7 years, to do an A-B comparison.  Same MOSFET's, check.  2 resistors per driver, check.  Rectifying diodes on each output... wait, where's the diodes?  

The lightbulb finally illuminated, and I began to realize my major oversight.  On the Modern Firepower power driver, these diodes allowed any excess voltage returning from the solenoids a straight path to the TVS diode.  But on my v4 design, the missing diode forced the excess voltage to travel backwards through the MOSFET - something MOSFETs can't handle.  Even worse, those mischievous electrons wouldn't escape through the MOSFET's Source but instead through the Gate, a common weakness of MOSFET's, and then travel through my N-channel transistors before finding it's way to the TVS diode.  If anything, the TVS diode was encouraging the voltage spike to tear through the transistors, leaving behind a wasteland.

MOSFET's respond poorly to a TVS, which is destructive not only because it is hundreds of volts higher than normal, but also because it is often running in reverse, with the solenoid as the power source and the MOSFET acting as the negative power connection.  So hundreds of volts are rushing straight to the MOSFET, and they want to go somewhere.  I thought, correctly as it turns out, that they would go to that TVS diode I had installed.

But what I hadn't considered is the path the electrons had to take to find the TVS diode.  The surge actually has to travel back through the MOSFET in order to reach positive power plane, making the MOSFET stuck in the middle and exposed to the full brunt of the TVS.  

These MOSFETs typically direct the surge to the gate (back towards the controlling signal, not the power supply).  This fries the MOSFET, and fries the PNP and NPN transistors controlling the gate.  Eventually the power surge reaches the high power plane, and the TVS diode shunts the excess voltage to ground.  But by then it's too late, the damage is done, and everything on the board is fried.

The solution is to give the TVS a shortcut to the TVS diode, so it doesn't travel back through the MOSFET in the first place.  I don't know why I didn't recognize this issue before, now it seems so obvious that allowing the surge to travel back through the MOSFET is a really bad approach, though I was naïve enough to think my design was immune to transient voltage spikes, so I guess it's not too surprising that my bolt-on TVS diode didn't actually fix the issue.


Quick fix:  1N4007 diodes soldered onto bottom of v4 Power Driver, connecting to large TVS diode

I quickly mocked up a fix, soldering on some spare 1N4007 diodes onto the bottom of a new power driver board.  Now the surge would bypass the transistors and head straight to the exit... right?


And Yet Another Flaw!

Just when I thought I was home free, having resolved a major PCB layout bug, reality said it wasn't done toying with me yet and promptly fried another board.  The symptoms were the same as before - it was working until it wasn't, all channels fried as if a TVS spike had slammed into the delicate silicon, and I was just standing there stupefied.  Did I solder the diodes on wrong?  What else could it be?...

I was momentarily alarmed when I realized that I had overpowered my 5v LED's with 5.5v on the Chameleon IO Controller, and this 5v level is what also triggering the power driver's circuits.  Perhaps it could only handle 5v.  I checked all the specs, and 5.5v was well within tolerance.

I double-checked my soldering work, and it seemed legit.  With the answer eluding me, I went back to basics.  I reverse engineered my PCB circuit, using a multi-meter and documenting the electrical schema on paper.  I then did the same for the Modern Firepower power driver board.  The first thing that popped out at me is that the resistor values were drastically different.  My memories from four years ago was faint, but eventually I recalled tweaking the resistors to allow the driver to operate fully over a wider range of voltages.  I used 5Spice to model the circuit behavior, so I felt sure I did it right, but the 2 resistors had a 10:1 ratio on the v4 board, and a 1:2'ish ratio on the old board.

Re-examining my v3 Power Driver, which has worked flawlessly for 7+ years in Modern Firepower

But resistor values weren't the only thing that popped out at me.  One of them seemed to be in a different location on the v4 board, on the controlling side of the middle N-channel transistor - a component that didn't even exist on the older Modern Firepower board, since that was instead handled by the LED-Wiz.

Once I section off the lower transistors and resistors (since they are all new components that replace the LED-Wiz), I finally realized that I was actually missing a resistor in my new design.  Where the Modern Firepower had two resistors per MOSFET, the v4 power driver design only had one.  The resistor that was missing should have stood guard between the N-Channel's output line and the MOSFET's gate.  But how important could that missing 82Kohm resistor be?

I fired up 5Spice and loaded up my circuit design from four years ago, thinking I would have to add the resistor into the design and model it.  But to my surprise, the missing resistor was already in the design.  A nice, 94Kohm resistor was in the perfect spot, between the N-Channel and the MOSFET.  Somehow it was forgotten in my PCB design.  Argh!

I spent some time playing with resistor values, including removing it altogether, to discover the impact on the circuit behavior.  With the resistor in place, the max voltage on the MOSFET's gate was around -13 volts, perfectly within the MOSFET's -20v max limit.  As soon as I removed the resistor, the gate saw the full voltage of the power supply, -38v in this case.  

Doubling the safe limit for the gate voltage is a sure-fire way to destroy a MOSFET real quick.  It seems I had finally uncovered the true cause of all my v4 power driver failures - a missing 2-cent resistor.

Trying to come up with a quick fix, I briefly considered placing a TVS diode on each output - eight per board - but those TVS diodes are way too big and expensive for that approach.  A better approach would be to use common 1N400x class diodes, probably a 1N4005, similar to what is typically installed across the solenoid coil's lugs.  I could place these small and cheap diodes between the power connection to each solenoid (my design uses high-side switching), and it would allow any excess power to flow from the surge straight to a single, shared TVS diode on the board.

But I finally accepted that all of this was nothing but band-aids, and that I need to address the root problem.


Designing the Chameleon Power Driver v5.0

This newest flaw was too big (and the components too small) for me to jerry rig a fix.  I would have  to design a new Power Driver v5, incorporating fixes for all of the issues I've discovered, plus some bonus enhancements.  Do-overs are expensive, so I figured I better make it count.

My wish-list was long: 

  • Add a TVS diode to give transient voltage spikes a safe path to ground.
  • Add 8 diodes to give the power surge a direct path to the TVS diode, bypassing the MOSFETs
  • Add 8 resistors between each N-Channel Transistor and the MOSFET it controls
  • Add a 2nd 'DANGEROUS VOLTAGE' LED that would stay lit even if the fuse popped
  • A faster discharging circuit, as the existing LED discharging circuit is way too slow for safety
  • Add a green LED on the 5v power line from the IO Controller to help with troubleshooting
  • Add a 4th screw/post hole for better PCB support to prevent twisting when tightening screws
  • A couple more screw terminals for extra ground connections
  • And some extra probe test points to aid in troubleshooting

I also revisited all of my transistor choices, to make sure that they were still the ideal solution.  I originally used a really small NPN transistor for direct connection to the IO controller, and perhaps that was a mistake.  An opto-isolator might actually make more sense here, to protect the IO Controller board from a failure of all other protection mechanisms.

I spent some time evaluating all my components and options, modelling circuit behavior at different voltages and with different resistors.  After reading a few too many datasheets, and seeing how crazy expensive opto-isolators are, I arrived at the conclusion that my 4 year old design was still the best choice of components, and all the issues were just with a few major PCB design flaws.

It took a while to select the new components.  I was happy to find a nice surface mount TVS diode that is quite small and a fraction of the price of the through-hole monsters I'd been testing.  All 1N400x series diodes are actually legacy products that are not suitable for new designs, so I found some nice S1G surface mount replacements, super small and just as cheap.  The other components were pretty basic, resistors and LED's and more of the same screw terminal blocks.

With all the additional components, the new the v5 board was going to end up larger, though I was hoping to keep size in check.  The end result was that the board is 10mm wider and 5mm longer.  It's no longer the size of a business card, and now is a smidge smaller than a credit card.  I printed out a life-size template and checked how the new size would fit on The Black Knight Rises, worried that perhaps I hadn't left enough room for larger power driver boards.  I need not have worried, space was plentiful, and the few extra millimeters made no difference.

It took me a couple days to refine my design in DipTrace, my preferred PCB design software.  I hadn't done any PCB design work in years, so it was slow going at first just to remember how to do it.  I vaguely recall that 4 years ago I was in quite the rush to complete my v4 design, and with all the issues I'd just uncovered I was in no rush to repeat that mistake.  I triple checked all connections, worked through my hit-list of changes, and took my sweet time to make sure this revision was going to be great.

With the new design completed, I sent it off to JLCPCB for fabrication.  I've never used them before, so this isn't a recommendation.  I've heard a lot of negative press, and even seen examples of common issues, but for their cost savings I decided to gamble on them.  After all, my "perfect" v4 PCB boards that were expensively manufactured elsewhere are all worthless now.


ChameLED v2.0

I found the process of designing the v5 Power Driver pretty enjoyable, so I spent an extra day designing a new ChameLED revision.  In an earlier post I mentioned my desire to design a smaller PCB around the larger Piranha / Superflux 5mm LED's, and this seemed like a perfect 1-day project.

One aspect that I liked about my v1 LED design was that it was compatible with multiple LED types.  It had 3 different landing pads for different types of LED's, one for RGB and two others for different types of single-color LED's.  From a cost standpoint this is a huge savings, making it easier to get volume discounts with only a single PCB design for all your LED designs.

This was even easier with the Piranha's, as they all have the exact same footprint - just four through-hole legs in a 5.08mm x 5.08mm square.  For RGB I would hook up all four legs, but for single-color LED's I would only have to hook up two.  During assembly I would only have to swap out resistors and wire terminals to adapt the single PCB to any color Piranhas including RGB.

I included the ChameLED v2 with my Power Driver v5 order at JCL.  This helped save on total shipping costs.  Even with the small volume prototype run, my costs per PCB are less than half what I paid for the v1 design. Hopefully the order will reach me in a week or so.


No comments:

Post a Comment