Sunday, June 12, 2022

The Black Knight Rises - June 2022 Progress Update

A lot has happened since the last full project update.  The Black Knight Rises playfield build is completed, the electronics are all working, the cabinet switches were wired up, the playfield finally left the rotisserie and lives in the cabinet, and gameplay debug and multi-media development are under way.

The Black Knight Rises has risen!

When time has allowed, over the past 8 months I have troubleshot some build issues, swapped out some mechanical assemblies and switches, refined controller firmware, and did a ton of code development.

Except for a few small items, the physical build is essentially complete.  The main task that remains for me to complete is the actual game multi-media elements like sound effects, videos cut scenes, DMD animations, plus some enhanced game rules.

Click through for the full update.  There were a lot of lessons learned.

Phantom Switch Hits

Last October I resolved the power driver issues with a PCB redesign, and since then the new Chameleon Power Driver v5 has been working flawlessly.

With a working power driver, I was finally able to start debugging gameplay and hardware.  One of the things I observed pretty early on is that several switches were experiencing phantom hits - they would trigger when solenoids fired, shaking the playfield.  I hoped that this problem was caused by the rotisserie, which isn't very stable, and that installing the playfield in the cabinet would minimize vibrations and phantom switch hits.  Unfortunately, the problem didn't go away in the cabinet.

Ultimately, I found all of the problematic switches to be the original 40 year old leaf switches - I had re-used these as a cost savings, big mistake.  Since I had some modern Cherry style microswitches on hand, I swapped them out and that has resolved this issue.  I still have a few original leaf switches in the game that are working okay, but I may eventually swap those out as well for more reliable performance.

Lesson learned, modern microswitches are well worth the cost.

A quick pic of the playfield finally nestled in the cabinet.  The two bright red lights on each power driver make it easy to see dangerous voltages are present.  Just like Modern Firepower, only 3 cables connect the playfield to the cabinet:  38v solenoid power, 5v LED power, and a USB cable.  Note that this pic is before some of the changes mentioned below.

Ball Trough

 When I first assembled The Black Knight Rises' playfield, I used the original '80s style ball trough with an extra solenoid to move drained balls over a central hump.  I had never worked with these before, and thought it was a good opportunity to get some hands-on experience with them, and to make sure my Chameleon Pinball Engine software knew how to handle this configuration.


The original '80s style ball trough is a total pain and not worth the hassle.

The software changes were minor, and it handles this style ball trough perfectly now.  But after spending a few months with this style ball trough, I discovered I hate it with a passion.  Primarily, the switches are the main weak point, and no amount of tweaking these 40 years old switches got them to work reliably.  Plus, having an extra solenoid just to move a ball over a hump seemed wastefully complex.


Even with micro-switches, I found it difficult to get reliable performance from these 40yr old switches.

I was eager to switch everything out for the modern sloped ball trough, and installing the 8-ball unit I got from Pinball Life has provided pure joy.  This was a much better experience than on Modern Firepower.  The Black Knight playfield already had ball trough openings compatible with the modern design, whereas for Firepower I had to use a router to enlarge the openings to let the ball trough fit.

I did have to trim the mounting bracket on the ball trough to fit around the flipper assembly, though this was mainly my fault as I should have installed the ball trough first, then the flipper mechs, then the LED's.  At this point the in the project, it was just easier to Dremel away some metal.

I had to notch out the metal on the mounting bracket to make the ball trough mount correctly, as the flipper mech was mounted in the wrong orientation.

My only complaint is that the Pinball Life ball trough does not have a ball lock, so you have to eject all the balls before lifting the playfield.  I also discovered that the ball lock I bought as an add-on is incompatible with this ball trough.  Maybe I can 3D print something in the future, but for now I just have to remember to eject the balls.

After installing the new ball trough, it occurred to me it might not actually fit in the cabinet.  My pinball cabinet is a custom, lightweight design with a much lower internal height than normal.

My custom, light-weight, low-height pinball cabinet has a couple inches to spare under the playfield.

Turns out I have plenty of clearance, so I guess I can make the next cabinet even smaller!

Auto Ball Launcher

A few years ago I installed an auto ball launcher on Modern Firepower, and I wanted to do the same for The Black Knight Rises.

With the removal of the old-style ball trough, I repurposed that solenoid connection to power the ball launcher.

I did find that installation of the ball launcher is a little tricky.  On The Black Knight, the ball has to loop up to the upper playfield at the end of the ball shooter lane, so you need lots of speed to make the loop.  When I first installed the ball launcher, it worked great in the rotisserie, but was too weak after I moved the playfield into the cabinet.  I finally realized that the ball was resting on the end of the manual ball shooter rod, and that the auto launcher was about 4mm too far towards the front of the cabinet.  If I pulled back the ball shooter rod, letting the ball rest fully in the auto-launcher bracket, it launched perfectly.  


The Pinball Life 8-ball Ball Trough and the Auto Ball Launcher were both awesome upgrades!

So I had to remount the ball launcher bracket, moving it forward by about 5mm, so now it launches perfectly every time.  Even though the ball is making the loop to the upper playfield, the speed is just a tiny bit slow, and any slower and it wouldn't make it up.  I may upgrade the solenoid to something more powerful to be on the safe side, even though for now it seems to be working fine.

I have added some preliminary code to handle the auto ball launcher.  Currently it fires only on replay, and in all other scenarios you have to manually launch the ball.  I need to expand this to multi-ball, as I would like those to auto-launch, and that would let me get beyond 4 ball multi-ball to 7 or 8 balls - I love crazy stuff like this.


Playfield Slides

On the Modern Firepower cabinet, I created playfield support rails with wood.  For The Black Knight Rises cabinet, I decided to use proper playfield slides.  I had never used these before, and I mistakenly installed them backwards.

In my mind, I thought the hook part of the slide was supposed to be towards the front of the cabinet, so that you wouldn't accidentally slide off while lifting the playfield - and in this regard that worked great.  But later I realized that the top end of the playfield was floating loosely, and could flop around and hit the glass during transport.  It was then I understood that the hook needs to be at the back end of the cabinet, to hold the playfield securely during transport.

Flipping the slides around was easy and only took a few minutes, but didn't immediately solve the issue.  This is because I had mounted the playfield slide pins too far towards the center of the playfield, so they didn't nestle into the slide hook.  With some careful measuring, I determined that  I had to move them by about half an inch.  Now the top end of the playfield is securely restrained and is safe for transport.

I originally mounted the slide pins too far forward on the playfield.  The line indicates the correct spot.

Cabinet Buttons

One complaint nearly everyone shares with me regarding Modern Firepower is that the flipper buttons are in the wrong spot.  That's partly because I put them where I felt comfortable, and partly because I put them low enough to clear the playfield.

Determined not to repeat this mistake, I used real Black Knight cabinet trim with pre-cut flipper button holes, which guaranteed I would mount them in the right spot.  But something I hadn't considered is that the Groovy Game Gear style arcade cabinet buttons are too deep and would interfere with the ball shooter rod.  Oops.

Reverting to pinball style flipper buttons would not be easy, as the cabinet hole size is much smaller for those.  I needed to find a way to make these arcade buttons work.

The Groovy Game Gear arcade leaf switch buttons are too deep and hit the ball shooter rod.  I 3D printed a lower profile mount (in white) which barely solved the problem, but it wasn't secure enough.

My first attempt to fix this was to redesign the leaf switch mounting bracket to a shallower angle, giving me the necessary clearance for the ball shooter rod.  While this worked, my 3D print felt a little flimsy and the clearance wasn't all that great. 

A better solution was to mount the leaf springs straight to the cabinet.  This provided even more clearance, and was much more robust than my flimsy button mount.

Mounting the leaf switch to the cabinet (bottom) was more secure and provided even more clearance.

In the future, I'm more likely to use real pinball cabinet buttons, but at least I have the Groovy Game Gear arcade buttons working perfectly.  Note, this solution only works with the optional micro-leaf switch, and is not compatible with the standard Cherry style modern micro-switches.


Power Supply Tweaks

I originally bought a 36V 11A power supply for The Black Knight Rises, but I worried it might be a tad bit undersized for this application.  Plus after a few minutes of play the fan would ramp up and was annoyingly loud.

So I bought a higher quality 36V 15A power supply that I hoped would be quieter, as I wouldn't be going as close to peak power with the extra current capacity.

Unexpectedly, this unit ramps up the fan instantly, and it is just as loud.  Even with the cabinet fully closed up, I could clearly hear the fan blowing.  Not cool.

I checked, and the fan it was using is a standard 60mm size, and I happened to have a spare Noctua 60mm fan handy.  The Noctua fan was thicker, and wouldn't fit inside the power supply enclosure, so I simply installed it on the outside.  I also used one of the fan speed/noise reduction cables that Noctua includes, and now I can barely hear the fan and haven't noticed any power supply issues.  

I might tweak it further, either using an even slower fan speed reduction cable, or alternatively using a much larger fan that moves more air silently.  To use a larger fan, I'd have to 3D print an adapter.


Windows PC Replacement

When I first started The Black Knight Rises project, I bought a very cheap Windows PC to use as the main game processor.  At just under $200, the Intel Atom powered Wintel Pro was tiny and underpowered.  My hope was that if I could get my software to run on such a low-end PC, it would run on pretty much anything.

This plan was trashed in an unexpected way.  While my software seemed to be running great on such a wimpy PC, my Chameleon IO Controller wasn't showing up correctly.  After much diagnosis, I traced the issue down to the USB 3.0 port on Wintel Pro as not having enough endpoints - possibly a limitation of Intel Atom CPU.  One potential solution was to change the USB mode to 2.0 in the BIOS, but when I attempted this fix all USB ports stopped working - including the keyboard connection, ouch!

Having fully trashed the PC with such a simple BIOS tweak (and resetting the BIOS didn't fix this...), my only option was to buy a new PC.

So if a $200 PC couldn't do the job, how about a $130 PC?  That's right, I went even cheaper.  I bought a Beelink T4 Pro Mini PC, which includes Win10, and runs on an Intel Celeron N3350, a dual-core CPU with peak 2.4GHz speeds and 4G of RAM.  You can find these on Amazon for $149, but they have a $20 coupon so your actual price is only $129.

The $129 Beelink T4 Pro Mini PC, runs Windows 10 on a Celeron N3350 and 4G RAM

Even with only 2 cores and no hyper-threading, the Chameleon Pinball Engine has been running perfectly on it.  Unlike the original Wintel Pro PC, this one doesn't seem to have any USB endpoint limitations, so all my USB devices show up correctly.

Believe it or not, this thing even plays video during a game with no stuttering.  I'm pretty stoked that such a cheap PC can run my software so well.

The only modification I may need to do is to tap into the power switch, to run a remote on/off switch to the front of the cabinet.  Otherwise, this little PC is the Beelink's knees.

One factor in choosing this PC was its diminutive size.  I had already designed my backbox around the Wintel Pro PC, and didn't have much spare room to put a taller PC in the backbox.  At only  18mm thick, this tiny PC would mount on the monitor's VESA mounts with plenty of room to spare.

The Beelink T4 Pro Mini PC mounted to the monitor's VESA mounting brackets in the backbox.

While I was working in the backbox to swap out the PC, I also finally wired up the Kenwood speakers.  I didn't bother installing any amps.  Rather, I simply connected the monitor's original speaker connections to the Kenwood speakers, and I use HDMI audio.

The speakers are working great.  One side effect of Windows 10 and this particular PC's sound drivers is that the HDMI audio ramps up slowly each time a new sound is played after 5-10 seconds of silence, part of the energy saving features.  I tried a few Windows settings to disable the slow audio ramp, to no avail.  I finally discovered that as long as I have the background music playing in the game, the speakers are always ready to play any new sounds instantly - problem solved.  I currently have the famous theme song from Monty Python's Holy Grail as the background music track, and it's awesome.

Chameleon IO Controller Gamepad Buttons Sleep Issue

On Modern Firepower, I used the Ultimarc U-HID gamepad adapter, which worked great except for a couple major issues.  It was limited to 32 gamepad buttons, so for more than 32 buttons I had to assign keyboard characters to support all the switches on Firepower. No surprise that pressing keyboard characters on a Windows PC is a finicky solution to reading switch inputs.  The other issue is that the gamepad would go to sleep when unused, and the only way to wake it up was to trigger button presses.  This meant that any balls resting in the ball trough or ball locks would not be seen when you first started up the machine, which caused lots of issues.   On Modern Firepower, I wrote a routine to "jiggle" the balls in the ball trough using the ball ramp thrower solenoid, which would usually wake the gamepad up.

For my own custom Chameleon IO Controller, I wanted to solve both of these problems.  The button limitation was easy, as I configured it as two gamepad controllers, one with 32 buttons and the 2nd with 24 more, for the full assortment of 56 true gamepad buttons.

The sleep issue was much harder to solve.  I had to program the logic in the controller's firmware.  I could get it to stay awake on one PC, but then it would go asleep (or possibly get caught in a loop) on another PC.  After spending way too much time on the issue, I finally found a combination of logic and features that allows the Chameleon IO Controller to always read out the current button states, even after weeks of inactivity.  The solution was to send constant updates at 100Hz, plus immediate updates on button state changes, plus adding in a sleep/wake handler and a watchdog handler.  The solution felt a little bit like black magic, and I basically stumbled onto a combination a values that worked.


Debounce Code Works

One of the new features I added to my Chameleon Pinball Engine rewrite was switch debouncing.  The logic is really simple - ignore any switch hits that come too quickly, using a configurable window of time for debouncing.  Even though I coded this years ago, this is the first time I've been able to test it, and it seems to be working perfectly, yay!  Not everything in custom pinball development goes well.  Rarely, in fact.  So it's good to get an easy win once in a while.

The most obvious impact of switch debouncing is on the kickers/slingshots.  Typically, the act of firing a kicker will re-trigger the switch when the rubber rebounds, and you end up with a rapid-fire staccato performance from your slingshots.  It's awesome not to have to deal with that anymore.


Next Steps

At this point, The Black Knight Rises is playable, and plays extremely well.  There are a couple minor bugs that occasionally cause the ball counting logic to fail, which freaks out the game.  

One bug is related to having a ball stuck in the ball jam position - I need to add that switch into my logic, a notable oversight at this point in my development.  Currently if a ball gets stuck there, it seems to disappear from the game, and then when balls are thrown into the ball shooter lane, you'll get extra balls - sounds fun is anything but...  I think there's another scenario that get's the ball count confused, but it only occurs during multi-ball which makes it extremely challenging to debug.

I also have a fair amount of code development to do.  The playfield configuration still has a ton of parameters that are only set in config files and are not exposed in the GUI.  I need to add everything to the GUI to finish that part of the design.  There's also still some gameplay features I need to code, like the ability to auto-launch more balls for multi-ball, and to relaunch balls during multi-ball if drained too quickly.

I've started adding in video cut scenes.  This is something I tried to do 8 years ago on Modern Firepower, but back then I struggled with game stuttering during video playback.  My current code is much better optimized, and I'm able to play full-screen 1080p video without any issues.  I'm using Monty Python's The Holy Grail as source material, so The Black Knight Rises should be a very entertaining game to play.

Plus I just have to do lots more testing.  Some bugs take a while to show up, and sometimes you have to experience them a few times before you understand what's even happening.  That just means lots and lots of playing, but this is fun work so I'm not complaining.  Now that The Black Knight Rises is running well, testing is extremely fun and it gets better every day.


No comments:

Post a Comment