Things were going fine, and it turned out to not be so scary! In fact, it was quite satisfying how clean the cuts were, and how easily aluminum got cut away by this cheap bit. Speaking of cheap bit, it was all fun and games till this happened...and it wasn't even under high stress! It pretty much just fell of, snapped clean through!

Anyways, I popped the next sized up, cheap bit and continued on without any issues. These were brand new, I'm pretty sure I only paid like $20 for a box of 6 different sizes.
So, this is what I did
*by hand*, and this was before I took a 3/4" file to the holes!!

I'm very proud of my work there. I did tidy it up and square it a hair more with a fine file - I didn't spend more than 10 minutes; then used some abrasive paper to soften the sharp edges.
Then grabbed a 2x4 and my palm router and began to hog out a test run!
My original plan was to use a 1/8" bit with a long shaft that was half smooth and half fluted. I'd run the smooth part up against the jig and just change the depth according to which square I was cutting. Unfortunately in practice this did not work out the way I had hoped. Too much slop in the palm router, in the 1/4 to 1/8" chuck adapter, and in the 1/8" bit itself. All conspired to make cuts that felt barely controlled, and I feared I'd wind up damaging my pretty jig. So I wound up putting a guide bushing (brass collar) on the palm router, which made my 1" square holes now 3/4". But hey, this is a test run/proof of concept.

You can see, even with the collar the cuts were wobbly. Nothing like the beautiful, tightly controlled free hand stuff I was able to do on the router table.

Final 2x4 test piece. The intention was to make mountains of these, as quickly and as accurately as possible, then just glue them to different height (probably hollowed out to save weight) 3.5 x 3.5 blocks to form the diffuser in the first picture; ultimately this would give me a diffuser, within a diffuser.

A few thoughts - making the jig was pretty awesome. I think I'm going to incorporate more aluminum works in my woodshop - though my shop is now all sparkly. I'll probably have to improve my air filtration. LOL. I think I was pushing the limits of what my tools, and especially the 1/8" bit can do. I think I'll just need to bite the bullet, and try again with a 1/4" bit, with the same jig. I'll have more rounded corners, but I'll deal.
Lastly, this was way, way slower than I wanted, to cut out those 8 slots. Changing depth and all that wasn't a big deal, it was really just hogging out that much material, clearing out the dust and chips, then hogging out more, clearing out, then the final clean cutting. I think the deepest I went was only 1.5"...maybe. I'll give this one more chance with a nice 1/4" bit, probably an all carbide spiral upcut, a more stable router platform, and I'll rig up a vacuum hose directly to the router. I bet I could do this in much less time then if this works out. Maybe be able to cut one block every 5 minutes? But then...I need 64 blocks, just for one 28 x 28 diffuser...ugh.
Anyone got a good CNC router for cheap?