[Atomic Shrimp] on Youtube posted a project he was working on here:

This is essentially the same sort of project as the original incarnation of this keyboard, but with the added simplicity of starting off with a chunk of protoboard instead of trying to hack an existing PCB.

He did a lot of things right:

• a mix of mapping out the matrix and brute-forcing the keyboard matrix is a much better process to map out the keyboard than my process of building a quasi-chorded keyboard with a button for each row and column hooked to a common point
• magnet wire is much neater than my solid-core breadboard wire (even though you then can't have all links for a particular row or column the same colour)
• designing a keyboard layout from scratch drastically simplifies the project - I'm sure that adapting the original Apple Keyboard matrix to suit my donor USB keyboard controller was a significant amount of mental effort

However at the end of this he did come to the correct conclusion: building a key matrix to suit an existing controller is far more complicated and difficult than building a matrix from scratch and programming your controller to suit.

Keyboard design is a compromise between hardware complexity, software complexity, and cost:

• QMK adds a lot of software smarts and this can be used to drastically simplify the hardware design
• But this requires "expensive" hardware, i.e. key matrices with diodes
• A bit of hardware cleverness - designing the matrix to minimise ghosting opportunities - can drastically reduce the hardware cost, but that greatly increases the complexity of the hardware and pretty-much precludes any attempt to hand-wire it
• That hardware cleverness, as well as mass-production efficiencies, can also drastically reduce the amount of software complexity required - no need to detect ghosting if it should never happen - and therefore reduce the requirements of the hardware running that software

Long story short, it's a big game of give and take - on one hand these cost optimisations have enabled us to be able to buy reasonable keyboards for $7 / £3, however the design optimisations required to achieve that have made what should be a very simple device mind-bogglingly complex.

I tend to think of design happening in this rough sequence of phases:

1. Proof of concept - works well enough to show critical people
2. First generation manufacturing - can be manufactured in large quantities
3. Later generation optimised manufacturing - has been extensively analysed to reduce costs

The problem for us hobbyists is that most consumer products are now multiple generations into phase 3 and have been streamlined and optimised to the point where we're unlikely to ever fully understand them.