A few more tests and it looks like I'm running into power issues. I was dreading the prospect that my battery isn't up to par (2.4A @ 5V, with a 10Ah rating), but the issues happen with a wired connection, too.

What happens is that I kick power to the motors, and the whole thing shuts down. Well, I can still see the motor controller is powered, but the Jetson is dead in the water - even on serial console. The power light's out, so it's probably an overcurrent situation. I need to break out the multimeter when I have more time.