Next-Next and Next-day brief Updates at the end...

...

I am by no means experienced nor knowledgeable in this realm... 

I have done a bit of inline asm in my otherwise avr-gcc progects. But this is very different.

Here we have a booting system written in assembly, and I plan to make a bootable utility in C that makes use of the initial-setup and functions that assembly-code provides. In a sense, I'll just replace its "main loop" with my own.

Initial experiments were very promising. I wrote a small main()-like function in C, wrote a "void function(void);" *declaration* (not definition) for one of the assembly functions, called that function from mine...

From that alone (not even "including" the original assembly file in any way) sdcc gave me assembly-output that was *easily* modified in such a way that I could little more than copy/paste its output into the original assembly file and run my code as though I'd written a new function in assembly in the original.

Basically, sdcc's assembly output did "calls" to labels, and those labels just happened to not exist within its awareness... and it didn't seem to care.

Which was *great* because when I pasted that output into the original assembly file, those labels now were available, and my assembler replaced them with the appropriate 16bit addresses. Just like assembly does.

I guess I'd expected sdcc to croak on the missing labels long before actually outputting a usable assembly file.

So... Awesome!

...

So then I bit off more than I could chew.

Wrote the entire utility in C, tested it with gcc as best as possible all along the way. Finished it, finally, today...

Then... yeah.

OK, so the big thing that ultimately stopped me from proceeding with my original plan (of copy/pasting sdcc's output into the original assembly file, then hand-modifying it as-necessary) is the fact that the sdcc output uses the same label-names in all functions. e.g. "00104$" which resulted in my assembler's complaining of duplicate labels 69 times. I *almost* considered hand-changing them until I realized it looks like it only complains about the first duplicate of each... Heh.

So then, obviously, just throw the thing into sdcc's assembler, instead, right? But apparently *that* didn't like the original assembly in an entirely different and equally difficult to repair way: Apparently it's so low-level that it doesn't handle things like "equ"... which would be a tremendous feat to remove all instances of. Heh.

Again, I'm no expert, maybe it was just a matter of find/replace... but this came after quite some effort dealing with many other incompatibilities, e.g. "immediate values" are prefixed with # in one, but not the other...

So, finally after much "hand-jobbing" I decided it was time to throw up my hands and come up with another way.

Now, I should probably interject that obviously there is some "right" way, otherwise we wouldn't have many of the fine things we have... I imagine "the linker" is a big part of it. I have history with that beast that prevents me from preferring trying another go at it over, say, hand-editting 69 labels.

But, I think I came up with another solution, which actually should be easier... just modify the original to call some specific address, say near the end of the ROM. Assemble that in the usual way. Then modify sdcc's assembly-output with actual addresses instead of labels pointing to the original code. There's only three functions and one buffer to be loaded into RAM. Four addresses to hand-enter. Oh, and a .org at the beginning of its output to somewhere after the original's.  And, finally, add a .org at the decided-upon jump-to-address, with a jump to my main(). Then, of course, use sdcc's tools to compile that, as though it was its own completely standalone thing. Merge the two ihex files, and we're done! Scripting that whole process should be easy-enough, too. And it allows for keeping the two codebases separate, which has many benefits.

I dunno what this all equates to in "the normal way". Probably a minimal linker-script, maybe a tiny C-Runtime...

I think it should work. I guess we'll see.

...

Oh, Almost forgot the days of trying to figure out sdcc's method for passing arguments to functions, getting back return values, keeping its registers from being mangled, etc... but that's another topic. Along with numerous others I'm forgetting.

...I think this could work... we'll see.

....

Next-Day Update:

It took a little learning of the linker, after all. But, actually, I looked into its documentation (again, some twenty years after my months-long failed ordeal with it) just a tad and saw a bulletted-list of what it's supposed to do (if you're not like me in your attempts to code it). On the list, nothing was there I hadn't already done by hand *except* converting it to ihex. Heh. I thought that was objcopy's job, so looked there, first. But, thankfully, I'd found it in a combined-manual for the whole process, which, scrolling caught my eye with "convert to ihex" and only later did I realize I was looking in the linker's section. Heh! 

Handy doc I've not seen elsewhere:

http://www.ee.nmt.edu/~rison/ee308_spr97/asmlnk_doc.html

Anyhow. the short of it is: It's not a huge nor difficult process at all... A small bash script could surely take care of everything I did. BUT. I dunno why it doesn't work. It looks like it's doing some pretty gnarly stuff right after it calls main... randomly resetting, hanging with the address bus going wild, ignoring interrupts... It's like it jumped to some random address somehow. But I've looked that bit over quite thoroughly.

Did I mention I *hate* trying to debug huge code changes? This aint small potatoes, here. And the friggin irony is that the whole reason I'm doing this particular subsubsubproject is to enable testing of small code changes, so I won't wind up in exactly the situation I'm in. GAH THAT FRICKIN CHICKEN AND ITS EGG!

I should make a list of all the things this blasted utility contains that really should've been tested on real hardware, every step of the way, rather'n trying to simulate. Believe me, I wouldn't've gone this route if it hadn't basically been week(s?!) of a dangling-carrot just inches away from completion. 

I've written yet another gnarly rant about debugger-culture I'll spare y'all from.

....

Next-Next day:

Well, I decided to settle in for the long haul to figure out where it was crashing, step-by-step starting from before calling main() to just after... Coded up wrappers for a couple led functions... Turned on some LEDs at different steps... What else? Nothing, really, far as I recall.

And It Just Works. (so far as I could test).

Heh!

At the same time I wrote-up a step-by-step hand-linking procedure. It's not at all difficult if you're just cross-calling a few functions, as long as you've got wrapper-functions for them, to interface C's call/arg/return convention with ours in assembly. After that it's just a matter of copy/pasting a few addresses and some other goofies like using just one section with absolute addressing via ".org". Oh, and I couldn't figure out how to typecast a uint8_t* to a void function(void) so as to execute the actual Flash-writing procedure from RAM. No matter what I tried, no matter where I put "const" sdcc insisted on using a "trampoline" function to call the absolute address. "ld hl, address ; call __sdcc_call_hl" which is goofy and I didn't include *any* external libraries, anyhow, so that resulted in a missing symbol. Anyhow, the solution is easy in assembly, so I just rewrote those two lines by hand in sdcc's assembly-output as simply "call address". Heh. (Hmm, I suppose inline-asm would've been far easier). Oh, right, and for some stupid reason its using jp instead of jr in the function that has to be copied-to/ran/from RAM... but, again, easy fix. Probably a few other things I'm forgetting... but not difficult nor unexpected.

So, now, I can use serial I/O and a few other things within C, while those functions, buffers, and interrupts were written [now seemingly long-ago] in assembly and loaded at boot...

Kinda like a BIOS, maybe? Kinda cool!

Still rather frustrating it seems like it just fixed itself... Non-repeatable bugs are the worst. My changes were minor. Not intended to fix anything, just to *find* it. Though I suppose it's possible I missed a step in the first go-round hand-linking... I don't think I'm bouts to disassemble the old broken ihex... unless... this keeps... nagging at me... like it is. ARGH.

As it stands, I couldn't test everything... The actual flash-writing fails because I've yet to modify the board/circuit to handle two 16k pages instead of just one. I expected that. Too dark to solder tonight. Though, seeing it actually working all the way through is also nagging at me.

....

Next Next Next day... And into the next thereafter:

I couldn't allow this bug to just magically fix itself...

14 hours later... amounting to little more than organizing files and diffing... It would seem there were two stupidities on my part and, yes, both were fixed... Properly. No magic involved.

There were three versions until one worked properly. The first acted quite weird, randomly resetting, disabling ints. I caught that I'd accidentally jumped to main before the global variable inits which were in the previous mainlike function. Changing that was what version2 was all about... And it *sorta* fixed some things, but not really much more in the way of functionality before it appeared to crash at basically the same point: right after jumping to main, but apparently before the "boot message." Version three was my hunker-down to begin debugging... The first steps amounted to nothing more than moving code from one big file into two smaller files. And... It Worked.

Now, that just don't make sense.

So today was all about investigating...

First, could the uninitialized globals *really* cause seeming crashing, random resetting, and Ints to be disabled? The only way I can perceive that possible is executing instructions outside of ROM, which took some time to figure out how that could possibly happen. 

The number of factors involved are crazy... Basically, because the ReceiveData buffer is not initted, its "length" (aka number of bytes currently in the buffer) could be anything... IF its length is greater than the number of bytes the buffer can store, then it will never stop returning data when requested. Now, I forgot to check, but I think the result would be just repeating the same 128 bytes over and over(?). Regardless, what it would mean (only if length just happens to power-up > size) is that the ihex parser thinks it's receiving data. *immediately* Now, the ihexParser ignores *everything* /before/ a colon... that space is for newlines and comments. 

(Did I mention that I do not enjoy this kind of sleuthing AT ALL? 14 friggin hours! But I DO care about the quality and reliability of my work. So, even though I could've actually *seen* my weeks of hard work Doing What It Was Designed For for the first time today, instead I spent 14 friggin hours doing something I darn-near despise doing. And keep in mind this was 14 hours with code I already know inside-and-out from weeks of writing half of it myself, and weeks prior of working with the other half.). 

OK, so the ihexparser gets random garbage that just happens to have been in the RAM at power-up (our RAM is battery-backed, which might help to explain the slight pattern in the randomness of its resetting, etc.) But, again, it totally discards everything before a colon. Which means, somewhere, randomly, it found a colon. And, again, I'm not certain, but that random colon might've had to have been in a specific 128bytes in the RAM.

Now, I didn't do anything fancy, like stop processing on CRC errors, or when something other than 0-9, A-F is received. (It *will* spit out a CRC-Error message, which if you see then you can just send the entire file again, to flash it again... Big whoop, 30 seconds). But, what that means, here, is that after it finds that random colon, it tries to parse the actual ihex data... which, of course, is garbage. But I didn't do fancy-math/testing, so any two-byte binary value would return *some* 8bit number.... Now, part of that ihex stream is a number of bytes in the ihex line. 0-255... So then the parser grabs *only* that many bytes (in hex) and the CRC. After those, it discards everything, again, until the next colon. No data is actually written to the flash until *at least* the next command comes through. (the last batch of data is written when the ihex "EndOfFile" is received, which, yes, starts with a colon). This Means: No attempts to write data will occur unless *two* colons are received, *at least* as far-apart as whatever it interprets the two bytes following the first colon to equate to in a uint8_t... So, somewhere between 0 and 255 bytes, as "specified" *at least* have to separate these colons. (Note that the parser ignores 0-byters... So, actually, we have to have a somewhat-specific sequence of ":XY" for this to work... And that somewhat-specific sequence has to occur at least twice... And at least a somewhat-specific distance apart).

 It's further-complicated by: it only writes once an entire flash-sector (128 bytes) is loaded *or* the second ":XYABCD" where ABCD (the write-to address) is outside the range of the currently-selected sector. 

So: Let's say the circular buffer just kept sending out the same 128byte sequence repeatedly... There's say a 50/50 chance the "number of ihex bytes" will actually be interpretted as greater than one sector... If it wasn't, then we'd not see this glitch, because it would just keep reloading the same <128 bytes to the sector buffer, and never attempt to write them.

Again, mind-you, we're talking about whatever just randomly happens to be in the RAM at power-up. And I never *didn't* see this glitch. (though, again, our RAM is battery-backed. The fact is, I think the battery is weak, as previous experiments showed its holding data between resets, but seldom long between power-cycles).

The only logical explanation for its malfunction is executing code from somewhere *other* than the ROM. Because, even if it were somehow executing some random address *in* ROM, it would have to get there, first. 

So, we're at only one reasonable explanation... (aside from maybe power-glitches, and they didn't put decoupling caps on any of the chips, and the power traces are surprisingly thin and snakey... but despite ground-bounce my logic-probe picks up, this thing *has* been surprisingly stable when the code's not buggy)

Somehow that seemingly ~50/50 chance, at best, of two bytes being interpretted as >128 (Greater-than *not* equal!), combined with whatever weird-low odds of there even being a colon to go before them, and even weirder odds of there being *another* colon (followed by a non-zero two-bytes) to trigger the sector-write procedure... I mean, again, the best chance seems to be if the circular buffer just happened to contain those three somewhat-specific bytes in sequence... in only 128 positions?

Oh, right, there's more...

If the random data in the RAM just happened to have a *zero* in the "transmitting" flag, then we'd see at least a few characters... But I never saw any transmission. Which Also implies that the random data in the transmit-buffer's "length" must *also* be outside the actual size of the buffer. Otherwise, it would've hung early, long before the ihexParser was called, due to waiting for the Tx Buffer to drain so it could finish sending the boot message!

Really, it seems the odds of all these things happening exactly right to cause this particular glitch, not once in a while, but *every time* seem downright infinitesimal.

*Hanging*, waiting for the TxBuff to empty, that'd surely be *far* more likely, but even in those cases that seemed to be the case (when it took a long time to reset?), RxInts weren't being handled. Which, again, shouldn't be the case unless the Wrong Code was executed somehow/where... (to disable the ints). Which, again, can only happen in one place in the code: When the sector is to be written... Because that particular function has to be loaded-into and executed-from RAM, whereas ALL other functions/jumps are within the ROM, half of which were auto-generated by C, the other half meticulously hand-written and looked-over countless time in Assembly. NO external libraries, compiler-provided runtimes, nor even use of the linker.

These odds seem incredible.

NOW I'm In No Way trying to say the code should work correctly with those globals uninitialized. Nor even that my code is robust in cases like random data errors. I AM saying that the Particular Way it's malfunctioning seems incredible. As in: lacking in credibility. Improbable.

So... Again, even after, now, 16 hours of looking into nitty-gritty and a couple more hours' pondering, here, I'm still not entirely convinced The Cause of The Glitch I've seen can be boiled down to "uninitialized globals". HAH!

...

Meanwhile, the bug in the second version *seems* almost crystal-clear... I'd forgotten to change the addresses, when hand-linking, after fixing the global-inits. The result is that all my assembly-function calls are three bytes too early... Which most-likely throws them into the return-area of the previous function.

I, frankly, figured something like that was the case somewhere in this... it didn't magically-fix-itself; I just used a different procedure in the third version (which was near-identical, and should've been functionally-identical, and later proved to be) when hand-linking, which reduced the chance for that sort of error.

And, frankly, I almost thought it'd be a complete waste of time to do any of this on that account...

But those globals... Five hours ago it seemed blatantly-obvious (again) they were the culprit, and again that it would be a waste of time to look into *after* their proper initializations seemed to fix it...

But now after writing this, I'm again not at all convinced the glitch I saw (and seemingly disappeared with the proper inits) could've possibly been caused, repeatedly, by such a low-odds string of just-right-conditions.

....

Day number three thousand four hundred and seventy six:

I've been strung-along on this voyage for what seems like an eternity. I don't recall anymore how long ago I could've just counted my blessings that it worked and moved forward, when instead I chose to analyze in depth what, truly, specifically, caused the particular failure I'd seen in the first place.

Today I contemplated how to determine whether the original "obvious" conclusion-- which I later determined nearly infinitesimally-possible--was truly the culprit. I eventually came up with the least-invasive modification I could think of: Find the call to that function in the original ihex, and block it out with NOPs. If, then, the problem stopped, then obviously that would indicate that either my infinitesimal-odds-estimate was wrong, or that I'd really, somehow, encountered infinitesimal-odds countless times back-to-back, during the original test.

I reprogrammed it with the original (not yet NOPed) ihex, after much debate as to whether re-running those original experiments was even worth the time---I surely remember what I saw, after all.

Turned it on...

And...

No more.

Frankly, I was a bit dumbfounded upon seeing it, since I'd seen the other SO MANY times, previously. And it took me a while to verify I'd uploaded the right version, then to piece-together (again) what *might've* changed to cause this "new" behavior.

Eventually I realized that this time (and after *many* manual resets) it acted exactly as I determined it would've if those infinitesimal-odds *weren't* met. I have those theories documented in my logs from the "deep investigation". Recalling that, eventually, I recalled how to *test* it... And sure-enough the darn thing now gets RxInts, but otherwise seems unresponsive... exactly as I'd predicted.

Hah.

This, mind-you, is *exactly* what I'd expected to see *after* NOPing-out that function-call, as well. So, I guess it was a good idea to run this experiment with the original version again before doing that, lest I'd've confused "infinitesimal-odds-theory" (aka iot) with "See? my test proves it."

Ooof.

So... I suppose it's plausible iot was correct... And, yet, somehow I "lucked out" in getting those odds the first time, because, again, it brought my attention to the uninitialized variable problem, which today's experiment seems to show rather clearly would've otherwise been one of those *very rare* glitches, had I continued-on from how it functions right now. Heh.

Now, how on earth did those infinitesimal-odds occur *repeatedly*? Theory goes something like "battery-backed RAM". And how on earth didn't they happen this time? Same theory; wherein yesterday I ran code that initialized them correctly. NOW, How On Earth did the infinitesimal-odds get there in the first place that first day, when all experiments prior would've *also* initted those variables? Theory goes? *shrug* Maybe the battery-theory combined with a few days of discharge?

Anyhow, these new findings make it exceptionally difficult to try to test these theories. I'm far too deep in this rabbit-hole... I could actually be *using* this thing, for what it was intended, this evening; could've /been/ using it for many days now... Instead of, apparently, hunting down some White Whale that actually worked-out in my favor... I dunno.

Was this worthwhile? I dunno. I dunno if I know anythin anymo