Close

Do or do not. There is no try.

A project log for Let's Build a Modern LLM - Entirely from Scratch

While there are several open source versions of various AI applications out there, there is still very little out there for doing training.

glgormanglgorman 07/15/2026 at 03:270 Comments

Well, sometimes we need try and catch.  They are convenient.  Pascal, on the other hand, has an arcane way of exiting a function by using an EXIT function.  So function A can call function B, which calls procedure C, which calls D, I think, and then procedure D can call EXIT(B) if it wants to, and then even if we are 999 levels of recursion deep with C calling D, calling C over and over again, it should all unwind back to wherever we would have been if we unwound everything normally, via the normal return paths.  This can be emulated in C++, in those situations where you have some legacy Pascal code that needs to be rewritten.

class EXIT_CODE
{
public:
    bool m_edit;
    int     err;
    char *m_str;

public:
    EXIT_CODE(char*str);
    EXIT_CODE(int n, bool edit);
};

 Just create an EXIT_CODE object and populate it with a string or an integer for the error code, if that is what is required, or else set a flag to invoke the editor.   Maybe.  Yet for me, the really fun thing is still knowing that I can get away with having a mostly Pascal-compatible version of write, and writeln, that not only works from regular Visual Studio Code but also seems to work from CUDA code.  Which might seem to defy explanation, but it does seem to work!

Believe it or not.  So, naturally, I think I need to do some more work on this concept.  Like maybe it should be able to do certain types of asynchronous IO automatically.  So that the user does not need to worry about locks and semaphores, buffering, and so on.  Most of that I am already doing in the Propeller Debug Terminal, but there are still some differences between how the CUDA interface works and the way I did things in a C++ port of a version of the UCSD Pascal compiler from 1979, and so on.

Likewise, when thinking about tokenizing an input stream, I have an Eliza-based method, as well as a complete rewrite of the UCSD Pascal compiler's INSYMBOL function to choose from.  Maybe.

This version shamelessly uses the use of goto in C++, but it also makes for a good use of a C++ implementation of Pascal-style sets to check for different character types.  All in good fun.  So you thought that writing a tokenizer is trivial?  Wouldn't it be nice if we could specify something that works like BNF, without having to use LEX or YACC or Regex?

Be very afraid.

I think it can be done.

But not quite there yet.

So, in the meantime, back to brute force methods.

Well, then again.  There is a version of Propeller Debug Terminal that I wrote back in 2019, which is able to host an instance of Eliza, as well as an instance of Mega Hal.  So, maybe now is a good time to work on also getting it to host an instance of Lotto Pro, when built as a library instead of as a standalone application.

So that will be the next step then.  Get "Lotto Pro running", as close to as-is as possible, inside Propeller Debug Terminal, that is, as a statically linked library.  Then it will be possible to start the aforementioned complete rewrite of the core of the application, so as to perhaps try to make some kind of collection of "mixture of experts models" that all run in parallel, and which can make use of each other's strengths and weaknesses.

"Do, or do not!"

Alright then.  Let's try this!  It is deep.  Dark.  Messy.  Be very afraid!

For a very long time now, like ever since Windows 95, there has been this technology called "Microsoft Foundation Class", and one of the very nice things that is available in MFC is something known as "Document View Architecture".  Which in turn is a VERY nice piece of tech, even to this day.  That is, when you can get it to do whatever it is that it is supposed to do.  Unfortunately, in practice, that often requires a minefield of tricks and hacks to get the framework to jump through all of the desired hoops.  Things like manually editing resource files across multiple projects, to try to get them to play nice with each other, and that is just the beginning.

Whether it is obvious, as to what is happening here, may not be so obvious.

When a multiple-document, multiple-view application is initialized, there are a bunch of data structures that are created, called "Document Templates," that tell the framework what to do when we want to open a document, whether from a file, or as a new instance.  So we need to specify the type of document, according to the class that handles that particular type, as well as associate that document with a view, and also an appropriate menu bar. When it works, it works very nicely, since we can have a document such as a "Terminal Document" that might have multiple views associated with it, such as a text view, or an oscilloscope view, or a spectrum analyzer view.  Likewise, each combination can have its own unique set of menus, or they can run off of multiple copies of the same menu system, with or without variations.

So this is how I think I am finally going to get Eliza to talk to Megahal, or to have instances of Megahal that can chat with each other, such as an instance trained on "Alice In Wonderland" that might be set up to have conversations with "The Federalist Papers"

So, yes.  Just getting to a "Hello World State" with "Lotto Pro" being hosted inside another application is a pretty big deal.  Not just for what it does, in and of itself, but for what it makes it possible to do with other applications, according to the time-honored method of "figure out how to do it" then "copy the technique.

To be continued.  Then again.  At least now it is doing something!  Right now I have come to realize that there is a massive amount of "accumulated" technical debt that needs to be overcome before any new "real work" gets done, and who knows?  That is probably a major problem across the industry as a whole.

Yet I have managed to get quite a bit done in the last couple of days.  Lotto Pro now works with Unicode, without crashing when I try to use the std libraries for things like iosstream, fstream, and so on, because I am no longer using those libraries to load the lottery history file.  Not getting 9999 or more linker errors from having incompatible libraries, duplicate functions "already defined" errors, with warnings to "use NODEFUALT LIB option," which then results in a few 1000 more "unresolved external symbol" errors when the linker can't find the right version strcpy, or whatever.  Also nice.  So what I did of course, was to "borrow" some code that I wrote for the C++ port of the UCSD Pascal compiler, in order to get around a nasty problem of the iostream libraries not wanting to accept the UNICODE string format when being passed a file name as generated by the file picker.  And that, of course, is just one of many rabbit holes that have to be dealt with when needing an ASCII-aware terminal mode, in an otherwise purely UNICODE application.  One method is to make shadow copies of everything, and only generate the UNICODE version of a string when it is time to display it.  The other is to want to go "full Monty" and completely re-invent string handling altogether, so as to have a concept of being able to have "text objects" that themselves might be managed linked lists of dictionary pointers, and so on, which can hopefully be extended to more fully embrace other "token" types, that is based on certain LISP concepts.

Thus, the original Pascal INSYMBOL technique retains some appeal, since it, or something like it, might be more easily adaptable to a more generic implementation, i.e.. one that can recognize not just Pascal keywords, operators, identifiers, numbers, etc., but something that can handle more general tokenizing problems, as if it were more general, like regex, on the one hand. but ideally we would like to be able to "infer when a flat file might just as well have been written out as a CSV", so why don't generate some JSON on the fly, which can be used to create the BNF-like metadata that the lexer needs to import the data.

Or just hard code it, again, and figure out the more generic problem later.

Discussions