Close

Back into the Maze of Twisty Little Passages

A project log for Deep Sneak: For Lack of a Better Name

The mixture-of-experts and multi-model meme is in play, so lets let the reindeer games begin, even if we missed Christmas with Deep-Seek.

glgormanglgorman 3 days ago0 Comments

So I am starting to figure out some of the issues with the original Atari Genetic algorithm code, and there are a bunch of issues, despite the fact that the original code does actually appear to work - sort of.   For those that love to stare at debugging sessions, this will explain volumes.  Maybe I could ask Deep Seek for an explanation of what is happening here?

First the good news.  I completely refactored the C code that Deep Seek gave me, so as to have something that for now is in C++, and which I think I can eventually adapt to run on a Propeller, or on an Arduino, or an NVIDIA GPU, as discussed in the previous log entry,  It is also quite easy to see that every element of the array m_out0 is getting filled with the same value, which I am pretty sure was also happening in the original Atari BASIC code.  So clearly, some more work needs to be done on the topology of this network.  Yet I have definitely had some substantial improvement in the performance of the algorithm by adding some gain and bias parameters.

Therefore, since I have a short, simple function that can easily be modified to work according to some different theory of operation, I can proceed therefore, with some of my ideas about running something like this in multiple threads, on the one hand, and/or adding some kind of evolutionary selection component, as previously discussed.  In Windows of course, a simple call to a training function via AfxBeginThread is all that would be required to do the actual model training and instance evaluation, this is to say - in and of itself.  Furthermore, all of that should work just fine, without needing a whole bunch of locks, semaphores, or event handles or whatever, that is to say - until we get to the reportage sections. 

 Obviously, we don't want multiple threads attempting to call WRITELN, at the same time.  Yet if I add a critical section to the code where the reporting takes place, then this could result in blocking problems from time to time.  Perhaps a simple mail slot system would be the way to go; so that a thread can create a report and then "submit it" to the IO queue as a completed text-object.  That doesn't seem all that hard, given that most of what I would need to do that, already exists in my PASCAL style IO classes for WRITELN, etc., and where I could perhaps modify the WRITELN class to allow writing to a TextObject, maybe something like this:

#include "../Frame Lisp/intrinsics.h"
#include "text_objects.h"

...
TextObject debug_text;

    lisp::begin_message(debug_text);
    for (i=0; i<np; i++)
    {
        x = m_ds->x(i);
        y = m_ds->y(i);
        arg = double(i)/m_ds->size();
        value = eval0(w1,i);
        writeln (debug_text," >>> X=",x," Y=",y, " G=",value);
    }
    lisp::end_message(debug_text);
    lisp::send_message(output,debug_text);
...

That way I can put all of the memory management, as well as the synchronization objects in the LISP and/or PASCAL library text manipulation and/or IO handlers.  Hopefully in such a way that there are no blocking calls that ever have to be made from within this otherwise critical loop.  

And I still need to get back to reading the code to see how Deep Seek really works, as well is putting some of this to use with my braiding functions.  Fortunately, there is quite a bit that is already done, at least as far as all of this other IO stuff is concerned,

Something to keep in mind therefore, is that there is already a project on Hackaday, as well as a source code repository on GitHub for a framework that I was working on, as hard as it is to believe right now, for bespoke application builder that I made in 2019 with the intent that it allow me to interact with Propeller and Arduino projects over a USB connection, among other things.  

Therefore, a lot of the heavy lifting has already been done, that is - since that code base made heavy use of my Frame-Lisp library, among other things.  So, yes - a lot of heavy lifting has already been done.  Maybe I should go back and rewrite the messaging system from that project, in such a way as to be able to support the new APIs that I just suggested that I would like to have. 

Or I could give the code to Deep Seek and see if it is up to the chore!  Ha! Ha! Ha! Ha!

All, fun aside, it is easy to see how the idea of figuring out how BlueSky works on the one hand, with its AT = Authenticated Transfer Protocol, vs. whatever Deep Seek is doing, with its Mixture of Experts Model, and whatever that actually does.

Yet there is some other news of course, and that is that I finally did get back on track with trying to create a CUDA-aware version of some of the stuff that I have been working on., and that includes an update to a very weird method for doing number base conversion according to a convolutional algorithm, that is instead of the typical "schoolboy" method commonly referred to as serial division.  Getting some of this stuff to just barely try to link in VS2022 took almost a whole day, yet finally, I have some progress in that direction.  That means of course, that I am also on track with the idea of further modifying my PASCAL-style IO in C++ library to eventually be able to be used concurrently be usable from multiple threads, that is to say - without getting blocked or garbled, whether the code is running on NVIDIA hardware, or on an Arduino.

And that also means that there should hopefully be a Git repo coming out of this .... real soon now.

And so, it goes.

Discussions