Why did I build this project?

I've always been into low-level development. Recently, I've been experimenting with AI, specifically creating models that can run on edge devices. I saw Andrej Karpathy's MicroGPT, and I was inspired by its "lack of libraries" - it seemed to conclude the entire AI industry in only 243 lines of code. I wanted to see what would happen if I took this further: demonstrating an entire AI stack, with every single line of code written by me: strictly no dependencies.

The Building Progress

KernelGPT started by taking the MooseOS source and removing unneeded functionality — filesystem drivers, ATA support, GUI, etc. The goal was a minimal, mostly barebones kernel to host the LLM. 

Before I started, I converted MicroGPT’s names.txt file into a C header so KernelGPT could access the character-level vocabulary entirely in memory. While it felt a bit "hardcoded" and unreadable, it ended up working so well that I ended up accepting it as a design choice.

Inspired by Andrej Karpathy’s MicroGPT, I implemented a GPT-2 style LLM directly in kernel space. Key GPT-2 modifications from MicroGPT were retained: RMSNorm instead of LayerNorm, and ReLU instead of GELU. All tensors are allocated from a pre-reserved arena for simple memory management. The only non-AI addition was the FPU, which was initialized in kernel mode to allow floating-point arithmetic .

The overall coding experience was very smooth. The only minor mishap I experienced was when I spent a full hour chasing a General Protection Fault — only to realize it was because I forgot to disable interrupts, and the PIT timer was still firing even though I had removed the system clock! Funnily enough, all my debugging code pointed to an issue with the matrix multiplication; it was only when I decided to check which files I had removed when I noticed the real cause of the issue.

Overall, this is one of the projects that really made me feel proud of myself. Personally, I think you shouldn't code for the money and fame. Instead, you should code for the sake of coding. The feeling of thinking "wait.. I'm 15 and I built the ENTIRE THING by myself?? Wow" is really unmatched; I haven't felt the feeling in any other subject.

I hope you enjoyed the read, and have a nice day!