VGA Graphics Over SPI and Serial - VGATonic

As of tonight, VGATonic has all of the initial promised specs working:

• Hardware supported: 640x480, 320x240, 160x120, 80x60
• Hardware supported: 8 bit depth, 4 bit depth (CGA/VGA original standard), 2 bit depth, 1 bit depth
• Hardware 'Acceleration' by Skipping Rows
• (Old Stuff):
  • Linux drivers, Arduino Drivers, Firmware published, etc.

When I first laid VGATonic out, I put a microcontroller onboard with a number of pins connected to the CPLD... I was, of course, planning for feature creep.

The ATTiny 2313a is an interesting little microcontroller - it only has 128 Bytes of RAM, but it has a hardware UART, and a hardware 'USI', or Universal Serial Interface. The USI we will use as 'SPI Out' to the CPLD (driving VGATonic just like the rest of our demos), and the UART we'll use to communicate with every piece of hardware for the last 50 years (hyperbole - but you know what I mean, right?).

Here's the relevant section of the schematic:

Since SPI is working perfectly on the CPLD, I wont be getting in the way. Instead, I'll use one of the AVR_CPLD_EXT pins to send a signal to the CPLD that the AVR wants to be the master, and avoid the entire mess by just shifting where MOSI/SCLK/CS come from. It's a clever trick (or at least I think so). Here's the relevant VHDL:

	if (SEL_SPI = '1') then
		SEL <= AVR_SEL;
		SCK <= AVR_SCK;
		MOSI <= AVR_MOSI;
	else
		SEL <= EXT_SEL;
		SCK <= EXT_SCK;
		MOSI <= EXT_MOSI;
	end if;

Again, very excited to be switching from VHDL to C here... let's see what we can make the Microcontroller do!

I've leveled up my "rewrite this VHDL to optimize for pterms/macrocells/function block input" skills, and I think I've got the hardware working perfectly as of tonight. I had to remove some 'nice to have' stuff - double buffering my clock domain crossing for writes and resets being the biggest - but the version I've got up on Github as of a few minutes ago is about my maximal effort: https://github.com/dqydj/VGAtonic/tree/master/First_Draft/CPLD Firmware

I'm a software guy at heart... bring on the Microcontroller! Let's add UART input, shall we?

(Exactly how we're going to arbitrate that I'll write up a new post in a few minutes).

Most everything I've got planned involves the Atmel silicon onboard, the ATTiny 2313a.

The ATTiny is the (128 Byte) brains of this setup - it has a Universal Serial Interface which I am using as SPI. For the early demo, the first thing the board does is program the LTC6903 to get 2x VGA frequency (target: 25.175x2 MHz).

Most of the next work will involve the ATTiny. Here's the fun I've got planned.

Tuning

• LTC6903 Frequency changing/tuning. (Hardcoded currently)

"Stretch" Goal

• Asynchronous Serial/UART control(!).
  • SPI is great, but asyncronous serial is universal! If I say 8-N-1, you know exactly what I mean, right?

"Super Stretch" Goal

• World's smallest terminal emulator*! (I have 128 Bytes of RAM, plus the above needs to fit...)

Notes:

• I called asynchronous serial a stretch goal when I started, but it's a real goal now. Consider this my promise to you. Serial is a language that almost everything electronic speaks - plus it'll be worth it for the "actually, we've got that.." aspects of driving VGA over a serial port. And the laughs.
  • (But, seriously, we have a black and white 80x60 mode - that only needs 4800 bits for a whole update. We can drive that with no problems, in theory...and it'll make a great demo that VGATonic is hitting its goals!)
• Maybe the smallest emulator, but since early teletype machines just printed out characters they could do it with barely any memory. Still - 128 Bytes.

I wanted to give you folks a progress report on how far I've come with the project, and what I've got planned for the next few drops. I've made a ton of progress in the last 15 days - fueled by a ton of motivation and (unhealthy amounts of?) caffeine.

Here's a log of our progress. For every instance, I've made an effort to comment on my efforts, both in entries and inside the code (I attempted to overcomment, so please let me know if it helped.)

For many hardware decisions, I have detailed them in last year's logs. For the complete picture (video?), you may want to take a look.

Starting Point:

• 640x480 VGA at 8 bit color
• 2.5V, 3.3V, 5V tolerant SPI
• Drivers:
  • Raspberry Pi 2 Model B (at 25 Frames Per Second), Linux
  • Arduino Uno (at 3.25 Frames Per Second), Bare Metal
  • Intel Edison in Arduino Mode (at 10 Frames Per Second), "Bare Metal"

2015 Completed Goals:

• Allow hardware assisted lower resolutions. Complete!
  • 320x240
  • 160x120
  • 80x60 (Yes, XServer accepts it and launches!)
• Allow hardware assisted lower bit depths. Complete!
  • 4bpp (16 Colors VGA/CGA Standard)
  • 2bpp (Black, Dark Grey, Light Grey, White)
  • 1bpp (Black, White)
• Allow hardware 'acceleration' to minimize serial writes. Complete!

Hardware Support (in order completed) (see Github):

• Arduino Uno (and a Nano!)
• Intel Edison in Arduino Mode
• Raspberry Pi 2 Model B at 62.5 MHz SPI
• Intel Edison at 100 kHz bitbang SPI (will revisit)
• Odroid C1 at 32 MHz SPI

I will post a roadmap in a few minutes.

I wanted to rush and get this checkbox checked tonight - I've got very basic 'Hardware Acceleration' working in VGATonic!

The quotes? Just to protect me from your arguments on the definition of 'acceleration' - we literally picked fruit from the lowest possible branches on this one. And, yes, it's a subtle feature anyway - I'll prove it not with a screenshot but with the code:

WRITE_ROW <=    '0' 				& -- 9
  ( WRITE_DATA(7) and WRITE_DATA(6) )  	 	& -- 8
  ( WRITE_DATA(7) and (not WRITE_DATA(6))) 	& -- 7
    '1'                                	 	& -- 6
    '1'					        & -- 5
  ( WRITE_DATA(7) xor WRITE_DATA(6) )      	& -- 4
  ( WRITE_DATA(6) )                        	& -- 3
   "000";                                         -- 2, 1, 0

Catch that? (bonus if you catch the wrong math - will be fixed, but after midnight PST, haha!).

That's right - all we're doing is letting the control bit move where we are writing to the framebuffer. With 2 bits, we can divide the screen into quarters (for tonight that was the goal: I only wanted 2 bits - if you move to 3 bits, 80x60 resolution won't divide evenly into 8s!). This presents a very, again, subtle improvement in some cases - say, moving the mouse near the bottom of the screen.

Previously, you were blasting an entire framebuffer every screen write, even if the top of the screen was static (say... a menubar?). That means the old math applied - if your bottom right pixel updated, you were doing 2.5 million transfers (for 640x480x8bpp). At 32 MHz, say, on an Odroid C1... that's 13 frames a second. Now you're doing 1/4 the transfers, which means your throughput can be 52 frames per second for many refreshes.

Remember I joked about Doom not being a good idea at 25 frames per second? It's a bit better at 100* - considering you're now outrunning the VGA refresh rate of 60 Hz on a Raspberry Pi refreshing 1/4 of a screen.

*(Of course, video games tend to redraw screens a bit more than your desktop or TV Tropes binge).

Okay, I'll post my status report in a few minutes here.

At least I'm publishing this post at a reasonable hour - I'm happy to announce I've got lower bit depth colors working!

If you just want to see the code, I posted it on Github.

Let's dive in for the rest of you; here's VGAtonic being meta and browsing on its own page in 640x480 at 1 bit color (translation: Black and White. If you're just following us now, we started with color and we're working backwards to support slower parts):

Gross, Black and White?

(For the record: This release also includes 4 bit color - 16 colors, and 2 bit - 4 'colors' consisting of black, white, and 2 grays)

Well, I'm excited about the Black and White anyway.

The point, with apologies to Dennis Rodman, is you can set up VGATonic "as bad as you want it to be".

If you've got fast SPI? Sure, 640x480x8bpp

If you've got slow SPI, or you're bitbanging it? Well, as of the release today you can go as low as 80x60x1bpp.

• 640x480x8bpp: 2,457,600 bpS
• 640x480x4bpp: 1,228,800 bpS
• 640x480x2bpp: 614,400 bpS
• 640x480x1bpp: 307,200 bpS
• ...
• 80x60x1bpp: 4,800 bpS

If you're willing to drop down to 80x60x1bpp, that's a 512x speedup. You could bitbang at 100 kilohertz and still get around 21 frames per second for full screen refreshes. The 56k modem you dialed into AOL with could do 11 and change frames per second.

The Theory

In the last article I talked about how we could do tricky things with our LSBs to move around the screen and sit on certain pixels, and when we wrote to VGAtonic we just skipped to those positions.

Colors, it turns out, are even easier - in a single SPI 8 bit 'transaction', you just pack more pixels in. So, once VGAtonic 'knows' it is in a lower bit depth mode, it will start to shift pixels out of the single byte.

That means something like this:

For Clock = "000" to "111" do this:

    Pixel <- Data >> Clock

If Clock == 0
  Pixel <- Data (shift right) 7
If Clock == 1
  Pixel <- Data (shift right) 6
If Clock == 2
  Pixel <- Data (shift right) 5
If Clock == 3
  Pixel <- Data (shift right) 4
If Clock == 4
  Pixel <- Data (shift right) 3
If Clock == 5
  Pixel <- Data (shift right) 2
If Clock == 6
  Pixel <- Data (shift right) 1
If Clock == 7
  Pixel <- Data (shift right) 0

A Note on the Platform Change

I hinted at it in my last post on hardware assisted lower-resolutions, but I switched platforms and I'm doing this current work with an Odroid C1 instead of the Raspberry Pi 2 B from the initial demo (the one I played Doom on). The Pi has nuked 4 SD cards - so, yeah, I'll port all the new features when I'm happy with the progress, but the C1 has been reliable since I switched.

But, hey, I've now seen it working on an Intel Edison (I'll revisit it soon - I was having SPI issues), an Odroid C1, a Raspberry Pi 2 B, and various flavors of microcontrollers. That's worth something!

I've got some good after-midnight Pacific Time news for you: hardware acceleration of lower resolutions is working!!! (Macrocell count: 137/144. Pushing it...)

This is a great result for slower-speed parts, since you can still get a usable resolution out of any VGA monitor you've got without forcing a "slide show" through 640x480 resolution.

Here are the stats for 8 bit color; bpS is 'bits per Screen'

• 640x480: 2,457,600 bpS
• 320x240: 614,400 bpS
• 160x120: 153,600 bpS
• 80x60: 38,400 bpS

So, if you only had 2 MHz SPI on your part, you were staring at .8 frames per second... but if you drop down to the hardware accelerated 160x120 you can now support 13!

Here's a quick snap of a desktop at 80*60, for the laughs and the proof (bonus: can you identify the single board computer at this puny resolution? I'll release a new Linux framebuffer driver drop after the VHDL is good!):

Pretty funny, eh? Well... I thought it was funny, anyway.

(Side note: everything is working beutifully in Linux; I can do my resolutions changes with 'fbset -fb /dev/<mine> -xres xxx -yres yyy' just fine. Props, too, for xserver dealing with 80x60...)

And the theory:

Rows are easy; for each drop in resolution, double the number of rows we 'loiter on'.

480 Rows = 1 Clock
240 Rows = 2 Clock
120 Rows = 4 Clocks
60 Rows  = 8 Clocks

Columns are a little trickier - I had to find a way to skip ahead, but the 'loitering' part isn't that bad - for each drop in resolution, I shift the LSB. I'll write out the bits so you can see the pattern:

640 = 0000 0010 1000 0000
320 = 0000 0001 0100 0000
160 = 0000 0000 1010 0000
80  = 0000 0000 0101 0000

You know, if I can fit them into 7 macrocells.

There was a bit of interest in some of the project's goals for this year, but it really boiled down to this question:

"If you're supporting 640x480x8bpp, why would you want to have anything worse?".

The reason, as always, is resource constraints.

I was able to demo a Raspberry Pi 2 playing Doom over SPI, but that part has excellent SPI speeds - I can reliably blast 62.5 MHz down SPI with only rare hiccups. This makes for good Doom-in-native-resolution demos, but isn't realistic for every platform.

Lots of platforms are more modestly spec'd: the Arduino Uno in your drawer probably can do 8 MHz SPI (and where are you going to store 307kB of VRam? You'd have something hacked up like my 640x480 Arduino demo project.) If you had to update the full screen every time (say, a flashing light) you'd get a slideshow at roughly 3.26 FPS. Maybe that's enough for many purposes, but many (most?) of you would certainly be itching for more speed. Similar math applies to other parts: take the speed of SPI in MHz and divide by 2.5 for the 'full frame update' FPS speed.

Okay, sure, you don't always update the full frame - that's why we'll (try to) add some basic hardware acceleration (the most rudimentary acceleration is already in the firmware - just stop writing the frame and you can reset to 0,0. If you're only updating the top left pixel you can get 1,000,000 updates out of the Arduino!).

Realistically, though, we should add more options. 320x240 requires 1/4 the writes - suddenly the Arduino is doing a respectable 13 FPS over SPI. 160x240 and 80x60 more still.

Bit depth works in a similar way. While 8 bit color requires the whole SPI byte, B&W, for example, can pack 8 pixels into the same byte!

I'll surely expand on this if I can actually fit this into our own resource constraint - my insistence on the 144 macrocell CPLD. (I enjoy the challenge, haha). But here's the math on one planned range:

• 640x480x8bpp = 2,457,600 bits per full screen
• --- all the way to our most modest setting ---
• 80x60x1bpp (B&W) = 4800 bits per screen

Now you see the real trick - the lower settings are acceptable even over the serial port. Interesting, huh?

Also, for those interested in graphics, follow and skull HOMER, another graphics entry in this year's contest by monnoliv! I'll be following along with the progress intently, and you should too.

Let's discuss this year's prompt: Building something that matters. And, yes, the secondary prompt: is this a useful product?

Is This Something That Matters?

Does an open source graphics card matter?

Maybe, I'm not a judge - but I'll try to make the case with a quote - as Vince Vaughn's Character Trent famously described Jon Favreau's character Mike in Swingers, "This is the guy behind the guy behind the guy."

The card itself probably doesn't matter much on its own - but, hey, if it saves someone some time developing something bigger, or getting them started on a GPU project, or easily adding graphics to something larger... it matters. Everything is documented - from the drivers on the platform, to the firmware on the CPLD and the Microcontroller, to the protocol itself - then even to the hardware level if you choose to build a board. That's worth something, and if time is fungible, it's worth freeing up someone else from the hours I spent - just as I didn't have to reinvent VGA to get VGATonic working.

So if you see this post and need easy and cheap graphics with a cheap and easy connector and protocol ubiquitous on cheap hardware, follow and click through to the GitHub link to see what we've got.

And, yeah, it works fine today in this limited release... if you want to integrate something quickly:

And the Product Part?

Well, that's the easy one. Since this year's logs will likely concentrate more on firmware, software, and platform support (and perhaps new protocols?) than board revisions you'll see the most improvement in these arenas - the "productization areas". Don't rule out hardware changes, though - we'll discuss that if/when it comes up.

Sure, forcing a processor to do graphics may seem like a bit of a step backwards and a relic of the 80s, but there are places where it makes sense to plug something like VGATonic in... you know, on hardware that doesn't need to play 3D games.

But an < $25 graphics card which can drive VGA (and maybe-even-NTSC) monitors? That's a product right there.

I'm back for round 2 after VGATonic version 1.0.

I finally got the motivation back to complete the framebuffer driver when I picked up a Raspberry Pi 2 B. Here is last year's effort (and, ahem, this month's accomplishment) running a game I think you may have seen before:

There's lots of work still to go, and we're going to move in an interesting direction at first: getting it to run on 'slower' hardware. Not every board can spit out reliable 62.5 MHz SPI, so it seems like aiming lower is better than higher, in this case!

Follow along and maybe we'll hook VGAtonic up to some really esoteric stuff by the end of Hackaday Prize 2015?