It's been a long time since I worked on this and my other projects. It's been very difficult to find time and space to work on things. But in the last few evenings I was successful in making small steps to achieve significant goals to advance some of my projects. Such as this one.

TXX, We Love You, but...

As you may remember, TXX is a module that I wrote to allow the Propeller 1 to generate data in various formats to a serial port at high speed, up to 8 megabits per second. It started here in this project because I needed a way to generate fast data logging, but it grew out into a fairly significant reusable module which was well-received in the Parallax forums. I even uploaded it to the OBEX website where Parallax previously made user-submitted projects public.

However, OBEX doesn't exist anymore (Parallax moved their user-submitted projects to GitHub) and though I can't find my TXX protect in there, I realized that it wasn't that easy to find it in my own GitHub repositories either: it was in the PropSPDIFDecoder repo, in a branch that kind of dead-ended, and that branch wasn't the default branch in the GitHub web-gui so you had to know where it was.

So I figured out how to do a filter-branch on a clone of the repo, and start a new repository with just the TXX module (and its demo module) in it, so it could have its own rightful place. You can find it at https://github.com/jacgoudsmit/TXX.

I pruned the dead-end branch and the txx project away from the PropSPDIFDecoder project. I will still use TXX as part of the project, but now I can just insert it as an external. That might give some trouble with the Propeller tool which doesn't like to import modules from other directories, but I'm sure there are ways around that.

Still Struggling with Subchannels

I've been thinking for a long time what the best way would be to get the subchannel data from an SPDIF signal from a DCC recorder into a PC or into another microcontroller. I thought of serializing the subchannel data by itself on the Propeller 1, and sending it over the serial port in blocks. Maybe even use multiple serial ports for the User and Channel subchannels.

But then I thought: "Why not both?" With TXX it should be possible to forward all the SPDIF data to a PC and a PC could process it.

Or not?

Snag 1: Bandwidth

The first snag to hit that plan is that it's apparently really difficult to find USB to serial converter chips that can be used with baud rates higher than 3 megabits per second. And of course the FTDI chip that Parallax puts on their FLiP modules is no exception and can't be easily replaced anyway, so if someone would want to replicate my circuit, they wouldn't be able to use the FLiP which would make it much harder to build the circuit on a breadboard.

3 mbps is pretty respectable for a serial port but it won't do for a 48kHz stereo SPDIF data stream. For that you need roughly 3.072 megabits per second: 48000 subframes per second, times two for stereo, with (for simplicity's sake) 32 bits per subframe. 48000 * 2 * 32 = 3072000. And that's continuous traffic, not including things such as start-bits, stop-bits and time for the Propeller to set things up in the TXX cog. So yeah, that's not going to work.

Snag 2: Synchronization

A second snag is that serial ports transfer their data as bytes, not as 32 bit words like SPDIF. Imagine we would somehow find a way of sending the 32 bit subframes as 4 bytes each without the bandwidth problem; it would be almost impossible for the receiver to reliably determine where each subframe starts, unless there's some sort of synchronization protocol. A 9-bit scheme where the first bit would indicate the start of a frame or subframe would be possible if another microcontroller is the receiver, but is pretty much impossible with a USB to serial converter.

So I thought about possible compromises. Let's see what kind of data is available in each subframe:

• There are potentially 24 bits of audio data. However, no DCC recorder generates more than 20 bits of audio data.
• There are two subchannel bits (User data and Channel Status), however the Channel Status subchannel only requires half the bandwidth: one subchannel bit per frame, not subframe.
• There are two bits that indicate whether a subframe is usable: The Parity bit and the Validity bit. But the transmitter can easily just mute the audio when it detects that either of these two bits is wrong.
• The preambles help us distinguish between the left channel and right channel, and indicate the beginning of a block, once every 192 frames. Internally, we store these as two bits in the subframe (and we store them twice so as not to influence the parity, but that's irrelevant).

There's some redundancy in there, and I thought it would be great if it would be possible to transfer each subframe as 3 bytes instead of 4 bytes in order to reduce the required bandwidth by 25%, and then switch the bits around a bit so that, for example, the most-significant bit of each byte could be used for synchronization. If I reduced the audio to 20 bits of a 3-byte frame, it would leave 4 bits per subframe that could be used for other data and synchronization.

I thought about setting the first msb of a 3-byte subframe to 1, and setting the last msb of a subframe to 0, so that the synchronization code could look for a byte with a cleared msb followed by a byte with a set msb. But on second thought, I didn't like that idea. I didn't work it out any further but let's just say I had a funny feeling that that wasn't going to work very well. Besides, it wouldn't leave enough space to put the other bits, unless I would reduce the audio data even further.

But if I would design a protocol that was a full frame (i.e. samples for the left and right channel), in 6 bytes, it would make things a little easier:

• There would not be a need to distinguish between a left subframe or a right subframe, because if the receiver could sync to a frame, it would know that the left subframe is always the first 3 bytes and the right subframe is the last 3 bytes.
• The beginning-of-block marker and the Channel Status only need to be transferred once per frame, 

So I came up with the matrix as shown above. I may make some changes later on depending on how practical it turns out to be once I implement it. As a matter of fact, I've already thought of a change to make synchronization easy:

The protocol consists of  6 bytes per frame. The lowest 7 bits of each byte contain audio bits as well as the User subchannel bits. The most significant bits contain the once-per-frame bits: The block detection and the Channel Status subchannel.

The first byte has its msb cleared to 0, and the last two bytes of a frame have their msb set to 1. The way the other bits are arranged in the msb's of each byte, makes it so that the beginning of a frame can easily be detected by finding the first byte that has its msb cleared, after at least two bytes that have their msb set.

Conclusion

All of this is not implemented yet, of course. I just wanted to get it out here because I'm excited that the project can make progress again after it got a bit stuck. I think it's a viable solution for getting SPDIF data including subchannels out of a DCC recorder (or another device), and into a PC for further processing, through a common interface that uses chips that are common and easy to get, and very few passive components. I'm looking forward to sharing recorded subchannel data with fellow DCC enthusiasts and especially those hackers (you know who you are!) who are working on ITTS decoders and are eager to get their hands on some real data, even without the need to reproduce my SPDIF decoder hardware or even the need to own the DCC's that contain the data that they want to work with.

EDIT: That's not going to work.

I did some tests with the code from my TXX project, and the news is not good. When sending a sustained stream of serial bytes at 3 megabits per second (with no flow control), the FTDI chip on the Parallax FLiP drops tons of characters because it can't keep up. I had to insert delays between the characters to drop the throughput down to about 72900 bytes per second, in order to make it not drop characters.

That means there's only enough time to send a single byte per S/PDIF frame.

A single byte is enough for the subchannel data (3 bits per frame, plus probably a bit to indicate start-of-block) but it definitely doesn't leave enough bandwidth for high-quality audio.

Maybe one day I'll figure a way to generate uLaw or aLaw encoded audio into the output, or use a different serial-to-USB converter with better performance, but for now, it's back to the drawing board (again).