Nyx: Edison - FPGA Devboard

Reverse Engineering

As I mentioned in my last post I decided to use SDIO as an interface between the Edison and the FPGA on Nyx. The only problem is that in order get the information to design a proper SD host controller and SDIO device you need to pay lots of monies to the SD Association and you have to sign an non-disclosure agreement.

This is no good!

We need a free version of both the SD host controller on a host computer and SDIO device in FPGAs so we can make our own devices.

There is a way!

Fortunately in 2006 the SD Association released a simplified version of the SD Specification for free. It doesn't give all the details of the various SD protocols but it has got me a long way. (EDIT, I just found a section in the simplified SDIO specifcation that documents the data bus protocol... ignore my gripes at the bottom of this post, I keep it there so people can see a SDIO logic analyzer capture)

As far as I know if you were to discover how to use a protocol by reverse engineer you can use the protocol for free. This is what the Linux community did when they wrote their own SD Host Controller.

Both the simplified specification and the various Linux drivers have helped a lot in writing an SDIO device core for the FPGA on Nyx.

The only real issue I am running into right now is that I don't have a Vendor ID number to uniquely identify my SDIO device. I can go rogue for a while when the application is small but eventually I'll need to figure out how to get a vendor ID or figure out some other way of uniquely identifying my SDIO device.

Although not finished the github page for the SDIO controller is here:

SDIO Device

There is a python script in the tools directory that will read in the "sdio_configuration.json" file to change the behavior of a core. So, hopefully, in the future this core can be used to create custom SDIO devices without spending a huge amount of time configuring the core. I plan to make more thorough documentation when it's finished up.

Simulated SD Host

After I wrote most of the SDIO device core I realized that in order to really verify it's functionality I also need a simulated SD Host Controller. I started out with this SD host core on opencores it helped a lot. I made a host controller based on this and so far I can send commands read responses to/from the simulated device.

Cocotb For Simulations

Cocotb has been really helpful because I can control the whole simulation with high level Python code. Here is what it looks like to test out a reading a few bytes from the configuration space on the SDIO device using the command bus.

@cocotb.test(skip = False)            
def receive_byte_test(dut):           
    """
    Description:
        Read 3 Bytes from the SD Configuration Space    
    
    Test ID: 3
    
    Expected Results:                                                                                                  
        Single Data Read
    """
    dut.test_id = 3
    SDIO_PATH = get_verilog_path("sdio-device")
    sdio_config = os.path.join(SDIO_PATH, "sdio_configuration.json")
    config = None
    with open (sdio_config, "r") as f:
        dut.log.warning("Run %s before running this function" % os.path.join(SDIO_PATH, "tools", "generate_config.py"))
        config = json.load(f)

    #print "SDIO PATH: %s" % SDIO_PATH
    #print "module path: %s" % MODULE_PATH
    nysa = NysaSim(dut, SIM_CONFIG, CLK_PERIOD, user_paths = [MODULE_PATH])
    setup_dut(dut)
    yield(nysa.reset())
    nysa.read_sdb()
    yield (nysa.wait_clocks(10))
    #nysa.pretty_print_sdb()
    driver = wb_sd_hostDriver(nysa, nysa.find_device(wb_sd_hostDriver)[0])
    #Enable SDIO
    yield cocotb.external(driver.enable_sd_host)(True)
    yield cocotb.external(driver.cmd_io_send_op_cond)(enable_1p8v = True)
    yield cocotb.external(driver.cmd_get_relative_card_address)()
    yield cocotb.external(driver.cmd_enable_card)(True)

    value = yield cocotb.external(driver.read_config_byte)(0x00)
    dut.log.info("Read value: 0x%02X" % value)
    value = yield cocotb.external(driver.read_config_byte)(0x01)
    dut.log.info("Read value: 0x%02X" % value)
    value = yield cocotb.external(driver.read_config_byte)(0x02)
    dut.log.info("Read value: 0x%02X" % value)


    yield (nysa.wait_clocks(1000))

At the end you can see I am reading from 3 address 0x00, 0x01, 0x02

Here is the output of the test, near the bottom you can see it read back

0x43: where the two nibbles mean: 4 = SD Version 3.0, 3 = CCCR Format Version number 3.0)

0x03: SD Format Version Number 3: Version 3.0

0x00: Bitmask of enabled functions (non are enabled)

     0.00ns INFO     cocotb.regression                         regression.py:254  in execute                         Running test 1/3: receive_byte_test
     0.00ns INFO     ...receive_byte_test.0x2b02b909b510       decorators.py:189  in send                            Starting test: "receive_byte_test"
                                                                                                                               Description: 
                                                                                                                                   Description:
                                                                                                                                       Read 3 Bytes from the SD Configuration Space
                                                                                                                               
                                                                                                                                   Test ID: 3
                                                                                                                               
                                                                                                                                   Expected Results:
                                                                                                                                       Single Data Read
                                                                                                                                   
     0.00ns WARNING  cocotb.tb_cocotb                            test_dut.py:172  in receive_byte_test               Run /home/cospan/Projects/sdio-device/tools/generate_config.py before running this function
Debug: NysaSim:__init__: nysa started
ERROR: /home/cospan/Projects/sdio-device/rtl/cia/sdio_cis.v:52: $readmemb: Unable to open sdio_cis_rom.rom for reading.
VCD info: dumpfile design.vcd opened for output.
Verbose: NysaSim:read_sdb: entered
Important: NysaSDBManager:read_sdb: Parsing Top Interconnect Buffer
Verbose: _parse_bus: Bus @ 0x00000000: Name: top
Verbose: _parse_bus: 	Found 2 Devices
Verbose: _parse_bus: Found Bridge:
Verbose: _parse_bus: Bus @ 0x00000040: Name: peripheral
Verbose: _parse_bus: 	Found 2 Devices
Verbose: _parse_bus: Found device SDB Type (0x01): SDB
Verbose: _parse_bus: Found device wb_sd_host Type (0x23): SD Host
Verbose: _parse_bus: Found Bridge:
Verbose: _parse_bus: Bus @ 0x00000080: Name: memory
Verbose: _parse_bus: 	Found 1 Devices
Verbose: _parse_bus: Found device mem1 Type (0x06): Memory
user read 00000000
ADDR: 00000000 user wrote 00000001
. . .CMD:Args 05:01ffff00
 . . . . . 
. . .CMD:Args 03:00000000
 . . . . . 
. . .CMD:Args 07:00010000
 . . . . . 
. . .CMD:Args 34:00000000
 . . . . . 
    16.74ns INFO     cocotb.tb_cocotb                            test_dut.py:191  in receive_byte_test               Read value: 0x43
. . .CMD:Args 34:00000200
 . . . . . 
    20.96ns INFO     cocotb.tb_cocotb                            test_dut.py:193  in receive_byte_test               Read value: 0x03
. . .CMD:Args 34:00000400
 . . . . . 
    25.19ns INFO     cocotb.tb_cocotb                            test_dut.py:195  in receive_byte_test               Read value: 0x00
    35.18ns INFO     cocotb.regression                         regression.py:201  in handle_result                   Test Passed: receive_byte_test

The gobbly goop of signals look like this:

Here is where it is returning back 0x43

Data Bus

Today I have been working on sending data back and forth between the host and SDIO core using a double data rate 4-bit bus. I am sure there is some documentation somewhere (Found it!) but I couldn't find a good timing diagram showing me the waveforms (it was in the simplified SD Phy Layer Specification) for the data bus so I hooked up my friends newer Saleae logic analyzer and captured these images of a SDIO transaction where the host is initiating a command using the command bus then sending data to the device using the data bus.

Here is a zoomed in view of the command and start of data

I have been writing all the 1's and 0's in my notebook and translating it to hex to verify that it really is command 53 (0x35) which is how to tell an SDIO device that we are going to send a large packet of data on the data bus.

Hopefully by the end of the day I can send large packets of data to/from the SDIO Device.

Dionysus is going the way of the Dodo

Dionysus is a Spartan 6 LX9 Development board with 8MB of 100MHz SDRAM and a fast USB - FIFO interface between the host computer and the FPGA. It has served me well, I've developed a camera and LCD demo (Video below) as well as exercised most of the Nysa tool. but it doesn't have the space to store larger FPGA images that I need to really exercise Nysa.

I found that Nysa would eat up about 6K of the FPGA resources. This is understandable because it provides so much infrastructure but Dionysus's 9K is just too small. I'm still working on reducing Nysa's footprint but right now is the time for fixing bugs and adding necessary features, not about optimizing.

Nyx

I designed Nyx as a replacement for Dionysus. Nyx uses a LX25 Spartan 6 boasting almost three times the FPGA image space as the LX9. To talk with the FPGA I initially though of using the same interface I used for Dionysus. A USB 2.0 FIFO which achieved data rates of about 25MBs. As you can see in the demo video linked above it is possible to do quite a lot but I felt USB just wasn't the perfect solution, What I really didn't like about it was that although large transfers were pretty fast whenever I wanted to perform small transactions I could spend up to 16ms waiting for the USB stack within the kernel to respond. I've tried the modification that reduced this response time to 1ms and it did yield better results but I was never happy with the non-deterministic behavior of USB.

I almost designed Nyx with Cypress's USB 3.0 FX3 Chip but I decided against it for two reasons.

• I would need to support the Cypress Software and Nysa itself is a handful already.
• The physical size of the chip is small! With .8mm pitch balls. This requires 4mil trace and space for fabrication which drives up the price of hardware development really fast!

<DIGRESSION>

As a side note if you are interested in working with the FX3 chip I've used it on another board of mine called Prometheus. Cypress does provide a pretty good board support package. I did need to massage the software a little in order to use it with Ubuntu but if you interested in playing with it I do have a build environment here:

Prometheus FX3 Build Tool

For the brave here is a GUI interface I wrote to program and talk to the FX3 chip using PyQt

Prometheus Comm Tool

Sorry for the digression.

<END DIGRESSION>

I really wanted a fast interface between the controlling processor and the FPGA. PCIE would have been awesome but adding the gigabit transceivers required to achieve this high speed communication would come at a cost of much more expensive chips and impedance controlled routing which also drives up the cost of PCB development. If you are interested in this I am working on Artemis which has this capability and I even got the board to talk with a SATA hard drive.

In the end I found SDIO is a really great interface to get data rates at around 50MBs and near deterministic timing. Now I just needed to find a platform that supports SDIO.

SDIO

Turns out SDIO is really common! It's on almost all of the single board computers including Raspberry Pi, Beaglebone Black and so on. I just needed to choose a devboard that works well with it.

Edison

I chose Edison primarily because it has WIFI out of the box. All I needed to do was load Ubilinux and immediately I had an Ubuntu environment I could SSH into. In fact I have terminal open to my Edison to Nyx right next to this window!

Edison is not the fastest of the devboards but I don't need fast I needed good software support and it looks like Intel is supporting it very well.

I am working on the SDIO interface right now and if I can catch a break from work I'm hoping I can start debugging the processor - FPGA interface in a little while.

Board Status

I tried to reduce the cost of this prototype by choosing a really long lead time which drove me crazy waiting for the board to come back but finally it arrived last week. The guys at Lightspeed Manufacturing helped me put on the BGA FPGA and spent a sleepless night on Thursday assembling the rest.

Usually when I do my first smoke test I use a current limited power supply with probes and touch the input rail. This allows me to break the connection immediately if the circuit draws more current than anticipated. Glad I did because the supply immediately crowbared. Turns out I had the 4V Switcher off by 180 degrees. After fixing that I turned it on everything has worked well so far.

I designed in an FTDI USB - UART converter into the design and after I plugged in the Edison and powered it up I ended up with this screen:

Debian GNU/Linux 7 ubilinux ttyMFD2

ubilinux login: 

Huzzah!

I then opened up Xilinx's PlanAhead and created a simple FPGA image to test out the clock, LEDs, reset and buttons and success

Huzzah²!

There are still a lot more things to test out including:

• SDRAM
• Edison FPGA programming interface
• Voltage Selector
• SDIO
• Edison USB OTG interface
• Differential Signals

But things a looking good so far.