jupyosys

It's time to update this, finally.

'myhdl2' was not a good name, the experiments 'myhdl.v2we' (van twee walletjes eten, a dutch pun) finally ended up in a merge which is now all under the name 'cyrite HDL'. And finally, it was time to say goodbye to the myhdl architecture.

The resulting ecosystem is now considered stable and is expected to have a longer life time.

Although I do generally not endorse to up-port working and verified code, it is now considered safe to migrate MyHDL legacy to the new cyHDL API. However, due to the modular nature and the strict interfaces, you may have to revisit and clean up your code.

The cyrite library comes with a few verification auxiliaries that make it easy to co-verify old MyHDL-generated HDL against a fresh cyHDL based port.

Some of the co-simulation methods from the jupyosys side will however no longer work, for example, Co-Simulation through VPI with GHDL or iverilog. The public cyrite version is as of now released only with a CXXRTL based co-simulation module only.

Also, all FPGA target specific setups are omitted. This is simply to cook down the memory usage and to avoid maintenance issues with specific boards.

After all, the link:

https://github.com/hackfin/cyrite.howto

Hit the Binder button and wait until the JupterLab starts up. Voila.

This is the prognosed roadmap for a next generation revamp. Jupyosys development has not completely stalled, but a number of issues will not fix with the current MyHDL policy, and more features not be added. The reasons for this are hinted further below.

• Explore a different 'myhdl2' kernel concept:
  • Modules are translated into MyIRL (a different representation, see below)
  • MyIRL compiles into code objects
  • No more conversion is taking place on the fly via AST visitors
  • Revisit ShadowSignals
  • Retain compatibility to MyHDL as far as:
    • Good code will work
    • Dirty code should spit out warnings
• Compiler options: PyPy and Cython
• Strong typing: Introduce `@component` API which will make use of type checking using the Python function argument annotation:

  @component
  def unit(a : BulkSignal.Input, b : Signal.Output, *, FAST = True : bool) -> int:
      ....
  

• Evaluate hdlConvertorAST for 'native' VHDL and Verilog output -- meanwhile, we can still use yosys RTLIL for output to V*.

• Easier to extend:
  • Functions/Blackboxes: procedural instancing of (customized) primitives
  • Bulk signal classes (VHDL records)
  • HLS options with special signals keeping delay pipelines into account

MyIRL - a reformed syntax approach for MyHDL

The general problem with `if..else` constructs: they are conditional at run time (how obvious...). For translation into hardware this is unpractical. Ways out:

• Collect all conditions, create coverage vector table, and run the code with all possible scenarios. Painful and slow.
• Translate conditional constructs to a different representation. The following is valid Python code:

def test_if_else(a, b, q):
    c = AutoSig("work_c")
    If((a < 2) & ~(a == 0)).Then(
        q.set(4),
        c.set(1)
    ).Elif(a >= 3).Then(
        If(b > 1).Then(
            VarAssign(v = 1, u = a ^ b),
            q.set(5),
            c.set(3)
        ).Elif(b > 0).Then(
            q.set(9),
        ).Else(
            q.set(2),
            c.set(4)
        )
    ).Else(
        q.set(0),
        c.set(0)
    )

The class construct behind resolves such that a structure is created internally that can elaborate into other language elements or multiplexer hardware.

Reminds you of migen/FHDL? Yes, it sure does!

The difference is, that we still author in MyHDL syntax and this intermediate representation will seldom be visible.

So we would still get the benefits of both sides:

• AST analysis before compilation
• Compileable IRL for code generation (closed source modules for evaluation by customer)

As this project is dependent on a number of other OpenSource projects that are subject to dynamic development, it has resided in a kind of frozen state concerning the toolchain.

A lot of effort has gone into building up a Python library of synthesizeable constructs to verify that the internal structures are sane enough for large projects. Also, verification of a novel FPGA technology was explored, however, economical circumstances have slowed everything down quite a bit on various ends, so it all takes a bit longer than prognosed.

Nevertheless, I am happy to announce that there'll be a presentation on the Embedded World Conference DIGITAL about jupyoys and introduction to creating hardware designs in a Jupyter Notebook.

This will take place 'online' on 2021/3/5 @ 11:00 CET.

For details, see Session 10.4 in the

Embedded World 2021 Conference Program

Participation in the conference requires registration incl. a fee. The Jupyter Notebooks to replay the entire session will be made available at a later point.

To provide a monthly update at least, a short summary on the current development lane progress:

• Blackbox API considered frozen for an upcoming 0.1(after summer break) release
• Board supply package for ECP5 versa board 'in sync' with a frozen yosys .deb package
• More real life designs tested:
  • pyrv32 (RISC-V core) elements post-map co-simulation
  • Memory inference scenarios verified

The `@blackbox` interface had undergone some extensive experimenting. In short, there are several usage scenarios, not all of them completely elaborated:

• Automated inference of a vendor blackbox primitive (hard PLL, Serdes block, etc.)
• Integration with an external Verilog/VHDL module: Creating a black box stub
• Implementing a top level design ('virtual board') in several ways and configurations:
  • Simulation, Co-Simulation with external models
  • ... for different target architectures and boards

What is also going to not change for a while - so you can safely base on it for own development:

• Co-Simulation setups
• Hierarchy interface (although there are issues with the name mangling)
• extension classes: BulkSignal, GeneratorClass

Still under scrutiny:

• Tri-State interface signals: These are not yet 100% nailed down, but chances will be that they are only allowed in top level modules for synthesis.
• VHDL std_logic compatible signals that know also know 'U', 'X': They might possibly never make it into the official MyHDL tree and will be kept in a separate package.
• Latch support: Likely to end up as an error when trying to infer one.
• Memory inference: Likely to end up as Blackbox model library

Finally, what requires some attention at this point is the underlying (p)yosys API, so it might be time to build a testing pipeline to make sure the supporting pillars don't break away with future revisions.

Close to summer break, where brains stop to function and the incentives tend to be ice cream rather than a successfully blinking but heat producing FPGA board. It's time to summarize what works and what doesn't, plus a small road map:

Working:

• Designs that stick to certain design rules (undocumented, but covered by a number of test cases under `test/conversion/toYosys`)
• Integrating external Verilog/VHDL implementations (VHDL support yet subject to volatilities)
• A few ECP5 primitives (blackbox wrappers)
• Simple `$mem` block inference

Missing:

• Functional simulation models for most of the ECP5 blackbox primitives
• Automated DSP element inference (must be explicit)
• Support for negative and dual edge events inference and several I/O standards
• Support for sequential programming (@instance style)

Flaws:

• Coverage for some corner cases with variable usage (some tests
  for it are marked to fail in the test suite)
• Support for the fully fledged multi port memory inference (this is is also a yosys issue being worked on)
• Handling of yosys exceptions: these terminate the python interpreter without error shown in the Notebook environment.

As of now, a lot of complexity is introduced by legacy MyHDL behaviour and support of existing structures. To support inference of all kinds of constructs, the complexity would be very much increased (which is considered 'dangerous' at this time, as the underlying architecture might still receive a redesign).

To be able to create working prototypes anyhow, the current strategy tries to follow the 'blackbox' approach:

Instead of trying to make the synthesizer guess what it should create from a functional description, we pick things from a library that is known to work, then we wrap them such that they look like our functional description (that plays right in simulation). Even better, in MyHDL we can pack them into the same functional block. We just need to verify that the actual hardware implementation behaves exactly the same as the functional simulation. This functional block also provides the support for high level constructs, for example, a class object that supports multiplication and infers to the matching black box multiplier element underneath.

For example, a PLL on an ECP5 FPGA is defined as @blackbox entity `EHXPLLL`, including a simple (currently imprecise) simulation model for its up to four clock outputs, however, for synthesis, this model is ignored and the explicit (yosys specific) inferral routine is called to reserve the matching primitive for the final wiring. For another implementation method (be that a different synthesizer or different silicon architecture), another implementation is added as python sub-function decorated with the @synthesis(method) decorator.

This blackbox API is now considered 'set' and will only change marginally with respect to some argument/parameter handling. Now here comes the roadmap:

• Creating a library of working modules that infer and verify against yosys synthesis output as correct, using Co-Simulation
• Improve framework to swap easily between external blackbox V*-HDL implementations and MyHDL implementations
• Get all memory inference scenarios completed (this is important to run a RISC-V design in pure MyHDL)
• Support I/O standards up to the interface
• gensoc integration (SoC generator for register maps, bus decoders, etc.)
• Testing, testing, testing...

The full goal: Running just everything out of a python shell

Still far away from synthesizing just any design, but when following the design rules, things basically bake into a *.svf file that downloads to the corresponding target board.

Last commits (inside the 'hacker space') add the mandatory blinky example for the Versa ECP5 development kit from Lattice Semi.

Those who have been dealing with the Diamond toolchain for a while may remember some pain: The downloads to the target could take up to 30 seconds already.

The current setup shortens the translate-synthesize-map-place'n'route iteration cycles significantly. So the newly added blinky example should download within five seconds (after installing all the necessary packages, setting permissions and running the container locally). Not that this should encourage trial&error coding, but it might stop you from having too many smoke breaks.

Starting out as a documentation project with tiny steps and rather frequent updates, it's now moving towards:

• Getting real world (existing) designs to spin
• Augmenting the test benches with plenty of uncovered corner cases
• Not breaking stuff (unit tests for Jupyter notebooks, too)

Also, so far all has been 'virtual', i.e. running in the browser only. The next milestone is to get a complex CPU design running (still undecided which one it is going to be, but certainly one of the MaSoCist choices).

The hard part (which needed a lot of playing and revamping) is to support a rather large number of corner cases that have to do with conditional code:

if state == t_state.RESET:
    a.next = 0xff
elif state == t_state.RUN:
    a.next = c
else:
    a.next = 0xaa

This snippet is just supposed to assert a signal, decided upon a state condition, let aside the context for now (whether behind a flip flop or combinatorial logic)

In the silicon, this basically creates multiplexers. And it gets very interesting, when a case is omitted (like the 'else' clause), you might remember issues with combinatorial loops.

Hopefully with the recent release, I've gotten the corner cases right.

Here's where the testing comes in:

• Testing against a working reference
• Testing that the examples are correct

And this is where Python performs exceptional: it provides probably the most convenient ecosystem to test stuff. This is now also found in the [ Binder based virtual machine ].