Software Blog
While working on an another similar project realised that access to lookup tables, e.g. sine, exp and log from FLASH is much more expensive than from RAM. Switching the high frequency access tables to RAM, (for now just by making them non-const) would give a very large performance boost. And it works! Have been able to upgrade to 8 voices per core for a total of 16 voices, like a real DX7, and seeing around 70% CPU usage.
To help test and debug the picoX7, have started using a proper DAW, Abelton Live. First proper DAW I've used and it's quite a revelation, unlike the free Apple software, everything I need to do seems possible and it doesn't crash or lock in stale/broken configs. Very nice! But will I cough up some hard earned cash at the end of the 30 day free trial? There appears to be a latency of around 90ms between MIDI out from the DAW to audio back into the DAW. There has to be some latency, but 90ms seems excessive, and audibly unpleasant. Suspect it's my software on the Pico and not the no-name cheap USB audio interface I'm using, but plan to upgrade to a better USB audio interface in the near future ... found I can tell Abelton about the large latency and it compensates, just amazing!
MIDI in over USB (device mode) starting to work. Not entirely happy to have cargo culted in the USB descriptors everyone else (including my AKAI MPK mini) seem to have copied out of the appendix in the USB MIDI spec. The conflicting semantics of USB IN/OUT and MIDI IN/OUT has also been somewhat challenging for my small brain. The bugs and opaqueness of Apple MIDI studio also hasn't helped. Shout out to "Aria Maestosa" which unlike Apple Garage Band and Apple MIDI studio does actually drive MIDI out over USB.
Thought I'd feedback some minor updates to AJXS DX7 firmware disassembly. Found I'd been using quite an old copy making it very hard to merge any changes back. Oops!
Have decided that emulation of the firmware is the best way to get the data required to achieve accurate simulation of the equivalent EGS inputs, keyboard scaling, pitch env, LFO etc. Have been developing a framework for modelling old computer systems for many years so creating a new model for the main CPU of the Yamaha DX7 was straightforward. Debugging was made unusually easy thanks to the AJXS disassembly of the firmware!
The emulation initialises ok, 7-seg LEDs and 16x2 text LCD respond correctly to the various button presses and writes to the EGS and OPS registers look believable. Currently debugging why, after simulation of the 6303 OCI timer and IRQ were added, the firmware now keeps calling EGS_RESET_VOICES and eventually locking up. 
Would love to open sources this too. Unfortunately am not currently able to as, for personal projects, my employer places restrictions on the type of software I can open source. May add as MacOS (on arm64) and Linux (on arm64) binaries to the next picoX7 release e.g. as freeware. Be cool to integrate this to the native build of the EGS+OPS simulation for a more authentic DX7 experience!
Started to rewrite the OPS simulation to work the same way as the real thing with respect to using a log-sine wavetable. Got to an intermediate working state but not ready to push any patches yet.
Updated the console baud rate to 115200! Turns out 8 voice at 191 MHz was too close to the edge ... dropped back to 6 voices and the glitches and ocassional cacophonic crash seems to be avoided. Suspect the current software architecture is not resilient enough to cope with peaks in demand, say when several operators switch envelope generator phase at the same time.
Hand translation of the DX7 firmware ROM continues. Some updates including a couple of drawio diagrams showing the call trees for interesting areas of the firmware. Small diversion to create a quick and dirty MIDI file player for driving the device from another computer. The Python player is stretching the picoX7 more than my keyboard skills ;-) and it is very noticeable that the audio glitches that occasionally occurred before are still present.
Recently been hand translating the DX7 v1.8 firmware ROM disassembly from AJXS to C++. See DX7_v1_8.h. This is to generate usable code that can drop into picoX7.
The UARTs for console debug and MIDI are being polled with no use of IRQs and buffers. This is a waste of CPU cycles on TX and potentially lossy on RX. UART API in Platform has been updated with 512 byte RX and TX buffers streamed via IRQs from the hardware FIFOs. Seems to have solved several glitches and issues. In particular overclocking at 191 MHz now seems reliable. (but will need to monitor)
The current number of voices is fixed at 4 via NUM_VOICES in picoX7_rpipico.cpp. This has been kept at a power of 2 to save some time in the voice mixer. i.e. division by a power of two is much cheaper. However the current max CPU usage, 65%, indicates that more voices would be possible. Tried increasing the voice count and also some over clocking to see what might be possible...
| Clock (MHz) | Overclock | Max voices | CPU Usage | Comments |
|---|---|---|---|---|
| 133.00 | 0.0% | - | - | Not suitable due to DAC sample rate requirements |
| 137.48 | 3.3% | 6 | 89% | |
| 157.10 | 18.1% | 7 | 91% | |
| 191.08 | 43.7% | 8+ | 85% | Sometimes fails to come out of reset. Not sure if this is the RP2040 pushed passed it's limits or a bug in the RP2040 bring up code in Platform |
NOTES:
- Although interesting, this is not a final indicator of what will be achieved as the simulation of each DX7 voice is incomplete and will probably require a few more cycles
- The 2nd core in the RP2040 is still idle
When the CPU is "saturated", by trying to play too many voices at once, everything locks up with an unpleasant cacophony from the DAC. This is because the main loop never gets to run and so note off events, that would resolve the overload, cannot be received. In addition, as there are not enough cycles to compute all the samples the audio stream will be discontinuous and sound terrible. Occasionally, even when CPU usage was quite low some temporary audio glitches were heard. Needs investigating.
Re-worked the MIDI SysEx implementation in Source/DX7/Synth.h so that single patch dumps can be received from voice editors like Dexed. In theory a full bank of 32-patches, and individual parameter edits could also be received, but that has not been tested.
Despite owning a USB to MIDI out adapter seem unable to get MIDI out working from my Mac book. So hacked Dexed to spool single patches directly to the Pico over the serial port used for debug; which on the RX side, despite being the wrong baud rate, has been connected as an extra MIDI-in. The MIDI interfaces on the Pico rely on the UARTs being polled regularly. They do not have a huge FIFO coupled with the lower 9600 default baud rate on the debug UART was leading to garbled SysEx messages, i.e. data loss, at the Pico end. Changing the send of the SysEx to write() no more than 8 bytes at a time with usleep()s in-between solved the problem. (subsequent updates to add software FIFOs to the UART interfaces has solved this problem and removed the need for the usleep()s)
Quick and dirty patch to change STORE button on dexed to SEND (to picoX7). Warning, untidy and contains hard-coded paths: dexed.patch