PI Used as Audio Card for ZX Next

This section is for discussing everything about Next hardware and latest updates.
Post Reply
Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

PI Used as Audio Card for ZX Next

Post by Alcoholics Anonymous » Tue Sep 24, 2019 6:26 pm

The video shows a terminal program running on the next connected to a bash terminal on the pi. Commands are sent to the pi, in this case to play various audio formats (sid, mp3, etc), the pi generates the audio and sends it back to the next via a stereo i2s interface. The digital audio sent by the pi is mixed into the next's internal audio stream. This uses no cpu time at all at the next end.


User avatar
Flinnster
Posts: 62
Joined: Fri Jun 02, 2017 5:35 pm

Re: PI Used as Audio Card for ZX Next

Post by Flinnster » Tue Oct 01, 2019 9:24 pm

This is rather special :) How much interactivity is there with the playback? Can you do all the usual play, stop, jump to, volume fade in and volume fade out commands from the NEXT itself? :)
Gamer. Pixelographist. Game Developer. Arcade Collector.

User avatar
Timbucus
Posts: 230
Joined: Tue May 30, 2017 7:43 pm

Re: PI Used as Audio Card for ZX Next

Post by Timbucus » Tue Oct 01, 2019 9:40 pm

I know people try to avoid FB but there is a thread that discusses this topic a bit https://www.facebook.com/groups/specnex ... 501053637/ - the group is open so you do not need to register. In summary more work is needed to finesse the systems but, all of this should be possible as the PI runs in parallel and communicates with a UART with the Next Z80N.
I'm Infinite Imaginations when not in work... PAWS for thought.

User avatar
Flinnster
Posts: 62
Joined: Fri Jun 02, 2017 5:35 pm

Re: PI Used as Audio Card for ZX Next

Post by Flinnster » Tue Oct 01, 2019 9:49 pm

Thanks ;)
Gamer. Pixelographist. Game Developer. Arcade Collector.

PiyoTaro
Posts: 181
Joined: Thu Jun 01, 2017 11:13 am

Re: PI Used as Audio Card for ZX Next

Post by PiyoTaro » Sat Oct 05, 2019 8:54 am

  • Functions required as a “game BGM player”:
    1. Transfer audio/music tracks from the Z80 host. Multiple tracks can be stored on the accelerator board. / 2. Play the song with the track number specified by the host. Pause and resume BGM playing. / 3. Playback volume can be controlled from the host. "Mute function". If possible, fader processing at the start and stop of BGM playing.
  • Playing “game sound effects”:
    1. Transfer PCM/MP3 audio tracks from Z80 host. Multiple tracks can be stored on the accelerator board. Data encryption. / 2. I want the ability to play only a part of a "containerized audio track". (If the format for separating the audio tracks cannot be determined, the “temporal position of the audio track” may be specified.) / 3. While playing an audio track, "Mute" the background music.

    * Mute (If done, change the volume to the minimum. When released, restore the original volume.)

P.S.
I should have a separate thread, but for now I just want to say “NextPI”.

I was silent when "Alpha Core v.3.00" was announced.
On FB, there were demo videos that routed tape image files to cassette tape input and demo videos that played "SID" format music files (it does not mean that sound chip was implemented). Unfortunately, there was no mention of whether the audio of this video was recorded from the audio interface on the ZXSpectrumNext side.

And “System/Next v.1.1 RTM” was announced. "NextPI" was written in the announcement text. The interface on the Z80 host side to the accelerator board is that a UART serialport is implemented.
Looking at FB posts, it is written that “files” such as tracker format music data and tape images need to be converted to BASE64 text and sent (instead of being sent as binary data). I read it and imagine that "NextPI" is like a command line operating system that processes messages sent via the serialport.
Also, in the FB post, there was a description of “RAMdisk 512K”.

At this time, since there is no information such as the author's site, I do not know what "NextPI" is.
Currently, the only explanation for accelerator boards is the Wiki item "RPi0 Acceleration".
I hope that a Wiki item will be created for "NextPI".
Last edited by PiyoTaro on Sat Oct 05, 2019 11:00 am, edited 1 time in total.

Ped7g
Posts: 111
Joined: Mon Jul 16, 2018 7:11 pm

Re: PI Used as Audio Card for ZX Next

Post by Ped7g » Sat Oct 05, 2019 10:34 am

I read it and imagine that "NextPI" is like a command line operating system that processes messages sent via the serialport.
Exactly. There's a limited customized linux installation with bash at the Pi0 running, the console is connected to the UART.

Next can send via UART regular console input, and read back console output.

The file system of the NextPi distro is read-only (to prevent any damage when powered off abruptly), but it does always initialize and mount 250MB ram-disk, which is RW, where the game can upload files each time when started.

The sound is I2S stereo-channel communication between Next and Pi0, the Next is sending its stereo channel to Pi0, where it can be mixed/transposed/added other sounds/music, and gets back the result, playing it on output, so the TZX works by adding the tape screeching to the output and sends it back to Next, where it is picked up by the regular `in a,(254)` in the LOAD routines, i.e. this is identical to actually playing the tape to the real spectrum from tape player.

For game BGM/effects:

For simple BGM the currently pre-installer players may be already sufficient... not sure what controls with which players are available, but I guess some of them may have even hot-keys for volume/etc, so you can send the commands through the console and control the playback (to some extent).

For effects I guess you need a player which can be controlled from console, some sort of "live mix table/tracker", and I don't think any of the default players already provides feasible solution (maybe some does?), because the overall latency, power consumption and performance was not good when ordinary OS-level mixing was tried (running multiple players), so the current out-of-box distro allows only single player playing the sound.

I *think* (pure theory, I may be wrong) it should be possible to create custom mod-player capable to mix game effects with reasonable latency and power consumption, but that must be first developed by somebody.

About the missing documentation and info:

Keep in mind there were hard date deadlines to deliver the sd-card images to the factory, so most of the effort of the guys working on these (Allen, Xalior, Tim, David and surely few more I'm not even aware of) was focused on delivering the images.

The docs and info will come in following weeks/months, it's just question of time. :)

PiyoTaro
Posts: 181
Joined: Thu Jun 01, 2017 11:13 am

Re: PI Used as Audio Card for ZX Next

Post by PiyoTaro » Sat Oct 05, 2019 4:27 pm

I received a reply that NextPi is based on Linux.

(The first thing I came up with was a “VGM file player” with a sound chip emulator.)

By the way, looking at Pi's textbook, there was a "MIDI file player" using a software synthesizer. The performance of the BGM of the MIDI file should be less loaded than the MOD tracker.
The open source project “FluidSynth” is compatible with the SoundFont and seems to be able to play sound effects.
(It is a MIDI synthesizer software device that supports "SoundFonts" and has a "MIDI player" that can be run from the command line. “SoundFont” seems to be able to handle sampling audio compressed in Ogg format. http://www.fluidsynth.org/ )

I think that playing music (encrypted) stored in RAMDISK and changing the volume of the music being played can be used for game BGM.

P.S. About "SoundFont". I found something like this (there is a video link in the article). It looks like the timbre used in the 'Sega Genesis Remix' YouTube video. Sound Blaster OPL3 FM-MIDI instrument sound can also be found. https://musical-artifacts.com/artifacts/24

Addendum 10/27/2019. "Sound font" explanatory article (Audio data + attribute text * .sfz)
"SoundFonts and SFZ files"https://musescore.org/en/handbook/3/sou ... -sfz-files
Last edited by PiyoTaro on Sun Oct 27, 2019 8:35 am, edited 1 time in total.

EtchedPixels
Posts: 18
Joined: Fri May 04, 2018 10:59 am

Re: PI Used as Audio Card for ZX Next

Post by EtchedPixels » Thu Oct 24, 2019 9:35 pm

Take a look at the NeoGS and related Russian cards. They have a well defined API, do MOD files and a reasonable amount of game support.

PiyoTaro
Posts: 181
Joined: Thu Jun 01, 2017 11:13 am

Re: PI Used as Audio Card for ZX Next

Post by PiyoTaro » Sun Oct 27, 2019 8:06 am

EtchedPixels wrote:
Thu Oct 24, 2019 9:35 pm
Take a look at the NeoGS and related Russian cards. They have a well defined API, do MOD files and a reasonable amount of game support.
How the accelerator board is used in the “ZXSpectrumNext RTM version” explained in this “NextPi” thread.

The interface to the accelerator board from the Z80 side is only UART (however, it seems that the state of GPIO pin can be known by the extended CPU instruction). It seems that the communication between the app called “NextPi” and Z80 is limited to the exchange of messages through the serial port of UART.

To emulate devices such as “NeoGS” sound cards with an accelerator board, it is necessary to implement a device that acts as a real interface to the FPGA (Z80 side).
(For example, it is a device that can read and write registers as 8255PPI from the Z80 side, and exchange the result with Pi app via I2C communication.)
(Of course, there is also an idea that there is no need for an interface device by monitoring the behavior of the Z80 CPU bus with the Pi app. An HDMI adapter connected to an edge-connector would be the case.)

Even though FM Sound and SID core development was not made in priority (I understood that I received the explanation that it was a priority issue), would there be anyone who makes such a Pi interface core ???

In the past, I asked the person who was in the other forum (ZXUno) to make an FPGA-CPU emulation-based core "JTx" for the FM sound chip "YM2203".
I think the only response on our forum was a reply from TheSmog (a series of replies are erased and can't be seen now) that "the ZXSpectrumNext core team doesn't want to adopt such a thing".

Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

Re: PI Used as Audio Card for ZX Next

Post by Alcoholics Anonymous » Mon Oct 28, 2019 3:53 am

PiyoTaro wrote:
Sun Oct 27, 2019 8:06 am
The interface to the accelerator board from the Z80 side is only UART (however, it seems that the state of GPIO pin can be known by the extended CPU instruction). It seems that the communication between the app called “NextPi” and Z80 is limited to the exchange of messages through the serial port of UART.
The interface to the pi is through all 28 gpio pins. The next has direct control of these for reading, output enables and writing. On top of this, "special functions" can be superimposed. This is hardware on the fpga that re-purposes some gpio pins when enabled. Currently the hardware available is i2c, spi, uart and i2s. i2c (bit-banged) and spi (simple hardware assist) are not very useful for communicating with the pi but could be useful for communicating with pi hats. The uart interface is an asynchronous communication that can go as fast as 2Mbps. It is a second instance of the uart module used for the esp wifi. The i2s is a two-way stereo pcm audio connection with the pi. The next's audio stream is sent to the pi and the pi sends audio to the next. The pi audio sent to the next can be routed to the tape port or it can be added to the next's internal audio stream so that it comes out on the internal speaker, audio jacks or hdmi.

The pi itself runs a linux variant so can run any of the audio software available for the pi. This includes players for mod, sid, mp3, and so on. I expect eventually programs will be written for the pi that provide an api to software running on the next.

This is not necessarily the final pi interface as there could be more "special functions" added to the fpga depending on available space.
Even though FM Sound and SID core development was not made in priority (I understood that I received the explanation that it was a priority issue), would there be anyone who makes such a Pi interface core ???

In the past, I asked the person who was in the other forum (ZXUno) to make an FPGA-CPU emulation-based core "JTx" for the FM sound chip "YM2203".
I think the only response on our forum was a reply from TheSmog (a series of replies are erased and can't be seen now) that "the ZXSpectrumNext core team doesn't want to adopt such a thing".
These audio options take up a lot of fpga space for little payback, especially given the pi is much more versatile as an audio option and puts to use a cheap & powerful add-on that is currently seeking purpose.

Even emulating the GS or NeoGS adds several io ports and then a special function or something similar to communicate changes to the pi. This is not free and there are currently things deemed more important on the to-do list. The option will always be there to connect one of these devices to the expansion bus if they are not implemented internally in the next.

Edit: About the video in the original post - this is recorded from a zx next. It is too clear to be recorded from the analog audio jack so I think it's captured using David's hdmi capture card.

Post Reply