popsicle + dawdreamer

[shakfu]

Hi, very cool project, thanks for sharing!! I just mentioned it in the discussions sections of another cool juce + python project. I think these two are complementary and I they work nice together.

I made a somewhat crude demo which is attached.

You will need to pip install dawdreamer and, of course, pip install popsicle

demo.py.zip

[shakfu]

Here's another iteration to address a segfaulting bug with some command line options

demo.2.zip

[kunitoki]

Sounds great! Thanks for mentioning it. What is the benefit of using dawdreamer over pythonized juce and the c++ capabilities of cppyy?

[shakfu]

Thanks for your reply @kunitoki

What is the benefit of using dawdreamer over pythonized juce and the c++ capabilities of cppyy

I would also add spotify's pedalboard to the same category as dawdreamer: both are pip-installable python packages and both use pybind11 to integrate some subset of JUCE's audio capabilities to enable the creation of audio processing graphs.

I'm sure you can achieve the same with cppyy, but the benefit is that they already exist and already provide novel and differentiated features: midi playback and faust-processing with dawdreamer, and 'strong thread-safety, memory usage, and speed guarantees' with pedalboard.

[kunitoki]

Yeah i understand, they are python packages afterall so they could integrate nicely

[turian]

@shakfu so what does your demo actually do?

[shakfu]

@turian It was some time since I posted it, and I tried to run popsicle examples just now but was getting "uncaught exception in JIT" warnings and execution issues on a Mac m1 machine.

But I think the point I was trying to make is evident from a function calling dawdreamer where a sine wave file is rendered from a daydreamer audio traph which contains a node containing faust code and then displayed on the popsicle sample player.

import dawdreamer as daw
...
def render_sine(path='sine.wav'):
    engine = daw.RenderEngine(SAMPLE_RATE, 512)  # 512 block size
    faust_processor = engine.make_faust_processor("faust")
    faust_processor.set_dsp_string(
        """
        declare name "MySine";
        freq = hslider("freq", 440, 0, 20000, 0);
        gain = hslider("vol[unit:dB]", 0, -120, 20, 0) : ba.db2linear;
        process = freq : os.osc : _*gain <: si.bus(2);
        """
        )
    print(faust_processor.get_parameters_description())
    engine.load_graph([
                       (faust_processor, [])
    ])
    faust_processor.set_parameter("/MySine/freq", 440.)  # 440 Hz
    faust_processor.set_parameter("/MySine/vol", -6.)  # -6 dB volume

    engine.set_bpm(120.)
    engine.render(4., beats=True)  # render 4 beats.
    audio = engine.get_audio()  # shaped (2, N samples)
    wavfile.write(path, SAMPLE_RATE, audio.transpose())
    return path

[kunitoki]

I've started a complete rewrite of popsicle, and i'm currently being addressing audio. The new popsicle should be more stable and break less often as it doesn't rely on cppyy and overall it is manually finetuned.

[shakfu]

Hi @kunitoki that sounds great!

[kunitoki]

I've released the current progress (not really 1.0 but we are approaching it). Feel free to try and report back any feedback !

[shakfu]

@kunitoki I just downloaded it, build it from source and ran the examples (I'm on a M1 MacBook Air).

On the build side: it was very easy and non-problematic to build, your CMakeExtension class is pretty cool, I liked the default universal2 architecture choice and the extension size is a nice 20Mb).

Examples ran great except for drawables.py which crashed on me... I'm not sure was the demo folder contained since it was in cpp.

But overall congrats.. it's looking good. I definitely prefer this pybind11 version of popsicle to the prior one.

[kunitoki]

Nice, glad to hear that!

[kunitoki]

Btw, i saw that you correctly embedded the standard library as a zip file, it would be great to be able to have a step in the demo app that bundles the python std lib so it's standalone. The tricky part is doing it cross platform.

[shakfu]

@kunitoki
Back in the day, I posted about this in stack-overflow.

I have evolved somewhat since then, and now I routinely embed the python standard lib as a zip file in my py-js project and this is one step of of a whole custom build system in python in the builder module.

The specific function in builder looks like the following:

def ziplib(self):
        """zip python package in site-packages to .zip archive"""
        temp_lib_dynload = self.prefix_lib / "lib-dynload"
        temp_os_py = self.prefix_lib / "os.py"

        self.cmd.move(self.site_packages, "/tmp/site-packages")
        self.lib_dynload.rename(temp_lib_dynload)
        self.cmd.copy(self.python_lib / "os.py", temp_os_py)

        zip_path = self.prefix_lib / f"python{self.product.ver_nodot}"
        shutil.make_archive(str(zip_path), "zip", str(self.python_lib))

        self.cmd.remove(self.python_lib)
        self.python_lib.mkdir()
        temp_lib_dynload.rename(self.lib_dynload)
        temp_os_py.rename(self.python_lib / "os.py")
        self.cmd.move("/tmp/site-packages", self.site_packages)

I've only tested this on macOS, but I don't see why this wouldn't work on other platforms.

[kunitoki]

Thanks for the headsup, really appreciated ! Btw i started hacking around a bit:

import os
import sys
import shutil
from pathlib import Path

if __name__ == "__main__":
    base_folder = Path(sys.argv[1])
    output_folder = Path(sys.argv[2])
    version_major = sys.argv[3]
    version_minor = sys.argv[4]

    version = f"{version_major}.{version_minor}"
    version_nodot = f"{version_major}{version_minor}"

    ignored_files = shutil.ignore_patterns(
        "*.pyc",
        "__pycache__",
        "__phello__",
        "*config-3*",
        "*tcl*",
        "*tdbc*",
        "*tk*",
        "Tk*",
        "_tk*",
        "_test*",
        "libpython*",
        "pkgconfig",
        "idlelib",
        "site-packages",
        "test",
        "turtledemo",
    )

    print("cleaning up...")
    shutil.rmtree(str(output_folder / "python"))

    print("copying library...")
    shutil.copytree(
        str(base_folder / "lib" / f"python{version}"),
        str(output_folder / "python"),
        ignore=ignored_files,
        dirs_exist_ok=True)

    print("creating site-packages...")
    os.makedirs(str(output_folder / "python" / "site-packages"), exist_ok=True)

    print("making archive...")
    shutil.make_archive(str(output_folder / f"python{version_nodot}"), "zip", str(output_folder / "python"))

This is executed from cmake and the resulting zip file is embedded in the app, but i've tried bootstrapping python and i cannot find the right way to do it (assuming i setup isolated mode to not look into the system):

libc++abi: terminating due to uncaught exception of type std::runtime_error: failed to get the Python codec of the filesystem encoding

[kunitoki]

This is the full output of python when it fails initialisation:

Python path configuration:
  PYTHONHOME = '/Users/kunitoki/Library/Caches/Popsicle Demo'
  PYTHONPATH = '/Users/kunitoki/Library/Caches/Popsicle Demo/python311.zip'
  program name = 'PopsicleDemo'
  isolated = 1
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/Users/kunitoki/Library/Caches/Popsicle Demo/lib/python3.11'
  sys._base_executable = '/Users/kunitoki/popsicle/demo/build/popsicle_demo_artefacts/Debug/Popsicle Demo.app/Contents/MacOS/Popsicle Demo'
  sys.base_prefix = '/Users/kunitoki/Library/Caches/Popsicle Demo'
  sys.base_exec_prefix = '/Users/kunitoki/Library/Caches/Popsicle Demo'
  sys.platlibdir = 'lib'
  sys.executable = '/Users/kunitoki/popsicle/demo/build/popsicle_demo_artefacts/Debug/Popsicle Demo.app/Contents/MacOS/Popsicle Demo'
  sys.prefix = '/Users/kunitoki/Library/Caches/Popsicle Demo'
  sys.exec_prefix = '/Users/kunitoki/Library/Caches/Popsicle Demo'
  sys.path = [
    '/Users/kunitoki/Library/Caches/Popsicle Demo/python311.zip',
    '/Users/kunitoki/Library/Caches/Popsicle Demo/lib/python311.zip',
    '/Users/kunitoki/Library/Caches/Popsicle Demo/lib/python3.11',
    '/Users/kunitoki/Library/Caches/Popsicle Demo/lib/python3.11/lib-dynload',
  ]