deep learning audio plugin Apple Silicon

1 次查看（过去 30 天）

Tommaso 2023-8-25

0
链接

此问题的直接链接

https://ww2.mathworks.cn/matlabcentral/answers/2013117-deep-learning-audio-plugin-apple-silicon

评论： Jimmy Lapierre 2023-8-31

I created an audio plugin to make predictions about two kinds of sounds with a pre-trained model that I did. It works on audioTestbench, and it's validated by validateaudioplugin, but in VST format, it crashes all my DAWs, or it's unrecognizable. I'm on an M2 Apple Silicon architecture. Is there somebody who has experienced this situation?

Thanks

classdef cnnpredict < audioPlugin

properties

bsize = 1;

end

properties (Access = private)

prediction = 2;

end

properties (Constant)

PluginConfig = audioPluginConfig( ...

'DeepLearningConfig',coder.DeepLearningConfig('none'));

PluginInterface = audioPluginInterface(...

audioPluginParameter('bsize',...

'DisplayName','Buffer size',...

'Mapping',{'lin',0.1,4},...

'Label','seconds'), ...

PluginName="GuessPreparedPiano", ...

VendorName="tommasorosati", ...

VendorVersion="1.1.1", ...

InputChannels=1, ...

OutputChannels=2)

end

properties (Access = private)

Mybuffer

end

methods

function plugin = cnnpredict()

plugin.Mybuffer = dsp.AsyncBuffer(44100*2);

end

function out = process(plugin, in)

fs = getSampleRate(plugin);

% Write signal to buffer

% plugin.Mybuffer=plugin.Mybuffer(1:fs*2);

write(plugin.Mybuffer,in(:,1:size(in,2)));

if plugin.Mybuffer.NumUnreadSamples == (fs*2)

frame = read(plugin.Mybuffer, (fs*2));

plugin.prediction = predictit(frame,fs);

end

if plugin.prediction == 1

out = [in(:,1),in(:,1)*0];

else

out = [in(:,1)*0,in(:,1)];

end

function yy = predictit(x,fs)

persistent trainedNet;

if isempty(trainedNet)

trainedNet = coder.loadDeepLearningNetwork('trained_network_1.mat');

end

x=resample(x,16e3,fs);

x=x(1:32000,:);

auditorySpectrogram = extractAudioFeatures(x);

auditorySpectrogram = log10(auditorySpectrogram + 1e-6);

yy = classify(trainedNet,auditorySpectrogram);

yy = double(yy == "prepared") + 1;

end

8 个评论
显示 6更早的评论隐藏 6更早的评论

Tommaso 2023-8-31

I tried your vst but I still got "reaper_host_x86_64 quit unexpectedly." when I try to load it.

And "“cnnpredict.vst” is damaged and can’t be opened. You should move it to the Bin." when I try to open it in MaxMSP 8

Versions:

Matlab R2023a Update 5

Xcode 14.3.1

Jimmy Lapierre 2023-8-31

Sounds like the MaxMSP issue might be caused by the file being downloaded, I wonder if #4 in this article would help.

For Reaper, with the VST I build, it shouldn't matter which version of MATLAB or Xcode you have. I tried it on a fresh download of Reaper on a M2 with Sonoma and it was running fine (playing one side or the other).

Is it crashing on loading the plugin or when playing back? Maybe there's an issue with the specific media file or project you have? I tried with the RockGuitar 16bit 44.1kHz stereo example file in the audio toolbox samples folder.

请先登录，再进行评论。

请先登录，再回答此问题。