deep learning audio plugin Apple Silicon

6 次查看（过去 30 天）
Tommaso
Tommaso 2023-8-25
评论： Jimmy Lapierre 2023-8-31
I created an audio plugin to make predictions about two kinds of sounds with a pre-trained model that I did. It works on audioTestbench, and it's validated by validateaudioplugin, but in VST format, it crashes all my DAWs, or it's unrecognizable. I'm on an M2 Apple Silicon architecture. Is there somebody who has experienced this situation?
Thanks
classdef cnnpredict < audioPlugin
properties
bsize = 1;
end
properties (Access = private)
prediction = 2;
end
properties (Constant)
PluginConfig = audioPluginConfig( ...
'DeepLearningConfig',coder.DeepLearningConfig('none'));
PluginInterface = audioPluginInterface(...
audioPluginParameter('bsize',...
'DisplayName','Buffer size',...
'Mapping',{'lin',0.1,4},...
'Label','seconds'), ...
PluginName="GuessPreparedPiano", ...
VendorName="tommasorosati", ...
VendorVersion="1.1.1", ...
InputChannels=1, ...
OutputChannels=2)
end
properties (Access = private)
Mybuffer
end
methods
function plugin = cnnpredict()
plugin.Mybuffer = dsp.AsyncBuffer(44100*2);
end
function out = process(plugin, in)
fs = getSampleRate(plugin);
% Write signal to buffer
% plugin.Mybuffer=plugin.Mybuffer(1:fs*2);
write(plugin.Mybuffer,in(:,1:size(in,2)));
if plugin.Mybuffer.NumUnreadSamples == (fs*2)
frame = read(plugin.Mybuffer, (fs*2));
plugin.prediction = predictit(frame,fs);
end
if plugin.prediction == 1
out = [in(:,1),in(:,1)*0];
else
out = [in(:,1)*0,in(:,1)];
end
end
end
end
function yy = predictit(x,fs)
persistent trainedNet;
if isempty(trainedNet)
trainedNet = coder.loadDeepLearningNetwork('trained_network_1.mat');
end
x=resample(x,16e3,fs);
x=x(1:32000,:);
auditorySpectrogram = extractAudioFeatures(x);
auditorySpectrogram = log10(auditorySpectrogram + 1e-6);
yy = classify(trainedNet,auditorySpectrogram);
yy = double(yy == "prepared") + 1;
end
  8 个评论
显示 6更早的评论
Tommaso
Tommaso 2023-8-31
I tried your vst but I still got "reaper_host_x86_64 quit unexpectedly." when I try to load it.
And "“cnnpredict.vst” is damaged and can’t be opened. You should move it to the Bin." when I try to open it in MaxMSP 8
Versions:
Matlab R2023a Update 5
Xcode 14.3.1
Jimmy Lapierre
Jimmy Lapierre 2023-8-31
Sounds like the MaxMSP issue might be caused by the file being downloaded, I wonder if #4 in this article would help.
For Reaper, with the VST I build, it shouldn't matter which version of MATLAB or Xcode you have. I tried it on a fresh download of Reaper on a M2 with Sonoma and it was running fine (playing one side or the other).
Is it crashing on loading the plugin or when playing back? Maybe there's an issue with the specific media file or project you have? I tried with the RockGuitar 16bit 44.1kHz stereo example file in the audio toolbox samples folder.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Audio Plugin Creation and Hosting 的更多信息

标签

产品


版本

R2023a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by