deep learning audio plugin Apple Silicon
1 次查看(过去 30 天)
显示 更早的评论
I created an audio plugin to make predictions about two kinds of sounds with a pre-trained model that I did. It works on audioTestbench, and it's validated by validateaudioplugin, but in VST format, it crashes all my DAWs, or it's unrecognizable. I'm on an M2 Apple Silicon architecture. Is there somebody who has experienced this situation?
Thanks
classdef cnnpredict < audioPlugin
properties
bsize = 1;
end
properties (Access = private)
prediction = 2;
end
properties (Constant)
PluginConfig = audioPluginConfig( ...
'DeepLearningConfig',coder.DeepLearningConfig('none'));
PluginInterface = audioPluginInterface(...
audioPluginParameter('bsize',...
'DisplayName','Buffer size',...
'Mapping',{'lin',0.1,4},...
'Label','seconds'), ...
PluginName="GuessPreparedPiano", ...
VendorName="tommasorosati", ...
VendorVersion="1.1.1", ...
InputChannels=1, ...
OutputChannels=2)
end
properties (Access = private)
Mybuffer
end
methods
function plugin = cnnpredict()
plugin.Mybuffer = dsp.AsyncBuffer(44100*2);
end
function out = process(plugin, in)
fs = getSampleRate(plugin);
% Write signal to buffer
% plugin.Mybuffer=plugin.Mybuffer(1:fs*2);
write(plugin.Mybuffer,in(:,1:size(in,2)));
if plugin.Mybuffer.NumUnreadSamples == (fs*2)
frame = read(plugin.Mybuffer, (fs*2));
plugin.prediction = predictit(frame,fs);
end
if plugin.prediction == 1
out = [in(:,1),in(:,1)*0];
else
out = [in(:,1)*0,in(:,1)];
end
end
end
end
function yy = predictit(x,fs)
persistent trainedNet;
if isempty(trainedNet)
trainedNet = coder.loadDeepLearningNetwork('trained_network_1.mat');
end
x=resample(x,16e3,fs);
x=x(1:32000,:);
auditorySpectrogram = extractAudioFeatures(x);
auditorySpectrogram = log10(auditorySpectrogram + 1e-6);
yy = classify(trainedNet,auditorySpectrogram);
yy = double(yy == "prepared") + 1;
end
8 个评论
Jimmy Lapierre
2023-8-31
Sounds like the MaxMSP issue might be caused by the file being downloaded, I wonder if #4 in this article would help.
For Reaper, with the VST I build, it shouldn't matter which version of MATLAB or Xcode you have. I tried it on a fresh download of Reaper on a M2 with Sonoma and it was running fine (playing one side or the other).
Is it crashing on loading the plugin or when playing back? Maybe there's an issue with the specific media file or project you have? I tried with the RockGuitar 16bit 44.1kHz stereo example file in the audio toolbox samples folder.
回答(0 个)
另请参阅
类别
在 Help Center 和 File Exchange 中查找有关 Audio Plugin Creation and Hosting 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!