How to handle a huge amount of data with surface and stair plots to avoid memory spikes

1 次查看（过去 30 天）
Ivy Chen
Ivy Chen 2017-11-13
评论： Ivy Chen 2017-11-13
When we try to handle very large amount of the data with batched bins (500) and averages (50) and stored in the matrices. It still consumes and have memory spikes when the ploting scenario reaches 20 minutes long.
Besides adjusting the bin and average sizes, is there a way to make this more efficient to plot a large timeframe of data by splicing multiple plots?
In both plots, X is referring to time with calcuated magnitude spectrum in dB.
  2 个评论
Jan
Jan 2017-11-13
The question is not clear. Please explain, what you are doing exactly: What is displayed how? Code? What are "memory spikes"? What takes 20 minutes? How large is "large"? What is "splicing"? Which "X" refers to what?
Ivy Chen
Ivy Chen 2017-11-13
Sorry about the confusion and here are some clarifications.
  • File size - Data collected in 20 seconds results about 265 MB
  • 20 minutes referring to the raw data collected in the bin file during the 20 minutes timeframe. Ideally, we would like to process at least data collect up to 1 hour.
  • We notice that the memory spikes when the plot opens.
  • X is referring to time vs calcuated magnitude spectrum in dB
  • Splicing meaning that we can have multiple plots respect to time (X) axis. For example, plot 1: time 0-20 minutes, plot 2: time 21-40 minutes, plot 3: time 41-60 minutes, etc.
Here is the code:
Determine RF data size based on defined batches (NsperBatch=500 in this case)
magSpectrumMAT_Sum=[];
magSpectrumMAT_Avg=[];
while ~feof(fid)
magSpectrum=0;
for k=1:Kavg
data=fread(fid,NsperBatch*2,'int16');
dataIQ = complex(data(1:2:end,:), data(2:2:end,:));
num_dataIQ = length(dataIQ);
dataIQ_GnOff = (Gain * dataIQ) + (Offset+Offset*1i); % Apply mx+b for Gain and Offset specified in the URT User Manaul
dataIQ_GnOff = dataIQ_GnOff * db2mag(Reference_L); % Normalize with respect to the reference level as indicated in the header (eg. -1.445 dbm = -31.445 dbw)
target_num_dataIQ = NsperBatch;
if num_dataIQ ~= target_num_dataIQ
fprintf('Note: complex data is not a multiple of %d samples long, padding\n', NsperBatch);
dataIQ_GnOff(target_num_dataIQ) = 0; % Fill zero automatically
end;
mag_dataIQ_GnOff = abs(fftshift( fft( reshape(dataIQ_GnOff, NsperBatch, []) ) ) / sqrt(NsperBatch)).^2; % Compute RF data batched Matrix in FFT, then ABS
magSpectrum = magSpectrum + mag_dataIQ_GnOff;
end
magSpectrumMAT_Sum = [magSpectrumMAT_Sum magSpectrum];
magSpectrum_avg = magSpectrum/Kavg;
magSpectrumMAT_Avg = [magSpectrumMAT_Avg magSpectrum_avg];
end
magSpectrumMat_dB=pow2db(magSpectrumMAT_Avg); %%
Define Frequncy Range for Batch Load
num_full_batches = size(magSpectrumMAT_Avg, 2);
dt = (NsperBatch*Kavg)/BW;
tvec = ((1:num_full_batches)-1)*dt;
df=BW/NsperBatch;
freqvec=(-BW/2):df:(BW/2-df);
Plot 1: Spectrum Power
X=tvec;
Y=freqvec/1e6;
Z=magSpectrumMat_dB;
[X, Y]=meshgrid(X,Y);
h=surf(X,Y,Z);
set(h,'EdgeColor','none');
grid off
axis tight
view([90 90]); %Set view angle as Top-Flat 2D plot
colormap(jet);
%%Plot 2: Sum Power
Power_Sum=sum(magSpectrumMAT_Sum)/Kavg;
CPower_W=Power_Sum/NsperBatch;
CPower_dBW=10*log10(CPower_W);
dt_sum = (NsperBatch*Kavg)/BW;
X_sum=((1:num_full_batches)-1)*dt_sum;
h_sum=stairs(X_sum,CPower_dBW);

请先登录，再进行评论。

请先登录，再回答此问题。

回答（0 个）

类别

在 Help Center 和 File Exchange 中查找有关 Spectral Measurements 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by