hasdata

Log Big Data from Model

Open the model sldemo_fuelsys.

mdl = "sldemo_fuelsys";
open_system(mdl)

Select Configuration Parameters > Data Import/Export > Log Dataset data to file to log data to persistent storage. Alternatively, you can log data using Dataset format to a MAT-file instead of the workspace programmatically.

set_param(mdl,"LoggingToFile","on")

Simulate the model.

sim(mdl);

The MAT-file out.mat appears in your current folder. Logged signal data is stored in the MAT-file with the variable name sldemo_fuelsys_out.

Create a DatasetRef object that refers to the logged signal data.

DSRef = Simulink.SimulationData.DatasetRef("out.mat","sldemo_fuelsys_output");

Preview Big Data

Use curly braces to return a SimulationDatastore representation of the fuel signal, which is the tenth element in the DatasetRef object DSRef. The SimulationDatastore object exists in the Values property of the returned Signal object.

SimDataSig = DSRef{10};
DStore = SimDataSig.Values;

Use the preview function to inspect the first ten samples of logged data for the fuel signal.

preview(DStore)

ans=10×1 timetable
         Time          Data 
    ______________    ______

    0 sec             1.2855
    0.00056195 sec    1.2855
    0.0033717 sec     1.2855
    0.01 sec          1.2398
    0.02 sec           1.199
    0.03 sec          1.1628
    0.04 sec          1.1309
    0.043098 sec      1.1309
    0.043098 sec      1.1309
    0.043098 sec      1.1309

Inspect Specific Sample

Suppose you want to inspect the 603rd sample of logged fuel data. Set the ReadSize property of DStore to a number that, considering memory resources, your computer can tolerate. For example, set ReadSize to 200.

DStore.ReadSize = 200;

Read from the datastore three times. Each read operation advances the reading position by 200 samples.

read(DStore);
read(DStore);
read(DStore);

Now that you are close to the 603rd sample, you can set ReadSize to a smaller number to make the target sample easier to find. For example, set ReadSize to 5.

DStore.ReadSize = 5;

Read from the datastore again. The third sample of read data is the 603rd sample in the datastore.

read(DStore)

ans=5×1 timetable
      Time       Data 
    ________    ______

    5.83 sec    1.6609
    5.84 sec    1.6733
    5.85 sec    1.6831
    5.86 sec     1.691
    5.87 sec    1.6975

Inspect Earlier Sample

You can also inspect an earlier sample. For example, inspect the 403rd sample of logged fuel data. Due to previous read operations, the datastore now reads starting from the 606th sample.

Use the reset function to reset DStore. Then, read from the first sample up to the 403rd sample.

reset(DStore);

Set ReadSize to 200.

DStore.ReadSize = 200;

Read from the datastore twice to advance the read position to the 401st sample.

read(DStore);
read(DStore);

Set ReadSize to 5 and read from the datastore. Now, the third sample of read data is the 403rd sample in the datastore.

DStore.ReadSize = 5;
read(DStore)

ans=5×1 timetable
      Time       Data  
    ________    _______

    3.86 sec     0.9895
    3.87 sec    0.98253
    3.88 sec    0.97559
    3.89 sec    0.96867
    3.9 sec     0.96178

Extract Multiple Samples

You can also use the read function to extract multiple samples. For example, extract samples 1001 through 1020.

Reset the datastore. Then, advance to sample 1001 by setting the ReadSize property to 200 and reading the datastore five times.

reset(DStore)

DStore.ReadSize = 200;
for i = 1:5
    read(DStore);
end

Set the ReadSize to 20 to extract 20 samples from the datastore.

DStore.ReadSize = 20;

Extract samples 1001 through 1020. Store the extracted data in a variable named targetSamples.

targetSamples = read(DStore)

targetSamples=20×1 timetable
      Time       Data 
    ________    ______

    9.77 sec      1.68
    9.78 sec    1.6856
    9.79 sec    1.6905
    9.8 sec     1.6948
    9.81 sec    1.6812
    9.82 sec    1.6714
    9.83 sec    1.6642
    9.84 sec     1.659
    9.85 sec    1.6553
    9.86 sec    1.6527
    9.87 sec    1.6684
    9.88 sec    1.6806
    9.89 sec    1.6904
    9.9 sec     1.6982
    9.91 sec    1.7047
    9.92 sec    1.7101
      ⋮

Find Maximum Value of Data in Datastore

Use the hasdata function as the condition for a while loop to incrementally analyze the data in chunks of 200 samples.

reset(DStore);
DStore.ReadSize = 200;
runningMax = [];
while hasdata(DStore)
    tt = read(DStore);
    rawChunk = tt.Data;
    runningMax = max([rawChunk; runningMax]);
end

The variable runningMax stores the maximum value in the entire datastore.

runningMax

runningMax = 
1.7291

Create sdidatastore for Signal

Simulate the sldemo_fuelsys model, which is configured to log several signals, to create data in the Simulation Data Inspector repository.

mdl = "sldemo_fuelsys";
sim(mdl);

Logged signal data is returned in a Simulink.SimulationData.Dataset object named sldemo_fuelsys_output.

sldemo_fuelsys_output

sldemo_fuelsys_output = 
Simulink.SimulationData.Dataset 'sldemo_fuelsys_output' with 10 elements

                         Name            BlockPath                                
                         ______________  ________________________________________ 
    1  [1x1 Signal]      ''              sldemo_fuelsys/EGO Fault Switch         
    2  [1x1 Signal]      air_fuel_ratio  sldemo_fuelsys/Engine Gas Dynamics      
    3  [1x1 Signal]      ''              sldemo_fuelsys/Engine Speed Fault Switch
    4  [1x1 Signal]      speed           sldemo_fuelsys/Engine_Speed_Selector    
    5  [1x1 Signal]      ''              sldemo_fuelsys/MAP Fault Switch         
    6  [1x1 Signal]      map             sldemo_fuelsys/MAP_Selector             
    7  [1x1 Signal]      ego             sldemo_fuelsys/O2_Voltage_Selector      
    8  [1x1 Signal]      ''              ...o_fuelsys/Throttle Angle Fault Switch
    9  [1x1 Signal]      throttle        sldemo_fuelsys/Throttle_Angle_Selector  
   10  [1x1 Signal]      fuel            sldemo_fuelsys/To Plant                 

  - Use braces { } to access, modify, or add elements using index.

Use the Simulation Data Inspector programmatic interface to get the signal ID for the signal named speed.

runCount = Simulink.sdi.getRunCount;
latestRunID = Simulink.sdi.getRunIDByIndex(runCount);
latestRun = Simulink.sdi.getRun(latestRunID);
speedSigID = getSignalIDsByName(latestRun,"speed");

Use the signal ID to create an sdidatastore object for the speed signal.

speedSDIds = matlab.io.datastore.sdidatastore(speedSigID);

Verify Contents of Datastore

Check the Name property of the sdidatastore object to verify that it matches your expectations.

speedSDIds.Name

ans = 
'speed'

You can also use the preview function to verify the first ten samples in the signal.

preview(speedSDIds)

ans=10×1 timetable
         Time          Data 
    ______________    ______

    0 sec             314.16
    0.00056195 sec    314.16
    0.0033717 sec     314.16
    0.01 sec          314.16
    0.02 sec          314.16
    0.03 sec          314.16
    0.04 sec          314.16
    0.043098 sec      314.16
    0.043098 sec      314.16
    0.043098 sec      314.16

Process Signal Data with sdidatastore Object

When your signal is too large to fit into memory, you can use the read function to read chunks of data from the Simulation Data Inspector repository to incrementally process your data. Use the hasdata function as the condition for a while loop to incrementally process the whole signal. For example, find the maximum signal value.

latestMax = [];

while hasdata(speedSDIds)
    speedChunk = read(speedSDIds);
    speedChunkData = speedChunk.Data;
    latestMax = max([speedChunkData; latestMax]);
end

latestMax

latestMax = 
314.1593

On each read operation, the read function updates the read position for the start of the next read operation. After reading some or all of the sdidatastore object, you can reset the read position to start again from the beginning of the signal.

reset(speedSDIds)

Process Signal Data in Memory

When the signal referenced by your sdidatastore object fits into memory, you can use the readall function to read all the signal data into memory for processing rather than reading and processing the data incrementally with the read function. The readall function returns a timetable with all the signal data.

speedTimetable = readall(speedSDIds);
speedMax = max(speedTimetable.Data)

speedMax = 
314.1593