Analyze TDMS Files Using Tall Tables

This example shows how to access a TDMS datastore using tall tables in MATLAB®.

For this example, you have measurement vibration measurements of a rotating shaft recorded across multiple TDMS files. Your goals is to extract portions of the files using tall tables and visually analyze them.

Set up the Datastore and Create a Tall Table

Identify the location of the TDMS datastore files. Create a tall table from the TDMS datastore with an applied transform. The tdmsDatastore function can be applied to either a set of multiple files, or a single large file. Here a transform is used to convert the returned datastore type from a cell array to a table.

folder = "RotatingShaftAnalysis";
mapreducer(0); % Sets the global execution environment to be the local MATLAB session.
tData = tall( transform(tdmsDatastore(folder), ...
    @(cellData)table2timetable(cellData{1}, TimeStep=milliseconds(0.1))) );

Identify Data of Interest

Extract Top and Bottom Rows of the TDMS Tall Table

You can extract the first or last N rows using head or tail, respectively.

tHead = head(tData, 1000);
tTail = tail(tData, 1000);

Extract Specific Data

You can also apply relational operators or colon expressions to filter the data. The ? characters indicate that evaluation is deferred until you call gather.

filteredPulse = tail(tData(tData.Pulse > 2,:), 1000)

filteredPulse =

  M×3 tall timetable

    Time    Pulse    Sensor_X    Sensor_Y
    ____    _____    ________    ________

     ?        ?         ?           ?    
     ?        ?         ?           ?    
     ?        ?         ?           ?    
     :        :         :           :
     :        :         :           :

Preview deferred. Learn more.

tDataInRange =  tData(5000:10000,:)

tDataInRange =

  5,001×3 tall timetable

    Time    Pulse    Sensor_X    Sensor_Y
    ____    _____    ________    ________

     ?        ?         ?           ?    
     ?        ?         ?           ?    
     ?        ?         ?           ?    
     :        :         :           :
     :        :         :           :

Preview deferred. Learn more.

Find the Number of Rows in the Table

All table operations can be applied to tall table. One such operation is height, to get the total number of rows in the collection.

tRows = height(tData);

Gather Data for Evaluation

Finally, evaluate the computations using gather, which triggers execution of the operations that were delayed earlier. You can use gather to pass all the variables of interest in a single function call.

[tHead, tTail, filteredPulse, tDataInRange, tRows] = gather(tHead, tTail, filteredPulse, tDataInRange, tRows)

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 2: Completed in 0.25 sec
- Pass 2 of 2: Completed in 0.046 sec
Evaluation completed in 0.46 sec

tHead=1000×3 timetable
       Time         Pulse       Sensor_X    Sensor_Y
    __________    __________    ________    ________

    0 sec         0.00087333    0.12865      0.5479 
    0.0001 sec     0.0007626    0.13118     0.55483 
    0.0002 sec    0.00085302    0.12971     0.53442 
    0.0003 sec    0.00081109    0.12775      0.5556 
    0.0004 sec    0.00078684    0.12813     0.55271 
    0.0005 sec    0.00084647    0.13173     0.54674 
    0.0006 sec    0.00081109     0.1281     0.52979 
    0.0007 sec    0.00078488    0.12769     0.52421 
    0.0008 sec    0.00074556    0.13112      0.5452 
    0.0009 sec    0.00084647    0.12977     0.55233 
    0.001 sec     0.00080191    0.13064     0.53133 
    0.0011 sec    0.00088578     0.1315     0.52998 
    0.0012 sec     0.0008314    0.13487     0.51265 
    0.0013 sec    0.00080388    0.13436      0.5269 
    0.0014 sec    0.00083074    0.13597     0.52844 
    0.0015 sec    0.00083336    0.13029     0.53499 
      ⋮

tTail=1000×3 timetable
       Time         Pulse       Sensor_X    Sensor_Y
    __________    __________    ________    ________

    1.9 sec       -0.0015896     0.1368      0.5503 
    1.9001 sec    -0.0016119    0.13764      0.5842 
    1.9002 sec    -0.0016119    0.14088     0.55338 
    1.9003 sec    -0.0016571    0.13696     0.55896 
    1.9004 sec    -0.0016938    0.13838     0.56551 
    1.9005 sec    -0.0016394    0.13716      0.5894 
    1.9006 sec    -0.0016401    0.13857     0.55954 
    1.9007 sec    -0.0017646    0.13809     0.56879 
    1.9008 sec    -0.0016237    0.13918      0.5711 
    1.9009 sec    -0.0016938    0.13934     0.53142 
    1.901 sec     -0.0016027    0.13822     0.55607 
    1.9011 sec    -0.0016217    0.13854     0.56821 
    1.9012 sec    -0.0016951    0.14184     0.57283 
    1.9013 sec    -0.0016283    0.13783     0.55858 
    1.9014 sec     -0.001743     0.1361     0.55627 
    1.9015 sec    -0.0017508    0.13786     0.54201 
      ⋮

filteredPulse=27×3 timetable
       Time       Pulse     Sensor_X    Sensor_Y
    __________    ______    ________    ________

    0.1985 sec    2.7111     0.17406    0.57833 
    0.6452 sec    3.3274     0.12907    0.53788 
    1.0919 sec    3.3197     0.11504    0.53865 
    1.5386 sec    2.7469     0.14389    0.51669 
    1.9853 sec    2.8597    0.075178    0.55406 
    0.2596 sec    3.3864     0.20214    0.52591 
    0.5694 sec     2.182     0.17233     0.5367 
    0.5695 sec    2.5165     0.17727    0.60874 
    0.8793 sec    3.4009     0.13664     0.5315 
    1.1892 sec    2.2825     0.12525    0.55307 
    1.1893 sec    2.4268    0.097772    0.60046 
    1.4991 sec    3.4114     0.14396    0.54325 
    1.8089 sec    2.3788     0.11822    0.55134 
    1.809 sec     2.3314    0.081532    0.58023 
    0.0948 sec    3.2981    0.090041    0.57322 
    0.3035 sec    2.3475     0.14608    0.56763 
      ⋮

tDataInRange=5001×3 timetable
       Time          Pulse       Sensor_X    Sensor_Y
    __________    ___________    ________    ________

    0.4999 sec    -0.00076734    0.13439     0.53519 
    0.5 sec       -0.00077651    0.13228     0.53557 
    0.5001 sec    -0.00081844    0.13125     0.54597 
    0.5002 sec    -0.00086627    0.12756     0.53557 
    0.5003 sec    -0.00079617    0.12939     0.54231 
    0.5004 sec    -0.00080272    0.12887     0.55175 
    0.5005 sec    -0.00075685    0.12964     0.55849 
    0.5006 sec    -0.00085841    0.12788     0.57198 
    0.5007 sec    -0.00084007    0.12406     0.54289 
    0.5008 sec    -0.00092131    0.12685     0.55792 
    0.5009 sec    -0.00078765    0.12656     0.54963 
    0.501 sec      -0.0007942    0.12801     0.56235 
    0.5011 sec    -0.00076472    0.12984     0.54732 
    0.5012 sec    -0.00074113    0.12968     0.57872 
    0.5013 sec    -0.00083744    0.13102     0.58065 
    0.5014 sec    -0.00078175    0.12935     0.58334 
      ⋮

tRows = 
60000

Plot the Data

Use a plot to visually analyze the filtered data.

plot(tDataInRange.Time, tDataInRange.Pulse);
hold on
plot(tDataInRange.Time, tDataInRange.Sensor_X);
hold on 
plot(tDataInRange.Time, tDataInRange.Sensor_Y);
legend('Pulse','Sensor X', 'Sensor Y');
xlabel('Time (s)') 
ylabel('Pulse, Sensor_X, Sensor_Y');

Data from sensor attached to rotating shaft

Summarize Tall Table Contents

You can call the summary function to trigger immediate evaluation of all the table variables.

summary(tData);

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: Completed in 0.1 sec
Evaluation completed in 0.13 sec

RowTimes:

    Time: 60,000×1 duration
        Values:

            Min         0 sec
            Max    1.9999 sec

Variables:

    Pulse: 60,000×1 double
        Values:

            Min    -0.69755
            Max      3.4114

    Sensor_X: 60,000×1 double
        Values:

            Min     -0.212 
            Max    0.50583 

    Sensor_Y: 60,000×1 double
        Values:

            Min    0.18122 
            Max    0.94865

To learn more about tall tables, see Tall Arrays. Note: tdmsDatastore is not enabled to work with Parallel Computing Toolbox.