Import Data into Diagnostic Feature Designer

Diagnostic Feature Designer is an interactive tool for processing ensemble measurement data and extracting features that indicate the condition, such as healthy or faulty, of the machines that produced the data. The data can come from measurements on systems using sensors such as accelerometers, pressure gauges, thermometers, altimeters, voltmeters, and tachometers. The main unit for organizing and managing multifaceted data sets in the app is the data ensemble. An ensemble is a collection of data sets, created by measuring or simulating a system under varying conditions. Each row within the ensemble is a member. Each member of the ensemble contains the same variables, such as Vibration or Tacho.

The first step in using Diagnostic Feature Designer is to import source data into the app from your MATLAB^® workspace. You can import data from tables, timetables, cell arrays, or matrices. You can also import an ensemble datastore that contains information that allows the app to interact with external data files. Your files can contain actual or simulated time-domain measurement data, spectral models or tables, variable names, condition and operational variables, and features you generated previously. Diagnostic Feature Designer combines all your member data into a single ensemble data set. In this data set, each variable is a collective signal or model that contains all the individual member values.

Before importing your data, it must already be clean, with preprocessing such as outlier and missing-value removal. For more information, see Data Preprocessing for Condition Monitoring and Predictive Maintenance.

During the import process, you select which variables you want to import, specify the variable types, and perform other operations to create the ensemble that you want to work with. When you click import, the app applies your specifications and creates an ensemble that contains your selected data. The following figure illustrates the overall import flow.

Notional import workflow. Workspace data sources are on the left. Import dialog box is in the middle. Post-import app variables pane is on the right.

After you import your data once, you do not need to import it again for subsequent sessions. Save your session to store both the initial data ensemble and any derived variables and features you compute prior to saving. You can also export your data set to the MATLAB workspace, and save your data as a file that you can import in a subsequent session.

If you have a large number of ensemble members, consider creating a representative subset of members when you first start exploring the data and potential features in the app. Because the app is interactive, importing a large number of members can result in slower performance. Instead, you can develop and rank features interactively with the smaller data set, and then generate code that repeats the computations on the original data set.

For more information on data sources, ensembles, and variable types in predictive maintenance, see Data Ensembles for Condition Monitoring and Predictive Maintenance.

Source Data Requirements

For signals, the app accepts individual member table arrays, timetable arrays, cell arrays, or numeric matrices, each member containing the same independent variables, data variables, and condition variables. For spectral data, the app accepts individual member table arrays or idfrd objects. This table describes the data requirements for variables within ensemble members.

Input Item	Content	Notes
Signal data	Timetables, tables, cell arrays, or numeric arrays	For time-based data, timetables are recommended.
Signal independent variables (IVs)	Double, `duration`, or `datetime`	For each signal variable, all member IVs must be of the same type, whether your signal is based on time or on another IV such as consumption or duty cycles. If your member data is stored in a matrix, you must have only one IV that applies to the full set of signals that you are importing. If your data was uniformly sampled in time and you do not have recorded timestamps, you can construct a uniform timeline during the import process.
Spectral data	Numeric (double) table or `idfrd` object	Each `idfrd` object must contain data for only one spectrum (the first dimension is 1). You cannot import spectra from matrices.
Condition variables (CVs)	Scalar — Numeric, string, cell, or categorical	You can import condition variables along with your data in tables, timetables, or cell arrays, but not in matrices.
Features	Scalar — Numeric, string, or cell	You can import features that you computed previously, either externally or within the app itself.
Matrices	Purely numeric array that contains columns representing a single IV and any number of signals that share that IV. Cannot accommodate spectra. Cannot generally accommodate condition variables or features.	Matrices cannot accommodate variable names. You can import condition variables and features from a matrix if your data set contains only scalars and does not contain signals.

You can import data members individually or as an ensemble that contains all your data members. This ensemble can be any of the following:

An ensemble table containing table arrays, cell arrays, or matrices. Table rows represent individual members.
An ensemble cell array containing tables, cell arrays, or matrices. Cell array rows represent individual members.
An ensemble datastore object such as a fileEnsembleDatastore or simulationEnsembleDatastore object that contains the information necessary to interact with files stored externally. Use an ensemble datastore object especially when you have too much data to fit into app memory. Reading and writing to external files during computations does impact performance, however. To create a representative subset of the ensemble datastore files to work with, see subset.
An ensemble matrix that contains only condition variables and features. Matrix rows represent individual members.
A labeled signal set (LSS), which is a labeledSignalSet object, that contains labeled data for signals, condition variables, and features. The LSS must have a label type of "attribute" and not "roi" or "point". If you are using Signal Labeler, you can export LSS items directly from that app to Diagnostic Feature Designer.

For more information about organizing your data for import, see Organize System Data for Diagnostic Feature Designer.

Typical Workflows for Data Import

The following sections describe typical workflows for different types of data import. The first workflow describes the overall import process and the subsequent workflows focus on variations that are specific to the import data format. The following table summarizes the cases for the workflows.

Import Type	Summary
Ensemble table	Core workflow that describes the import of an ensemble table, including ensemble specification, configuration, import, and confirmation in the app
Individual Ensemble Members	Use of the filtering capability within the import dialog process to quickly select and import individual ensemble members that are represented by tables or timetables
Matrices	Use of matrix column indices for variable identification. Alternative specification of matrix as feature rather than signal.
Spectral Data	Import of spectral data in `idfrd` object or in a table with columns that contain frequency or order values and corresponding data values
Signal with no IV	Import of signal with no explicit time stamps by generating virtual IV
Signal with multiple IVs	Specification of an alternative IV so that the signal can be analyzed in the app using either IV
Ensemble Datastore	Specification of an ensemble datastore that contains information on interacting with external files.
Labeled Signal Set	Import of signal, condition variable, and feature information from a `labeledSignalSet` object. The object must have a label type of `"attribute"`, and not `"roi"` or `"point"`.

Core Workflow—Import Ensemble Table

This workflow illustrates the steps for importing an ensemble table into the app.

Load Ensemble Table into MATLAB Workspace

Load your ensemble table into the MATLAB workspace. You can preview the data in the workspace variable browser, as this example shows. In this case, the data set contains two time-based signals and a scalar condition variable named faultCode.

Example ensemble table. Vibration signal is in the leftmost column. Tacho signal is in the center column. Fault code is in the rightmost column.

Open App and Start New Session

Open the app by entering diagnosticFeatureDesigner at the command line. Then, click New Session. This action opens the import dialog box.

New Session button

Select Ensemble Table from Source Variables

In the Select dataset from workspace pane, select your ensemble table as the Source.

A menu displays the one available variable, which is dataTable.

View Source Variable Components and Change Variable Type and Units

The Select source variables pane displays the variables from your ensemble. In the following example figure, the app identifies the Vibration and Tacho variables as time-based signals that each contain Time and Data variables. The third variable in the variable set for each signal, Sample (Virtual), is not checked. Sample (Virtual) allows you to generate an IV, especially when your source data set does not contain an explicit IV. For more information on Sample (Virtual), see Import Signal with No Time Variable.

Source variables pane with Vibration, Tacho, and FaultCode in a column

The icons identify the variable type that the app assumes. The icon next to faultCode, which illustrates a histogram, represents a feature. Both features and condition variables are scalars, and the app cannot distinguish between the two unless the condition variable is categorical. To change a variable type, click the variable name to open the variable properties in the Configure source variable properties pane. Then, in Variable type, change Feature to Condition Variable.

In the source variables list on the left, the faultCode variable row is gray. In the source variable properties on the right, a menu displays Feature, Condition Variable, and Independent Variable.

The icon for faultCode now illustrates a label, which represents a condition variable.

For signal and spectrum variables, you can also change the units that the app uses for plotting and for other operations within the app. To do so, in the lower level variable list of the signal or spectrum variable, click the name of the IV or data variable. The Configure source variable properties pane provides a menu of options for each property that you can modify. In the following example figure, Configure source variable properties displays properties for the Time variable of the Vibration signal. The figure illustrates the selection of Minutes from the menu of Units options.

The Select source variables pane is on the left. The Time variable in the Vibration signal is highlighted. The Configure source variable properties pane on the right displays options for variable name, variable type, IV type, and unit. The unit menu ranges from seconds to years.

Preview Data Variables

In addition to providing options for variable properties such as Type, the Configure source variable properties pane displays a preview of your import data when you click the name of a signal or spectrum variable. The following example figure shows the preview of the Vibration data. The preview pane in the figure displays source data for the first ten Vibration samples of the first ensemble member, and includes values for IV, data, and the sample index.

The preview pane displays source properties only. The preview pane does not reflect any property modifications that you make in the pane. For example, if you change the units of the Vibration signal from seconds to minutes, the preview pane still displays source units of seconds. When you complete the import, the app converts the time data to minutes for use in the app.

The Select source variables pane is on the left. The variable at the top of the pane, Vibration, is highlighted. The Configure source variable properties pane on the right displays the preview table.

Confirm Ensemble Specification and Execute Import

Confirm the ensemble specification in the Summary pane at the bottom of the dialog box. Click Import to execute the data import.

Summary displaying the ensemble name near the top and three columns that contain, from left to right, Variable Name, Variable Type, and Independent Variable.

Confirm Successful Import into App

Confirm the import in the variables pane. In this example, the two signals appear in the Signals list. The condition variable FaultCode appears in the Condition Variables list. Beneath the variables pane is the Details pane, which provides additional information about the selected variable.

The Variables pane contains a Signals list above a Condition Variables list. The details pane is on the bottom.

Select Vibration/Data and click Signal Trace to plot the data and view the imported signals, as the following example figure shows. For more information on plotting data, see Import and Visualize Ensemble Data in Diagnostic Feature Designer.

Plot of the imported vibration signal.

Import Individual Members

This workflow describes the steps associated with importing ensemble members individually.

Initiate Import Process

Load the individual member variables into your MATLAB workspace, open the app, and click New Session. In the Select dataset from workspace pane, the Source menu displays a list of the files in your workspace. The following example figure shows 10 member files and one additional file, sens2, that is not an ensemble member.

List of all workspace variables, arranged vertically in a menu. The list contains member files that start with the letter "d" and another file named "sens2".

Select Ensemble Members

Select one of the variables that represents an ensemble member. In this example, select d1. The app opens a list of all compatible workspace variables that contain the same internal variables. You can select any combination of these variables or click Select All to select all of them at once.

On the left is a list of variables that have similar format to selected variable "d1". On the right is the list of the variables—Vibration, Tacho, and faultCode—within d1.

Import Individual Timetables

If your individual members are packaged as timetables with a scalar value for each time point, specify Use as signal to have the app interpret the timetable variable as a signal rather than as a feature.

The list of timetables, which are all selected, is on the left. A list of the lower level variables within each timetable is on the right.

Complete Import Process

Once you select your variables, the remaining import steps are the same as in Core Workflow—Import Ensemble Table. The app combines the members you import into a single ensemble.

Import Matrix Data

This workflow describes the import of signals from individual member matrices. When you import data in matrices, each signal in the ensemble must share one independent variable, such as a time variable, with all signals in the ensemble. You cannot import condition variables, features, or spectral data in matrices.

Select and Preview Matrices to Import

Load your matrices into the MATLAB workspace, and then initiate the import process by clicking New Session. In the Select dataset from workspace pane, select one of your matrices and click Select All. For matrices, this pane also displays a Use as feature option. This option applies to the special case where a matrix contains only scalar condition variables and features, and no signal data.

Because matrices are numeric, the app identifies each variable column by its column index. The following example figure shows four member matrices. The first column of each matrix contains the IV representing time, and the second and third columns contain the data values for vibration and tacho data. To preview the contents of the ensemble, in the Select Source variables pane, select the Matrix row.

On the left is a list of the matrices to import. In the middle are the variables for each matrix, with the names "Col1", "Col2", and "Col3". On the right are the source variable properties, which include the variable type "Signal" and a preview of the matrix contents.

Confirm Variable Types

The app interprets Col1 as the IV because it is monotonic, and Col2 and Col3 as signal data variables.

Matrix variable assignments. Col1 is on the top. The properties display on the right contains "Independent Variable" for variable type. Col2 is on the bottom. The variable type on the right is "Data Variable".

If you cannot accurately represent your signals with a single time variable, convert your matrices into tables before import.

Complete and Confirm Import

Click Import to complete the import. Confirm that the variables pane contains the signals you want, as shown in the following example figure.

Variables pane. The top box contains Col2 and Col3. The bottom box contains the details pane, which has no information because no signals are selected.

The app merges the matrices into an ensemble data set that contains the four matrices.

This workflow demonstrates that you can import matrices, but only with limitations. If you want to identify your variables by name, import condition variables or features, or use independent timelines for independent signals, convert your matrices to tables or cell arrays prior to import. For an example of converting a set of matrices to an ensemble table, see Prepare Matrix Data for Diagnostic Feature Designer.

Import Spectral Data

This workflow illustrates how to import spectral data. You can import spectral data in two forms:

An idfrd object that contains frequency and spectrum data for a single spectrum in the Frequency and SpectrumData properties, respectively
A table that contains columns with the frequency and spectral data

When you import an idfrd object, the app recognizes that the data source is spectral, and defaults to the Spectrum variable type, as the following example figure shows.

The source variables are on the left. The spectral variable Vibration_ps, which is selected, is the third main variable from the top. The properties pane on the right displays the variable name and type at the top. A preview table of Vibration_ps is beneath the variable type.

When you import spectral data in a table, the app defaults to the Signal variable type, but provides additional options for the Spectrum and Order Spectrum variable types. The following example figure illustrates the import of spectral data in a table.

The variable Vibration_ps_1 is selected in the variable list on the left. The variable type Spectrum is highlighted in the menu on the right.

You can also import order spectra that contain order rather than frequency information. Order spectra are useful for analyzing rotating machinery. Each order is a multiple of a reference frequency, such as the primary-shaft rotational frequency. The following example figure illustrates the import of an order spectrum.

The variable Vibration_os is selected in the variable list on the left. The variable type Order Spectrum is highlighted in the menu on the right.

When you complete the import, the Variables pane displays the spectra, as the following example figure shows.

Variables pane shows the imported variables vertically in order of Signals, Spectra, and Features

Import Signal with No Time Variable

This workflow illustrates how to generate a virtual IV if your data set includes signals that contain no IV. For instance, you might have a signal that was measured or generated in uniform time samples, but which does not include a vector of the actual time stamps. The app can generate a virtual timeline that contains the same sample rate.

Select Data Source and View Source Variables

Initiate the import process and select workspace variables to import. In the following example figure, the data source is a table that contains Vibration and Tacho variables. However, these variables contain only measurement data and no time information. As always, the app provides a Sample (Virtual) option. In this case, since the data has no IV, the app automatically selects this variable.

The name of the workspace variable is selected on the left. The source variables within that data set are listed vertically in the middle. A preview of the vibration data is on the right.

View Virtual IV Properties

The default unit for a virtual IV is the sample index. You can modify this default setting by selecting the Sample (Virtual) name, which opens the source properties. The following example figure displays the default properties.

The list of source variables are in a vertical list on the left. Sample is highlighted. The Configure source variables properties pane on the right displays, from top to bottom, the name as Sample, the IV type as Index, and the unit as samples.

Reconstruct Signal Time Variable Using Sample Time

If you know the sample time of the signal, you can reconstruct the time variable. To do so, change Independent variable name to the name you want, IV type to Time, Unit to the time units you want, and Sampling interval to the sample time. For example, consider that you know the sample time for both Vibration and Tacho is 0.001 seconds. The following figure shows how to set this sample time for Vibration. Note that these settings do not affect Tacho.

The source variables are on the left. The top signal is Vibration, which has Time (Virtual) as the second variable. The middle signal is Tacho, which has Sample (Virtual) as the second variable. The Configure source variable properties pane on the right shows, from top to bottom, the name as Time, the IV type as Time, the unit as seconds, and the sampling interval as 0.001

Once you have reconstructed the IVs that you want, complete the import process. You can view your reconstructed timelines by plotting your imported signals in the app. The following figure shows plots for Vibration, which has the reconstructed timeline, and Tacho, which retains the default IV of Sample.

The vibration signal is on the left and the tacho signal is on the right.

Specify Sample Index as Alternative IV

This workflow describes the steps for specifying the signal sample index as an alternative IV when you also import a time variable or some other signal IV.

Specify Sample Index as IV

Initiate the import and select the data to import. In the Select source variables pane, select Sample (Virtual) and view the properties. The following example figure illustrates this step. In this figure, Vibration now has all three lower level variables selected. The Configure source variable properties pane displays the default IV type and unit for Sample (Virtual), which are Index and samples, respectively.

The source variables are on the left. Sample (Virtual), which is selected, is the third item beneath Vibration. The Configure source variable properties pane is on the right.

Perform the same operation on all variables for which you want to include the sample index. Complete the import process.

Switch to Sample Index the App

The app defaults to the IV type that you imported with the data. To switch to the sample index, in Options, select Index, as the example figure illustrates.

The app toolstrip is on the top. The Options menu is positioned in the right third of the toolstrip. In the menu, Index is the bottom item on the right .

All signals that include Index as an alternative IV type now use the sample indices rather than the time values. When you plot the data, the signal trace uses the sample data, as the following example figure shows.

Plot of the vibration data along an x-axis of sample

Import Ensemble Datastore

This workflow describes the steps for importing a fileEnsembleDatastore object or a simulationEnsembleDatastore object. Ensemble datastore objects provide information that allows the app to interact with external files. They include specifications for the variables you want to read, the variable types, and the source file locations. When you import an ensemble datastore, you can choose whether to store results within the app memory or write the results back to the ensemble datastore. For more information about ensemble datastores, see Data Ensembles for Condition Monitoring and Predictive Maintenance.

Select Data Source and View Source Variables

Initiate the import process. In the Select dataset from workspace pane, select the ensemble datastore. The app uses the ensemble datastore properties for SelectedVariables to select the variables to display. The app also uses the DataVariables, IndependentVariables, and ConditionVariables properties to determine which variables belong to which of these variable types. The example figure illustrates the import of a simulationEnsembleDatastore object ens.

The ensemble ens is selected on the left. The SimulationInput variable is on the top. It is not selected, and has an orange triangle with an exclamation point next to it. Beneath SimulationInput, the variables are, from top to bottom, Flow, Pressure, and CombinedFlag.

In the preceding example figure, the app interprets ens as follows:

The data variables Flow and Pressure appear identical in form to timetable-based variables extracted from tables.
ens includes the standard simulationEnsembleDatastore variable SimulationInput in the SelectedVariables property. However, the app does not support the SimulationInput data format and displays an orange warning icon. The app also automatically clears the selection and deletes SimulationInput from ens.SelectedVariables.
CombinedFlag appears as a condition variable in accordance with ens.ConditionVariables.

Choose How App Interacts with External Files

You can choose whether the app interacts with the external files referenced in your ensemble datastore. In the Select more variables pane, use Append data to file ensemble to specify your choice.

The Source menu is on the top of the Select dataset from workspace pane. The Append data to file ensemble is beneath the Source menu.

If you select this option, the app interacts directly with the external files and writes results, such as derived variables or features, to the same folder as the original data. If you are using a fileEnsembleDatastore object, the object must include a reference to a write function that is specific to your data structure. You do not need a write function if you are using a simulationEnsembleDatastore object.
If you clear this option, the app stores results in local app memory for the duration of the session. Select this option if, for example:
- You want to keep your source files pristine at least until you have finalized your processing and feature generation.
- You do not have write permission for your source files.
- You do not have a write function, and you are using a fileEnsembleDatastore.
- The process of writing the results back to the source files is slow.
To retain your local results at the end of the session, use Save Session. You can also export your results as a table to the MATLAB workspace. From the workspace, you can store the results in the file, or integrate results selectively using ensemble datastore commands.

Import Labeled Signal Set (LSS)

This workflow describes the steps for importing an LSS, or labeledSignalSet object, from Signal Labeler in Signal Processing Toolbox™. You can import only LSS objects with a LabelType property of "attribute", and not "roi" or "point". The signals you import must be full signals and not framed (segmented) signals.

Select Data Source and View Contents

Initiate the import process. In the Select dataset from workspace pane, select the LSS. The app retains the LSS labels and interprets the variables as signals, condition variables, and features, according to the variable roles in the LSS and their data type.

New Session dialog box. Dataset lss1 is selected on the left. Select source variables identify the signal, the condition variable, and the features in the middle. The summary on the bottom displays the imported variable names and types on the bottom.

View Imported Variables and Features in Variables Pane

View the imported variables and features in the variables pane. The imported data retains the labels and the variable and feature designations.

Variables pane showing the features extracted from Signal_lss

Export LSS data directly from Signal Labeler to Diagnostic Feature Designer

You can export LSS data directly from Signal Labeler to Diagnostic Feature Designer. To do so, in Signal Labeler, select the signal sets you want to import.

Export menu in Signal Labeler. In the middle section, To Diagnostic Feature Designer is selected. The Export button is at the top right corner.

A New Session window opens up in Diagnostic Feature Designer. Use the guidance in the previous two sections to complete the import.