Main Content

testStream

Create connection to event stream hosted by MATLAB with schema processing applied

Since R2022b

expand all in page

    This object requires Streaming Data Framework for MATLAB® Production Server™.

    Description

    The testStream function creates a TestStream object, which you can use to test reading from and writing to event streams hosted by MATLAB. TestStream objects store events in MATLAB variables, which are exported to event streams based on the data type values specified by the ExportOptions property. TestStream applies the standard configurable schema processing when reading and writing timetable data. Use this object to test your schema management before deployment, without requiring a streaming service host. The data in a TestStream object disappears when you exit MATLAB.

    Use TestStream to develop your streaming analytic function without having to connect to or consume resources from a streaming service such as Kafka®. TestStream stores data using MATLAB memory space, so use this object with finite-size data sets. Because the data is stored in MATLAB, TestStream objects typically stream data faster than streams that have data stored on a network or file system.

    Creation

    Syntax

    ts = testStream
    ts = testStream(Rows=numevents)
    ts = testStream(Duration=timespan)
    ts = testStream(___,Name=Value)

    Description

    Row-Based Event Window

    ts = testStream creates a default TestStream object connected to an event stream hosted by MATLAB. The object reads 100 stream event rows at a time.

    example

    ts = testStream(Rows=numevents) creates a TestStream object that reads numevents stream event rows at a time.

    Duration-Based Event Window

    ts = testStream(Duration=timespan) creates a TestStream object that reads stream events occurring during the specified timestamp span, timespan.

    Additional Options

    ts = testStream(___,Name=Value) sets event stream properties using one or more name-value arguments and any of the previous syntaxes.

    Input Arguments

    expand all

    Number of events in the event window, specified as a positive integer. Rows=numevents specifies the number of rows that a call to the readtimetable function returns. If there are less than the number of specified rows available for reading, then readtimetable times out and returns an empty timetable.

    Example: Rows=500 specifies that each call to readtimetable returns a timetable with 500 rows.

    Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Timestamp span in the event window, specified as a duration scalar. Duration=timespan determines the events that the readtimetable function returns based on their timestamp. timespan specifies the difference between the last and first timestamps of events in the event window.

    Example: Duration=minutes(1) specifies that each call to readtimetable returns a timetable that has one minute's worth of events, where the timestamp of the last event is no more than one minute later than the timestamp of the first event.

    Data Types: duration

    Properties

    expand all

    Name of the event stream, specified as a string scalar or character vector. This property is provided for compatibility with other stream objects. If you do not specify a name, the testStream function generates one when you create the TestStream object.

    Example: CoolingFan

    Data Types: string | char

    This property is read-only.

    Event window size, specified by a fixed amount of time (using the Duration argument) or a fixed number of messages (using the Rows argument).

    Data Types: duration | single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

    Strategy to wait for a response from the stream, specified as one of these values:

    • "Size" — Client prioritizes filling the event window. Using this strategy, the client might wait longer than the RequestTimeout time period as long as it is still receiving the expected number of messages. The default number of messages is 50. If the client receives no messages within the RequestTimeout time period, it no longer waits.

    • "Time" — Client strictly adheres to the RequestTimeout limit, even if it has not received the expected number of messages. RequestTimeout specifies the amount of time the stream object waits between receiving events. If the stream is actively receiving data, it does not time out during that operation.

    Note

    This object does not implement the ReadLimit property and does not have a RequestTimeout property. It is provided for compatibility with other stream connector objects. By setting the wait strategy in this object, you can more easily update your code to switch between objects that do implement this property, such as KafkaStream.

    Unit of event timestamp, specified as one of these values:

    • "Milliseconds"

    • "Seconds"

    • "Minutes"

    • "Hours"

    • "Days"

    Interpret the event timestamp as the number of corresponding units before or after the UNIX® epoch.

    Data Types: string | char

    Import and Export Options

    Rules for transforming stream events into MATLAB data, specified as an ImportOptions object. This object controls the import of stream events into MATLAB.

    Rules for transforming MATLAB data into stream events, specified as an ExportOptions object. This object controls the export of MATLAB data into streams.

    Flag to indicate whether the export schema is written to the output stream, specified as a logical scalar.

    The schema is embedded in each event, which can significantly increase the size of the event. If downstream applications do not require the schema, set this flag to false to reduce the number of bytes in your stream.

    Data Types: logical

    Event Key and Body Encoding

    Name of the key variable in the event stream, specified as a string scalar or character vector.

    Data Types: string | char

    Character encoding format used to interpret the bits in an event key, specified as one of the following:

    • utf8 — UTF-8 encoding format

    • utf16 — UTF-16 encoding format

    • base64— Base 64 encoding format

    • uint8 — Eight-bit unsigned binary bytes

    If KeyEncoding is utf8 or utf16, then the KeyType property must be text. If KeyEncoding is base64 or uint8, then KeyType must be one of the numeric encoding formats.

    Character encoding scheme used to interpret the bytes in an event key, specified as one of these values:

    • uint8 — One-byte unsigned integer

    • int8 — One-byte signed integer

    • uint16 — Two-byte unsigned integer

    • int16 — Two-byte signed integer

    • uint32 — Four-byte unsigned integer

    • int32 — Four-byte signed integer

    • uint64 — Eight-byte unsigned integer

    • int64 — Eight-byte signed integer

    • single — Single-precision IEEE 754 floating point number

    • double — Double-precision IEEE 754 floating point number

    • text — String

    If KeyType is text, then the KeyEncoding property must be either utf8 or utf16. If KeyType is any of the other numeric encoding formats, then KeyEncoding must be either base64 or uint8.

    Order for storing bits in the event key, specified as one of the following.

    • LittleEndian — Least significant bit is stored first

    • BigEndian — Most significant bit is stored first

    • MatchHost— Bits are stored in the same order as is used by the host computer on which the streaming data framework is running

    • NotApplicable — Not an integer key

    This property is applicable only for integer keys and not applicable to floating point or text keys.

    Character encoding format used to interpret the bits in the event body, specified as one of the following:

    • utf8 — UTF-8 encoding format

    • utf16 — UTF-16 encoding format

    • base64— Base 64 encoding format

    • uint8 — Eight-bit unsigned binary bytes

    This property determines the size and encoding of the bytes used in the event body, which are in the format specified by BodyFormat.

    Format of bytes in event body, specified as one of the following:

    • JSON — JSON string

    • Array — MATLAB array

    • Text — String data

    • Binary — Binary data

    Depending on the encoding specified by BodyEncoding, bytes can be larger than eight bits.

    Object Functions

    readtimetableRead timetable from event stream
    writetimetableWrite timetable to event stream
    seekSet read position in event stream
    previewPreview subset of events from event stream
    identifyingNameEvent stream name
    detectImportOptionsCreate import options based on event stream content
    detectExportOptionsCreate export options based on event stream content

    Examples

    collapse all

    Create a TestStream object to read from and write events to an event stream hosted by MATLAB.

    ts = testStream
    t = 

  TestStream with properties:

                   Name: "8ee84682-dcb6-4dc7-a10a-873f80dc5f98"
          ImportOptions: "None"
          ExportOptions: "Source: function eventSchema"
          PublishSchema: "true"
             WindowSize: 100
            KeyVariable: "key"
            KeyEncoding: "uint8"
                KeyType: "text"
           KeyByteOrder: "BigEndian"
           BodyEncoding: "utf8"
             BodyFormat: "JSON"
              ReadLimit: "Size"
    TimestampResolution: "Milliseconds"

    Write sample timetable data to the event stream.

    load indoors
writetimetable(ts,indoors)

    Preview data from the timetable. The timetable data has no key column, but writetimetable generates an empty key column in the stream by default. This makes it easier to transition your code from using a TestStream object to a KafkaStream object, which includes this key column to identify the event source.

    preview(ts)
    ans =

  8×3 timetable

         timestamp          Humidity    AirQuality    key
    ____________________    ________    __________    ___

    15-Nov-2015 00:00:24       36           80        "" 
    15-Nov-2015 01:13:35       36           80        "" 
    15-Nov-2015 02:26:47       37           79        "" 
    15-Nov-2015 03:39:59       37           82        "" 
    15-Nov-2015 04:53:11       36           80        "" 
    15-Nov-2015 06:06:23       36           80        "" 
    15-Nov-2015 07:19:35       36           80        "" 
    15-Nov-2015 08:32:47       37           80        ""

    Version History

    Introduced in R2022b

    See Also

    |

    Topics