Skip to main content
Version: 2.12.X

Event Pattern Authoring

The majority of pattern detection logic is configured in the event pattern view. In this environment, you define how to find matches from incoming data via elements, either by themselves, or in relation with other elements by means of constraints. Once a pattern is detected, outcomes are used to produce output data based on the input data or computational results of the input data.

Working with Elements​

An element is a unit of pattern detection logic that defines what parts of the incoming data constitute a match. In other words, elements tell the system what kinds of things to look for in the data. For more information about elements, see Model Artifacts.

Creating Elements​

To create an element:

  1. In the tool palette, click the Element (⬭) icon.
  2. Click the event type to associate with the new element. Click New Event Type to create new event types, and then follow the instructions in Creating New Event Types.
  3. Click any open space in the diagramming area to place the element.
  4. In the details panel, edit the element’s properties as needed. For more information, see Editing Elements.

Elements may also be created directly on the canvas as follows:

  1. Right-click any empty space in the canvas.
  2. In the selection menu, click Add Element.
  3. In the dialog, select an event type. To create a new event type, click New, then follow the instructions in Creating New Event Types.

Note

Depending on the source of the input data, elements in the event pattern view may have differently shaded colors:

  • An element associated with a raw event type is deep blue.
  • An element associated with a derived event type (that is, one that comes from an output of a lower-level event pattern) is brighter blue.

Editing Elements​

When an element is selected in the event pattern view, its information is displayed in the details panel, where the element's details can be viewed and modified as needed.

To edit an element:

  1. In the diagramming area, click the element to edit.
  2. In the details panel, click the Contents tab if it is not already selected.
  3. Edit the element's details as necessary:
    1. Under Attributes:
      1. From the Event Types dropdown menu, select the event type to modify the event type associated with the element.
      2. In the Alias field, enter an alias for labeling the element. (It is recommended that the alias describe the element's unique role in its event pattern that differentiates it from other elements using the same event type.)
      3. In the Min field, specify the minimum number of events the element must receive.
      4. In the Max field, specify the maximum number of events the element must receive.
      5. Under Time to Live, define an expiration period for the element (i.e., how long it should exist in the system before being discarded) if needed:
        1. Click +. The Create Time-To-Live dialog opens. (The Create Time-To-Live dialog is also accessible by right-clicking an element and selecting Add Time to Live from the context menu.)
        2. In the Time Value field, enter a number indicating how long the element should remain valid.
        3. From the Unit Type dropdown menu, select the appropriate unit of time for the specified time value.
        4. Check the Apply to Matched Events checkbox to drop any matched records after the configured time-to-live period. Leave it unchecked to let matched records persist after the configured time-to-live period.
        5. Click Create to create the expiration period, or Cancel to discard it.
        6. Edit the expiration period by changing the field values, if needed.
        7. Click the Delete (trash can) icon to clear the entered values, if necessary.
    2. Under the Event Type section (which uses the name of the event type selected for the element):
      1. Click the pencil (✎) icon to edit the details.
      2. Follow the procedure in Editing Event Types.

Working with Outcomes​

An outcome is a unit of pattern detection logic that represents output data. In other words, once the system finds a pattern established in an element, an outcome tells the system what to do after that discovery. For more information about outcomes, see Model Artifacts.

Creating Outcomes​

To create an outcome:

  1. In the tool palette, click the Outcome (▭) icon.
  2. Select the event type to associate with the new outcome. If a new event type should be created for the outcome, follow the process in Creating New Event Types.
  3. Click any open space in the diagramming area to place the outcome.
  4. In the details panel, edit the outcome’s properties as needed. For more information, see Editing Outcomes.

Outcomes may also be created directly on the canvas as follows:

  1. Right-click any empty space in the canvas.
  2. In the selection menu, click Add Outcome.
  3. In the dialog, select an event type. If a new event type should be created, click New, then follow the instructions in Creating New Event Types.

Editing Outcomes​

When an outcome is selected in the event pattern view, its information is displayed in the details panel, where the outcome's details can be viewed and modified as needed.

To edit an outcome:

  1. In the diagramming area, click an outcome to edit.
  2. In the details panel, click the Content tab if it is not already selected.
  3. Edit the outcome’s details as needed:
    1. Under Attributes:
      1. Under Sources, click a source to edit its details. For more information, see Editing Elements.
      2. Under Computations, click a computation to check its details. Click the Delete (trash can) icon beside a computation to delete it. For more information about computations, see Built-In Computation Functions.
      3. Under Risk, choose the appropriate risk type:
        1. If the risk factors (elements) are independent from one another, select Weighted to assign a weight to each element.
        2. If the risk factors (elements) are dependent upon one another, select Bayesian to specify the conditions used to compute risk.
      4. Under Batch Output, click the Enable Batch Output toggle to enable batch output if desired. (By default, the toggle is set to "Off.") For more information, see Configuring Batch Output.
    2. Under the Outcome section (which uses the name of the selected outcome):
      1. Click the pencil (✎) icon to edit the details.
      2. Follow the procedure in Editing Event Types.

Configuring Batch Output​

Batch output defines an interval of time during which the system listens to incoming data streams, and only publishes the data's updated (latest) state during that interval.

Note

The batch output interval refers to server time (time elapsed during the operation of the server), not data time (time specified in the data's timestamps or metadata).

To configure batch output for event patterns:

  1. In the diagramming area, click the desired event pattern's outcome.
  2. In the details panel, click the Contents tab if not already selected.
  3. Click the Batch Output section to expand it.
  4. Click the Enable Batch Output toggle to set it to "On".
  5. In the Time Value field, enter the interval of time that the output window should span.
  6. From the Unit Type dropdown menu, select the appropriate unit for the time value.
  7. Click outside of the details menu to save the changes.

Working with Constraints​

A constraint is a condition placed upon an input event for an event pattern to be considered a match. For more information about constraints, see Model Artifacts.

Creating Constraints​

To create a constraint:

  1. In the diagramming area, click an element.
  2. Click the Connection (🔀) button, then drag the line to another element.
  3. In the Create a Constraint dialog:
    1. For the first (green) element:
      1. Click the Fields dropdown menu.
      2. Select the appropriate field name. If the list is too long, type the first few characters of the field's name to narrow down the list.
      3. Click Computations if computations based on field values are needed to produce a result for comparison. For more information, see Constraint Computations Authoring.
    2. Repeat steps 3.i.a–3.i.c for the second (yellow) element.
    3. Click the Comparison Option dropdown menu to select a comparison option for the two elements.
      1. In the Offset field, if necessary, specify a value to add to the source event for the comparison. (For example, if you knew that all the source event's dates would be five minutes behind due to a server outage, you could use Offset to add five minutes to the source value.)
      2. In the Tolerance field, if necessary, specify a value to add and subtract from the source event to create a range of valid values for comparison operations. (For example, if you knew that the source event's dates would be either five minutes ahead or five minutes behind, and wanted to cover all possible dates within that range, then you could use Tolerance to establish this as the acceptable range.)
  4. Click OK to create the constraint, or click Cancel to discard it.

Selecting Constraints​

Constraint options vary depending on the datatypes involved. For example, if a constraint is created between two scalar strings, the constraints are limited to "equal" and "not equal."

The available constraints are as follows:

  • Contains – Checks whether one element is contained within the other.
  • Count equal – Checks whether the number of elements in an array is equal to the specified number.
  • Count greater than – Checks whether the number of elements in an array exceeds the specified number.
  • Count greater than or equal – Checks whether the number of elements in an array exceeds or matches the specified number.
  • Count less than – Checks whether the number of elements in an array is less than the specified number.
  • Count less than or equal – Checks whether the number of elements in an array is less than or equal to the specified number.
  • Count not equal – Checks whether the number of elements in an array does not match the specified number.
  • Equal – Checks whether both elements have the same value.
  • Greater than – Checks whether one element represents a greater number than the other.
  • Greater than or equal – Checks whether one element represents a greater or equivalent number than the other.
  • In network – Checks whether one element exists in the same network as the other.
  • Inside – Checks whether one element is situated inside the other.
  • Less than – Checks whether one element represents a lesser number than the other.
  • Less than or equal – Checks whether one element represents a lesser or equivalent number than the other.
  • Not equal – Checks whether the elements are not the same.
  • Outside – Checks whether one element is situated outside the other.

The possible datatypes are as follows, and can be either scalars or arrays:

  • Boolean – A true or false value.
  • Date/Time – A datatype that represents a date and time together as a single data unit.
  • Float – A floating point number (decimal value or fractional number).
  • Geo Coordinate – A GeoJSON-formatted pair of numerical coordinates indicating a single geographical point.
  • Geo Polygon – A GeoJSON-formatted set of coordinate pairs describing a geographical area.
  • Integer – A whole number, such as 25 or -25.
  • IP – A unique address that identifies a device on the internet or a local network.
  • IP Range – A set of consecutive IP addresses.
  • Null – An empty value.
  • String – A sequence of (often alphabetical) characters.
  • Unique ID (UUID) – An alphanumeric string for labeling information in a computer system.
  • URL – An address for a unique resource on the Internet.

For certain Cogynt builds, several datatypes beginning with the prefix COG_ may be available. These datatypes represent Cogynt-level metadata or extra features that invoke particular Cogynt behaviors. The COG_ datatypes include:

  • COG_confidence: The place where risk scores are published. Manually mapping values here allows bypassing risk tables if desired.
  • COG_filter: A Boolean field that, if it receives True, publishes the event to the output topic. If it receives False, it suppresses the event instead.
  • COG_id: A unique identifier for items. COG_id only exists in input events, and cannot be overwritten from the UI.
  • COG_matches: The behavior of COG_matches varies depending on whether it is an input or output element:
    • As input: If the element or event type has a lexicon filter on it, then COG_matches produces an array of words that matches the lexicon. Similarly, for matched nodes, it lists the name of the lexicon node.
    • As output: If mapped to an outcome, words indicating lexicon matches are highlighted when shown in Cogynt Workstation.
  • COG_timestamp: An optional field. Datetime values can be mapped to this field. When ingested, Cogynt Workstation takes the mapped value to mean the timestamp to use to denote events in chronological order. (COG_timestamp has no impact on how Authoring patterns behave.)

The constraints available for the various datatype combinations are outlined in the following tables.

Note

Only valid datatype pairings are listed in the tables. If a pairing is not listed, then there are no constraints available for it.

The same constraints are available regardless of which datatype is the first and which is the second in the pairing.

Scalar-to-Scalar Constraints​

First DatatypeSecond DatatypeAvailable Constraints
BooleanBoolean
  • equal
  • not equal
Date/TimeDate/Time
  • equal
  • not equal
  • less than
  • less than or equal
  • greater than
  • greater than or equal
FloatFloat
  • equal
  • not equal
  • less than
  • less than or equal
  • greater than
  • greater than or equal
Geo CoordinateGeo Coordinate
  • equal
  • not equal
Geo CoordinateGeo Polygon
  • inside
  • outside
Geo PolygonGeo Polygon
  • equal
IntegerInteger
  • equal
  • not equal
  • less than
  • less than or equal
  • greater than
  • greater than or equal
IPIP
  • equal
  • not equal
  • in network
IP RangeIP
  • equal
  • not equal
  • in network
IP RangeIP Range
  • equal
  • not equal
  • in network
StringString
  • equal
  • not equal
Unique IDUnique ID
  • equal
  • not equal
URLURL
  • equal
  • not equal

Array-to-Array Constraints​

First DatatypeSecond DatatypeAvailable Constraints
Geo CoordinateGeo Coordinate
  • equal
  • not equal
Geo CoordinateGeo Polygon
  • inside
  • outside
Geo PolygonGeo Polygon
  • equal

Array-to-Scalar Constraints​

First Datatype (Array)Second Datatype (Scalar)Available Constraints
BooleanBoolean
  • contains
BooleanInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
Date/TimeDate/Time
  • contains
Date/TimeInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
FloatFloat
  • contains
FloatInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
Geo CoordinateGeo Coordinate
  • equal
  • not equal
Geo CoordinateGeo Polygon
  • inside
  • outside
Geo PolygonGeo Coordinate
  • inside
  • outside
Geo PolygonGeo Polygon
  • equal
IntegerInteger
  • contains
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
IPInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
IPIP Range
  • contains
StringInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
StringString
  • contains
Unique IDInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
Unique IDUnique ID
  • contains
URLInteger
  • count equal
  • count not equal
  • count less than
  • count less than or equal
  • count greater than
  • count greater than or equal
URLURL
  • contains

Editing Constraints​

To edit a constraint:

  1. In the diagramming area, click a constraint to edit.
  2. In the details panel, click the Contents tab if it is not already selected.
  3. For the first (green) element:
    1. Click the Fields dropdown menu.
    2. Select the appropriate field name. If the list is too long, type the first few characters of the field’s name to narrow down the list.
    3. Click Computations if computations based on field values are needed to produce a result for comparison. For more information, see Constraint Computations Authoring.
  4. Repeat steps 3.i-3.iii for the second (yellow) element.
  5. Click the Comparison Option dropdown menu to a select a comparison option for the two elements.
  6. Click anywhere outside the details panel to save the changes.

Editing Groups​

The only editable attribute of a group is its color.

Note

Groups are a legacy feature in Cogynt, and no longer serve any necessary functions.

To change the color of a group:

  1. In the diagramming area, click the group to edit.
  2. In the details panel, click the Contents tab if it's not already selected.
  3. Click the color bar above the Members section.
  4. From the color palette, select the desired color.
  5. Click anywhere outside the color palette to close it and save the change.

For more information about groups, see Model Artifacts.

Creating Partitions​

Partitions instruct the system to analyze events based on a particular field within a specified event pattern.

To create a partition for an event pattern:

  1. In the diagramming area, click an event pattern to give a partition.
  2. In the details panel, click the Contents tab if it is not already selected.
  3. Under Attributes, click the Partitions section to expand it.
  4. Click +.
  5. In the Create Partition dialog:
    1. From the Elements dropdown menu, select an element containing the field to partition over.
    2. From the Fields dropdown menu, select the field to partition over. (The Fields dropdown menu cannot be opened until a selection is made from Elements.)
  6. Click Create to create the partition, or click Cancel to discard it.

Setting Windowing​

Windowing separates the data from an input stream into discrete sections, making it possible to process and calculate data over time. For more information, refer to Windows in the official Apache Flink documentation.

To add windowing to an event pattern:

  1. In the diagramming area, click an event pattern to set windowing for.
  2. In the details panel, click the Contents tab if it is not already selected.
  3. Under Attributes, click the Windowing section to expand it.
  4. From the Window Type dropdown menu, select the desired window type ("Sliding" or "Tumbling"). For more information about these windowing types, refer to Tumbling Windows and Sliding Windows in the official Apache Flink documentation.
  5. From the Input Event Type dropdown menu, select the event type to use as the input.
  6. From the Referenced Timestamp dropdown menu, select the timestamp for the window to use as its referencing timeframe.
  7. If the Referenced Timestamp selection is anything other than "Kafka Ingestion Time", the Lateness field appears. In the Lateness field, specify the length and unit of time that the system should continue accepting and processing events from a window after it has closed.
  8. In the Window Length field, specify the minimum length of time that the window should span.
  9. In the Slide Length field, specify the length of time for each sliding window. (Note: This field appears only if Window Type is set to "Sliding".)
  10. From the Unit Type dropdown menu to the right of the Minimum Length field, specify the unit of time for the Minimum Length value.
  11. In the Timeout field, specify how long the system should wait before closing a window if no new input data is being received. (Note: The Timeout value should not exceed the Minimum Length value.)
  12. From the Unit Type dropdown menu to the right of the Timeout field, specify the unit of time for the Timeout value.
  13. Click outside of the details menu to save the changes.