Data Modification

This section details how you can manipulate the data to better meet your Telemetry goals. This includes the new Actions Chain, which can contain the includeData and excludeData options introduced in F5 BIG-IP Telemetry Streaming 1.8.0, and the previously introduced setTag property.

Filtering

If a configuration contains multiple items, filtering is not recommended as filtering disables endpoints. For example, if your configuration contains 250 pools, TS will take all 250 pools at the same time then applies filtering by name. However, filtering is useful if you want to exclude specific data such as pools. In this case TS would not make an HTTP request to a REST API endpoint, reducing the load on BIG-IP.

Actions chain

Version Notice:

Support for Actions chain is available in BIG-IP TS 1.8.0 and later.

This section describes BIG-IP Telemetry Streaming Action chains, and how to use them. Action chains can be thought of as a pipeline of actions for data post-processing (and pre-processing for System Poller only).

These actions use the setTag, includeData, and excludeData options described in detail later on this page. See each section for details on the individual properties.

The order of execution is deterministic; actions are executed as they are defined - one after another.

The following is an example of an Action chain with a description after the example.

{
    "actions": [
        {
            "excludeData": {},
            "locations": {
                "system": true
            }
        },
        {
            "setTag":
        },
        {
            "includeData": {},
            "locations": {
                "virtualServers": {
                    ".*":
                        "name": true,
                        "clientside.bitsIn": true
                }
            }
        }
    ]
}

1. First BIG-IP Telemetry Streaming will exclude system.
2. Next BIG-IP Telemetry Streaming will apply tags to known locations (if the locations property is not specified, then the tag is applied to virtualServers, pools etc.)
3. Finally BIG-IP Telemetry Streaming keeps all virtualServers data with properties defined in locations only.

As result of execution output will look like:

{
    "virtualServers": {
        "/Common/app.app/virtualServer":
            "name": "/Common/app.app/virtualServer",
            "clientside.bitsIn": "100"
    }
}

If the action has the property enable with value false then this action will be skipped.

Another example:

{
    "actions": [
        {
            "includeData": {},
            "locations": {
                "system": true
            }
        },
        {
            "setTag":
        },
        {
            "includeData": {},
            "locations": {
                "virtualServers": {
                    ".*": {
                        "name": true,
                        "clientside.bitsIn": true
                    }
                }
            }
        }
    ]
}

As result of execution output will look like:

{} - empty object

This is because

1. First action includeData will keep only system.
2. Second action setTag will try to assign tag to known locations (if the ‘locations’ property is not specified, then the tag is applied to ‘virtualServers’, ‘pools’ etc.)
3. Third action includeData should keep only virtualServers, but after execution of action #1 only the system property was left in the output - so, BIG-IP Telemetry Streaming removed from the output everything that not matched virtualServers and as result the output is empty object.

Pre-optimization (System Poller only)

BIG-IP Telemetry System tries to analyze the actions chain before fetching data from the BIG-IP in order to reduce number of requests to BIG-IP.

Example 1:

{
    "actions": [
        {
            "includeData": {},
            "locations": {
                "system": true
            }
        },
        {
            "setTag":
        },
        {
            "includeData": {},
            "locations": {
                "virtualServers": {
                    ".*": {
                        "name": true,
                        "clientside.bitsIn": true
                    }
                }
            }
        }
    ]
}

The BIG-IP Telemetry System sees that first action is includeData and it should include only system. So, as result of the Actions chain analysis, the BIG-IP Telemetry System will fetch only system data and not virtualServers.

Example 2:

{
    "actions": [
        {
            "excludeData": {},
            "locations": {
                "system": true
            }
        },
        {
            "setTag":
            "ifAllMatch": {
                "pools": {
                    ".*": {
                        "name": "poolName"
                    }
                }
            }
        },
        {
            "includeData": {},
            "locations": {
                "virtualServers": {
                    ".*": {
                        "name": true,
                        "clientside.bitsIn": true
                    }
                }
            }
        }
    ]
}

1. Telemetry System sees that first action is excludeData and it should exclude system property. So, as result of the Actions chain analysis, Telemetry System will not fetch system information, but will still fetch everything else - pools, virtualServers and etc.
2. Telemetry System sees that seconds action is setTag with ifAllMatch conditions. So, the Telemetry System fetches pools already (see step 1) - no additional action required.
3. Telemetry System sees that third action is includeData. So, Telemetry System looks into locations and determines that it should keep virtualServers only.

As result of the actions chain analysis, the Telemetry System will fetch virtualServers only and not pools (and not anything else) because only virtualServers should be included in the result’s output.

Value-based matching

Version Notice:

Support for value-based matching is available in BIG-IP TS v1.10.0 and later

F5 BIG-IP BIG-IP Telemetry Streaming v1.10 adds the ifAnyMatch functionality to the existing value-based matching logic. Value-based matching means that BIG-IP TS can filter based on the value of ifAnyMatch instead of just the presence of the field. You can provide multiple values, and the Action (includeData, excludeData or setTag, described in detail in the section starting with Using the includeData property) is triggered if any of the blocks in the array evaluate to true.

The following example snippet uses the includeData action, so if any of the virtual servers in the test tenant are either enabled or disabled (and have a state of available), then only the virtualServer data is included. And because it uses includeData, the action must evaluate to true to occur, so if none of the virtualServers have a state of available, then ALL data is included.

"actions": [
    {
        "includeData": {},
        "ifAnyMatch": [
            {
                "virtualServers": {
                    "/test/*": {
                        "enabledState": "enabled",
                        "availabilityState": "available"
                    }
                }
            },
            {
                "virtualServers": {
                    "/test/*": {
                        "enabledState": "disabled",
                        "availabilityState": "available"
                    }
                }
            }
        ],
        "locations": {
            "virtualServers": {
                ".*": true
            }
        }
    },

For a complete declaration with value-based matching, see Value-based matching.

Tag property

Beginning in F5 BIG-IP Telemetry Streaming 1.6.0, tagging is an actions array (the old Tag property is still available). Inside this actions array, you can add tagging objects.

This table shows the parameters available for the Tag property.

Parameter	Required	Type	Description/Notes
setTag	true	Object	The setTag property is the tag(s) that will be applied (each additional property inside setTag is a tag that will be applied).
enable	false	Boolean	This value is used to enable an action.
ifAllMatch	false	Object	This property contains conditions you specify for the tag. If you use this property, TS verifies the conditions inside ifAllMatch and checks against the data. All conditions inside this property must match the data for tagging to be performed. If you do not use this property, then the system tags everything in the locations property.
locations	false	Object	This property is used to specify where the tags should be applied. If you used ifAllMatch, and all of the conditions inside ifAllMatch pass, then the locations are where the tags are applied (or to default objects if no locations are provided). If you do not use this property, the following locations are used by default: virtualServers, pools, ltmPolicies, httpProfiles, clientSslProfiles, serverSslProfiles, networkTunnels, deviceGroups, and iRules. If you use this property with an empty array, no tagging occurs.

The following is a snippet that includes this tagging action.

 "actions": [
     {
         "enable": true,
         "setTag": {
             "tag1": {
                 "prop1": "tag1prop1",
                 "prop2": "tag1prop2"
             },
             "tag2": "Another tag"
             },
         "ifAllMatch": {
             "virtualServers": {
                 ".*": {
                     "serverside.bits.*": true
                 }
             }
         },
         "locations": {
             "virtualServers": {
                 ".*": {}
             }
         }
     }
 ]

Example declaration using setTag

The following is an example declaration using the tagging action. In this example:

• BIG-IP Telemetry Streaming tags all virtualServers with the two tags in setTag if the conditions in ifAllMatch pass.
• For the conditions in ifAllMatch to match, all virtualServers that match the regular expression .*, must have the property that matches serverside.bits.*.
• If all the conditions pass, the two tags are applied.

 {
     "class": "Telemetry",
     "My_System_Poller": {
         "class": "Telemetry_System",
         "systemPoller": {
         "interval": 60,
         "actions": [
             {
                 "enable": true,
                 "setTag": {
                     "tag1": {
                         "prop1": "hello",
                         "prop2": "goodbye"
                     },
                     "tag2": "Another tag"
                 },
                 "ifAllMatch": {
                     "virtualServers": {
                         ".*": {
                             "serverside.bits.*": true
                         }
                     }
                 },
                 "locations": {
                     "virtualServers": {
                         ".*": {}
                     }
                 }
             }
         ]
         }
     }
 }

Note that you can still use `A` and `T` as tag values. For example:

"setTag": {
  "applicationTag": "`A`",
  "tenantTag": "`T`"
}

"setTag": {
    "appInfo": {
        "applicationTag": "`A`",
        "tenantTag": "`T`"
    }
}

Tag property for BIG-IP TS versions prior to 1.6.0

For F5 BIG-IP Telemetry Streaming versions 1.5.0 and earlier, the tag property provides a way to add additional properties (key:value) to the output. If not provided, the property will default to:

{
    "tenant": "`T`",
    "application": "`A`"
}

Using the includeData property

Version Notice:

Support for includeData is available in BIG-IP TS 1.8.0 and later.

You can use the includeData property to output only the data you specify, and exclude everything else.

The following table shows the possible parameters for includeData. After the table, there are three examples.

Parameter	Required	Type	Description/Notes
enable	false	Boolean	This value is used to enable an action.
ifAllMatch	false	Object	This property contains the conditions you specify for the includeData. If you use this property, TS verifies the conditions inside ifAllMatch and checks against the data. All conditions inside this property must match the data for includeData to be performed.
locations	false	Object	This property is used to specify what data should be included. If you used ifAllMatch, and all of the conditions inside ifAllMatch pass, then the locations will be included.

Example 1

The following is an example of Telemetry output without using includeData:

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    }
}

This is an example of an includeData Action definition:

{
    "includeData": {},
    "locations": {
        "system": {
            "version": true
        }
    }
}

And this is an example of the output from the Action definition.

{
    "system": {
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    }
}

Note

includeData treats ‘version’ as a regular expression, so as result both ‘version’ and ‘versionBuild’ are included in output.

Example 2

The following is an example of Telemetry output without using includeData:

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    }
}

This is an example of an includeData Action definition:

{
    "includeData": {},
    "locations": {
        "system": {
            "version$": true
        }
    }
}

And this is an example of the output from the Action definition.

{
    "system": {
        "version": "14.0.0"
    }
}

Example 3

The following is an example of Telemetry output without using includeData:

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    },
    "virtualServers": {
        "virtual1": {
            "clientside.bitsIn": "100",
            "clientside.bitsOut": "200"
        }
    }
}

This is an example of an includeData Action definition:

{
    "includeData": {},
    "locations": {
        "virtualServers": {
            ".*": {
                "clientside.bitsIn": true
            }
        }
    }
}

And this is an example of the output from the Action definition.

{
    "virtualServers": {
        "virtual1": {
            "clientside.bitsIn": "100"
        }
    }
}

Using the excludeData property

Version Notice:

Support for excludeData is available in BIG-IP TS 1.8.0 and later.

You can use the excludeData property to exclude only the data you specify, and include everything else.

The following table shows the possible parameters for excludeData. After the table, there are three examples.

Parameter	Required	Type	Description/Notes
enable	false	Boolean	This value is used to enable an action.
ifAllMatch	false	Object	This property contains the conditions you specify for the excludeData. If you use this property, TS verifies the conditions inside ifAllMatch and checks against the data. All conditions inside this property must match the data for excludeData to be performed.
locations	false	Object	This property is used to specify what data should be excluded. If you used ifAllMatch, and all of the conditions inside ifAllMatch pass, then the locations will be excluded.

Example 1

The following is an example of Telemetry output without using excludeData:

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    }
}

This is an example of an excludeData Action definition:

{
    "excludeData": {},
    "locations": {
        "system": {
            "version": true
        }
    }
}

And this is an example of the output from the Action definition.

{
    "system": {
        "hostname": "hostname",
    }
}

Note

includeData treats ‘version’ as a regular expression, so as result both ‘version’ and ‘versionBuild’ are excluded in output.

Example 2a

The following is an example of Telemetry output without using excludeData. Note that excludeData tries to find an exact match first, and if no exact match exists, then treats the property as a regular expression (see example 2b).

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    }
}

This is an example of an excludeData Action definition:

{
    "excludeData": {},
    "locations": {
        "system": {
            "version*": true
        }
    }
}

And this is an example of the output from the Action definition.

{
    "system": {
        "hostname": "hostname"
    }
}

Example 2b

This example highlights how BIG-IP Telemetry Streaming treats a non-exact match as a regular expression (this example uses the same example without using excludeData).

This is an example of an excludeData Action definition:

{
    "excludeData": {},
    "locations": {
        "system": {
            "versio": true
        }
    }
}

And this is an example of the output from the Action definition.

{
    "system": {
        "hostname": "hostname"
    }
}

Example 3

The following is an example of Telemetry output without using excludeData:

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    },
    "virtualServers": {
        "virtual1": {
            "clientside.bitsIn": "100",
            "clientside.bitsOut": "200"
        }
    }
}

This is an example of an excludeData Action definition:

{
    "excludeData": {},
    "locations": {
        "virtualServers": {
            ".*": {
                "clientside.bitsIn": true
            }
        }
    }
}

And this is an example of the output from the Action definition.

{
    "system": {
        "hostname": "hostname",
        "version": "14.0.0",
        "versionBuild": "0.0.1"
    },
    "virtualServers": {
        "virtual1": {
            "clientside.bitsOut": "200"
        }
    }
}