Use Transformation Rules to lower the costs of BC Telemetry

Telemetry is one of the most important tools for any partner to see what’s happening in Business Central. However, unmanaged telemetry (or analytics) workspace costs could be huge. To lower the cost, you just need to spend some time configuring relevant signals you want to use and signals that should be ignored/suppressed.

Important update 15.10.2023

The bug was replicated by Microsoft and will be fixed in the future (no ETA set). To avoid this issue, always use tostring() whenever working with dynamic data type (for example tostring(Properties.eventId)). It’s working as expected when querying logs, but there are some issues with the same queries when used as a transformation rule.


Update 28.09.2023

There is a bug in the Log Analytics if you use dynamics types in transformation rules (such as Properties). In some cases, the transformation rule can start ignoring all incoming signals, even those that should be accepted and stored. The main issue is that this problem is only in transformation rules, if you run the same problematic query as a query against your log entries, it will return correct data. Furtunately, there is an easy solution how to avoid this problem – always cast the dynamic value to string using the tostring() function.

It’s currently under investigation by Microsoft.

To create a transformation rule, open your “Log Analytics Workspace“, in the left menu, choose “Tables”, and select three dots for the table for which you want to create the transformation rule (for BC, the only relevant tables are AppPageViews and AppTraces).

Our current transformation rule for the AppTraces table

You can see our transformation rules below. The main goal is to lower the data ingestion cost by suppressing signals that we have no usage for. We also want to receive signals from the production environments only (+ selected Sandboxes) – be aware that the application string defined in the BC Admin Center is copied with the environment, so if you have telemetry enabled in the Production environment and you create a sandbox as a copy of the production, the newly created sandbox will have telemetry enabled too.

The comments next to each line should be self-explanatory. We are suppressing web service calls for the large ISV solutions (Jet Reports, Continia, etc.) as we have no usage for these signals right now, and it can generate a huge number of signals (especially the Jet Reports).

We are also temporarily suppressing AL0000CTE – Sensitive field value has changed (it’s not in the example below) as we are currently looking for the best way to see only relevant fields as some of our customers are tracking tables with many changes that make sense for them but not for us (capacity planning etc.)

source
| where 
    // RT0003 - Pre-Authorization Succeeded, log for webclient only, otherwise too many records
    (tostring(Properties.eventId) <> "RT0003" or tostring(Properties.clientType) == "WebClient") and
    // RT0004 - Authorization Succeeded, log for webclient only, otherwise too many records
    (tostring(Properties.eventId) <> "RT0004" or tostring(Properties.clientType) == "WebClient") and
    // RT0006 - Report rendered: 1320 - Notification Email
    (tostring(Properties.eventId) <> "RT0006" or tostring(Properties.alObjectId) <> "1320") and
    // RT0008 - Incoming WS calls for specific publishers that generates large amount of API calls
    (tostring(Properties.eventId) <> "RT0008" or (tostring(Properties.extensionPublisher) !in ("PUBLISHER1", "PUBLISHER2"))) and
    // RT0019 - Outcoming WS calls for specific publishers that generates large amount of API calls
    (tostring(Properties.eventId) <> "RT0019" or tostring(Properties.extensionPublisher) !in ("PUBLISHER1", "PUBLISHER2")) and
    // RT0030 - Error dialogs with empty text (not really errors)
    (tostring(Properties.eventId) <> 'RT0030' or tostring(Message) !startswith "Error dialog shown:" or Properties has "alErrorMessage") and
    // LC0042 - Task XZY removed: ID
    (tostring(Properties.eventId) <> 'LC0042' or tostring(Properties.codeunitObjectId) <> "448") and
    // AL0000GDP - Feature Workflows is Used not needed
    // AL0000CTM - Feature Emailing is Used not needed
    // AL0000GNO - Feature Check documents and journals while you work is Used
    // AL0000GNP - Feature Check run
    // AL0000H7M - Feature Dataverse is Used
    // AL0000H7F - Feature Dynamics 365 Sales is Used
    // AL0000KMT - Feature Dataverse Base Entities is Used
    // AL0000JBO - Posting Preview called
    // AL0000E24 - Job Queue Lifecycle	Job queue entry enqueued
    // AL0000E25 - Job Queue Lifecycle	Job queue entry started
    // AL0000E26 - Job Queue Lifecycle	Job queue entry finished
    // AL0000H7N - Entity sync
    // AL0000CTV - Email sent
    // AL0000H7G - Sales entity synch
    // AL0000H7P - Custom entity synch
    // AL0000H7O - Base entity synch
    // AL0000IIL - Base entity synch
    // LC0040 - Task XYZ created: TIME scheduled to run after DATETIME. Ready to run: True
    // LC0043 - Task XYZ main codeunit ID completed.
    tostring(Properties.eventId) !in (
        "AL0000GDP", "AL0000CTM", "AL0000GNO", "AL0000E24", "AL0000E25", "AL0000E26", "AL0000GNP", "AL0000H7M", "AL0000H7N", "AL0000CTV", "AL0000H7F", "AL0000JBO", "AL0000H7G", "LC0040", "LC0043", "AL0000H7P", "AL0000H7O", "AL0000IIL", "AL0000KMT"
    ) and
    // ALFS5XXX-XXX - Our custom signals we do not want to receive by default
    tostring(Properties.eventId) !in (
        "OUR OWN SIGNAL"
    ) and
    (
        tostring(Properties.environmentType) == "Production" or 
        (
            tostring(Properties.environmentType) == "Sandbox" and 
            (
                // Enabled UAT environment
                (tostring(Properties.aadTenantId) == ('TENANT1') and tostring(Properties.environmentName) == 'UAT-XYZ') or
                // Enabled UAT environment
                (tostring(Properties.aadTenantId) == ('TENANT2') and tostring(Properties.environmentName) == 'UAT-XYZ')
            )
        )
    )

We don’t have any specific transformation rules for the AppPageViews table, as we’ve just enabled this table. We are only suppressing Sandbox signals except for the signals from specific UAT environments.

Note: The “Properties.environmentType” is not currently available in AppPageViews, but there is a similar information in (Measurements.environmentType, 0 = Production, 1 = Sandbox). This will be fixed with the next telemetry update (last update 09.09.2023) to have the same information as for AppTraces.

source
| where 
    tostring(Measurements.environmentType) == "0" or
    (
        tostring(Measurements.environmentType) == "1" or
        (
            // Enabled UAT environment
            (tostring(Properties.aadTenantId) == ('TENANT1') and tostring(Properties.environmentName) == 'UAT-XYZ') or
            // Enabled UAT environment
            (tostring(Properties.aadTenantId) == ('TENANT2') and tostring(Properties.environmentName) == 'UAT-XYZ')
        )
    )