Introduction

Diginsight telemetry is a set .Net packages that that provides automatic observability for dotnet applications.
In particular, the full application flow is made available to local text based streams such as traditional file logs, the Console Log or the Azure Streaming Log and also to remote analysis tools such as Azure Monitor and Prometheus/Grafana.

Diginsight allows observability of the full application lifecycle, including static methods, injection sequences and the application startup and shutdown sequences where configuration problems and much complexity are often hidden.

Diginsight telemetry is produced by standard ILogger<> and System.Diagnostic activity classes so it integrates (without replacing) other logging systems telemetry. Also, diginsight telemetry fully integrated with Opentelemetry and the W3C Trace Context Specification so traceids are preserved across process invocations of a distributed system.

Diginsight telemetry targets all dotnet framework versions starting from netstandard2.0.
Samples are available on telemetry_samples repository to demonstrate use of telemetry on .net 4.8 up to [blazor webassembly,].net6 and .net8+ assemblies.

Additional Information

Diginsight telemetry is readable, consistent and efficient:

readable: telemetry is easily readable on local troubleshooting tools, live server consoles and remote analysis tools such as the Azure monitor. The generated application flow is consistent with application code so that problems understanding is much simplified.

consistent: the application flow is rendered consistently on local troubleshooting tools, where maximun flow detail is available, and to the remote analysis tools such as the Azure monitor, where metrics and data from past executions can be easily compared.

efficient: application flow is generated with fully optimized techniques (such as dynamic compilation). Also, the full application flow observability can be enabled dynamically and only on the specific executions that are under analysis. this ensures that diginsight can be leveraged with no pratical impact on application performance.

The image below shows the text based stream associated to to a Web API call.

The following image shows the same call on the Azure Monitor Transaction Detail where the call structure is shown as a hierarchy of activities (also called spans) and trace details:

Diginsight uses dynamic logging to support full observability on live environments.
Live environments logging level is normally limited to Warning or Information levels to limit telemetry volumes produced by the applications.
With dynamic logging Log level can be raied to debug or trace for a single call, for example, by means of the Log-Level http headers.

The image below shows a call to a live environment where the log level is set to Debug or Trace for 2 categories:

The image below shows the live environment AKS console where our call is traced with full datail, while other calls are being processed with limited Log level.

Performance information gathered by diginsight can be analyzed in the form of metrics.
The following image shows the Azure Monitor Metrics dashboard where method invocations and latencies can be analized in value and frequency:

Intelligent sampling, dynamic compilation, automatic truncation and other strategies are used to maximize application efficiency and minimize telemetry cost
For these reasons Local analysis and analysis on the remote tools can be supported without compromises on performance and without compromises on cost of telemetry in test and production environments.

HowTo: Use diginsight telemetry with no impact on Application performance an telemetry cost
Explores how diginsight telemetry can be used without impact on application performance and telemetry cost.

Example analysis

The following image shows a diginsight application flow on a text based stream for DataAnalyticsReportsController.GetDevices method. The flow can be easily obtained from developer machine log file, or from application live console such as Azure app streaming log or a Kubernetes console log:

Starting from its traceid (42488cedb33da51726293a70c3463c71), the same flow can be found as an Azure Monitor Application transaction:

from the image we can observe that internal component calls are shown into the transaction flow and not just interactions across different components.
Also, note that the transaction flow structure is consistent with the transaction flow rendered on the live console log, where more detail is available.

Latencies for the same function can be analyzed in a chart with the span_duration metric, filtered on DataAnalyticsReportsController.GetDevices method.

In facts, the span_duration metric allows analyzing latencies of any method within code.
Also, we'll see that the developer can easily add other metrics and metric tags to split and compare values in different conditions (eg. by site properties, user properties etc).

Learn more

The following articles provide easy steps to integrate diginsight into our code, how to configure telemetry for the local text based strams and how to configure telemetry for the remote analysis tools.
Also, details are provided to use its relevant features such as Dynamic configuration and Dynamic Logging.

Example code used in the articles is also available in the telemetry_samples repository.

Getting Started: explores basic steps we can follow to integrate diginsight telemetry

Observability Concepts: Explores basic concepts for application observability and Opentelemetry.

HowTo: configure diginsight telemetry to the local text based streams
Explores how we configure diginsight telemetry to the local analysis tools.

HowTo: use dynamic logging to manage loglevel dynamically, at runtime
Explores how we can troubleshoot applications by means of dynamic logging.

HowTo: use dynamic configuration to manage configurations and feature flags dynamically, at runtime
Explores how we can troubleshoot applications by means of dynamic configuration.
4

HowTo: configure diginsight telemetry to the remote tools
Explores how we configure diginsight telemetry to the remote analysis tools.

HowTo: maximize application performance and minimize telemetry cost with diginsight
Explores diginsight telemetry can be used without impact on application performance and telemetry cost.

Advanced topics:

HowTo: customize entities rendering on diginsight log streams: Explores how diginsight entities rendereing can be fully customized.

HowTo: customize metrics sent to the remote tools: Explores how custom metrics sent to the remote tools can be integrated and fully customized with tags.

HowTo: customize diginsight log streams row content: Explores how diginsight row content can be fully customized.

HowTo: troubleshoot the startup sequence: Explores how full observability can be enabled on static methods and the startup sequence.

HowTo: use class aware configurations to support comonent level or class level configurations: Explores how class aware configurations can be used to implement feature flags or configurations that can be enabled with class level granularity.

Previous versions

diginsight v3 is now available
the following article describes improvements of diginsight v3 over the previous version. Introduction to Diginsight v3.
Features such as old frameworks support (eg. .Net Framework 4.5+) or observability for startup and static sections may still have limited support on Diginsight v3.
In these cases the developer may decide to keep the old model.
Diginsight v2 will not be discontinued until feature parity is reached.
Documentation about v2 packages is still available here Diginsight v2 documentation.

Summary

diginsight telemetry is a set .Net packages that that provides automatic observability for dotnet applications.
In particular, the full application flow is made available to local text based streams such as traditional file logs, the Console log or the Azure Streaming log.

Enabling Opentelemetry, the same information can be made available to remote tools for troubleshooting or performance analysis such as Azure Monitor or Grafana.

Samples

You can start using diginsight telemetry by running the samples on the telemetry_samples repository.

Article HOWTO - Use Diginsight Samples: explores how we can use diginsight samples to test and understand integration of Diginsight telemetry in our own projects.