Exposing Diagnostic Counters

Diagnostic counters are internal variables of FrameWorX servers and its point managers. They can be set up to generate alarms and to log warnings in Windows Event Log.

The diagnostic counters may be of a single instance or of multiple instances. A multi-instance counter means the counter exists for multiple objects of the same kind.

The diagnostic counters can be exposed using the following tools:

  • GraphWorX

  • Windows Performance Monitor

  • Dotnet-counters

  • OpenTelemetry

GraphWorX

By default, diagnostic counters are exposed as data points in the Data Browser, and you can visualize them in GraphWorX displays.

To visualize diagnostic counters in a GraphWorX display:

  1. Open GraphWorX and go to the Data Browser in the left panel.

  2. Expand Diagnostics > Control and Diagnostics > Metric. The Metric folder contains the diagnostic counters.

  3. On the Dynamics tab, select Process Point, click in the canvas, and drag diagonally to insert the process point box. In the Data Browser, select the desired counter and drag and drop it into the process point. The selected data point is added into the DataSource field in the right panel.

  4. View the current value of the counter in runtime. Use the options on the upper right of the GraphWorX screen to switch to the runtime mode or to open the display in a web browser .

Windows Performance Monitor

Diagnostic counters, except Session and SessionByPID categories, are exposed as Windows Performance Counters (not available in Azure) and can be viewed and logged with Windows Performance Monitor (perfmon.exe).

Session and SessionByPID are not exposed as Windows Performance Counters because their number grows over time, and the total number of created Windows Performance Counters is limited.

Dotnet-counters

You can use the dotnet-counters utility to monitor point manager counters in Linux. The functionality of the dotnet-counters utility is equivalent to the performance counters, but it uses a new cross-platform technology.

Follow the documentation on dotnet-counters (including the download link) on Microsoft Learn.

Usage:

dotnet-counters monitor -n <process-name> --counters <meter-name>

Example:

dotnet-counters monitor -n IcoFwxServer --counters GENESIS.FwxServer_PointManagers.SimulationPointManager

OpenTelemetry

OpenTelemetry is a cross-platform, open standard observability tool used for collecting and displaying telemetry data. It can be used with a variety of backends.

To get the OpenTelemetry (OTLP) metrics, you need an OTLP collector and a separate tool that displays the collected data. The following procedure describes visualizing diagnostic counters in Grafana using Prometheus as the OTLP collector. It sends batches with up to 1024 messages up to every 15 seconds (it might be faster if the queue is full). Learn more about using Prometheus as your OpenTelemetry backend.

To connect to Prometheus in Grafana:

  1. Open Grafana and select Drilldown > Metrics. .

  2. In Data Source, select Prometheus.

  3. In Filter, enter job = IcoFwxServer (or any other process you want to monitor).

  4. Set the environment variable, ICO_METRIC_COLLECTOR, to the OTLP collector endpoint (http://localhost:9090/api/v1/otlp/v1/metrics) to activate the OTLP metric export.

    The Prometheus OLTP collector endpoint URI is different from standard OTLP collectors. The standard default URI (http://localhost:4318/v1/metrics) is typically used with a different OTLP collector than Prometheus.

  5. (Optional) Set up additional parameters:

    • Enable gzip compression using the suffix #gzip. For example:

      http://localhost:9090/api/v1/otlp/v1/metrics#gzip

      On local networks where bandwidth is not a problem, gzip is not recommended because it is slower.
    • Enable basic authentication using the suffix #user=<username>&pass=<password>. For example:

      http://localhost:9090/api/v1/otlp/v1/metrics#user=admin&pass=pass

    • Enable writing extra debug information into stdout (console) for troubleshooting using the suffix #debug. For example:

      http://localhost:9090/api/v1/otlp/v1/metrics#user=admin&pass=pass&debug