What is Instrumentation?
Instrumentation is the process of modifying or configuring an application so that it produces observability data about how it behaves while running.
When an application is instrumented, it emits data that helps operators and developers understand:
- What the application is doing
- How it is performing
- Where failures occur
This data typically includes:
- Logs – textual records of events occurring in the application
- Metrics – numerical measurements such as request count, latency, or memory usage
- Traces – end-to-end records showing how a request travels through services in a distributed system
In practice, instrumenting an application means enabling it to expose logs, metrics, and traces in a standardized way so that monitoring and observability tools can collect and analyze this information.
At BYM, instrumentation follows the OpenTelemetry specification, which provides a standard approach for generating and exporting observability data across services and programming languages.
Levels of Instrumentation Automation
| Level of Automation | Also Called | Description |
|---|---|---|
| Manual Instrumentation | Library Instrumentation Partially Automatic Instrumentation |
OpenTelemetry SDKs and supporting libraries are installed as needed and configured in the application's startup code. |
| Automatic Instrumentation | Automatic Instrumentation Zero-code Instrumentation |
OpenTelemetry binaries are installed to handle instrumentation without requiring changes to the application code. The binaries are typically installed during the build process, for example in the Dockerfile using scripts. |
BYM Recommendation: Manual Instrumentation
At BYM, we have chosen to prioritize manual instrumentation over automatic instrumentation.
This decision was made after extensive testing and several proof-of-concept (POC) implementations. During our testing, we found that automatic instrumentation was difficult to debug due to its “black box” setup. In practice, this often led to troubleshooting issues through manual workaround configurations, which ultimately resulted in having to instrument the application manually anyway.
Manual instrumentation gives developers more control over the instrumentation, which we found to be easier to configure and debug if problems occur.
For example, in .NET, manual instrumentation is typically implemented using various NuGet packages, which our developers are already familiar with. This approach allows developers to explicitly choose which NuGet packages should be included.
In contrast, automatic instrumentation attempts to load the same packages automatically, but in our experience this did not work reliably in practice and sometimes caused issues that could break the application during deployment.