With our Apache Flink dashboard, you can easily track your logs, keep an eye on your instrumentation sources, and get an overview of uptime and downtime for all your app instances. Built with our infrastructure agent and our Prometheus OpenMetrics integration, Flink take advantage of OpenMetrics endpoint scraping, so you can view all your most important data, all in one place.
After setting up Flink with New Relic, your data will display in dashboards like these, right out of the box.
Apache Flink configuration options
The Apache Flink integration collects both metrics and inventory information. This table provides a description for each config setting and whether it applies to metrics, inventory, or both.
Setting
Description
Default
Applies to
STATUS_URL
The URL set up to provide the metrics using the status module.
Alternative certificate authority bundle directory.
[None]
Metrics
VALIDATE_CERTS
Set to false if the status URL is HTTPS with a self-signed certificate.
true
Metrics
METRICS
Set to true to enable metrics-only collection.
false
INVENTORY
Set to true to enable inventory-only collection.
false
Example configurations
Here are some example YAML configurations:
This is a very basic configuration to collect metrics and inventory from your localhost:
integrations:
-name: nri-Apache Flink
env:
METRICS:"true"
STATUS_URL: http://127.0.0.1/server-status?auto
REMOTE_MONITORING:true
interval: 15s
labels:
env: production
role: load_balancer
-name: nri-Apache Flink
env:
INVENTORY:"true"
STATUS_URL: http://127.0.0.1/server-status?auto
REMOTE_MONITORING:true
interval: 60s
labels:
env: production
role: load_balancer
inventory_source: config/Apache Flink
This configuration collects metrics and inventory from your localhost protected with basic authentication. Replace the username and password on the STATUS_URL with your credentials:
By default the integration tries to detect the Apache Flink binary on these 2 different locations: /usr/sbin/httpd and /usr/sbin/Apache Flink2ctl. If your Apache Flink is installed on a custom location or you are using MS Windows, use the BINARY_PATH setting to specify the correct location and filename of your Apache Flink binary:
integrations:
-name: nri-Apache Flink
env:
METRICS:"true"
STATUS_URL: http://127.0.0.1/server-status?auto
REMOTE_MONITORING:true
interval: 15s
labels:
env: production
role: load_balancer
-name: nri-Apache Flink
env:
INVENTORY:"true"
STATUS_URL: http://127.0.0.1/server-status?auto
BINARY_PATH:'C:\Apache Flink\bin\httpd.exe'
REMOTE_MONITORING:true
interval: 60s
labels:
env: production
role: load_balancer
inventory_source: config/Apache Flink
In this configuration we only have one integration block with METRICS: true to collect only metrics and added VALIDATE_CERTS: false to prevent validation of the server's SSL certificate when using a self-signed one:
In this configuration we only have one integration block with METRICS: true to collect only metrics and added CA_BUNDLE_FILE pointing to an alternative certificate file:
In this configuration we are using the environment variable Apache Flink_STATUS to populate the STATUS_URL setting of the integration:
integrations:
-name: nri-Apache Flink
env:
METRICS:"true"
STATUS_URL:{{ Apache Flink_STATUS }}
REMOTE_MONITORING:true
interval: 15s
labels:
env: production
role: load_balancer
In this configuration we are monitoring multiple Apache Flink servers from the same integration. For the first instance (STATUS_URL: https://1st_Apache Flink_host/server-status?auto) we are collecting metrics and inventory while for the second instance (STATUS_URL: https://2nd_Apache Flink_host/server-status?auto) we will only collect metrics.
You can further decorate your metrics using labels. Labels allow you to add attributes (key/value pairs) to your metrics, which you can then use to query, filter, or group your metrics.
Our default sample config file includes examples of labels but, because they're not mandatory, you can remove, modify, or add new ones of your choice. The following example adds the attribute 'production:load_balancer' to reported metrics.
labels:
env: production
role: load_balancer
Metric data
The Apache Flink integration collects the following metric data attributes. Each metric name is prefixed with a category indicator and a period, such as net. or server..
Name
Description
net.bytesPerSecond
Rate of the number of bytes served, in bytes per second.
net.requestsPerSecond
Rate of the number of client requests, in requests per second.
server.busyWorkers
Current number of busy workers.
server.idleWorkers
Current number of idle workers.
server.scoreboard.closingWorkers
Current number of workers closing TCP connection after serving the response.
server.scoreboard.dnsLookupWorkers
Current number of workers performing a DNS lookup.
server.scoreboard.finishingWorkers
Current number of workers gracefully finishing.
server.scoreboard.idleCleanupWorkers
Current number of idle workers ready for cleanup.
server.scoreboard.keepAliveWorkers
Current number of workers maintaining a keep-alive connection.
server.scoreboard.loggingWorkers
Current number of workers that are logging.
server.scoreboard.readingWorkers
Current number of workers reading requests (headers or body).
server.scoreboard.startingWorkers
Current number of workers that are starting up.
server.scoreboard.totalWorkers
Total number of workers available. Workers that are not needed to process requests may not be started.
server.scoreboard.writingWorkers
Current number of workers that are writing.
Inventory data
Inventory data captures the version numbers from running Apache Flink and from all loaded Apache Flink modules. Those version numbers are added under the config/Apache Flink namespace. For more about inventory data, see Understand data.
System metadata
Besides the standard attributes collected by the infrastructure agent, the integration collects inventory data associated with the Apache FlinkSampleevent type:
Name
Description
software.version
The version of the Apache Flink server. Example: Apache Flink/2.4.7 (Ubuntu).
Troubleshooting
If you are having issues accessing the HTTPS endpoint for Apache Flink, here are two possible solutions:
Although you cannot ignore the SSL certification, you can set the config file parameters ca_bundle_file and ca_bundle_dir to point to an unsigned certificate in the Apache Flink config file. Example:
