telegraf gpu_在Windows上使用Telegraf监控nvidia gpu_windows

telegraf gpu

Telegraf | 英伟达| 监控| Grafana | InfluxDB (Telegraf | Nvidia | Monitoring | Grafana | InfluxDB)

This Small Guide Will Cover

本小指南将涵盖

install Telegraf
安装Telegraf
configure Telegraf (assuming InfluxDB as output)
配置Telegraf(假设InfluxDB作为输出)
monitor NVIDIA GPU
监控NVIDIA GPU

安装Telegraf (Install Telegraf)

To install Telegraf on Windows, you simply need to download the Zip-File and extract it to your Program Files folder.

要在Windows上安装Telegraf，您只需要下载Zip文件并将其解压缩到Program Files夹中。

download the zip from the Telegraf downloads or releases page: https://portal.influxdata.com/downloads/
从Telegraf的下载或发布页面下载zip ： https ： //portal.influxdata.com/downloads/

download the zip from the Telegraf downloads or releases page: https://portal.influxdata.com/downloads/https://github.com/influxdata/telegraf/releases
从Telegraf的下载或发行页面下载zip ： https ： //portal.influxdata.com/downloads/ https://github.com/influxdata/telegraf/releases
extract it so that your Telegraf configuration file is located here: C:\Program Files\telegraf\telegraf.conf(along with the .exe)
解压缩它，以便您的Telegraf配置文件位于此处： C:\Program Files\telegraf\telegraf.conf (以及.exe )

配置Telegraf (Configure Telegraf)

基本设置 (Basic Settings)

You can add your own [global_tags] which are then added to all metrics sent to InfluxDB.

您可以添加自己的[global_tags] ，然后将其添加到发送到InfluxDB的所有指标中。

[global_tags]
  os = windows

Other than that there are only few settings you want to change in the [agent] category:

除此之外，您只想在[agent]类别中更改几项设置：

[agent]
  interval = "30s" # may want to lower it from default 10s...
  metric_buffer_limit = 10000 # if your InfluxDB can be aways for a few minutes, increase the local metrics buffer
  debug = true # useful for the first run to check what gets sent
  logfile = "/Program Files/Telegraf/telegraf.log" # check here for errors
  hostname = host # value of the `host` tag

Ah! And don’t forget to configure your output, a docker InfluxDB container running locally in our case:

啊! 并且不要忘了配置您的输出，在我们的例子中是一个在本地运行的Docker InfluxDB容器：

[[outputs.influxdb]]
  urls = ["http://localhost:8086"]

My previous article covers how to setup and run InfluxDB and Grafana locally within minutes:

我的前一篇文章介绍了如何在几分钟内在本地设置和运行InfluxDB和Grafana：

标准性能计数器 (Standard Performance Counters)

First of all, this is how collection of Windows performance counters is configured, as an example: Processor.

首先，这是配置Windows性能计数器集合的方式，例如： Processor 。

[[inputs.win_perf_counters]]
  [[inputs.win_perf_counters.object]]
    # Processor usage, alternative to native, reports on a per core.
    ObjectName = "Processor"
    Instances = ["*"]
    Counters = [
      "% Idle Time",
      "% Interrupt Time",
      "% Privileged Time",
      "% User Time",
      "% Processor Time",
      "% DPC Time",
    ]
    Measurement = "win_cpu"
    # Set to true for _Total instance when querying for all (*).
    IncludeTotal=true

With this you can monitor basic metrics such as “Processor”, “LogicalDisk”, “PhysicalDisk”, “Network Interface”, “System”, “Memory” and “Paging File” to get a similar basic dashboard than for Linux or macOS.

使用此工具，您可以监视基本指标，例如“ Processor”，“ LogicalDisk”，“ PhysicalDisk”，“ Network Interface”，“ System”，“ Memory”和“ Paging File”，以获得与Linux或macOS相似的基本仪表板。

Here’s the input configuration with the above counters, add it to your telegraf.conf:

这是带有上述计数器的输入配置，将其添加到您的telegraf.conf ：

# Windows Performance Counters plugin.
# These are the recommended method of monitoring system metrics on windows,
# as the regular system plugins (inputs.cpu, inputs.mem, etc.) rely on WMI,
# which utilize more system resources.
#
# See more configuration examples at:
#   https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters
# If metrics are missing run `lodctr /r` as Administrator


[[inputs.win_perf_counters]]
  [[inputs.win_perf_counters.object]]
    # Processor usage, alternative to native, reports on a per core.
    ObjectName = "Processor"
    Instances = ["*"]
    Counters = [
      "% Idle Time",
      "% Interrupt Time",
      "% Privileged Time",
      "% User Time",
      "% Processor Time",
      "% DPC Time",
    ]
    Measurement = "win_cpu"
    # Set to true to include _Total instance when querying for all (*).
    IncludeTotal=true


  [[inputs.win_perf_counters.object]]
    # Disk times and queues
    ObjectName = "LogicalDisk"
    Instances = ["*"]
    Counters = [
      "% Idle Time",
      "% Disk Time",
      "% Disk Read Time",
      "% Disk Write Time",
      "Current Disk Queue Length",
      "% Free Space",
      "Free Megabytes",
    ]
    Measurement = "win_disk"
    # Set to true to include _Total instance when querying for all (*).
    #IncludeTotal=false


  [[inputs.win_perf_counters.object]]
    ObjectName = "PhysicalDisk"
    Instances = ["*"]
    Counters = [
      "Disk Read Bytes/sec",
      "Disk Write Bytes/sec",
      "Current Disk Queue Length",
      "Disk Reads/sec",
      "Disk Writes/sec",
      "% Disk Time",
      "% Disk Read Time",
      "% Disk Write Time",
    ]
    Measurement = "win_diskio"


  [[inputs.win_perf_counters.object]]
    ObjectName = "Network Interface"
    Instances = ["*"]
    Counters = [
      "Bytes Received/sec",
      "Bytes Sent/sec",
      "Packets Received/sec",
      "Packets Sent/sec",
      "Packets Received Discarded",
      "Packets Outbound Discarded",
      "Packets Received Errors",
      "Packets Outbound Errors",
    ]
    Measurement = "win_net"


  [[inputs.win_perf_counters.object]]
    ObjectName = "System"
    Counters = [
      "Context Switches/sec",
      "System Calls/sec",
      "Processor Queue Length",
      "System Up Time",
    ]
    Instances = ["------"]
    Measurement = "win_system"
    # Set to true to include _Total instance when querying for all (*).
    #IncludeTotal=false


  [[inputs.win_perf_counters.object]]
    # Example query where the Instance portion must be removed to get data back,
    # such as from the Memory object.
    ObjectName = "Memory"
    Counters = [
      "Available Bytes",
      "Cache Faults/sec",
      "Demand Zero Faults/sec",
      "Page Faults/sec",
      "Pages/sec",
      "Transition Faults/sec",
      "Pool Nonpaged Bytes",
      "Pool Paged Bytes",
      "Standby Cache Reserve Bytes",
      "Standby Cache Normal Priority Bytes",
      "Standby Cache Core Bytes",


    ]
    # Use 6 x - to remove the Instance bit from the query.
    Instances = ["------"]
    Measurement = "win_mem"
    # Set to true to include _Total instance when querying for all (*).
    #IncludeTotal=false


  [[inputs.win_perf_counters.object]]
    # Example query where the Instance portion must be removed to get data back,
    # such as from the Paging File object.
    ObjectName = "Paging File"
    Counters = [
      "% Usage",
    ]
    Instances = ["_Total"]
    Measurement = "win_swap"


  [[inputs.win_perf_counters.object]]
    ObjectName = "Network Interface"
    Instances = ["*"]
    Counters = [
      "Bytes Sent/sec",
      "Bytes Received/sec",
      "Packets Sent/sec",
      "Packets Received/sec",
      "Packets Received Discarded",
      "Packets Received Errors",
      "Packets Outbound Discarded",
      "Packets Outbound Errors",
    ]

And a link to the dashboard I created for these metrics. If you don’t want to spend time setting up your own dashboard, simply import it into your Grafana — you can always adjust it to your needs afterwards.

以及指向我为这些指标创建的仪表板的链接。如果您不想花费时间来设置自己的仪表板，只需将其导入到Grafana中即可-之后您始终可以根据需要进行调整。

You can also add the win_services input plugin to monitor your services, just follow the steps in the documentation:

您还可以添加win_services输入插件来监视您的服务，只需按照文档中的步骤操作：

自定义脚本或二进制执行 (Custom Script or Binary Execution)

This is where it gets interesting, because you can pretty much do whatever you want. There are just a few things you need to consider from my experience:

这就是有趣的地方，因为您几乎可以做任何您想做的事情。根据我的经验，您只需要考虑几件事：

The command must only return metrics. In case of influxdb protocol, one per line.
该命令只能返回指标。如果是influxdb协议，则每行一个。
The command should exit or fail quickly. Make sure to configure reasonable timeouts in any network calls! For instance: five seconds is more than enough for an API or HTTP call to return (usually).
该命令应退出或Swift失败。确保在任何网络呼叫中配置合理的超时！例如：五秒钟足以使API或HTTP调用返回(通常)。
Hence don’t try to do complicated stuff when gathering metrics. Keep it simple or gather the metrics “offline” and just return them when the command is executed by Telegraf.
因此，在收集指标时不要尝试做复杂的事情。 保持简单或“离线”收集指标，并在Telegraf执行命令时返回它们。
Don’t mix metrics, create generic metrics (e.g requests per second) or use dedicated measurements.
不要混合指标，创建通用指标(例如每秒请求)或使用专用指标。
Keep the cardinality low. Cardinality is basically the number of different combinations of tags. The higher this number, the bigger and slower your measurement becomes and also the queries. Compare these two cases:
保持低基数。 基数基本上是标签的不同组合的数量。该数字越高，测量值越大，查询也越慢。比较这两种情况：

a)
一种)

host tag: 10 values * cpu-core tag: ~16 values = 160 combinations
host标签：10个值* cpu-core标签：〜16个值= 160个组合

b)
b)

host tag: 10 values * pid tag: ~2000 = 20000 combinations (if not more)
host标签：10个值* pid标签：〜2000 = 20000个组合(如果不更多)

→
→

Do Not Use Unique Values For Tags. Pid (process ID) is a bad idea, so are UUIDs and generated docker container names — basically anything that can have a lot of different values. Only use “categories” for tags.
不要为标签使用唯一值。 Pid (进程ID)是个坏主意，UUID和生成的Docker容器名称也是一个坏主意-基本上任何可以具有很多不同值的东西。仅将“类别”用于标签。
Normalize tag values. Either lowercase or uppercase or any other defined case, not MixED cAse
标准化标签值。小写或大写或任何其他定义的大小写，而不是混合大小写

I will cover more on InfluxDB maintenance and useful queries in another article…

我将在另一篇文章中介绍有关InfluxDB维护和有用查询的更多信息……

配置示例 (Example Configuration)

[[inputs.exec]]
  commands = ['"C:\Program Files\telegraf\metrics-script"']
  timeout = "5s"
  data_format = "influx"

指标输出示例： (Example Metrics Output:)

my_measurement,host=foo,mytag=measure onevalue=5,val2=3234.34 1609459200000000000

my_measurement is the “table” (InfluxDB calls it “measurement”) in which the metrics gets written
my_measurement是在其中写入指标的“表”(InfluxDB称其为“测量”)
host=foo,mytag=measure are tags by which you can group and filter
host=foo,mytag=measure是可以分组和过滤的标签
onevalue=5,val2=3234.34 are values, which you can display in graphs
onevalue=5,val2=3234.34是可以在图形中显示的值
1609459200000000000 is the current unix timestamp + 9 x “0" — to make it a nanosecond timestamp. It’s 01/01/2021 UTC.
1609459200000000000是当前的unix时间戳+ 9 x“ 0” —使其成为纳秒级的时间戳，它是01/01/2021 UTC。

More on the protocol in the InfluxDB documentation:

InfluxDB文档中有关协议的更多信息：

监控NVIDIA GPU (Monitor NVIDIA GPU)

Luckily there is an extra input plugin for the Nvidia SMI interface. If you have Nvidia drivers installed, the nvidia-smi.exe program should be already in place:

幸运的是，Nvidia SMI界面还有一个额外的输入插件。如果安装了Nvidia驱动程序，则nvidia-smi.exe程序应该已经存在：

[[inputs.nvidia_smi]]
  bin_path = "C:\\Program Files\\NVIDIA Corporation\\NVSMI\\nvidia-smi.exe"

That’s it, really! Now startup up your telegraf service and you’ll see metrics arriving in the measurement nvidia_smi:

就是这样，真的！现在启动您的telegraf服务，您将看到度量标准nvidia_smi到达的指标：

你得到什么 (What You Get)

Install and run your Telegraf as a service so it will be started when you boot your machine.

安装并运行Telegraf作为服务，以便在启动计算机时将其启动。

cd "C:\Program Files\telegraf"# test your config
telegraf.exe --test# install telegraf as a service
telegraf.exe --service install# update performance counters
# you may have to rerun this if counters are missing
lodctr /rnet start telegraf