project Utilities / Python Influxdb Downsample avatar

utilities/python_influxdb_downsample#16: Collect stats

Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Milestone: v0.2
Created: 08-Feb-23 08:30



Description

As we're writing to InfluxDB anyway, it'd probably be useful to (optionally) collect some statistics:

  • Overall runtime
  • Number of jobs
  • Number of rows processed?
  • Time spent processing (per job)?
  • Time for query to run
  • Number of rows output

Having those collected will probably make it easier to check and improve performance too.



Toggle State Changes

Activity


08-Feb-23 08:30

assigned to @btasker


08-Feb-23 08:38
verified

mentioned in commit 9bbea5083dc8e6d577c251abcec0f48390cdb9f6

Commit: 9bbea5083dc8e6d577c251abcec0f48390cdb9f6 
Author: B Tasker                            
                            
Date: 2023-02-08T08:36:22.000+00:00

Message

Implement config support for stats collection (utilities/python_influxdb_downsample#16)

+23 -0 (23 lines changed)

08-Feb-23 08:38
verified

mentioned in commit be49254a249547caffcc62e29f6599d3c5f54d86

Commit: be49254a249547caffcc62e29f6599d3c5f54d86 
Author: B Tasker                            
                            
Date: 2023-02-08T08:37:59.000+00:00

Message

docs: update README with stats example utilities/python_influxdb_downsample#16

+7 -0 (7 lines changed)

08-Feb-23 08:54
verified

mentioned in commit e853386b23291bed0bff64538128fc53b82f55fe

Commit: e853386b23291bed0bff64538128fc53b82f55fe 
Author: B Tasker                            
                            
Date: 2023-02-08T08:53:51.000+00:00

Message

Implement ability to push run-time stats to Influx utilities/python_influxdb_downsample#16

This doesn't currently push anything, but provides the framework to do so - by calling write_stat() and passing a dict of fields and one of tags

+58 -2 (60 lines changed)

08-Feb-23 09:05
verified

mentioned in commit 919930b033c7f67bc8670ed122558a785b9b3144

Commit: 919930b033c7f67bc8670ed122558a785b9b3144 
Author: B Tasker                            
                            
Date: 2023-02-08T09:05:06.000+00:00

Message

Write final stats when closing the object utilities/python_influxdb_downsample#16

It wrote it's first stat!

+24 -7 (31 lines changed)

08-Feb-23 09:14
verified

mentioned in commit d8ae80083d82346dce493555b7eb43b5c26790e9

Commit: d8ae80083d82346dce493555b7eb43b5c26790e9 
Author: B Tasker                            
                            
Date: 2023-02-08T09:13:26.000+00:00

Message

Add job-level stats utilities/python_influxdb_downsample#16

This adds

  • Points processed
  • Points output
  • Outputs used

We probably also want to look at calculating job run-time, but that'll be a seperate commit as the intention is to break the times down

+26 -2 (28 lines changed)

08-Feb-23 09:21
verified

mentioned in commit fa25d7fdca2ebed508246875470afcc9d741b617

Commit: fa25d7fdca2ebed508246875470afcc9d741b617 
Author: B Tasker                            
                            
Date: 2023-02-08T09:21:21.000+00:00

Message

Add stats providing times taken to run each stage of the job (measured in nanos) - utilities/python_influxdb_downsample#16

+11 -2 (13 lines changed)

08-Feb-23 18:03

This could probably do with a dedicated doc creating to detail what each of the stats are.

Tagset

  • job: name of the job (or main-handler for stats originating from the main handler)
  • state: describes the state when the stat was generated (currently closing or in-job)

Fields

  • aggregate_run_time: the time (in nanoseconds) taken to apply aggregates in the current job
  • enabled_aggregates: The number of aggregates enabled in the current job
  • flush_run_time: the time (in nanoseconds) required to flush write buffers at script shutdown
  • jobs_run: only generated by main-handler - the number of jobs processed in this run
  • new_rows: the number of rows in the generated output (a row may contain multiple fields)
  • outputs_used: the number of InfluxDB outputs that we've attempted to write to (excluding the stats output)
  • points_processed: the number of points received from the input
  • q_period: the time period (in minutes) queried
  • window_period: the time period (in minutes) over which results are aggregated
  • query_run_time: the time (in nanoseconds) taken for the upstream query to complete
  • total_run_time: the time (in nanoseconds) taken for the job to run
  • write_run_time: the time (in nanoseconds) taken to queue the points for write (actual writes are processed in the background and will take longer)

08-Feb-23 23:41
verified

mentioned in commit 72cf3456c187b6a5d8324d22c74552a0206ad00c

Commit: 72cf3456c187b6a5d8324d22c74552a0206ad00c 
Author: B Tasker                            
                            
Date: 2023-02-08T23:40:59.000+00:00

Message

docs: create document detailing statistics utilities/python_influxdb_downsample#16

+58 -0 (58 lines changed)

08-Feb-23 23:41

The example configuration is now as follows

stats:
    active: True
    influx: home1x
    bucket: telegraf/autogen
    measurement: downsample_stats
    # Key-value pairs of tags to add to any stats writes
    #
    # if used, $HOSTNAME will be replaced with the system's 
    # hostname at run time
    #
    tags:
        role: "hourlydownsamples"
        host: "$HOSTNAME"

08-Feb-23 23:53
verified

mentioned in commit eb9f826d89bf7e658c7e1b81f112476440ef4c10

Commit: eb9f826d89bf7e658c7e1b81f112476440ef4c10 
Author: B Tasker                            
                            
Date: 2023-02-08T23:52:20.000+00:00

Message

Add process level stats utilities/python_influxdb_downsample#16

This adds the following

  • config_load_time: the time taken to parse and normalise the YAML config
  • startup_time: the time taken by the script to startup and be ready to start jobs

It also adds a total_run_time entry for main-handler to reflect the total process run time

+26 -4 (30 lines changed)

09-Feb-23 09:10

mentioned in issue #17