utilities/telegraf-plugins#16: Design Downsampling



Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Created: 06-Jul-23 19:28



Description

Probably worth thinking about what downsampling jobs might look like



Toggle State Changes

Activity


assigned to @btasker

The data's already at a 30 minute granularity, so in some ways there isn't much downsampling to be done.

I do, however, want to have a downsampling job which calculates cost per half-hour (saving me having to write queries to do it) based on consumption and the unit price at that time.

downsample_octopus_calculate_cost:
    name: "Downsample Octopus Costings"
    influx: home1x
    period: 10
    window: 10
    query: >
        from(bucket: "Systemstats")
        |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
        |> filter(fn: (r) => r._measurement == "octopus_pricing" or r._measurement == "octopus_consumption")
        |> filter(fn: (r) => r._field == "consumption" or (
        r._field == "cost_inc_vat" and r.charge_type == "usage-charge"
        and r.payment_method == "DIRECT_DEBIT"
        )
        )
        |> keep(columns: ["_time",  "_field", "_value", "tariff_code"])
        |> pivot(rowKey: ["_time"], columnKey: ["_field"], valueColumn: "_value")
        |> map(fn: (r) => ({
        _time: r._time,
        _measurement: "octopus_cost",
        _field: "usage_cost_pence",
        _value: r.consumption * r.cost_inc_vat
        }))

    aggregates:
        copy:

    output_influx: 
        - influx: home2xreal
    output_bucket: Systemstats/rp_720d       

I do, however, want to copy the data into my longer-lived RP, so have added a max job

downsample_octopus_stats_max:
    # Name for the task
    name: "Downsample Octopus Stats (Max)"
    influx: home1x

    # Query the last n mins
    period: 120

    # Window into n minute blocks
    window: 30

    # taken from in_bucket
    bucket: Systemstats
    measurement:
        - octopus_pricing
        - octopus_consumption        

    fields:
        - cost_exc_vat
        - cost_inc_vat
        - consumption

    aggregates: 
        max:

    output_influx: 
        - influx: home2xreal
    output_bucket: Systemstats/rp_720d

mentioned in commit sysconfigs/downsample_configs@f745c80c85ea702f168e02c1ffc3b0bee348b70c

Commit: sysconfigs/downsample_configs@f745c80c85ea702f168e02c1ffc3b0bee348b70c 
Author: ben                            
                            
Date: 2023-07-06T20:33:39.000+01:00 

Message

Add downsampling jobs for utilities/telegraf-plugins#16

+59 -0 (59 lines changed)