project Utilities / Python Influxdb Downsample avatar

utilities/python_influxdb_downsample#19: Testing notes

Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Milestone: vnext
Created: 11-Feb-23 00:18


Doing some stats comparison of the RC.

Toggle State Changes


assigned to @btasker

The stats on the left were written into 1 bucket via a Kapacitor Flux Task, the stats on the right by my cronjob


They weren't matched when I first checked them though, which looks to have been the result of a timing issue.

The cronjob is scheduled to run every 15 minutes.

The telegraf plugin which fetches the underlying stats runs once an hour (apparently around the 15 minute mark), but is sometimes a little delayed.

Because the downsample task was configured to query a range of 1 hour, we sometimes missed out on the stat.

Updating the job to use a range of 2 hours fixed it.

The requests graph highlights a curious difference


Both use

 |> fill(usePrevious: true)

But, the flux-task originated one has a point at 22:00 with a value of 0 - the cronjob task does not (so the earlier previous value continues).

This seems to be something that the Fluxtask has done, if I run the same query against the source

from(bucket: "websites/autogen")
    |> range(start: 2023-02-10T20:30:00Z, stop: 2023-02-10T23:30:00Z)
    |> filter(fn: (r) => r._measurement == "bunnycdn")
 |> filter(fn: (r) => r._field == "requests_served")   
 |> filter(fn: (r) => r.edge_zone == "btaskerwww")
    |> aggregateWindow(every: 15m, fn: max)

I don't get a window at 22:00.

It's a tricky one to address, but we probably do want to look at whether there's a way to detect missing windows and create them so that we can insert a 0. If we pass createEmpty: true into window it'll create the tables for us, but we'll have to discern column values from the group key.

mentioned in issue #20

The above has been fixed in #20

mentioned in issue jira-projects/LAN#88