project Utilities / Python Influxdb Downsample avatar

utilities/python_influxdb_downsample#22: Add support for remote aggregation

Issue Information

Issue Type: issue
Status: opened
Reported By: btasker
Assigned To: btasker

Milestone: backlog
Created: 11-Feb-23 23:44


The primary aim of this script is to downsample in a way that moves resource usage (primarily CPU and RAM) off the node being queried.

However, there are use-cases where that isn't necessary: querying against Cloud, for example.

It'd be useful to be able to push that computation back out to the remote end in some cases (the default should still be local processing)

Issue Links

Toggle State Changes


assigned to @btasker

It'd need to only support a constrained set of aggregates, and would probably require a job per remote-aggregate (it is possible to run a Flux query which does multiple, but I don't think we really want to be adding the complexity of generating those).

Once the results are received, they need to be passed to the output unchanged, so implementation of this is blocked by the need for a copy mode (#21)

marked this issue as related to #21

mentioned in issue #24

This was blocked by needing #21 to be implemented. Now that that's in place we probably just need a job config flag to note that aggregation should be done remotely - when that's active we should push a mean()/count()/whatever() call into the query and then locally treat the aggregate as if it were copy