The primary aim of this script is to downsample in a way that moves resource usage (primarily CPU and RAM) off the node being queried.
However, there are use-cases where that isn't necessary: querying against Cloud, for example.
It'd be useful to be able to push that computation back out to the remote end in some cases (the default should still be local processing)
| #21 | Add new "copy" aggregate |
Activity
11-Feb-23 23:44
assigned to @btasker
11-Feb-23 23:46
It'd need to only support a constrained set of aggregates, and would probably require a job per remote-aggregate (it is possible to run a Flux query which does multiple, but I don't think we really want to be adding the complexity of generating those).
Once the results are received, they need to be passed to the output unchanged, so implementation of this is blocked by the need for a
copymode (#21)11-Feb-23 23:46
marked this issue as related to #21
16-Feb-23 13:33
mentioned in issue #24
28-Mar-23 15:28
This was blocked by needing #21 to be implemented. Now that that's in place we probably just need a job config flag to note that aggregation should be done remotely - when that's active we should push a
mean()/count()/whatever()call into the query and then locally treat the aggregate as if it werecopy