Although not technically an aggregate, it'd be useful to add a copy mode - where points are queried from the input and queued for writing to the output.
This would allow the downsampler to be used for replicating from one source to the other.
It also means that more complex queries can be put into the query attribute, safe in the knowledge that the structure will simply be copied out to the output
| #22 | Add support for remote aggregation |
Activity
11-Feb-23 23:40
assigned to @btasker
11-Feb-23 23:42
It would also open the door to possibly adding support for injecting
aggregateWindow- that's not desirable when running against a local install (the whole point is to use resource usage off the InfluxDB node), but might be when querying against Cloud (where the concern about resource usage doesn't exist in the same way).11-Feb-23 23:46
mentioned in issue #22
11-Feb-23 23:46
marked this issue as related to #22
12-Feb-23 17:58
Support for this would need to go into the record loop (currently around line 457).
If the
copyaggregate was active, we'd need to push an entire copy of the record intonew_records.In theory it'd be possible to have this and other aggregates active.
13-Feb-23 09:09
mentioned in commit a8540b4ad62b004de5bc1130fbdf06f29eaf4720
Commit: a8540b4ad62b004de5bc1130fbdf06f29eaf4720 Author: B Tasker Date: 2023-02-13T09:07:22.000+00:00Message
fix: Convert row to a dict when copying (utilities/python_influxdb_downsample#21)
The influxdb-client doesn't support having data passed into it as a FluxRow (to be fair, it doesn't claim to) so we have to convert to a dict when queuing the point to a write.
This is more or less the same logic used later on, so to help keep things DRY, have moved out to a function
13-Feb-23 09:09
mentioned in commit 0a88f6ed234e2611bd4a9788e3f1a28db01fa688
Commit: 0a88f6ed234e2611bd4a9788e3f1a28db01fa688 Author: B Tasker Date: 2023-02-13T08:55:01.000+00:00Message
NotWorking: Start to implement copy aggregate utilities/python_influxdb_downsample#21
This adds code to the correct place, however, we're not currently seeing the new data at the database end - it may be that the client doesn't accept it's own format, or might be that a conditional is failing later in the process.
13-Feb-23 09:12
The first commit above introduces changes in the correct place, but unfortunately the type
FluxRowisn't one accepted by the write client, so the outputs weren't appearing in the database.The second commit above introduces conversion to a dict.
The drawback of this, though, is it means that the
copyaggregate cannot be used with pivoted data - it looks for_fieldand_valuein order to determine what fields should be. Definitely worth noting in documentation.13-Feb-23 09:18
mentioned in commit f17fe67e8486be8f6ccbcd4db4d2046e8ea80c82
Commit: f17fe67e8486be8f6ccbcd4db4d2046e8ea80c82 Author: B Tasker Date: 2023-02-13T09:18:22.000+00:00Message
docs: Add details on copy aggregate (utilities/python_influxdb_downsample#21)
13-Feb-23 09:34
mentioned in commit d239b46cbff4877266d808bae822f3f7f6b84e4b
Commit: d239b46cbff4877266d808bae822f3f7f6b84e4b Author: B Tasker Date: 2023-02-13T09:34:03.000+00:00Message
docs: add document on setting up replication using
copy(utilities/python_influxdb_downsample#21)