project Utilities / Python Influxdb Downsample avatar

utilities/python_influxdb_downsample#4: Add support for an "as" to aggregates



Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Milestone: vnext
Created: 04-Feb-23 16:41



Description

Currently, we auto-calculate the output field name, taking field_suffix into account.

It'd probably be good to support an as so that fields can be renamed:

    interface_stats:
        # Name for the task
        name: "downsample_interface_stats"
        influx: home1x
        period: 15
        bucket: telegraf/autogen
        measurement: interface
        fields: 
            - ifHCInOctets

        group_by: []

        # Any additional filters to inject into the query
        filters: []
        # Which aggregates to generate
        # field_suffix is optional, if not included
        # the source fields name will be used
        aggregates: 
            mean:
              as: bytes_in

In the above example, the output field would be called bytes_in



Toggle State Changes

Activity


assigned to @btasker

verified

mentioned in commit c0b12e87bc30d3e697ac8f7310c598336809d862

Commit: c0b12e87bc30d3e697ac8f7310c598336809d862 
Author: B Tasker                            
                            
Date: 2023-02-05T10:09:34.000+00:00 

Message

chore: max field name calculations DRY in preparation for utilities/python_influxdb_downsample#4

+20 -10 (30 lines changed)
verified

mentioned in commit 38cfe2cd0c16a8487c4e93e70e98d248d567d0ad

Commit: 38cfe2cd0c16a8487c4e93e70e98d248d567d0ad 
Author: B Tasker                            
                            
Date: 2023-02-05T10:20:15.000+00:00 

Message

Implement support for "as" for utilities/python_influxdb_downsample#4

+50 -3 (53 lines changed)

Although these commits implement support for as, I'm not sure it's currently as efficient as it could be.

Currently, you can specify one as per aggregate. That's fine if the job's only processing one field, but not so fine if it's doing multiple.

That could be worked around by creating a job per field, but that's not an efficient use of resources.

I think instead, as should (also) accept a map of fieldname -> new name, so that you could do something like

aggregates:
   mean:
     as:
      usage_user: user_cpu
      usage_system: system

Ideally that'd be whilst continuing to support the simplified version we've just built

verified

mentioned in commit 0a283554c9b1deb39b68579b7cf8404c3df07e41

Commit: 0a283554c9b1deb39b68579b7cf8404c3df07e41 
Author: B Tasker                            
                            
Date: 2023-02-05T11:35:51.000+00:00 

Message

Adjust "as" so that it can also be a dict providing a mapping when multiple fields are being processed utilities/python_influxdb_downsample#4

+25 -5 (30 lines changed)
verified

mentioned in commit 1eddedd3cefc35cc9a948dacd6ac6f95c4e6fc8e

Commit: 1eddedd3cefc35cc9a948dacd6ac6f95c4e6fc8e 
Author: B Tasker                            
                            
Date: 2023-02-05T11:39:06.000+00:00 

Message

docs: update README for utilities/python_influxdb_downsample#4

+12 -3 (15 lines changed)

OK, so we now support both string and dict in as.

So, for a single field job a string can be used to achieve a sort of short-hand markup. For tasks with multiple fields a dict can be used to map over.

This probably has some overlap with #1 as a few of the custom call examples given there are concerned with renaming fields - this functionality can also be used for those (obviously it's less use where arithmetic is required).