utilities/gadgetbridge_to_influxdb#11: Possible Syncing Issues

Can see in Nextcloud that the database has been uploading/changing overnight

So the auto-export stuff is working.

The logs imply that data's being retrieved and submitted from the DB to InfluxDB without issue

ben@bumblebee:~$ kubectl logs job/gadgetbridge-to-influxdb-28218720
ERROR:influxdb_client.client.write_api:The batch item wasn't processed successfully because: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Request-Id': 'bcf76fad-44af-11ee-9039-0242ac130002', 'X-Influxdb-Build': 'OSS', 'X-Influxdb-Error': 'partial write: points beyond retention policy dropped=682', 'X-Influxdb-Version': '1.8.10', 'X-Request-Id': 'bcf76fad-44af-11ee-9039-0242ac130002', 'Date': 'Sun, 27 Aug 2023 08:00:07 GMT', 'Content-Length': '70'})
HTTP response body: {"error":"partial write: points beyond retention policy dropped=682"}


ERROR:influxdb_client.client.write_api:The batch item wasn't processed successfully because: (400)
Reason: Bad Request
HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json', 'Request-Id': 'bcfd146b-44af-11ee-903c-0242ac130002', 'X-Influxdb-Build': 'OSS', 'X-Influxdb-Error': 'partial write: points beyond retention policy dropped=19', 'X-Influxdb-Version': '1.8.10', 'X-Request-Id': 'bcfd146b-44af-11ee-903c-0242ac130002', 'Date': 'Sun, 27 Aug 2023 08:00:07 GMT', 'Content-Length': '69'})
HTTP response body: {"error":"partial write: points beyond retention policy dropped=19"}

(Those partial writes are because of #9 and can be ignored).

As soon as I unlocked my phone to check that, it's synced the overnight data to the watch (because I have Auto fetch activity data set).

So, looks like there isn't an issue, it's just that nothing has prompted GB to sync overnight.

That's not the end of the world - don't actually need the data until we're awake - but might cause some headaches once #9 is fixed: the script will only try and extract the last 2 hours of data from the database.

I have been thinking that that's a bit short anyway.

I'll raise a ticket later for changing the default of QUERY_DURATION.

I'll hold this ticket open as a reminder to do that.

OK, we now have readings - as we should - but there's a gap in the data for the queries that QUERY_DURATION is effective for (i.e. those that have a 1 second precision on their timestamps)

I'm going to bump the QUERY_DURATION up via the chart just to prove the gap does get filled

Perfect:

The query duration has been adjusted to 1 day in #12

That should be more than enough to prevent gaps in data from this.

As this isn't actually a bug, I'm going to re-label the issue to informational. Before I close it though, I want to go through the process of creating a Tasker task/profile to trigger a sync in the early hours.

The bit of the Intents API that we want is nodomain.freeyourgadget.gadgetbridge.command.ACTIVITY_SYNC

It first needs to be enabled in Gadgetbridge:

• Settings
• Intent API
• Allow activity sync trigger

Then, in Tasker

• Create new task
• Set the name to Gadgetbridge Sync
• Add action
• System
• Send Intent
• Set Action to nodomain.freeyourgadget.gadgetbridge.command.ACTIVITY_SYNC
• Hit the back arrow in the app twice

Switch to Profiles

• Add, Create
• Set the Trigger type to Time
• Set to start at 11pm
• Run every 2 hours
• Stop at 8am
• Hit the Back arrow
• Choose the Gadgetbridge Sync Task
• Name it Gadgetbridge Sync
• Hit the tick to save

Technically, we could leave off the start/end times and have it trigger a sync every couple of hours 24/7, but I'm OK with it syncing less often in the day (if nothing else, the phone's less likely to be plugged into a charger during the day)