utilities/gadgetbridge_to_influxdb#19: Recent Data Not Being Written

Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Created: 20-Aug-25 10:03



Description

Data stopped showing up in my dashboards a week ago

image

I thought it was the Nextcloud sync breaking because I was away, but that seems to be fine. The data's not caught up since I returned from the US, nor has more recent data started appearing.

This morning I pointed the exporter at my InfluxDB v2 instance and recent data doesn't appear there either.

Need to figure out what's going on



Toggle State Changes

Activity


20-Aug-25 10:03

assigned to @btasker


20-Aug-25 10:06

changed the description


20-Aug-25 10:08

OK, have run a packet capture whilst triggering a job.

There are some 400s in there:

HTTP/1.1 400 Bad Request
Content-Type: application/json
Request-Id: a2655ffc-7dac-11f0-ae1e-0242ac130002
X-Influxdb-Build: OSS
X-Influxdb-Error: partial write: points beyond retention policy dropped=1000
X-Influxdb-Version: v1.11.8
X-Request-Id: a2655ffc-7dac-11f0-ae1e-0242ac130002
Date: Wed, 20 Aug 2025 10:01:26 GMT
Content-Length: 71

{"error":"partial write: points beyond retention policy dropped=1000"}

But, that's expected and there are also successful writes.

Which sort of suggests that maybe the issue is with the dashboard's queries rather than the writes?


20-Aug-25 10:12

Or not:

SELECT sum("steps") AS "total_steps" FROM "telegraf"."autogen"."gadgetbridge" WHERE  $timeFilter group by time($interval)

20-Aug-25 10:14

Looking at the last write in the payload, it does seem like they're older points

gadgetbridge,device=Amazfit\ Bip\ 3\ Pro,sample_type=periodic_samples intensity=0i,raw_intensity=0i,raw_kind=243i,steps=0i 1754780280000000000
gadgetbridge,device=Amazfit\ Bip\ 3\ Pro,sample_type=periodic_samples intensity=0i,raw_intensity=0i,raw_kind=243i,steps=0i 1754780340000000000
gadgetbridge,device=Amazfit\ Bip\ 3\ Pro,sample_type=sync_check last_seen=1754805668000000000i,last_seen_age=878418002315310i 1755684086002315310HTTP/1.1 204 No Content

The last seen age is today, and the graphs reflect that:

Last seen age graph

So, I guess we need to go and have a look in Gadgetbridge's sqlite database


20-Aug-25 11:43

Worth noting that there are currently 2 DBs which should try and write in - my main watch and my travel one.

Looking in Nextcloud, last upload dates are

  • Main: 10 Aug
  • Travel: 13 Aug

So, the data is probably right for the Main (and we need to look at why it's not uploaded).

I'd expect there to be some more recent data for the travel watch, though it's probably worth making it upload and then going from there.


20-Aug-25 11:52

OK, manually triggered an export in Gadgetbridge on my travel phone.

It wrote the DB out, which then auto-uploaded into Nextcloud. I'll kick off a job and run a packet capture at the same time.

That ran, but doesn't seem to have resulted in any newer data appearing. Gadgetbridge itself shows activity on the 16th.

Ahh, I appear to have fucked something up:

image

The Bip 3 pro is my main watch rather than the travel one - I think the job for my travel watch is actually fetching the DB for my main one.


20-Aug-25 12:02

The config definitely seems to be right though:

            - name: WEBDAV_PATH
              value: "files/service_user/GadgetBridgebkup"
            - name: EXPORT_FILENAME
              value: "Gadgetbridge.db"

Code does this

# Path to the export file
WEBDAV_PATH = os.getenv("WEBDAV_PATH", "files/service_user/GadgetBridge/")

# Creds
WEBDAV_USER =  os.getenv("WEBDAV_USER", False)
WEBDAV_PASS =  os.getenv("WEBDAV_PASS", False)

# What's the filename of the file on the webdav server?
EXPORT_FILE = os.getenv("EXPORT_FILENAME", "gadgetbridge")

Checking the access logs, it does seem to have fetched the wrong file:

192.168.3.23 - service_user [20/Aug/2025:11:47:42 +0000] "PROPFIND /remote.php/dav/files/service_user/GadgetBridge/ HTTP/1.1" 207 445 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:42 +0000] "HEAD /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 200 0 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:43 +0000] "PROPFIND /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 207 363 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:43 +0000] "HEAD /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 200 0 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:43 +0000] "PROPFIND /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 207 363 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:43 +0000] "HEAD /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 200 0 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:43 +0000] "PROPFIND /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 207 363 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:43 +0000] "HEAD /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 200 0 "-" "python-requests/2.32.4"
192.168.3.23 - service_user [20/Aug/2025:11:47:45 +0000] "GET /remote.php/dav/files/service_user/GadgetBridge/gadgetbridge HTTP/1.1" 200 24875008 "-" "python-requests/2.32.4"

The variables do get used within the code, so it's not like there's a path been left hardcoded.

Oh, no, it's me being a muppet:

kubectl create job gbridge2-manual --from=cronjob/gadgetbridge-to-influxdb

Created it from the wrong cron :face_palm:

With that corrected, the data gap is filled in:

image

So, outstanding

  • [x] Fix syncing of main DB (presumably, Gadgetbridge just needs a kick)
  • [x] Copy data to long term storage

On the second one, we should probably look at whether to just point the syncer directly at long term storage from now on - there's no downsampling going on, we literally just copy the raw data over.


20-Aug-25 12:44

OK, so yeah, on my main phone the mtime of the gadgetbridge export is 10 Aug.

Nextcloud's autoupload didn't seem to pick it up, so I pushed it up manually - will have to keep an eye on that.

I've switched the job to write into long-term storage so that we can check that it all works as expected.


20-Aug-25 12:51

mentioned in commit sysconfigs/bumblebee-kubernetes-charts@5d99e1175687af79d6775f9f4ed00fb96f0382c7

Commit: sysconfigs/bumblebee-kubernetes-charts@5d99e1175687af79d6775f9f4ed00fb96f0382c7 
Author: ben                            
                            
Date: 2025-08-20T13:50:52.000+01:00

Message

chore: switch to writing directly to long term storage (utilities/gadgetbridge_to_influxdb#19)

+9 -9 (18 lines changed)

20-Aug-25 12:51

That run seems to have worked OK, so I've switched both jobs to writing into long term storage


20-Aug-25 12:54

Closing this as Fixed/Done

Technically, it wasn't actually a bug, but it was the result of infra issues.