misc/pod-as-a-service#1: InfluxDB 1.x Pod definition

Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Milestone: PoC
Created: 09-Nov-24 12:01



Description

We've now got some of the basics in place - the system can create a customer namespace (and list customers).

The next thing to worry about is having it create the user's pod/volume etc in the customer namespace.

This needs a bit of thought to make sure we don't hem ourselves in, but

  • We should create a single replica statefulset (to ensure that storage/names persist between rolls)
  • We need to define some kind of storage
  • We'll need to generate a domain name for the service and expose an ingress to handle that

The catch with StatefulSet though is that we won't easily be able to resize the storage

I also want it to be relatively easy to add new application types - although my first "app" will likely be InfluxDB OSS 2.x, I'd like it to be relatively easy to add support for something else too.

So, each "app" should probably come in the form of a class so that calls are normalised.



Toggle State Changes

Activity


09-Nov-24 12:01

assigned to @btasker


09-Nov-24 12:07

changed the description


09-Nov-24 12:35
verified

mentioned in commit 6b3c7e2c61b3809d87794de7ca0291e10c419efb

Commit: 6b3c7e2c61b3809d87794de7ca0291e10c419efb 
Author: B Tasker                            
                            
Date: 2024-11-09T12:34:50.000+00:00

Message

feat: create an InfluxDB app (#1)

Note: this doesn't currently do anything for persistent storage

ben@bumblebee:~/charts/pod-as-a-service$ kubectl -n paas-abcdef get pods
NAME                                   READY   STATUS    RESTARTS   AGE
statefulset-influxdb1x-hello-world-0   1/1     Running   0          52s
+83 -2 (85 lines changed)

09-Nov-24 12:42

The system can now successfully define an InfluxDB 1.x install (I started with 1.x because it's much easier to get up and running).

    # Create an app
    influxdbv1.create(apps_api, name, c["ns_name"], c['id'])

There are some essentials that it doesn't currently do

  • There's no persistent storage
  • There's currently no ingress
  • There isn't currently any preconfiguration - it'll come up without auth enabled, which is far from ideal

The persistent storage presents a little bit of a challenge, because, at home, I tend to back pods with NFS. So, I need to choose from a couple of options

  • Back with NFS for the time being (we can still provide a PV/PVC template for that)
  • Move to using DOKS/EKS/AKS/GKE/Whatever so that we can launch straight into defining PVs and PVCs

The second, obviously comes with a tangible cost - I'd have to pay for a cluster. I think the best bet is probably to try and make the storage definition pluggable - that way I can have a single file which makes it use NFS but can later be changed to use EBS/whatever.

The ingress definition shouldn't pose too many issues.

Preconfig shouldn't be an issue either, but need to decide which of the following to do (I'd kinda like to support both):

  • Have the module support a post-up call, essentially placing HTTP requests against the pod to preconfigure it
  • Define a configmap to set everything up

09-Nov-24 15:04
verified

mentioned in commit 34b4aac871a30b2f715fbe2a9e185b00ce4570e3

Commit: 34b4aac871a30b2f715fbe2a9e185b00ce4570e3 
Author: B Tasker                            
                            
Date: 2024-11-09T15:04:35.000+00:00

Message

feat: add ability to list customers apps (#1)

+37 -4 (41 lines changed)

09-Nov-24 15:25
verified

mentioned in commit 19a629c1b74e064443ae4ff5f8b2edb798f71641

Commit: 19a629c1b74e064443ae4ff5f8b2edb798f71641 
Author: B Tasker                            
                            
Date: 2024-11-09T15:24:55.000+00:00

Message

feat: implement support for deleting customer apps (#1)

+44 -3 (47 lines changed)

09-Nov-24 17:40
verified

mentioned in commit 874918b634a3f50077f188054a8fb04b07297d9c

Commit: 874918b634a3f50077f188054a8fb04b07297d9c 
Author: B Tasker                            
                            
Date: 2024-11-09T17:40:32.000+00:00

Message

feat: configure service and ingress when standing up an app (#1)

+162 -8 (170 lines changed)

10-Nov-24 11:02
verified

mentioned in commit a55dbf2bc41d1c060b122e2e0e0ad525339a41a8

Commit: a55dbf2bc41d1c060b122e2e0e0ad525339a41a8 
Author: B Tasker                            
                            
Date: 2024-11-10T11:01:40.000+00:00

Message

feat: add persistent storage (#1)

Note: this is currently very much tailored to my home-lab, using NFS for the mounts. I decided to just chuck it in for now so that it stopped being a blocker, but we should absolutely revisit later so that it can create PVs in cloud providers etc.

+33 -1 (34 lines changed)

10-Nov-24 11:10

We now have most of the recipe:

  • StatefulSet gets created
  • Storage is persistent
  • Service gets created
  • Ingress gets updated to include a rule to match the new app domain

There are a couple of outstanding bits though:

  • There isn't currently any preconfiguration of the app (which, amongst other things, means that we'd be putting an Influx instance on the net without authentication enabled...)
  • After creating the app, we return some details to the caller, but they're not very useful

I did, however, chuck together a video of going through the process

Video of creating an app


10-Nov-24 11:18
verified

mentioned in commit 56a44c5563202e45cf376fa1b9a8a7e9de103a5e

Commit: 56a44c5563202e45cf376fa1b9a8a7e9de103a5e 
Author: B Tasker                            
                            
Date: 2024-11-10T11:18:19.000+00:00

Message

feat: return details of the new app after creation (#1)

+8 -6 (14 lines changed)

10-Nov-24 11:31

Pre-config of 1.x is a bit more of a challenge, because there isn't a way to automatically populate an admin user.

If we were doing it manually, the workflow would be

  • Spin up pod
  • Run CREATE USER to create an admin user
  • Enable auth (in config file or via env var)
  • Restart

To be able to do that automatically, we're going to need to

  • Mount a script into the container
  • have it check if auth's enabled (or if it's otherwise run)
  • consume creds from somewhere
  • update config
  • restart the pod

I think that's a complex enough task that we should spin it out to its own issue (#2)


10-Nov-24 12:39

changed title from Pod definition to {+InfluxDB 1.x +}Pod definition


10-Nov-24 13:31

Just need to go through and make sure that deletion is also deleting all the relevant bits.

I also think the statefulset name needs to change - everything else uses app_name, but the SFS gets prefixed with "sfs"


10-Nov-24 13:32
verified

mentioned in commit b853a90612d67b2cab127f916812a921651d5cbc

Commit: b853a90612d67b2cab127f916812a921651d5cbc 
Author: B Tasker                            
                            
Date: 2024-11-10T13:32:15.000+00:00

Message

fix: make StateFulSet name consistent with other names (#1)

+2 -4 (6 lines changed)

10-Nov-24 13:45
verified

mentioned in commit 862c9964914a2e17d50f2435436b64bc8b1fade8

Commit: 862c9964914a2e17d50f2435436b64bc8b1fade8 
Author: B Tasker                            
                            
Date: 2024-11-10T13:44:16.000+00:00

Message

fix: tidy up all resources when deleting an app (#1)

This updates the influxdb1x module to also remove service etc

The main framework also removes associated ingress rules.

+17 -6 (23 lines changed)