utilities/file_location_listing#13: Build and Deploy into Kubernetes

There are a couple of components to this, then:

• We need a service for the search portal
• We need a cronjob for the crawler
• They both need to be able to access a shared storage area

Also need a sane way to switch the container between modes. At the moment, we'd need to override the command - I'd prefer to be able to switch with an environment variable though

Running the container on my desktop is pretty straightforward:

docker run --rm -it \
-v /home/ben/tmp/search_db/:/search_db \
-e DB_PATH=/search_db \
-p 5000:5000 \
test

With the new multimode support, I can trigger a crawl with

docker run --rm -it \
-v /home/ben/tmp/search_db/:/search_db \
-e DB_PATH=/search_db \
-e MODE=crawl \
-p 5000:5000 \
test

So, need to tag the container and push into a registry so that I can start thinking about what the kube deployment would look like.

Image has been tagged as registry.bentasker.co.uk/utilities/file_location_listing:alpha1.0

Creating a cronjob with the following has worked with a crawl running at the moment

apiVersion: batch/v1
kind: CronJob
metadata:
  name: filelisting-spider
spec:
  schedule: "0 4 * * *"
  failedJobsHistoryLimit: 5
  successfulJobsHistoryLimit: 5
  jobTemplate:
    spec:
      template:
        spec:
            restartPolicy: OnFailure
            volumes:
            - name: searchstore
              nfs:
                server: 192.168.3.233
                path: /volume1/searchfiles
                readOnly: false

            hostname: search-crawler
            containers:
            - name: filelisting-spider
              image: registry.bentasker.co.uk/utilities/file_location_listing:alpha1.1
              imagePullPolicy: IfNotPresent
              volumeMounts:
                    - mountPath: /search_db
                      name: searchstore
                      readOnly: false
              env:
                - name: MODE
                  value: "crawl"
                - name: DB_PATH
                  value: "/search_db"

Crawl was triggered with

kubectl create job spider-run-2 --from=cronjob.batch/filelisting-spider

Added a deployment with the following

---

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: searchportal
  name: searchportal
spec:
  selector:
    matchLabels:
      app: searchportal
  template:
    metadata:
      labels:
        app: searchportal
    spec:
      containers:
        - name: searchportal
          image: registry.bentasker.co.uk/utilities/file_location_listing:alpha1.1
          imagePullPolicy: IfNotPresent
          env:
          - name: DB_PATH
            value: "/search_db"
          ports:
            - containerPort: 5000
              name: http-searchport
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /
              port: 5000
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 30
            successThreshold: 1
            timeoutSeconds: 2
          livenessProbe:
            failureThreshold: 3
            initialDelaySeconds: 30
            periodSeconds: 10
            successThreshold: 1
            tcpSocket:
              port: 5000
            timeoutSeconds: 1
          resources:
            requests:
              cpu: 250m
              memory: 750Mi
          volumeMounts:
            - mountPath: /search_db
              name: searchstore
      volumes:
        - name: searchstore
          nfs:
            server: 192.168.3.233
            path: /volume1/searchfiles
            readOnly: true            

---

apiVersion: v1
kind: Service
metadata:
  name: searchportal
spec:
  ports:
    - port: 5000
      protocol: TCP
      targetPort: http-searchport
  selector:
    app: searchportal
  sessionAffinity: None
  type: LoadBalancer

The portal's up and running

The manifest will obviously need updating one we do an actual release, but they're working, so I think we're done here.