Kubernetes storage location information

Contents

• Overview
• 1. Storage Classes
• 2. Persistent Volume Claims (PVCs)
• 3. StatefulSet Volume Claim Templates
• 4. Per-Container Storage Details
• 5. ConfigMap-Backed File Mounts
• 6. Secret Mounts
• 7. Ephemeral (In-Memory) Volumes
• 8. Deployment Options Affecting Storage
• 9. Downloading Logs
• 10. Checking disk usage of volumes
• 11. Storage Summary Table

Overview

This document describes all storage locations used across the containers in a CAST Imaging Kubernetes deployment: persistent volumes, ConfigMap-backed configuration files, Secrets, and ephemeral (in-memory) volumes.

1. Storage Classes

Two storage class types are used, depending on the feature set enabled.

1.1 Block Storage Class (castimaging-ds)

Used for all standard persistent volumes (databases, logs, archives, etc.).

CSI provisioner and parameters vary by cloud provider:

Storage class creation is controlled by the CreateStorageClass value (default: true). Set to false if the storage classes already exist in the cluster.

1.2 File Storage Class (castimaging-fs)

Only created and used when shared file storage for the Analysis Node is enabled (AnalysisNodeFS.enable: true). This allows the Analysis Node’s shared data directory to use ReadWriteMany (RWX) access mode, enabling horizontal scaling of analysis nodes.

2. Persistent Volume Claims (PVCs)

All PVCs are created in the deployment namespace with the annotation helm.sh/resource-policy: keep, ensuring they are not deleted on helm uninstall.

All PVC sizes are configurable through values.yaml (e.g. size_db_data, size_shared_datadir, etc.).

3. StatefulSet Volume Claim Templates

These volumes are provisioned automatically per pod replica via volumeClaimTemplates in StatefulSets. They use the castimaging-ds block storage class.

console-analysis-node (StatefulSet)

viewer-neo4j (StatefulSet)

A single neo4jdata volume (100Gi) is provisioned per pod. It is sub-divided using subPathExpr into three logical directories:

4. Per-Container Storage Details

4.1 console-postgres

The PostgreSQL database server used by all Console services and Keycloak.

The db-data PVC is only created when CastStorageService.enable: true. When using an external PostgreSQL instance (CustomPostgres.enable: true), no PVC is created and a separate init script (init-db-custom-pg.sh) is executed as an init container to set up the required users and databases on the external server.

4.2 console-analysis-node

The CAST analysis engine. Runs as a StatefulSet.

The pvc-shared-datadir sub-directories used at runtime:

When AnalysisNodeFS.enable: true, pvc-shared-datadir switches to ReadWriteMany (file storage class), allowing multiple Analysis Node replicas to share the same workspace. When disabled (default), it uses ReadWriteOnce block storage and only a single Analysis Node replica is supported.

4.3 viewer-neo4j

The graph database for the Imaging Viewer. Runs as a StatefulSet.

All three paths share a single underlying 100Gi PVC via sub-paths.

4.4 viewer-server

The main Imaging Viewer frontend and backend service, fronted by Nginx.

4.5 viewer-etl

The ETL (Extract, Transform, Load) service responsible for importing analysis data into Neo4j.

4.6 viewer-aimanager

The AI/enrichment manager service.

4.7 viewer-api

The public REST API service for querying the Imaging graph.

4.8 mcp-server

The Model Context Protocol server. Only deployed when McpServer.enable: true.

4.9 extendproxy

The CAST Extend proxy service. Only deployed when ExtendProxy.enable: true.

4.10 console-control-panel

No persistent volumes are mounted in the main container. Init containers use ConfigMap-backed SQL scripts to update context URLs in the PostgreSQL database on startup (see Section 5).

4.11 console-authentication-service

No persistent volumes are mounted in the main container. When UseCustomTrustStore: true, a ConfigMap (authcacrt) holding the CA certificate is projected into the container (see Section 5).

4.12 console-service, console-gateway-service, console-sso-service, console-dashboards

None of these services mount persistent volumes. They rely entirely on database-backed state and environment variable configuration.

5. ConfigMap-Backed File Mounts

The following ConfigMaps are mounted as files into containers or init containers.

6. Secret Mounts

The Secret imaging-pwd-sec is used both as environment variable source and as a file-system mount.

The file-system mount (/opt/secrets) in console-postgres is read-only and used by the init script to read passwords at startup.

7. Ephemeral (In-Memory) Volumes

8. Deployment Options Affecting Storage

8.1 Cloud Provider (K8SProvider)

Controls which CSI drivers are used for the castimaging-ds (and optionally castimaging-fs) storage classes. Accepted values: EKS, AKS, GKE.

8.2 Built-in vs. External PostgreSQL

8.3 Analysis Node Shared File Storage (AnalysisNodeFS.enable)

When enabling file storage on EKS, the EFSsystemID value is required. Optionally, providing EFSaccessPointID causes an explicit PersistentVolume (pv-shared-datadir) to be created.

On AKS, if auto-provisioning is not available, set AnalysisNodeFS.AKSsharedDatadirPV.create: true and provide secretName and shareName to create an explicit PV backed by an Azure File Share.

8.4 Optional Components

8.5 Storage Class Creation

Set CreateStorageClass: false to skip storage class creation (e.g., if the cluster admin has pre-provisioned them). The DiskClassName and FileClassName values must still match the names of the existing storage classes.

9. Downloading Logs

There are 2 types of logs:

• Standard output logs: correspond to the log avialable from kubernetes and containing anything services are writing to the standard output.
• Specific log files: some services have addtional logs available, stored in persistent files: console-postgres, console-analysis-node, viewer-neo4j, viewer-server, viewer-etl

The following commands can be used to download those logs. Run them from the machine where kubectl is configured and authenticated against your cluster.

Namespace: All commands assume the namespace castimaging-v3. Adjust if your deployment uses a different namespace.

Standard output logs (from all pods)

kubectl logs -l app=castimaging --all-containers=true --prefix=true --tail=-1 -n castimaging-v3 > all-pods-std-output.log

Specific log files stored inside pods

Analysis Node — CAST engine logs stored in the castdir volume:

Linux:

kubectl cp -n castimaging-v3 \
  console-analysis-node-core-0:/usr/share/CAST/CAST/Logs \
  ./Logs/console-analysis-node

Windows (PowerShell):

kubectl cp -n castimaging-v3 `
  console-analysis-node-core-0:/usr/share/CAST/CAST/Logs `
  ./Logs/console-analysis-node

Neo4j — graph database logs stored in the neo4jdata volume:

Linux:

kubectl cp -n castimaging-v3 \
  viewer-neo4j-core-0:/var/lib/neo4j/logs \
  ./Logs/viewer-neo4j

Windows (PowerShell):

kubectl cp -n castimaging-v3 `
  viewer-neo4j-core-0:/var/lib/neo4j/logs `
  ./Logs/viewer-neo4j

Postgres — postgres database logs stored in the db-data volume:
Adjust the console-postgres pod name if needed.
Linux:

kubectl cp -n castimaging-v3 \
  console-postgres-0:/var/lib/postgresql/data/log \
  ./Logs/postgres

Windows (PowerShell):

kubectl cp -n castimaging-v3 `
  console-postgres-0:/var/lib/postgresql/data/log `
  ./Logs/postgres

Viewer Server — Nginx access/error logs and application logs:

Linux:

POD=$(kubectl get pod -n castimaging-v3 -l imaging.service=viewer-server -o jsonpath='{.items[0].metadata.name}')
kubectl cp -n castimaging-v3 $POD:/opt/imaging/imaging-service/logs ./Logs/viewer-server

Windows (PowerShell):

$POD = kubectl get pod -n castimaging-v3 -l imaging.service=viewer-server -o jsonpath='{.items[0].metadata.name}'
kubectl cp -n castimaging-v3 "${POD}:/opt/imaging/imaging-service/logs" ./Logs/viewer-server

Viewer ETL — ETL service operational logs:

Linux:

POD=$(kubectl get pod -n castimaging-v3 -l imaging.service=viewer-etl -o jsonpath='{.items[0].metadata.name}')
kubectl cp -n castimaging-v3 $POD:/opt/imaging/imaging-etl/logs ./Logs/viewer-etl

Windows (PowerShell):

$POD = kubectl get pod -n castimaging-v3 -l imaging.service=viewer-etl -o jsonpath='{.items[0].metadata.name}'
kubectl cp -n castimaging-v3 "${POD}:/opt/imaging/imaging-etl/logs" ./Logs/viewer-etl

Collect all logs in one shot

Linux — save the following as collect-logs.sh and run with bash collect-logs.sh:

#!/bin/bash
NAMESPACE="castimaging-v3"
DEST="./Logs/$(date +%Y%m%d_%H%M%S)"
mkdir -p "$DEST"

# neo4j
kubectl exec viewer-neo4j-core-0 -n $NAMESPACE -- tar cf - /var/lib/neo4j/logs > $DEST/viewer-neo4j-logs.tar

# Analysis nodes
for pod in $(kubectl get pods -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}' | tr ' ' '\n' | grep '^console-analysis-node-core-'); do
  echo "=== Pod: $pod ==="
  kubectl exec $pod -n $NAMESPACE -- tar cf - /usr/share/CAST/CAST/Logs > $DEST/$pod-Logs.tar
done

# Other pods
declare -A LOG_PATHS=(
  [viewer-server]="/opt/imaging/imaging-service/logs"
  [viewer-etl]="/opt/imaging/imaging-etl/logs"
  [console-postgres]="/var/lib/postgresql/data/log"

)
for SVC in "${!LOG_PATHS[@]}"; do
  POD=$(kubectl get pod -n $NAMESPACE -l imaging.service=$SVC -o jsonpath='{.items[0].metadata.name}' 2>/dev/null)
  if [ -z "$POD" ]; then
    echo "No pod found for $SVC, skipping."
    continue
  fi
  echo "Copying logs from $POD ($SVC)..."
  kubectl cp -n $NAMESPACE $POD:${LOG_PATHS[$SVC]} $DEST/$SVC
done

# Retrieving standard output log from all pods
for pod in $(kubectl get pods -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}'); do
  echo "=== Pod: $pod ===" >> $DEST/all-pods.log
  kubectl logs $pod -n $NAMESPACE --all-containers=true --prefix=false >> $DEST/all-pods.log
done

echo "All logs collected in $DEST"

Windows — save the following as collect-logs.ps1 and run with powershell -File collect-logs.ps1:

$NAMESPACE = "castimaging-v3"
$DEST = "./Logs/$(Get-Date -Format 'yyyyMMdd_HHmmss')"
New-Item -ItemType Directory -Force -Path $DEST | Out-Null

# neo4j
kubectl exec viewer-neo4j-core-0 -n $NAMESPACE -- tar cf - /var/lib/neo4j/logs > $DEST\viewer-neo4j-logs.tar

# Analysis nodes
$pods = kubectl get pods -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}' | 
    ForEach-Object { $_ -split ' ' } | 
    Where-Object { $_ -match '^console-analysis-node' }

foreach ($pod in $pods) {
    Write-Host "=== Pod: $pod ==="
    kubectl exec $pod -n $NAMESPACE -- tar cf - /usr/share/CAST/CAST/Logs > $DEST\$pod-Logs.tar
}

# Other pods
$SERVICES = @{
  "viewer-server"    = "/opt/imaging/imaging-service/logs"
  "viewer-etl"       = "/opt/imaging/imaging-etl/logs"
  "console-postgres" = "/var/lib/postgresql/data/log"

}
foreach ($SVC in $SERVICES.Keys) {
  $POD = kubectl get pod -n $NAMESPACE -l "imaging.service=$SVC" -o jsonpath='{.items[0].metadata.name}' 2>$null
  if (-not $POD) {
    Write-Host "No pod found for $SVC, skipping."
    continue
  }
  Write-Host "Copying logs from $POD ($SVC)..."
  kubectl cp -n $NAMESPACE "${POD}:$($SERVICES[$SVC])" "$DEST/$SVC"
}

# Retrieving standard output log from all pods
foreach ($pod in $(kubectl get pods -n $NAMESPACE -o jsonpath='{.items[*].metadata.name}').Split(' ')) {
  Add-Content $DEST/all-pods.log "=== Pod: $pod ==="
  kubectl logs $pod -n $NAMESPACE --all-containers=true --prefix=false | Add-Content $DEST/all-pods.log
}

Write-Host "All logs collected in $DEST"

10. Checking disk usage of volumes

The following commands check the disk usage of main volumes. Run them from the machine where kubectl is configured and authenticated against your cluster.

kubectl exec -it console-analysis-node-core-0 -n castimaging-v3 -- df -h /opt/cast/shared
kubectl exec -it console-analysis-node-core-0 -n castimaging-v3 -- df -h /usr/share/CAST
kubectl exec -it console-postgres-0           -n castimaging-v3 -- df -h /var/lib/postgresql/data
kubectl exec -it viewer-neo4j-core-0          -n castimaging-v3 -- df -h /var/lib/neo4j/config/neo4j5_data
kubectl exec -it viewer-etl-xxxxx             -n castimaging-v3 -- df -h /opt/imaging/imaging-etl/logs
kubectl exec -it viewer-etl-xxxxx             -n castimaging-v3 -- df -h /opt/imaging/imaging-etl/upload/archive
kubectl exec -it extendproxy-xxxxx            -n castimaging-v3 -- df -h /opt/cast_extend_proxy/data

11. Storage Summary Table

Notes:

1. db-data PVC only exists when CastStorageService.enable: true.
2. authcacrt ConfigMap only mounted when UseCustomTrustStore: true.
3. mcp-server and its PVC only deployed when McpServer.enable: true.
4. extendproxy and its PVC only deployed when ExtendProxy.enable: true.