Enabling PhariaOS dynamic model management

Dynamic model management enables you to deploy a variety of models upon request using PhariaOS.


Activating dynamic model management

To use dynamic model management, activate the following flag:

phariaos-manager:
  kserve:
    enabled: true

When this chart is installed, custom resource definitions are installed to your cluster.

Enabling model deployment from the Aleph Alpha registry

To enable deployment of models from the Aleph Alpha model registry, you must provide your registry token. The token can be provided as a Kubernetes secret or directly in the Helm values:

phariaos-manager:
  kserve:
    storage:
      http:
        # The name of the secret that contains your Aleph Alpha credentials
        existingSecret: "your-secret"
        secretKeys:
            # The name of the key in the secret
            httpToken: "key-name"

If the token is provided directly as a Helm value, you only need to set the following when installing PhariaAI:

--set phariaos-manager.kserve.storage.http.token=$AA_REGISTRY_PASSWORD

Enabling model deployment from Hugging Face

To enable deployment of models from Hugging Face buckets, you must provide your Hugging Face token. The token can be provided as a Kubernetes secret or directly in the Helm values:

phariaos-manager:
  kserve:
    storage:
      huggingFace:
        enabled: true
        # The name of the secret that contains your Hugging Face credentials
        existingSecret: "your-secret"
        secretKeys:
            # The name of the key in the secret
            huggingFaceToken: "key-name"

If the token is provided directly as a Helm value, you only need to set the following when installing PhariaAI:

--set phariaos-manager.kserve.storage.huggingFace.token=$HUGGINGFACE_TOKEN

Enabling model deployment from S3 buckets

To enable deployment of models from S3 buckets, you must provide your S3 credentials. The credentials can be provided as a Kubernetes secret or directly in the Helm values:

phariaos-manager:
  kserve:
    storage:
      s3:
        enabled: true
        # The name of the secret that contains your S3 credentials
        existingSecret: "your-secret"
        secretKeys:
          # The secret key for access key id of the S3 storage.
          accessKeyId: "access-key-id"
          # The secret key for secret access key of the S3 storage.
          secretAccessKey: "secret-access-key"
          # The secret key for endpoint of the S3 storage.
          endpoint: "endpoint"
          # The secret key for region of the S3 storage.
          region: "region"

If the credentials are provided directly as a Helm value, you only need to set the following when installing PhariaAI:

--set phariaos-manager.kserve.storage.s3.accessKeyId=$S3_ACCESS_KEY_ID
--set phariaos-manager.kserve.storage.s3.secretAccessKey=$S3_SECRET_ACCESS_KEY
--set phariaos-manager.kserve.storage.s3.endpoint=$S3_ENDPOINT
--set phariaos-manager.kserve.storage.s3.region=$S3_REGION