Skip to main content

How to enable Dynamic Model Management

Dynamic Model Management enables users to deploy a variety of models upon request using PhariaOS.

Activating the feature​

In order to enable the Dynamic Model Management feature, activate the following flag:

phariaos-manager:
  kserve:
    enabled: true

When this chart is installed, Custom Resource Definitions will be installed to your cluster.

How to enable model deployment from the Aleph Alpha registry​

To enable deployment of models from the Aleph Alpha model registry, you must provide your registry token. The token can be provided as a Kubernetes secret or directly in the helm values.

phariaos-manager:
  kserve:
    storage:
      http:
        # The name of the secret that contains your Aleph Alpha credentials
        existingSecret: "your-secret"
        secretKeys:
            # The name of the key in the secret
            httpToken: "key-name"

If the token is provided directly as a helm value, you only need to set the following when installing PhariaAI.

--set phariaos-manager.kserve.storage.http.token=$AA_REGISTRY_PASSWORD

How to enable model deployment from HuggingFace​

To enable deployment of models from HuggingFace buckets, you must provide your HuggingFace token. The token can be provided as a Kubernetes secret or directly in the helm values.

phariaos-manager:
  kserve:
    storage:
      huggingFace:
        enabled: true
        # The name of the secret that contains your HuggingFace credentials
        existingSecret: "your-secret"
        secretKeys:
            # The name of the key in the secret
            huggingFaceToken: "key-name"

If the token is provided directly as a helm value, you only need to set the following when installing PhariaAI.

--set phariaos-manager.kserve.storage.huggingFace.token=$HUGGINGFACE_TOKEN

How to enable model deployment from S3 buckets​

To enable deployment of models from S3 buckets, you must provide your S3 credentials. The credentials can be provided as a Kubernetes secret or directly in the helm values.

phariaos-manager:
  kserve:
    storage:
      s3:
        enabled: true
        # The name of the secret that contains your S3 credentials
        existingSecret: "your-secret"
        secretKeys:
          # The secret key for access key id of the S3 storage.
          accessKeyId: "access-key-id"
          # The secret key for secret access key of the S3 storage.
          secretAccessKey: "secret-access-key"
          # The secret key for endpoint of the S3 storage.
          endpoint: "endpoint"
          # The secret key for region of the S3 storage.
          region: "region"

If the credentials are provided directly as a helm value, you only need to set the following when installing PhariaAI.

--set phariaos-manager.kserve.storage.s3.accessKeyId=$S3_ACCESS_KEY_ID
--set phariaos-manager.kserve.storage.s3.secretAccessKey=$S3_SECRET_ACCESS_KEY
--set phariaos-manager.kserve.storage.s3.endpoint=$S3_ENDPOINT
--set phariaos-manager.kserve.storage.s3.region=$S3_REGION