Managing models

With PhariaOS dynamic model management, you can deploy and manage Aleph Alpha-supported and open-source models without needing to modify your infrastructure. A unified interface allows you to explore, install, and control models in one place, reducing the risk of deployment errors.

note

Model support is limited to the following:

• Aleph Alpha-supported models, which are currently:
  • Pharia 1 LLM 7B Control
  • Llama 3.1 8B
  • Llama 3.3 70B
  • Llama Guard 3 8B
  • Luminous base
• vLLM-compatible models from Hugging Face.
• GPU resource prediction and adapter support will be included in future versions of PhariaOS!

Viewing installed models

To view the models installed in your PhariaAI instance, click Models in the PhariaOS sidebar.

Models are listed with their details:

The status of your model can be one of the following:

• Running: the model has been successfully deployed and is ready for use
• Deploying: downloading model weights and waiting for the scheduler to find enough resources to deploy the model
• Failed: the model was not deployed; this could be caused by timeout, resources not available, or other reasons

Click on a model in the list to view details about its deployment.

Installing a model

To install a new model:

1. Click Models in the PhariaOS sidebar.
2. Click Add model at the top right.
   The Select a New Model list looks something like the following:
   
   
3. Click Add in the card of the Aleph Alpha model you wish to install, or click Custom model from Hugging Face.
4. If you selected to install a model from Hugging Face, specify a name and the URL for the model:
   
5. For either the Aleph Alpha or Hugging Face model you selected, specify the hardware requirements and tolerations.
   Contact your Kubernetes operator to determine which hardware and tolerations to use for desired outcome. You can also see the Kubernetes documentation for requests and limits and tolerations.

6. When you are finished, click Download & deploy model.

The model takes the status Deploying:

Editing a model

To change the configuration parameters of an installed model:

1. Click Models in the PhariaOS sidebar.
2. Click the model in the list that you want to edit.
   The model's deployment status and configuration are displayed.
3. Click Edit to the right of the page.
4. Edit the hardware requirements and tolerations as required.
5. Click Save.

The model is redeployed, and takes the status Deploying.

Removing a model

To remove an installed model:

1. Click Models in the PhariaOS sidebar.
2. Click the model in the list that you want to remove.
   The model's deployment status and configuration are displayed.
3. Click the kebab menu icon at the top right, and select Remove.

The model is undeployed and removed from the list.