Skip to main content

Announcing release of Pharia embedding model

We are happy to bring to you our new Pharia Embedding model (Pharia-1-Embedding-4608-control) that builds on our latest Pharia LLM. The model is trained with adapters on top of (frozen) Pharia LLM weights and thus can be served on the same worker for both completion requests and embedding requests (see figure below). You can read more about the training details and evaluations of the embedding model in our model card.

Pharia Embedding Model

The new Pharia embedding model was trained using generative representational instruction tuning to allow for the specification of user-defined instructions along with the input text to embed. The instruction field provides context to the model to embed text and allows for flexible use of the model for different use cases without the need for additional finetuning. To illustrate the effect of instruction, let's consider an example with two sentences "How long is nile?" and "The Nile is a major north-flowing river in northeastern Africa". The cosine similarity between their embedding vectors will vary based on the instruction provided. Note that the following numbers are for illustration purposes only - actual similarity scores will differ based on your specific use case.

  1. Without instruction

Cosine Similarity: 0.80

  1. With instruction "represent the text to query a river database"

Cosine Similarity: 0.99

We get much higher cosine similarity when the model has additional context about the use case (in this case, querying a river database).

Pharia embedding model will be available via a new instructable_embed endpoint that allows you to pass an additional instruction field. Here is an example to get started.

curl -L -X POST 'https://api.aleph-alpha.com/instructable_embed' \
-H 'Content-Type: application/json' \
-H "Accept: application/json" \
-H "Authorization: Bearer <TOKEN>" \
--data-raw '{
  "model": "pharia-1-embedding-4608-control",
  "input": "How long is the nile?", 
  "instruction": "Embed the text to search a river database"
}'

For more details, check out the documentation!