Semantic Embed

With our semantic_embed-endpoint you can create semantic embeddings for your text. This functionality can be used in a myriad of ways. For more information please check out our blog-post on Luminous-Explore, introducing the model behind the semantic_embed-endpoint.

There are two possible ways to use the semantic_embed-endpoint. If you have texts with a dissimilar structure (e.g. a Document and a Query) you would want to use asymmetric embeddings. Contrary, if you have texts of similar structure, you would want to use symmetric embeddings. The illustration below shows how they differ and what a workflow with semantic embeddings could look like.

Code Example

Before we show how to use symmetric and asymmetric embeddings, we define a function to calculate the cosine similarity of two vectors. You can also use any similar function from the package of your choice.

import math
def compute_cosine_similarity(
    embedding_1,
    embedding_2
):
    sumxx, sumxy, sumyy = 0, 0, 0
    for i in range(len(embedding_1)):
        x = embedding_1[i]
        y = embedding_2[i]
        sumxx += x * x
        sumyy += y * y
        sumxy += x * y
    return sumxy / math.sqrt(sumxx * sumyy)

The following code snippet demonstrates how you can use symmetric embeddings for texts with a similar structure.

import os
from aleph_alpha_client import Client, Prompt, SemanticRepresentation, SemanticEmbeddingRequest
# If you are using a Windows machine, you must install the python-dotenv package and run the two below lines as well.
# from dotenv import load_dotenv
# load_dotenv()


client = Client(token=os.getenv("AA_TOKEN"))
texts = [
    "The gray wolf is a large carnivorous mammal.",
    "Wolves hunt in packs.",
    "I take the train to work everyday."
]

symmetric_embeddings = []

for text in texts:
    symmetric_params = {
        "prompt": Prompt.from_text(text),
        "representation": SemanticRepresentation.Symmetric,
        "compress_to_size": 128
    }
    symmetric_request = SemanticEmbeddingRequest(**symmetric_params)
    symmetric_response = client.semantic_embed(request=symmetric_request, model="luminous-base")
    symmetric_embeddings.append(symmetric_response.embedding)

print("""'{text_1}' and '{text_2}' similarity: {score_1}
'{text_2}' and '{text_3}' similarity: {score_2}
'{text_1}' and '{text_3}' similarity: {score_3}""".format(
    text_1 = texts[0],
    text_2 = texts[1],
    text_3 = texts[2],
    score_1 = compute_cosine_similarity(symmetric_embeddings[0], symmetric_embeddings[1]),
    score_2 = compute_cosine_similarity(symmetric_embeddings[1], symmetric_embeddings[2]),
    score_3 = compute_cosine_similarity(symmetric_embeddings[0], symmetric_embeddings[2])
))
# prints:
# 'The gray wolf is a large carnivorous mammal.' and 'Wolves hunt in packs.' similarity: 0.601247315089725
# 'Wolves hunt in packs.' and 'I take the train to work everyday.' similarity: 0.04440147971258914
# 'The gray wolf is a large carnivorous mammal.' and 'I take the train to work everyday.' similarity: -0.12290839164669595

This code snippet demonstrates how you can use asymmetric embeddings for texts with a dissimilar structure.

import os
from aleph_alpha_client import Client, Prompt, SemanticRepresentation, SemanticEmbeddingRequest

query_text = "When did wolves first appear in the fossil record?"
documents = [
    "The gray wolf (Canis lupus) is a species of placental mammal of the carnivore order. The earliest fossil record dates back eight hundred thousand years. Wolves are native to North America and Eurasia, where they were once widely distributed and abundant. Today, they inhabit only a very limited portion of their former territory.",
    "Game theory is a discipline concerned with mathematical models of strategic interaction between rational actors. Game theory is applied in various areas of the social sciences as well as in logic, systems theory, and computer science. Although it originally focused on zero-sum games in which each participant's gains or losses are perfectly balanced by those of the others, modern game theory applies to a wide range of behavioral relationships."
]

query_params = {
    "prompt": Prompt.from_text(query_text),
    "representation": SemanticRepresentation.Query,
    "compress_to_size": 128
}
query_request = SemanticEmbeddingRequest(**query_params)
query_response = client.semantic_embed(request=query_request, model="luminous-base")
query_embedding = query_response.embedding

document_embeddings = []

for document in documents:
    document_params = {
        "prompt": Prompt.from_text(document),
        "representation": SemanticRepresentation.Document,
        "compress_to_size": 128
    }
    document_request = SemanticEmbeddingRequest(**document_params)
    document_response = client.semantic_embed(request=document_request, model="luminous-base")
    document_embeddings.append(document_response.embedding)

print("""query_text and document_1 similarity: {score_1}
query_text and document_2 similarity: {score_2}""".format(
    score_1 = compute_cosine_similarity(query_embedding, document_embeddings[0]),
    score_2 = compute_cosine_similarity(query_embedding, document_embeddings[1])
))
# prints:
# query_text and document_1 similarity: 0.5142405613255223
# query_text and document_2 similarity: -0.13230054712395245

tip

Should I Use Embeddings or Semantic Embeddings?

Technically, you can work with both types of embeddings. Our embedding endpoint should be used as a feature input for downstream models (e.g., classifiers). There are cases where the semantic meaning of a text matters. These could be, but are not limited to, semantic search systems or Q&A applications. In these scenarios, we recommend using our semantic embeddings endpoint.

What is the Difference Between Embeddings and Semantic Embeddings?

Our embeddings are the pooled activations for the input tokens at the layer of your choice. While the embedding representation has some semantic meaning, the semantic embeddings have been optimized to find semantic relationships between two inputs (text and image). This was achieved by training additional layers using datasets curated for this task and a contrastive loss. There is an optional dimensionality reduction layer that reduces the embedding dimension to a 128-dimensional vector, making it more efficient to compare two vectors.

Code Example Attention Manipulation

You can also create semantic embeddings with our attention manipulation method, AtMan. For example, you might want to use asymmetric semantic embeddings to find a gaming console among a set of product descriptions. Without attention manipulation, the query scores nearly equally against both documents.

import os
import re
import math
from aleph_alpha_client import (
    Client,
    Prompt,
    SemanticEmbeddingRequest,
    SemanticRepresentation,
    TextControl,
)

# If you are using a Windows machine, you must install the python-dotenv package and run the two below lines as well.
# from dotenv import load_dotenv
# load_dotenv()

def compute_cosine_similarity(
    embedding_1,
    embedding_2
):
    sumxx, sumxy, sumyy = 0, 0, 0
    for i in range(len(embedding_1)):
        x = embedding_1[i]
        y = embedding_2[i]
        sumxx += x * x
        sumyy += y * y
        sumxy += x * y
    return sumxy / math.sqrt(sumxx * sumyy)

client = Client(token=os.getenv("AA_TOKEN"))


query_text = "I want to play on my TV."

documents = [
    "Your favorite movies and TV programs are automatically upscaled to 4K using our new 4K upscaling technology. You can quickly discover content and navigate streaming services with our entertainment hub via the internet.",
    "Experience unbelievable adventures with our new gaming console. Just plug your console into your TV to enjoy more than 130 fps and connect to the internet for unlimited gaming fun."
]

query_params = {
    "prompt": Prompt.from_text(query_text),
    "representation": SemanticRepresentation.Query,
    "compress_to_size": 128
}
query_request = SemanticEmbeddingRequest(**query_params)
query_response = client.semantic_embed(request=query_request, model="luminous-base")
query_embedding = query_response.embedding

document_embeddings = []

for document in documents:
    document_params = {
        "prompt": Prompt.from_text(document),
        "representation": SemanticRepresentation.Document,
        "compress_to_size": 128
    }
    document_request = SemanticEmbeddingRequest(**document_params)
    document_response = client.semantic_embed(request=document_request, model="luminous-base")
    document_embeddings.append(document_response.embedding)

print("""query_text and document_1 similarity: {score_1}
query_text and document_2 similarity: {score_2}""".format(
    score_1 = compute_cosine_similarity(query_embedding, document_embeddings[0]),
    score_2 = compute_cosine_similarity(query_embedding, document_embeddings[1])
))

# prints:
# query_text and document_1 similarity: 0.44637539708542545
# query_text and document_2 similarity: 0.44946165354098017

With attention manipulation we steer the model into the direction we want it to go. So we will amplify the word "play" in the query to find the product description matchting a gaming console.

import os
import re
import math

from aleph_alpha_client import (
    Client,
    Prompt,
    SemanticEmbeddingRequest,
    SemanticRepresentation,
    TextControl,
)

# If you are using a Windows machine, you must install the python-dotenv package and run the two below lines as well.
# from dotenv import load_dotenv
# load_dotenv()

def compute_cosine_similarity(
    embedding_1,
    embedding_2
):
    sumxx, sumxy, sumyy = 0, 0, 0
    for i in range(len(embedding_1)):
        x = embedding_1[i]
        y = embedding_2[i]
        sumxx += x * x
        sumyy += y * y
        sumxy += x * y
    return sumxy / math.sqrt(sumxx * sumyy)

client = Client(token=os.getenv("AA_TOKEN"))

query_text = "I want to play on my TV."

documents = [
    "Your favorite movies and TV programs are automatically upscaled to 4K using our new 4K upscaling technology. You can quickly discover content and navigate streaming services with our entertainment hub via the internet.",
    "Experience unbelievable adventures with our new gaming console. Just plug your console into your TV to enjoy more than 130 fps and connect to the internet for unlimited gaming fun."
]


matching_string = re.search("play", query_text)

begin_match = matching_string.regs[0][0]
end_match = matching_string.regs[0][1]

control = TextControl(start=begin_match, length=end_match-begin_match, factor=7)

query_params = {
    "prompt": Prompt.from_text(query_text, controls=[control]),
    "representation": SemanticRepresentation.Query,
    "compress_to_size": 128
}
query_request = SemanticEmbeddingRequest(**query_params)
query_response = client.semantic_embed(request=query_request, model="luminous-base")
query_embedding = query_response.embedding

document_embeddings = []

for document in documents:
    document_params = {
        "prompt": Prompt.from_text(document),
        "representation": SemanticRepresentation.Document,
        "compress_to_size": 128
    }
    document_request = SemanticEmbeddingRequest(**document_params)
    document_response = client.semantic_embed(request=document_request, model="luminous-base")
    document_embeddings.append(document_response.embedding)

print("""query_text and document_1 similarity: {score_1}
query_text and document_2 similarity: {score_2}""".format(
    score_1 = compute_cosine_similarity(query_embedding, document_embeddings[0]),
    score_2 = compute_cosine_similarity(query_embedding, document_embeddings[1])
))

# prints:
# query_text and document_1 similarity: 0.3567054956894866
# query_text and document_2 similarity: 0.4028220746612426

If you need more information on the parameters you can use, please checkout our HTTP API.