JSON Completion

POST 
https://api.aleph-alpha.com/complete/json

Will complete a prompt using a specific model.

In contrast to the /complete endpoint, some guardrails are applied which steer the model towards generating a completion of valid JSON format, even if not requested explicitly. This is achieved by appending a builtin helper prompt to the user prompt and by using a specialized sampling method that favors JSON-compatible tokens.

To obtain a valid model, use GET /models_available.

Request

Query Parameters

nice boolean

Setting this to True, will signal to the API that you intend to be nice to other users by de-prioritizing your request below concurrent ones.

application/json

Bodyrequired

modelstringrequired
The name of the model from the Luminous model family. Models and their respective architectures can differ in parameter size and capabilities. The most recent version of the model is always used. The model output contains information as to the model version.
hostingHosting (string)nullable
Optional parameter that specifies which datacenters may process the request. You can either set the parameter to "aleph-alpha" or omit it (defaulting to null).
Not setting this value, or setting it to null, gives us maximal flexibility in processing your request in our own datacenters and on servers hosted with other providers. Choose this option for maximum availability.
Setting it to "aleph-alpha" allows us to only process the request in our own datacenters. Choose this option for maximal data privacy.
Possible values: [aleph-alpha, null]
prompt objectrequired
This field is used to send prompts to the model. A prompt can either be a text prompt or a multimodal prompt. A text prompt is a string of text. A multimodal prompt is an array of prompt items. It can be a combination of text, images, and token ID arrays.
In the case of a multimodal prompt, the prompt items will be concatenated and a single prompt will be used for the model.
Tokenization:
• Token ID arrays are used as as-is.
• Text prompt items are tokenized using the tokenizers specific to the model.
• Each image is converted into 144 tokens.
oneOf
Text Prompt
string
The text to be completed. Unconditional completion can be started with an empty string (default). The prompt may contain a zero shot or few shot task.
Multimodal
Array [
oneOf
Text
typestringrequired
Possible values: [text]
datastringrequired
controls object[]
Array [
startintegerrequired
Starting character index to apply the factor to.
lengthintegerrequired
The amount of characters to apply the factor to.
factornumberrequired
Factor to apply to the given token in the attention matrix.
• 0 <= factor < 1 => Suppress the given token
• factor == 1 => identity operation, no change to attention
• factor > 1 => Amplify the given token
token_overlapstring
What to do if a control partially overlaps with a text token.
If set to "partial", the factor will be adjusted proportionally with the amount of the token it overlaps. So a factor of 2.0 of a control that only covers 2 of 4 token characters, would be adjusted to 1.5. (It always moves closer to 1, since 1 is an identity operation for control factors.)
If set to "complete", the full factor will be applied as long as the control overlaps with the token at all.
Possible values: [partial, complete]
Default value: partial
]
Image
typestringrequired
Possible values: [image]
datastringrequired
An image send as part of a prompt to a model. The image is represented as base64.
Note: The models operate on square images. All non-square images are center-cropped before going to the model, so portions of the image may not be visible.
You can supply specific cropping parameters if you like, to choose a different area of the image than a center-crop. Or, you can always transform the image yourself to a square before sending it.
xinteger
x-coordinate of top left corner of cropping box in pixels
yinteger
y-coordinate of top left corner of cropping box in pixels
sizeinteger
Size of the cropping square in pixels
controls object[]
Array [
rect objectrequired
Bounding box in logical coordinates. From 0 to 1. With (0,0) being the upper left corner, and relative to the entire image.
Keep in mind, non-square images are center-cropped by default before going to the model. (You can specify a custom cropping if you want.). Since control coordinates are relative to the entire image, all or a portion of your control may be outside the "model visible area".
leftnumberrequired
x-coordinate of top left corner of the control bounding box. Must be a value between 0 and 1, where 0 is the left corner and 1 is the right corner.
topnumberrequired
y-coordinate of top left corner of the control bounding box Must be a value between 0 and 1, where 0 is the top pixel row and 1 is the bottom row.
widthnumberrequired
width of the control bounding box Must be a value between 0 and 1, where 1 means the full width of the image.
heightnumberrequired
height of the control bounding box Must be a value between 0 and 1, where 1 means the full height of the image.
factornumberrequired
Factor to apply to the given token in the attention matrix.
• 0 <= factor < 1 => Suppress the given token
• factor == 1 => identity operation, no change to attention
• factor > 1 => Amplify the given token
token_overlapstring
What to do if a control partially overlaps with an image token.
If set to "partial", the factor will be adjusted proportionally with the amount of the token it overlaps. So a factor of 2.0 of a control that only covers half of the image "tile", would be adjusted to 1.5. (It always moves closer to 1, since 1 is an identity operation for control factors.)
If set to "complete", the full factor will be applied as long as the control overlaps with the token at all.
Possible values: [partial, complete]
Default value: partial
]
Token Ids
typestringrequired
Possible values: [token_ids]
datainteger[]required
controls object[]
Array [
indexintegerrequired
Index of the token, relative to the list of tokens IDs in the current prompt item.
factornumberrequired
Factor to apply to the given token in the attention matrix.
• 0 <= factor < 1 => Suppress the given token
• factor == 1 => identity operation, no change to attention
• factor > 1 => Amplify the given token
]
]
maximum_tokensintegernullable
The maximum number of tokens to be generated. Completion will terminate after the maximum number of tokens is reached.
Increase this value to generate longer texts. A text is split into tokens. Usually there are more tokens than words. The sum of input tokens and maximum_tokens may not exceed the model's context window size.
minimum_tokensinteger
Generate at least this number of tokens before an end-of-text token is generated.
Default value: 0
echoboolean
Echo the prompt in the completion. This may be especially helpful when log_probs is set to return logprobs for the prompt.
Default value: false
temperaturenumbernullable
A higher sampling temperature encourages the model to produce less probable outputs ("be more creative"). Values are expected in a range from 0.0 to 1.0. Try high values (e.g., 0.9) for a more "creative" response and the default 0.0 for a well defined and repeatable answer. It is advised to use either temperature, top_k, or top_p, but not all three at the same time. If a combination of temperature, top_k or top_p is used, rescaling of logits with temperature will be performed first. Then top_k is applied. Top_p follows last.
Default value: 0
top_kintegernullable
Introduces random sampling for generated tokens by randomly selecting the next token from the k most likely options. A value larger than 1 encourages the model to be more creative. Set to 0.0 if repeatable output is desired. It is advised to use either temperature, top_k, or top_p, but not all three at the same time. If a combination of temperature, top_k or top_p is used, rescaling of logits with temperature will be performed first. Then top_k is applied. Top_p follows last.
Default value: 0
top_pnumbernullable
Introduces random sampling for generated tokens by randomly selecting the next token from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Set to 0.0 if repeatable output is desired. It is advised to use either temperature, top_k, or top_p, but not all three at the same time. If a combination of temperature, top_k or top_p is used, rescaling of logits with temperature will be performed first. Then top_k is applied. Top_p follows last.
Default value: 0
presence_penaltynumbernullable
The presence penalty reduces the likelihood of generating tokens that are already present in the generated text (repetition_penalties_include_completion=true) respectively the prompt (repetition_penalties_include_prompt=true). Presence penalty is independent of the number of occurrences. Increase the value to reduce the likelihood of repeating text. An operation like the following is applied:
logits[t] -> logits[t] - 1 * penalty
where logits[t] is the logits for any given token. Note that the formula is independent of the number of times that a token appears.
Default value: 0
frequency_penaltynumbernullable
The frequency penalty reduces the likelihood of generating tokens that are already present in the generated text (repetition_penalties_include_completion=true) respectively the prompt (repetition_penalties_include_prompt=true). If repetition_penalties_include_prompt=True, this also includes the tokens in the prompt. Frequency penalty is dependent on the number of occurrences of a token. An operation like the following is applied:
logits[t] -> logits[t] - count[t] * penalty
where logits[t] is the logits for any given token and count[t] is the number of times that token appears.
Default value: 0
sequence_penaltynumber
Increasing the sequence penalty reduces the likelihood of reproducing token sequences that already appear in the prompt (if repetition_penalties_include_prompt is True) and prior completion.
Default value: 0
sequence_penalty_min_lengthinteger
Minimal number of tokens to be considered as sequence
Default value: 2
repetition_penalties_include_promptbooleannullable
Flag deciding whether presence penalty or frequency penalty are updated from tokens in the prompt
Default value: false
repetition_penalties_include_completionboolean
Flag deciding whether presence penalty or frequency penalty are updated from tokens in the completion
Default value: true
use_multiplicative_presence_penaltybooleannullable
Flag deciding whether presence penalty is applied multiplicatively (True) or additively (False). This changes the formula stated for presence penalty.
Default value: false
use_multiplicative_frequency_penaltyboolean
Flag deciding whether frequency penalty is applied multiplicatively (True) or additively (False). This changes the formula stated for frequency penalty.
Default value: false
use_multiplicative_sequence_penaltyboolean
Flag deciding whether sequence penalty is applied multiplicatively (True) or additively (False).
Default value: false
penalty_biasstringnullable
All tokens in this text will be used in addition to the already penalized tokens for repetition penalties. These consist of the already generated completion tokens and the prompt tokens, if repetition_penalties_include_prompt is set to true.
Default value: null
penalty_exceptionsstring[]nullable
List of strings that may be generated without penalty, regardless of other penalty settings. By default, we will also include any stop_sequences you have set, since completion performance can be degraded if expected stop sequences are penalized. You can disable this behavior by setting penalty_exceptions_include_stop_sequences to false.
penalty_exceptions_include_stop_sequencesbooleannullable
By default we include all stop_sequences in penalty_exceptions, so as not to penalise the presence of stop sequences that are present in few-shot prompts to give structure to your completions.
You can set this to false if you do not want this behaviour.
See the description of penalty_exceptions for more information on what penalty_exceptions are used for.
Default value: true
best_ofintegernullable
If a value is given, the number of best_of completions will be generated on the server side. The completion with the highest log probability per token is returned. If the parameter n is greater than 1 more than 1 (n) completions will be returned. best_of must be strictly greater than n.
Possible values: <= 100
Default value: 1
nintegernullable
The number of completions to return. If argmax sampling is used (temperature, top_k, top_p are all default) the same completions will be produced. This parameter should only be increased if random sampling is used.
Default value: 1
logit_biasobjectnullable
log_probsintegernullable
Number of top log probabilities for each token generated. Log probabilities can be used in downstream tasks or to assess the model's certainty when producing tokens. No log probabilities are returned if set to None. Log probabilities of generated tokens are returned if set to 0. Log probabilities of generated tokens and top n log probabilities are returned if set to n.
Possible values: <= 20
Default value: null
stop_sequencesstring[]nullable
List of strings that will stop generation if they're generated. Stop sequences may be helpful in structured texts. Say the user has specified "tor away" as one of the requested stop sequences and the model has generated the following sequence of tokens ["An", " apple", " a", " day", " keeps", " the", " doctor", " away"]. The user will see "An apple a day keeps the" as the model's response, omitting the last two tokens which contain the stop sequence. Note that even though " doc" is not part of the stop sequence "tor away", it won't appear in the user output since it is part of the token " doctor" which contains part of the stop sequence.
tokensbooleannullable
Flag indicating whether individual tokens of the completion should be returned (True) or whether solely the generated text (i.e. the completion) is sufficient (False).
Default value: false
raw_completionboolean
Setting this parameter to true forces the raw completion of the model to be returned. For some models, we may optimize the completion that was generated by the model and return the optimized completion in the completion field of the CompletionResponse. The raw completion, if returned, will contain the un-optimized completion. Setting tokens to true or log_probs to any value will also trigger the raw completion to be returned.
Default value: false
disable_optimizationsbooleannullable
We continually research optimal ways to work with our models. By default, we apply these optimizations to both your prompt and completion for you. Our goal is to improve your results while using our API. But you can always pass disable_optimizations: true and we will leave your prompt and completion untouched.
Default value: false
completion_bias_inclusionstring[]
Bias the completion to only generate options within this list; all other tokens are disregarded at sampling
Note that strings in the inclusion list must not be prefixes of strings in the exclusion list and vice versa
Default value: []
completion_bias_inclusion_first_token_onlyboolean
Only consider the first token for the completion_bias_inclusion
Default value: false
completion_bias_exclusionstring[]
Bias the completion to NOT generate options within this list; all other tokens are unaffected in sampling
Note that strings in the inclusion list must not be prefixes of strings in the exclusion list and vice versa
Default value: []
completion_bias_exclusion_first_token_onlyboolean
Only consider the first token for the completion_bias_exclusion
Default value: false
contextual_control_thresholdnumbernullable
If set to null, attention control parameters only apply to those tokens that have explicitly been set in the request. If set to a non-null value, we apply the control parameters to similar tokens as well. Controls that have been applied to one token will then be applied to all other tokens that have at least the similarity score defined by this parameter. The similarity score is the cosine similarity of token embeddings.
Default value: null
control_log_additiveboolean
true: apply controls on prompt items by adding the log(control_factor) to attention scores. false: apply controls on prompt items by (attention_scores - -attention_scores.min(-1)) * control_factor
Default value: true
streamboolean
When set to true, the model will transmit all completions tokens as soon as they become available via the server-sent events protocol.
Default value: false
steering_conceptsSteeringConcept (string)[]
Specifies how the output of the model should be steered. This steers the output in the direction given by positive examples associated to the steering concept and away from the negative examples.
Possible values: Value must match regular expression ^_worker/[a-zA-Z0-9-_]{1,64}$
Default value: []

Responses

200

OK

application/json

Schema

Schema

model_versionstring

model name and version (if any) of the used model for inference

completions object[]

list of completions; may contain only one entry if no more are requested (see parameter n)

Array [

log_probsobjectnullable

list with a dictionary for each generated token. The dictionary maps the keys' tokens to the respective log probabilities. This field is only returned if requested with the parameter "log_probs".

completionstring

generated completion on the basis of the prompt

raw_completionstringnullable

For some models, we may optimize the completion that was generated by the model and return the optimized completion in the completion field of the CompletionResponse. The raw completion, if returned, will contain the un-optimized completion. Setting the parameter raw_completion in the CompletionRequest to true forces the raw completion of the model to be returned. Setting tokens to true or log_probs to any value will also trigger the raw completion to be returned.

completion_tokensstring[]

completion split into tokens. This field is only returned if requested with the parameter "tokens".

finish_reasonstringnullable

reason for termination of generation. This may be a stop sequence or maximum number of tokens reached.

]

optimized_prompt object[]

Describes prompt after optimizations. This field is only returned if the flag disable_optimizations flag is not set and the prompt has actually changed.

Array [

oneOf

Text

typestring

Possible values: [text]

datastring

Image

typestring

Possible values: [image]

datastring

base64 encoded image

Token Ids

typestring

Possible values: [token_ids]

datainteger[]

]

num_tokens_prompt_totalinteger

Number of tokens combined across all completion tasks.

In particular, if you set best_of or n to a number larger than 1 then we report the combined prompt token count for all best_of or n tasks.

Tokenization:

• Token ID arrays are used as as-is.
• Text prompt items are tokenized using the tokenizers specific to the model.
• Each image is converted into a fixed amount of tokens that depends on the chosen model.

num_tokens_generatedinteger

Number of tokens combined across all completion tasks. If multiple completions are returned or best_of is set to a value greater than 1 then this value contains the combined generated token count.

Example (auto)

{
  "completions": [
    {
      "completion": "keeps the doctor away.",
      "finish_reason": "maximum_tokens"
    }
  ],
  "model_version": "2021-12",
  "optimized_prompt": "An apple a day",
  "num_tokens_prompt_total": 4,
  "num_tokens_generated": 5
}

Authorization: http

name: tokentype: httpscheme: bearerdescription: Can be generated in your [Aleph Alpha profile](https://app.aleph-alpha.com/profile)

• csharp
• curl
• dart
• go
• http
• java
• javascript
• kotlin
• c
• nodejs
• objective-c
• ocaml
• php
• powershell
• python
• r
• ruby
• rust
• shell
• swift

• HTTPCLIENT
• RESTSHARP

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Post, "https://api.aleph-alpha.com/complete/json");
request.Headers.Add("Accept", "application/json");
request.Headers.Add("Authorization", "Bearer <token>");
var content = new StringContent("{\n  \"model\": \"llama-3.1-8b-instruct\",\n  \"prompt\": \"An apple a day\"\n}", null, "application/json");
request.Content = content;
var response = await client.SendAsync(request);
response.EnsureSuccessStatusCode();
Console.WriteLine(await response.Content.ReadAsStringAsync());

Request Collapse all

Base URL

https://api.aleph-alpha.com

Auth

Bearer Token

Parameters

nice — query

---truefalse

Body required

{
  "model": "llama-3.1-8b-instruct",
  "prompt": "An apple a day"
}

ResponseClear

Click the Send API Request button above and see the response here!