Skip to main content

Completion

POST 

https://api.aleph-alpha.com/complete

Will complete a prompt using a specific model. To obtain a valid model, use GET /model-settings.

Request

Query Parameters

    nice boolean

    Setting this to True, will signal to the API that you intend to be nice to other users by de-prioritizing your request below concurrent ones.

Bodyrequired

    modelstringrequired

    The name of the model from the Luminous model family. Models and their respective architectures can differ in parameter size and capabilities. The most recent version of the model is always used. The model output contains information as to the model version.

    hostingHosting (string)nullable

    Optional parameter that specifies which datacenters may process the request. You can either set the parameter to "aleph-alpha" or omit it (defaulting to null).

    Not setting this value, or setting it to null, gives us maximal flexibility in processing your request in our own datacenters and on servers hosted with other providers. Choose this option for maximum availability.

    Setting it to "aleph-alpha" allows us to only process the request in our own datacenters. Choose this option for maximal data privacy.

    Possible values: [aleph-alpha, null]

    prompt objectrequired

    This field is used to send prompts to the model. A prompt can either be a text prompt or a multimodal prompt. A text prompt is a string of text. A multimodal prompt is an array of prompt items. It can be a combination of text, images, and token ID arrays.

    In the case of a multimodal prompt, the prompt items will be concatenated and a single prompt will be used for the model.

    Tokenization:

    • Token ID arrays are used as as-is.
    • Text prompt items are tokenized using the tokenizers specific to the model.
    • Each image is converted into 144 tokens.
    oneOf
    string

    The text to be completed. Unconditional completion can be started with an empty string (default). The prompt may contain a zero shot or few shot task.

    maximum_tokensintegernullable

    The maximum number of tokens to be generated. Completion will terminate after the maximum number of tokens is reached.

    Increase this value to generate longer texts. A text is split into tokens. Usually there are more tokens than words. The sum of input tokens and maximum_tokens may not exceed the model's context window size.

    minimum_tokensinteger

    Generate at least this number of tokens before an end-of-text token is generated.

    Default value: 0
    echoboolean

    Echo the prompt in the completion. This may be especially helpful when log_probs is set to return logprobs for the prompt.

    Default value: false
    temperaturenumbernullable

    A higher sampling temperature encourages the model to produce less probable outputs ("be more creative"). Values are expected in a range from 0.0 to 1.0. Try high values (e.g., 0.9) for a more "creative" response and the default 0.0 for a well defined and repeatable answer. It is advised to use either temperature, top_k, or top_p, but not all three at the same time. If a combination of temperature, top_k or top_p is used, rescaling of logits with temperature will be performed first. Then top_k is applied. Top_p follows last.

    Default value: 0
    top_kintegernullable

    Introduces random sampling for generated tokens by randomly selecting the next token from the k most likely options. A value larger than 1 encourages the model to be more creative. Set to 0.0 if repeatable output is desired. It is advised to use either temperature, top_k, or top_p, but not all three at the same time. If a combination of temperature, top_k or top_p is used, rescaling of logits with temperature will be performed first. Then top_k is applied. Top_p follows last.

    Default value: 0
    top_pnumbernullable

    Introduces random sampling for generated tokens by randomly selecting the next token from the smallest possible set of tokens whose cumulative probability exceeds the probability top_p. Set to 0.0 if repeatable output is desired. It is advised to use either temperature, top_k, or top_p, but not all three at the same time. If a combination of temperature, top_k or top_p is used, rescaling of logits with temperature will be performed first. Then top_k is applied. Top_p follows last.

    Default value: 0
    presence_penaltynumbernullable

    The presence penalty reduces the likelihood of generating tokens that are already present in the generated text (repetition_penalties_include_completion=true) respectively the prompt (repetition_penalties_include_prompt=true). Presence penalty is independent of the number of occurrences. Increase the value to reduce the likelihood of repeating text. An operation like the following is applied:

    logits[t] -> logits[t] - 1 * penalty

    where logits[t] is the logits for any given token. Note that the formula is independent of the number of times that a token appears.

    Default value: 0
    frequency_penaltynumbernullable

    The frequency penalty reduces the likelihood of generating tokens that are already present in the generated text (repetition_penalties_include_completion=true) respectively the prompt (repetition_penalties_include_prompt=true). If repetition_penalties_include_prompt=True, this also includes the tokens in the prompt. Frequency penalty is dependent on the number of occurrences of a token. An operation like the following is applied:

    logits[t] -> logits[t] - count[t] * penalty

    where logits[t] is the logits for any given token and count[t] is the number of times that token appears.

    Default value: 0
    sequence_penaltynumber

    Increasing the sequence penalty reduces the likelihood of reproducing token sequences that already appear in the prompt (if repetition_penalties_include_prompt is True) and prior completion.

    Default value: 0
    sequence_penalty_min_lengthinteger

    Minimal number of tokens to be considered as sequence

    Default value: 2
    repetition_penalties_include_promptbooleannullable

    Flag deciding whether presence penalty or frequency penalty are updated from tokens in the prompt

    Default value: false
    repetition_penalties_include_completionboolean

    Flag deciding whether presence penalty or frequency penalty are updated from tokens in the completion

    Default value: true
    use_multiplicative_presence_penaltybooleannullable

    Flag deciding whether presence penalty is applied multiplicatively (True) or additively (False). This changes the formula stated for presence penalty.

    Default value: false
    use_multiplicative_frequency_penaltyboolean

    Flag deciding whether frequency penalty is applied multiplicatively (True) or additively (False). This changes the formula stated for frequency penalty.

    Default value: false
    use_multiplicative_sequence_penaltyboolean

    Flag deciding whether sequence penalty is applied multiplicatively (True) or additively (False).

    Default value: false
    penalty_biasstringnullable

    All tokens in this text will be used in addition to the already penalized tokens for repetition penalties. These consist of the already generated completion tokens and the prompt tokens, if repetition_penalties_include_prompt is set to true.

    Default value: null
    penalty_exceptionsstring[]nullable

    List of strings that may be generated without penalty, regardless of other penalty settings. By default, we will also include any stop_sequences you have set, since completion performance can be degraded if expected stop sequences are penalized. You can disable this behavior by setting penalty_exceptions_include_stop_sequences to false.

    penalty_exceptions_include_stop_sequencesbooleannullable

    By default we include all stop_sequences in penalty_exceptions, so as not to penalise the presence of stop sequences that are present in few-shot prompts to give structure to your completions.

    You can set this to false if you do not want this behaviour.

    See the description of penalty_exceptions for more information on what penalty_exceptions are used for.

    Default value: true
    best_ofintegernullable

    If a value is given, the number of best_of completions will be generated on the server side. The completion with the highest log probability per token is returned. If the parameter n is greater than 1 more than 1 (n) completions will be returned. best_of must be strictly greater than n.

    Possible values: <= 100

    Default value: 1
    nintegernullable

    The number of completions to return. If argmax sampling is used (temperature, top_k, top_p are all default) the same completions will be produced. This parameter should only be increased if random sampling is used.

    Default value: 1
    logit_biasobjectnullable
    log_probsintegernullable

    Number of top log probabilities for each token generated. Log probabilities can be used in downstream tasks or to assess the model's certainty when producing tokens. No log probabilities are returned if set to None. Log probabilities of generated tokens are returned if set to 0. Log probabilities of generated tokens and top n log probabilities are returned if set to n.

    Possible values: <= 20

    Default value: null
    stop_sequencesstring[]nullable

    List of strings that will stop generation if they're generated. Stop sequences may be helpful in structured texts. Say the user has specified "tor away" as one of the requested stop sequences and the model has generated the following sequence of tokens ["An", " apple", " a", " day", " keeps", " the", " doctor", " away"]. The user will see "An apple a day keeps the" as the model's response, omitting the last two tokens which contain the stop sequence. Note that even though " doc" is not part of the stop sequence "tor away", it won't appear in the user output since it is part of the token " doctor" which contains part of the stop sequence.

    tokensbooleannullable

    Flag indicating whether individual tokens of the completion should be returned (True) or whether solely the generated text (i.e. the completion) is sufficient (False).

    Default value: false
    raw_completionboolean

    Setting this parameter to true forces the raw completion of the model to be returned. For some models, we may optimize the completion that was generated by the model and return the optimized completion in the completion field of the CompletionResponse. The raw completion, if returned, will contain the un-optimized completion. Setting tokens to true or log_probs to any value will also trigger the raw completion to be returned.

    Default value: false
    disable_optimizationsbooleannullable

    We continually research optimal ways to work with our models. By default, we apply these optimizations to both your prompt and completion for you. Our goal is to improve your results while using our API. But you can always pass disable_optimizations: true and we will leave your prompt and completion untouched.

    Default value: false
    completion_bias_inclusionstring[]

    Bias the completion to only generate options within this list; all other tokens are disregarded at sampling

    Note that strings in the inclusion list must not be prefixes of strings in the exclusion list and vice versa

    Default value: []
    completion_bias_inclusion_first_token_onlyboolean

    Only consider the first token for the completion_bias_inclusion

    Default value: false
    completion_bias_exclusionstring[]

    Bias the completion to NOT generate options within this list; all other tokens are unaffected in sampling

    Note that strings in the inclusion list must not be prefixes of strings in the exclusion list and vice versa

    Default value: []
    completion_bias_exclusion_first_token_onlyboolean

    Only consider the first token for the completion_bias_exclusion

    Default value: false
    contextual_control_thresholdnumbernullable

    If set to null, attention control parameters only apply to those tokens that have explicitly been set in the request. If set to a non-null value, we apply the control parameters to similar tokens as well. Controls that have been applied to one token will then be applied to all other tokens that have at least the similarity score defined by this parameter. The similarity score is the cosine similarity of token embeddings.

    Default value: null
    control_log_additiveboolean

    true: apply controls on prompt items by adding the log(control_factor) to attention scores. false: apply controls on prompt items by (attention_scores - -attention_scores.min(-1)) * control_factor

    Default value: true
    streamboolean

    When set to true, the model will transmit all completions tokens as soon as they become available via the server-sent events protocol.

    Default value: false
    steering_conceptsSteeringConcept (string)[]

    Specifies how the output of the model should be steered. This steers the output in the direction given by positive examples associated to the steering concept and away from the negative examples.

    Possible values: Value must match regular expression ^_worker/[a-zA-Z0-9-_]{1,64}$

    Default value: []

Responses

OK

Schema
    oneOf
    model_versionstring

    model name and version (if any) of the used model for inference

    completions object[]

    list of completions; may contain only one entry if no more are requested (see parameter n)

  • Array [
  • log_probsobjectnullable

    list with a dictionary for each generated token. The dictionary maps the keys' tokens to the respective log probabilities. This field is only returned if requested with the parameter "log_probs".

    completionstring

    generated completion on the basis of the prompt

    raw_completionstringnullable

    For some models, we may optimize the completion that was generated by the model and return the optimized completion in the completion field of the CompletionResponse. The raw completion, if returned, will contain the un-optimized completion. Setting the parameter raw_completion in the CompletionRequest to true forces the raw completion of the model to be returned. Setting tokens to true or log_probs to any value will also trigger the raw completion to be returned.

    completion_tokensstring[]

    completion split into tokens. This field is only returned if requested with the parameter "tokens".

    finish_reasonstringnullable

    reason for termination of generation. This may be a stop sequence or maximum number of tokens reached.

  • ]
  • optimized_prompt object[]

    Describes prompt after optimizations. This field is only returned if the flag disable_optimizations flag is not set and the prompt has actually changed.

  • Array [
  • oneOf
    typestring

    Possible values: [text]

    datastring
  • ]
  • num_tokens_prompt_totalinteger

    Number of tokens combined across all completion tasks.

    In particular, if you set best_of or n to a number larger than 1 then we report the combined prompt token count for all best_of or n tasks.

    Tokenization:

    • Token ID arrays are used as as-is.
    • Text prompt items are tokenized using the tokenizers specific to the model.
    • Each image is converted into a fixed amount of tokens that depends on the chosen model.
    num_tokens_generatedinteger

    Number of tokens combined across all completion tasks. If multiple completions are returned or best_of is set to a value greater than 1 then this value contains the combined generated token count.

Authorization: http

name: tokentype: httpscheme: bearerdescription: Can be generated in your [Aleph Alpha profile](https://app.aleph-alpha.com/profile)
var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Post, "https://api.aleph-alpha.com/complete");
request.Headers.Add("Accept", "application/json");
request.Headers.Add("Authorization", "Bearer <token>");
var content = new StringContent("{\n \"model\": \"llama-3.1-8b-instruct\",\n \"prompt\": \"An apple a day\"\n}", null, "application/json");
request.Content = content;
var response = await client.SendAsync(request);
response.EnsureSuccessStatusCode();
Console.WriteLine(await response.Content.ReadAsStringAsync());
Request Collapse all
Base URL
https://api.aleph-alpha.com
Auth
Parameters
— query
Body required
{
  "model": "llama-3.1-8b-instruct",
  "prompt": "An apple a day"
}
ResponseClear

Click the Send API Request button above and see the response here!