Amazon Bedrock - Fundamental
Back
Amazon Bedrock
- fully managed, serverless service
- available to base Foundation Models through an API.
Invoek Model
Inference Parameters
Parameters
- used to control the response generated by the model
-
Common Parameters
- Randomness and diversity
Temperature:
- Controls randomness.
- Lower values:
- close to 0
- make the model more deterministic and focused, by selecting the higher-probability words.
- higher values:
- close to 1
- make it more creative or chaotic, by selecting a lower-probability word
Top-K:
- Limits token selection to the most likely next tokens
- Cutoff where the model no longer selects the words.
Top-P (Nucleus Sampling):
- Limits token selection to a subset whose cumulative probability reaches
- Caps choices based on the sum of their probabilities.
Frequency Penalty
- Penalizes tokens based on how many times they have already appeared.
- Higher numbers reduce repetitive phrasing.
Presence Penalty
- Penalizes tokens based on whether they have appeared at all.
- Higher numbers encourage the model to talk about new topics.
- Length
Length:
- Controls the length of the generated response.
- Helps control cost.
Max Tokens:
- Specify the maximum number of tokens to use in the generated response.
Stop Sequences:
- A string that tells the model to stop generating immediately when it hits a specific character or word.
- Model-decided stop (end-of-sequence token) – Model training
- System-enforced stop (stop sequences or max tokens)
- After a stop sequence, the model stops generating further tokens.
- The returned text doesn’t contain the stop sequence.
{"firstName":"John", "lastName":"Doe"}