Skip to content

Using AI Prompts

To effectively use AI tasks in workflows, you must provide clear and structured instructions called prompts that define the model’s behavior and output. Orkes Conductor allows you to create, manage, and securely share AI prompts within your organization.

In Orkes Conductor, AI prompts are reusable and can include variables, allowing you to customize the input to AI models based on runtime data. This approach acts like parameterized API calls, where variables in the prompt are replaced with actual values during workflow execution. It ensures flexibility, reusability, and consistency when integrating AI models into complex workflows.

In this guide, you will learn how to create, test, and use AI prompts in your workflows.

Create AI prompts

Prerequisites

  • Integrate the required AI/LLM provider with Orkes Conductor. Refer to the Integration Guides for detailed steps.

Use the Create or Update Prompt API or go to Definitions > AI Prompts, select + Add AI prompt, and enter the following details:

Parameter Description
Prompt Name A name for the prompt.
Model Select the AI models to use with this prompt.

Note: Only models added here can generate responses based on this prompt in a workflow.
Description A description for the prompt.
Prompt Template The prompt, which can be formulated as context, instructions, or questions. The prompt can also contain variables, ${someVariable}, which are replaced with actual values at runtime before it is sent to the AI model.
Example
“What is the current population of ${country}? What was the population in ${year}?”
In the above prompt, ${country} and ${year} are variables that can be dynamically set based on the runtime data.

The following is a sample JSON for an AI prompt:

{
 "name": "population_prompt",
 "integrations": [
   "<YOUR-AI-PROVIDER>:<YOUR-AI-MODEL>"
 ],
 "description": "A prompt to retrieve the population of countries for specific years.",
 "template": "What is the current population of ${country}?\n\nWhat was the population in ${year}?"
}

Manage AI prompt versions

You can create and manage multiple versions of an AI prompt. By default, versioning starts at 1 and increments with each new version.

To create a new version:

Use the Create Prompt Version API with autoIncrement=true, or, after making changes to the AI prompt definition in the Conductor UI, select the ⏷ (down arrow) icon beside Save > Save as new version. Selecting Save without choosing Save as new version overwrites the version you are editing, so workflows that use that version pick up the change.

To view available versions:

Use the Get All Prompt Versions API or select the Version dropdown in the AI prompt editor and choose the required version. To retrieve one version, use the Get Prompt API with the version parameter.

Test AI prompts

After creating an AI prompt, test it to ensure it produces the expected results.

Test parameters

Use the Test Prompts API or open the AI prompt in the Conductor UI and define the following parameters in the testing interface:

Parameter Description
Select model to test Select an AI model for testing. Only the models added to the prompt configuration get listed here.
Temperature Set the temperature based on your requirements. Temperature is a parameter that controls the randomness of the model’s output.
  • Higher temperatures, such as 1.0, produce random and more creative responses. Useful in generating creative content.
  • Lower temperature produces a more stable and focused response. Useful in situations like text classification, where consistency is important.
TopP Set the topP parameter based on your requirements. TopP is a parameter to control the randomness of the model’s output. This parameter defines a probability threshold and then chooses tokens whose cumulative probability exceeds this threshold.
Stop words Enter one or more stop sequences. The model stops generating text when it produces any configured sequence. For example, END stops generation when the model produces that sequence.
Token limit The maximum number of tokens to be generated by the LLM and returned as part of the result. A token is approximately four characters.

Example of using the TopP parameter

Suppose you want to complete the sentence: “She walked into the room and saw a __.” The model considers the following top four words based on their probabilities:

  • Cat - 35%
  • Dog - 25%
  • Book - 15%
  • Chair - 10%

If you set the topP parameter to 0.70, the model selects tokens until their cumulative probability reaches or exceeds 70%. Here's how it works:

  • Adding "Cat" (35%) to the cumulative probability.
  • Adding "Dog" (25%) to the cumulative probability, totaling 60%.
  • Adding "Book" (15%) to the cumulative probability, now at 75%.

At this point, the cumulative probability is 75%, exceeding the specified topP value of 70%. Therefore, the model will randomly select one of the tokens from the list of "Cat," "Dog," and "Book" to complete the sentence because these tokens collectively account for approximately 75% of the probability.

To test an AI prompt:

  1. Define the variables in the testing interface.
  2. Select Test.
  3. Refine the prompt and test it again. When the results align with your expectations, save the prompt.

The AI prompt is now ready for use in workflows.

Use AI prompts in workflows

AI prompts can be used in workflows with the following LLM tasks:

To use a saved AI prompt, add one of these tasks to your workflow using the Create Workflow Definition API or from Definitions > Workflow in the Conductor UI, and configure the following parameters:

Parameter Description
Prompt Name The name of the saved AI prompt to use.
Prompt variables Key-value pairs that fill the variables in the prompt. Each key must match a variable name in the prompt template, such as country or year. Values can be static or mapped to workflow data.
Prompt version The version of the saved AI prompt to use. If not set, the task uses the latest version. Optional.
LLM provider and Model The AI provider and model that runs the prompt. Select a model that is associated with the prompt.
Temperature, TopP, Token limit, and Stop words Optional settings that control the model's output, as described in Test parameters.

The following is a sample LLM Text Complete task that uses the population_prompt prompt:

{
  "name": "llm_text_complete",
  "taskReferenceName": "population_ref",
  "type": "LLM_TEXT_COMPLETE",
  "inputParameters": {
    "llmProvider": "<YOUR-AI-PROVIDER>",
    "model": "<YOUR-AI-MODEL>",
    "promptName": "population_prompt",
    "promptVariables": {
      "country": "${workflow.input.country}",
      "year": "${workflow.input.year}"
    },
    "temperature": 0.1,
    "maxTokens": 150
  }
}

At runtime, Conductor replaces ${country} and ${year} in the prompt with the mapped values, sends the result to the model, and returns the response in the task output as result. Refer to the LLM Text Complete and LLM Chat Complete pages for all task parameters.