The notion to create workflows (chains) which leverage Large Language Models (LLMs) are necessary and needed. But there are a few considerations, one of which is Prompt Drift.

Chaining, also known as Prompt Chaining, is a way of employing a programming tool (often with a graphical interface) to arrange large language model prompts in an application which often creates a conversational user interface.

The essential element of prompt chaining involves transferring tasks from one chain to another, which is likely to continue for the entirety of the conversation with the user.

Prompt Drift is the process of cascading inaccuracies which can be caused by:

  • Model-inspired tangents,
  • Incorrect problem extraction,
  • LLMs’ randomness and creative surprises
Chaining can act as a safeguard against model-inspired tangents, because each step of the Chain defines a clear goal. ~ Source

The image below shows how a single node or prompt, forming part of a larger chain, can be impacted to produce prompt drift.

  1. The user input can be unexpected or unplanned producing an unforeseen output from the node.
  2. The previous node output can be inaccurate or produce drift which is exacerbated in the current node.
  3. The LLM Response can also be unexpected, due to the fact that LLMs are non-deterministic.

One of the ways to counter prompt drift (error cascading) is to ensure the prompt template used is comprehensive and enough contextual information is supplied to negate LLM hallucination.

In Closing

It is important to not see prompt chaining in isolation, but rather consider Prompt Engineering as a discipline which consists of eight legs, as depicted below.

Source

Prompt Engineering is the foundation of Chaining and the discipline of Prompt Engineering is very simple and accessible.

However, as the LLM landscape develops, prompts are becoming programable and incorporated into more complex structures. These structures should be a combination of available affordances.

Hence chaining should be supported by elements like Agents, Pipelines, Chain-of-Thought Reasoning, etc.

I’m currently the Chief Evangelist @ HumanFirst. I explore & write about all things at the intersection of AI and language. Including NLU design, evaluation & optimisation. Data-centric prompt tuning & LLM observability, evaluation & fine-tuning.