ChatGPT is currently powered by gpt-3.5-turbo-0301, the most advanced OpenAI language model. Although OpenAI has made the API for this model accessible, the API does not automatically manage conversations contextually...

GPT-3.5-Turbo-0301 is effective at single-turn tasks, even without any conversational dialog turns.

Therefore, a ChatGPT implementation should be able to keep track of conversation context and logical connections in order to respond appropriately.

The ChatGPT models hold no memory of past requests, all relevant information must be supplied via the conversation.

The ChatML document submitted must include conversational history in order to maintain conversational context and manage dialog state. This allows the model to answer contextual questions by leveraging prior dialog turns.

OpenAI explicitly indicates that their models lack the capacity to retain memory of prior and subsequent queries. Therefore, all necessary information must be provided in the conversation.

Remember, if the conversation exceeds the model's token limit, it must be condensed. This can be done by creating a rolling log that stores the last n dialog turns for re-submission.

Below are a few practical examples. You can enter a sequence of messages and the model will return a text output, as seen below:

pip install openai

import os
import openai
openai.api_key = "xxxxxxxxxxxxxxxxxxxxxxx"

completion = openai.ChatCompletion.create(
  model="gpt-3.5-turbo", 
  messages = [{"role": "system", "content" : "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2023-03-02"},
{"role": "user", "content" : "How are you?"},
{"role": "assistant", "content" : "I am doing well"},
{"role": "user", "content" : "How long does light take to travel from the sun to the eart?"}]
)
print(completion)

The Output:

{
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "It takes about 8 minutes and 20 seconds for light to travel from the sun to the earth.",
        "role": "assistant"
      }
    }
  ],
  "created": 1678039126,
  "id": "chatcmpl-6qmv8IVkCclnlsF5ODqlnx9v9Wm3X",
  "model": "gpt-3.5-turbo-0301",
  "object": "chat.completion",
  "usage": {
    "completion_tokens": 23,
    "prompt_tokens": 89,
    "total_tokens": 112
  }
}

Below is a different question relating to motor sport and the ChatML document submitted to the gpt-3.5-turbo-0301 model:

[{"role": "system", "content" : "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2023-03-02"},
{"role": "user", "content" : "How are you?"},
{"role": "assistant", "content" : "I am doing well"},
{"role": "user", "content" : "When was the last Formula One championship in South Africa?"}]

The correct response given to the last question:

[
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "The last Formula One championship race held in South Africa was on October 17, 1993.",
        "role": "assistant"
      }
    }
  ]

Note, when a follow-up question is asked with no context provided in the ChatML document…as seen below:

[{"role": "system", "content" : "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2023-03-02"},
{"role": "user", "content" : "How are you?"},
{"role": "assistant", "content" : "I am doing well"},
{"role": "user", "content" : "Who won the race in South Africa?"}]

The context is lost and the ChatGPT API states that fact.

{
      "finish_reason": null,
      "index": 0,
      "message": {
        "content": "I'm sorry, but I don't have access to up-to-date information on current events. My training data only goes up until September 2021, and I am not capable of browsing the internet or accessing real-time data.",
        "role": "assistant"
      }
    }

What is the correct way to ask a follow-up question, providing the necessary contextual reference to the system?

[{"role": "system", "content" : "You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2023-03-02"},
{"role": "user", "content" : "How are you?"},
{"role": "assistant", "content" : "I am doing well"},
{"role": "user", "content" : "When was the last Formula One championship in South Africa?"},
{"role": "assistant", "content" : "The last Formula One championship race held in South Africa was on October 17, 1993."},
{"role": "user", "content" : "Who won the race in South Africa?"}]

The correct response is given:

  {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "The 1993 South African Grand Prix was won by Alain Prost of France, driving for the Williams-Renault team.",
        "role": "assistant"
      }
    }

Finally

OpenAI states that gpt-3.5-turbo-0301 does not pay strong attention to system messages, so extra focus is needed for instructions in a user message in addition to context management.

If the model's generated output is not satisfactory, it is necessary to try different approaches to improve it.

This could include making the instructions more clear, specifying the answer format, asking the model to go through the steps sequentially (chain-of-thought prompting), and even fine-tuning the base GPT-3 model (GPT-3.5-turbo models are not able to be fine-tuned).

I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language; ranging from LLMs, Chatbots, Voicebots, Development Frameworks, Data-Centric latent spaces and more.