Traditional NLU Can Be Leveraged By Following A Hybrid NLU & NLG Approach

I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language.  Including NLU design, evaluation & optimisation.  Data-centric prompt tuning and LLM observability, evaluation and fine-tuning.

This article considers how Foundation LLMs can be used to leverage existing NLU investments, and improve chatbot experiences.

Considering the Conversational AI landscape and the inroads LLMs are making, there has been a few market phenomenons:

  1. Traditional NLU based chatbot frameworks are adopting Foundation LLM functionality. But mostly in an unimaginative way to generate intent training examples, bot response fallback dialogs or rewriting bot messages. The only exception here is Cognigy, and to some degree Yellow AI.
  2. LLMs are mostly being used in a generative capacity, and not in conjunction with existing predictive capability.
  3. The predictive power of traditional NLU engines with regard to intent classification should not be overlooked.
  4. Hybrid NLU and LLM based NLG implementations are not receiving the consideration in deserves.
Leverage the best of NLU, which is intent detection, with the most accessible LLM feature, which is response generation.

This article considers the following:

  • Increasing the granularity and sophistication of existing inbound user utterance NLU models, by complimenting it with NLG.
  • Using human supervision to monitor generative responses in the same way intents are monitored and maintained.
  • Engineered and curated LLM prompts can be used to create organisation specific fine-tuned LLM models.

Recently Stephen Broadhurst and Gregory Whiteside demonstrated an effective hybrid NLU/NLG approach which combines traditional intent-based logic with dynamic LLM responses.

When it comes to supervision hints, there are three key principles to keep in mind:

  1. Inbound supervision is typically more efficient than outbound supervision.
  2. Light supervision can be a great way to create bot messages that are both responsive and contextual.
  3. Highly directive guidance, while reliable, can lead to more restricted replies.

Consider the following scenario…

When a user utters a statement, it is usually intercepted by a chatbot (1).

After the statement is intercepted, the intent of the statement is retrieved (2).

Then, example prompts or hints associated with the intent are retrieved (3).

These hints help the Large Language Model (LLM) generate accurate responses without any hallucination (4).

In order to keep the hints and prompts up to date, they can be reviewed and maintained, though not in real-time (5).

The hints are sent to the LLM with relevant entities injected via templating, a process known as prompt engineering (6).

The LLM generates a response (7) and sends it back to the user.

The NLU/NLG trainers review the generative results and update the prompt templates daily, in order to give better hints and templating (8).

Below is a detailed diagram of a supervised generative bot process flow:

In Conclusion

It is clear that the successful and established architecture of chatbots and the advanced capabilities of NLU engines should be utilised through a hybrid approach.

This is because companies have consistently demonstrated a tendency to converge around the most effective ideas, as evidenced by the current chatbot landscape.

I’m currently the Chief Evangelist @ HumanFirst. I explore and write about all things at the intersection of AI and language.  Including NLU design, evaluation & optimisation.  Data-centric prompt tuning and LLM observability, evaluation and fine-tuning.