An educational chatbot trained to answer your questions about exhibits at the Exploratorium, a wondrous “learning lab” in San Francisco

1. Summary
2. Project details
   1. Code
   2. Data
      1. Knowledgebase
      2. Training data
   3. Modeling
      1. NLU and dialog management
      2. Custom actions
   4. Deployment

Summary

Explorer AI is a simple web-based chatbot which is trained to function as a virtual docent to the Exploratorium. Broadly speaking, its purpose is to answer basic questions that a human docent at the Exploratorium might be asked by a patron. In particular, Explorer AI can recognize the following categories of user intents:

1. Get a basic explanation for a concept or phenomenon which is illustrated by an exhibit
2. Get basic information about a specific exhibit:
   • A brief description of the exhibit
   • Current location of the exhibit (e.g. the gallery in which it’s located)
   • Names of the exhibit’s creator(s)
   • Year the exhibit opened at the Exploratorium
3. Get exhibit recommendations (randomly, or based on another exhibit)

In order to respond to exhibit-related user intents, Explorer AI references a database containing content scraped from exploratorium.edu, as well as AI-curated summaries of this content. To effectively explain a concept, Explorer AI essentially “looks it up” in the encyclopedia (using a process including lightning fast Faiss ranking, requests via the Encyclopedia Britannica API, and natural language processing). Explorer AI’s model is built on Rasa, an open-source framework for conversation-driven development in Python. The application was deployed on a Google Cloud virtual machine.

Project details

Code

The source code for this project is available on GitHub.

Data

Knowledgebase

Explorer AI’s knowledgebase data (which it draws upon when constructing many of its responses) consists of both institutional data (facts about each of the museum’s exhibits and galleries), as well as subject-matter data (general information about the concepts and phenomena illustrated by the exhibits).

The institutional data is built through a process that begins with web-scraping (I used the Scrapy framework) to collect data from exploratorium.edu related to the Exploratorium’s exhibits and galleries. After thoroughly processing this data to extract and encode exhibit location and byline details, the summary/description field for each exhibit is obtained through a method which leverages scraped data and clever prompt engineering to construct API requests to Open AI’s GPT-3. The resulting collection of of data is stored in a Pandas dataframe object, which persists in the bot’s memory.

The subject matter data consists of articles published by the Encyclopedia Britannica, and it is obtained via their API client. In order to keep the model lightweight (in one sense), article contents do not persist in Explorer AI’s memory. Rather, the bot stores a lookup table of article titles by article ID, and performs API calls in real-time in order to fetch data relevant to a user’s request (fetching the most appropriate article given the user’s query is a nice challenge; more on this process in the modeling section below). It then processes the article’s XML contents and and constructs a request to the user. All in a matter of seconds!

Training data

In addition to the knowledgebase that Explorer AI must maintain (either in memory or online), a good deal of conversational training data is also required in order to teach the bot how to interact with the user so that the conversation goes well (along a so-called happy path). This data, manually constructed, is organized into the following categories:

• A “domain” consisting of a sort of check list of everything the bot needs to keep track of
• Many examples of each type of user intent, annotated with any relevant entities that should be recognized
• A collection of if-then rules and sample conversation “stories” to train the bot’s next-action classifier

To adhere with Rasa’s training data formatting conventions, all of the conversation-relevant training data is serialized in YAML files located at prescribed locations inside the main Rasa project directory.

Modeling

NLU and dialog management

Explorer AI’s conversational model is multi-faceted. In order to manage its end of the dialog flow, the agent must (1) understand the user’s message, and (2) decide what to do in response. There are multiple classifiers involved in even the simplest examples of this process:

• An intent classifier assigns the most appropriate category to a user’s message
• An entity extractor detects important information in a user’s message
• An action classifier decides the most appropriate course of action at each conversation turn
• A fallback classifier determines when the model doesn’t have enough information to confidently classify the user’s intent or the next action to take

The NLU (natural language understanding) components of the bot’s model (i.e. intents and entities) are trained with Rasa’s DIET classifier, a state-of-the-art multi-task transformer architecture which provides plug-and-play support for features like pre-trained word embeddings and custom tokenization methods.

For dialog management (action classification and fallback behavior), the model relies on a collection of “policies”, which include strategies ranging from straightforward if-then rules to the implementation of transformer-based algorithms that learn patterns in conversation history and/or training data; see Rasa’s TED policy for details.

Custom actions

Each of Explorer AI’s responses is defined by what Rasa calls an “action”. Some actions have default definitions (e.g. uttering a canned response, like responding to a greeting), but other actions must be carefully customized to achieve specialized goals. One such action is implemented by Explorer AI’s model each time the user requests an explanation for a concept. Since subject-matter data is not stored, the custom action followed by the model in this case contains each of the following steps:

• Use Faiss-indexing to quickly rank the list of Encyclopedia Britannica articles titles by their relevance to user’s query
• Perform a lookup to obtain article IDs corresponding to the top-ranked article titles
• Iteratively process the list of top articles, ordered by their ranking:
  • Use Encyclopedia Britannica’s API client to obtain XML content for the article
  • Parse XML to extract data relevant to user’s request, then perform NLP tasks to transform data into a meaningful response
  • If the user confirms that the provided response answers their question, exit the loop.

Deployment

For consistency in the deployment experience, Explorer AI was packaged into a Docker network which could be composed inside just about any local or remote environment. The application consisted of four interrelated services:

1. A backend server, which performs language understanding and prediction tasks. This is where the Rasa model, aka the chatbot’s brain, lives.
2. An action server, which defines and performs all custom actions the chatbot must perform during the conversation. This service (which also stores Explorer AI’s institutional knowledgebase and instructions for subject matter recall) is frequently consulted by the backend server for help with dialog management.
3. An Nginx web server, which is configured to serve static (HTML) content, as well as proxy all communication between the client and backend service via a socket.io channel. The web server is also configured for SSL encryption.
4. A process for automating the renewal of SSL certificates using Certbot.

At the time of development, the service was deployed to run on a Google Cloud E2 Ubuntu 16.04 virtual machine, accessible from any web browser. However, due to the high cost of cloud computing, Explorer AI was taken offline indefinitely.