Answers have nowhere to hide

Answers are the true masters of hide-and-seek. No matter how certain you are about their whereabouts, all too often you're left frustrated, wishing you didn't have to waste so much time hunting them down.

In fact, it's estimated we spend 1.8 hours per day (more than 1 work-day per week) searching for information! The scary thing is this projection is from 2012, before cloud tools fragmented our work (and knowledge) all over the place.

Thankfully the tables are about to turn. We (the knowledge workers of the world) will soon be able to ask questions on any topic and get answers back — not links to where they may be! This ability will finally give us the long promised "ambient intelligence" that makes us significantly more productive and effective in our jobs.

The technology enabling this "superpower" is referred to as open domain question and answering. Open domain Q/A has long been a focus within the natural language processing research community. However, it wasn't until Google Research introduced transformer-based deep neural network language models in mid-2017 that significant progress towards human level performance took hold.

Today, Google and IBM have already commercialized this technology in the consumer and enterprise domains respectively. Over the next 3-4 years, it will become ubiquitous in any search tool we use. But let's take a step back and look at how this important AI powered capability has evolved.

Corporate AI research teams mature open domain Q/A

Open domain question and answering allows knowledge workers to find answers to questions on any topic. This is a particularly important NLP research area because it represents a core sign of general intelligence. You can think of this capability as being similar to the reading comprehension skills we teach students. Students are measured by how effectively they can comprehend a text and answer questions related to it. Open domain Q/A is not much different. NLP models are measured by how effectively they can surface answers to questions found in text and also identify when answers aren't present.

Google runs the leading competition measuring open domain question and answering performance. They've measured the upper bounds of human performance for finding the correct short answers in the competition data at 76%. Today, state-of-the-art deep learning models achieve 64%. What's remarkable though is that performance improved 120% over the first year of the competition, going from 29% to 64%.

Note: Precision is the proportion of answers correctly found. Recall is the proportion of answers correctly identified as not existing in the text. The F1 score is the harmonic mean of both recall and precision.

The practical impact of this research is the ability to get answers to ‘where’, ‘when’, and ‘what’ type questions. You can think of these as fact based questions.

Examples include:

• A Systems Engineer conducting a quality review of a requirements specification
  Q: What is the European loading gauge?
  A: 3,150 mm (10 ft 4 in) wide by 3,175 mm (10 ft 5.0 in) rising to 4,280 mm (14 ft 1 in) in the center
• A customer asking a virtual agent a technical question
  Q: What versions of Firefox does InfoSphere Information Server 1.3 support?
  A: ESR 17 and 24
• A Mortgage Broker verifying a loan application
  Q: Where is the property located?
  A: Lot 5-A, Block B, RESUBDIVISION OF LOT 5 BLOCK B METRO CENTER SECTION6, a subdivision in the City of Austin, Travis County, Texas.

Google brings short answer search to consumers

The examples above may seem similar to search results Google provides today. This is because they are. If you search for Joe Biden's age, you'll get the short, specific answer back of 78 years old. You don't even need to submit the search!

Google can even surface answers to more complex questions by referencing public domain data it indexes. In the example below you see the correct response pulled from IBM's support documentation.

IBM brings short answer search to businesses

Not all information is publicly available, indexed, and searchable through Google though. Most corporate data remains within a company and is used by the employees of that company. Corporate knowledge is also spread across chats, knowledge bases, wikis, blogs, work orders, etc., with each company implementing some different combination of tools.

IBM's Reading Comprehension feature within its Watson Discovery AI-powered search and text analytics platform is the first enterprise-centric solution to incorporate open domain question and answering into an enterprise offering.

Imagine you're a customer support rep and a customer asks how long they have to file a claim on a defective product. Today you'd either rely on knowledge gained through experience handling this type of call before or you'll have to search through a knowledge base looking for the answer, which happens after you put the customer on a "brief" hold.

With Reading Comprehension, you'd immediately get the answer —"within sixty (60) days of failure." Better still, the customer could have asked this question to a virtual agent and received the same answer without ever having look through a knowledge base or scourer a webpage before placing the call!

Open domain question and answering is a significant AI advancement that will improve the productivity of knowledge workers. With AI eliminating the cost of finding answers (i.e. worker time), we'll no longer waste time finding answers to fact based questions. Instead, we'll have the confidence to ask more questions – thus enhancing our experience, knowledge, and productivity.

Note: I work for IBM but these thoughts are my own.