Two New Challenging Resources to Evaluate Natural Language Interfaces to Databases Generated Based on Geobase and Geoquery

Abstract

Databases and corpora are essential resources to evaluate the performance of Natural Language Interfaces to Databases (NLIDB). The Geobase database and the Geoquery corpus (Geoquery250 and Geoquery880) are among the most commonly used. In this chapter, the authors analyze both resources to offer two elaborate resources: 1) N-Geobase, which is a relational database, and 2) the corpus Geoquery270. The former follows the standard normalization procedure, then N-Geobase has a schema similar to enterprise databases. Geoquery270 consists of 270 queries selected from Geoquery880, preserving the same kind of natural language problems as Geoquery880, but with more challenging issues for an NLIDB than Geoquery250. To evaluate the new resources, they compared the performance of the NLIDB using Geoquery270 and Geoquery250. The results indicated that Geoquery270 was the harder corpus, while Geoquery250 is the easier one. Consequently, this chapter offers a broader range of resources to NLIDB designers.