MiniSeek, a simplified version of the production methods used by Seek AI, is the first model to break 90% accuracy on the Spider Leaderboard. MiniSeek achieved 91.2% on execution accuracy and is also the leader for Exact Set Match without Values with 80.3% on Dev and 81.5% on Test.
Spider is a “large-scale complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 Yale students.” As Seek is working towards a natural language interface to databases (NLID), this is a significant milestone showing that the methods used by Seek AI are state-of-the-art.
You can learn more about the Seek AI Labs team at seek.ai/research. If you are interested in our mission to build a NLID, join us!
