Speech to Visualization using Transformer based Deep Learning Model

Patwardhan, Vishal Gajanan (2022) Speech to Visualization using Transformer based Deep Learning Model. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration manual) Download (1MB) | Preview

Abstract

Nowadays, so many studies are being carried out to ease the use of the systems to make them touchless. In this research, a speech-driven system called Speech-to-Visualization draws the visualizations to gather insights from the specified database by speaking natural language questions with the chart's information. To support this research, the nvBench dataset supports the natural language to visualization tasks. Using the Google speech recognizer, speech is converted to text and processing it to train the transformer-based Seq2Seq model with the forcing of attention and SQL-Aware translation module to help predict the SQL query components (select column, aggregate function, aggregate column, table name, where condition, order by column, order by type, group by column and limit value) and form the SQL query and also the information of chart types like a bar graph, scatterplot, line graph and pie. The predicted query is then executed on the SQLite3 database to get the desired structured data, i.e., rows and columns and transformed to feed to the matplotlib library of python’s visualization library to draw the chart. The quantitative evaluation shows that the Seq2Seq model with the forcing of attention achieves the overall accuracy of 77% using the nvBench dataset.

Item Type:	Thesis (Masters)
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Tamara Malone
Date Deposited:	28 Feb 2023 17:18
Last Modified:	01 Mar 2023 17:49
URI:	https://norma.ncirl.ie/id/eprint/6258

Actions (login required)

View Item