Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/springer_journal/text-normalization-with-convolutional-neural-networks-hc3vtlEqu2
References (49)
-
Daniel Weber, D. Zhekova (2016)
TweetNorm: Text Normalization on Italian Twitter Data -
R. Sproat, Keith Hall (2014)
Applications of maximum entropy rankers to problems in spoken language processing -
Christian Szegedy, Wei Liu, Yangqing Jia, P. Sermanet, Scott Reed, Dragomir Anguelov, D. Erhan, Vincent Vanhoucke, Andrew Rabinovich (2014)
Going deeper with convolutions2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
-
J. Zilly, R. Srivastava, J. Koutník, J. Schmidhuber (2016)
Recurrent Highway Networks -
Jonathan Allen, M. Hunnicut, K. Klatt (1987)
Text-to-speech: the mitalk system -
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann Dauphin (2017)
Convolutional Sequence to Sequence LearningArXiv, abs/1705.03122
-
D. García-Pérez, Juan Castillo, Yahya Al-Hazmi, Josep Martrat, K. Kavoussanakis, Alastair Hume, Celia López, G. Landi, T. Wauters, M. Gienger, D. Margery (2014)
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) -
(2016)
Keras : Theano - based deep learning library -
Holger Schwenk, Loïc Barrault, Alexis Conneau, Yann LeCun (2016)
Very Deep Convolutional Networks for Text Classification -
Joachim Daiber, R. Goot (2016)
The Denoised Web Treebank: Evaluating Dependency Parsing under Noisy Input Conditions -
W Shang, J Chiu (2017)
Exploring normalization in deep residual networks with concatenated rectified linear units batch normalization in ResNetsProceedings of the 31th Conference on Artificial Intelligence (AAAI 2017), 1
-
Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermüller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, A. Belikov, A. Belopolsky, Yoshua Bengio, Arnaud Bergeron, J. Bergstra, Valentin Bisson, Josh Snyder, Nicolas Bouchard, Nicolas Boulanger-Lewandowski, Xavier Bouthillier, A. Brébisson, Olivier Breuleux, P. Carrier, Kyunghyun Cho, J. Chorowski, P. Christiano, Tim Cooijmans, Marc-Alexandre Côté, Myriam Côté, Aaron Courville, Yann Dauphin, Olivier Delalleau, Julien Demouth, Guillaume Desjardins, S. Dieleman, Laurent Dinh, Mélanie Ducoffe, Vincent Dumoulin, S. Kahou, D. Erhan, Ziye Fan, Orhan Firat, M. Germain, Xavier Glorot, I. Goodfellow, M. Graham, Çaglar Gülçehre, P. Hamel, Iban Harlouchet, J. Heng, Balázs Hidasi, S. Honari, Arjun Jain, Sébastien Jean, Kai Jia, Mikhail Korobov, Vivek Kulkarni, Alex Lamb, Pascal Lamblin, Eric Larsen, César Laurent, S. Lee, S. Lefrançois, S. Lemieux, Nicholas Léonard, Zhouhan Lin, J. Livezey, C. Lorenz, J. Lowin, Qianli Ma, Pierre-Antoine Manzagol, Olivier Mastropietro, R. McGibbon, R. Memisevic, Bart Merrienboer, Vincent Michalski, Mehdi Mirza, A. Orlandi, C. Pal, Razvan Pascanu, M. Pezeshki, Colin Raffel, D. Renshaw, M. Rocklin, Adriana Romero, Markus Roth, Peter Sadowski, J. Salvatier, F. Savard, Jan Schlüter, J. Schulman, Gabriel Schwartz, Iulian Serban, Dmitriy Serdyuk, S. Shabanian, Étienne Simon, Sigurd Spieckermann, S. Subramanyam, Jakub Sygnowski, Jérémie Tanguay, Gijs Tulder, Joseph Turian, S. Urban, Pascal Vincent, Francesco Visin, Harm Vries, David Warde-Farley, Dustin Webb, M. Willson, Kelvin Xu, Lijun Xue, Li Yao, Saizheng Zhang, Ying Zhang (2016)
Theano: A Python framework for fast computation of mathematical expressionsArXiv, abs/1605.02688
-
AiTi Aw, Min Zhang, Juan Xiao, Jian Su (2006)
A Phrase-Based Statistical Model for SMS Text NormalizationProceedings of the COLING/ACL on Main conference poster sessions -
-
J Allen, MS Hunnicutt, D Klatt (1987)
From text to speech—The MITalk system -
Klaus Greff, R. Srivastava, J. Schmidhuber (2016)
Highway and Residual Networks learn Unrolled Iterative EstimationArXiv, abs/1612.07771
-
A. Mousa, Björn Schuller (2016)
Deep Bidirectional Long Short-Term Memory Recurrent Neural Networks for Grapheme-to-Phoneme Conversion Utilizing Complex Many-to-Many Alignments -
Jonas Gehring, Michael Auli, David Grangier, Yann Dauphin (2016)
A Convolutional Encoder Model for Neural Machine Translation -
M. Sundermeyer, Tamer Alkhouli, Joern Wuebker, H. Ney (2014)
Translation Modeling with Bidirectional Recurrent Neural Networks -
V. Sridhar (2015)
Unsupervised Topic Modeling for Short Texts Using Distributed Representations of Words -
Sergey Ioffe, Christian Szegedy (2015)
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftArXiv, abs/1502.03167
-
Ilya Sutskever, Oriol Vinyals, Quoc Le (2014)
Sequence to Sequence Learning with Neural NetworksArXiv, abs/1409.3215
-
J. Elman (2004)
Distributed representations, simple recurrent networks, and grammatical structureMachine Learning, 7
-
G. Pundak, Tara Sainath (2017)
Highway-LSTM and Recurrent Highway Networks for Speech Recognition -
Yoav Goldberg (2015)
A Primer on Neural Network Models for Natural Language ProcessingArXiv, abs/1510.00726
-
Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, Gang Wang, Jianfei Cai, Tsuhan Chen (2015)
Recent advances in convolutional neural networksArXiv, abs/1512.07108
-
V. Sridhar (2015)
Unsupervised Text Normalization Using Distributed Representations of Words and Phrases -
(2015)
Training very deep networks training very deep networks. In NIPS’15 Proceedings of the 28th international conference on neural information processing systems (pp 2377–2385) -
(2017)
Recent trends in deep learning based natural language processing. arXiv: preprint arXiv:1708.02709v4 -
Catherine Kobus, François Yvon, Géraldine Damnati (2008)
Normalizing SMS: are Two Metaphors Better than One ? -
Ramón Astudillo, Silvio Amir, Wang Ling, Mário Silva, I. Trancoso (2015)
Learning Word Representations from Scarce and Noisy Data with Embedding Subspaces -
R. Sun, C. Giles (2001)
Sequence learning: from recognition and prediction to sequential decision makingIEEE Intelligent Systems, 16
-
P. Zhou, Zhenyu Qi, Suncong Zheng, Jiaming Xu, Hongyun Bao, Bo Xu (2016)
Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max PoolingArXiv, abs/1611.06639
-
Cagil Sonmez, Arzucan Özgür (2014)
A Graph-based Approach for Contextual Text Normalization -
R. Sproat, N. Jaitly (2016)
RNN Approaches to Text Normalization: A ChallengeArXiv, abs/1611.00068
-
Vinod Nair, Geoffrey Hinton (2010)
Rectified Linear Units Improve Restricted Boltzmann Machines -
Diederik Kingma, Jimmy Ba (2014)
Adam: A Method for Stochastic OptimizationCoRR, abs/1412.6980
-
Wenling Shang, Justin Chiu, Kihyuk Sohn (2017)
Exploring Normalization in Deep Residual Networks with Concatenated Rectified Linear Units -
Tyler Baldwin, Yunyao Li (2015)
An In-depth Analysis of the Effect of Text Normalization in Social Media -
Sercan Arik, Mike Chrzanowski, Adam Coates, G. Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, Shubho Sengupta, M. Shoeybi (2017)
Deep Voice: Real-time Neural Text-to-SpeechArXiv, abs/1702.07825
-
Brian Roark, R. Sproat, Cyril Allauzen, M. Riley, Jeffrey Sorensen, Terry Tai (2012)
The OpenGrm open-source finite-state grammar software libraries -
Liang Lu, Xingxing Zhang, S. Renals (2016)
On training the recurrent neural network encoder-decoder for large vocabulary end-to-end speech recognition2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
-
Cyril Allauzen, M. Riley, Brian Roark (2016)
Distributed representation and estimation of WFST-based n-gram models -
Xiang Zhang, J. Zhao, Yann LeCun (2015)
Character-level Convolutional Networks for Text Classification -
R. Sproat, A. Black, Stanley Chen, Shankar Kumar, Mari Ostendorf, Christopher Richards (2001)
Normalization of non-standard wordsComput. Speech Lang., 15
-
Tomas Mikolov, Kai Chen, G. Corrado, J. Dean (2013)
Efficient Estimation of Word Representations in Vector Space -
Paul Cook, S. Stevenson (2009)
An Unsupervised Model for Text Message Normalization -
Alex Graves, Santiago Fernández, F. Gomez, J. Schmidhuber (2006)
Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networksProceedings of the 23rd international conference on Machine learning
-
Congle Zhang, Tyler Baldwin, C. Ho, B. Kimelfeld, Yunyao Li (2013)
Adaptive Parser-Centric Text Normalization -
Marina Sokolova, G. Lapalme (2009)
A systematic analysis of performance measures for classification tasksInf. Process. Manag., 45
- Publisher
- Springer Journals
- Copyright
- Copyright © 2018 by Springer Science+Business Media, LLC, part of Springer Nature
- Subject
- Engineering; Signal,Image and Speech Processing; Social Sciences, general; Artificial Intelligence (incl. Robotics)
- ISSN
- 1381-2416
- eISSN
- 1572-8110
- DOI
- 10.1007/s10772-018-9521-x
- Publisher site
- See Article on Publisher Site
Abstract
Text normalization is a critical step in the variety of tasks involving speech and language technologies. It is one of the vital components of natural language processing, text-to-speech synthesis and automatic speech recognition. Convolutional neural networks (CNNs) have proven their superior performance to recurrent architectures in various application scenarios, like neural machine translation, however their ability in text normalization was not exploited yet. In this paper we investigate and propose a novel CNNs based text normalization method. Training, inference times, accuracy, precision, recall, and F1-score were evaluated on an open-source dataset. The performance of CNNs is evaluated and compared with a variety of different long short-term memory (LSTM) and Bi-LSTM architectures with the same dataset.
Journal
International Journal of Speech Technology – Springer Journals
Published: May 30, 2018