TY - JOUR
AU - Hovy, Eduard
AB - In this work, we introduce a novel local au- toregressive translation (LAT) mechanism into non-autoregressive translation (NAT) models so as to capture local dependencies among tar- get outputs. Specifically, for each target decod- ing position, instead of only one token, we pre- dict a short sequence of tokens in an autore- gressive way. We further design an efficient merging algorithm to align and merge the out- put pieces into one final output sequence. We Figure 1: An example of the LAT mechanism. For each integrate LAT into the conditional masked lan- decoding position, a short sequence of tokens is gener- guage model (CMLM; Ghazvininejad et al., ated in an autoregressive way. hsopi is the special start- 2019) and similarly adopt iterative decod- of-piece symbol. ‘pos*’ denotes the hidden state from ing. Empirical results on five translation tasks the decoder at that position. show that compared with CMLM, our method achieves comparable or better performance incomplete translation (Wang et al., 2019). There with fewer decoding iterations, bringing a 2.5x speedup. Further analysis indicates that our have been various methods in previous work (Stern method reduces repeated translations and per- et al., 2019; Gu et al., 2019; Ma et al., 2018; 
TI - Incorporating a Local Translation Mechanism into Non-autoregressive Translation
JF - Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
DO - 10.18653/v1/2020.emnlp-main.79
DA - 2020-01-01
UR - https://www.deepdyve.com/lp/unpaywall/incorporating-a-local-translation-mechanism-into-non-autoregressive-9rWDadVuk7
DP - DeepDyve
ER -