SOFTMAX FUNCTION IN A SELF-ATTENTION MECHANISM
Main Article Content
Abstract
A machine translation technique called neural machine translation (NMT) uses artificial neural networks to predict the probability of word sequences. Typically, it incorporates all sentence models into one integrated model. This is the dominant approach today[1, 2] and under certain conditions can produce translations that compete with human translations when translating between languages with few resources[3]. However, especially in languages where high-quality data is less available[1, 4, 5] and there are still problems with domain switching between the data the system is trained on and the text to be translated. available[1]. NMT systems also tend to produce literal translations[5].
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
References
Koehn, Philipp (2020). Neural Machine Translation. Cambridge University Press.
Stahlberg, Felix (2020-09-29). “Neural Machine Translation: A Review and Survey”.
M.Popel, M.Tomkova, J.Tomek, L.Kaiser, J.Uszkoreit, O.Bojar, Z.Zabokrtsky,(2020-09-01). "Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals". Nature Communications. 11 (1): 4381. ISSN 2041-1723.
B.Haddow, R.Bawden, B.Miceli, V.Antonio, J.Helcl, A.Birch(2022). "Survey of Low-Resource Machine Translation". Computational Linguistics. 48 (3): 673–732.
T.Poibeau, N.Calzolari, F.Bechet, P.Blache, K.Choukri, Ch.Cieri, T.Declerck, S.Goggi, H.Isahara, B.Maegaard. "On "Human Parity" and "Super Human Performance" in Machine Translation Evaluation". Proceedings of the Thirteenth Language Resources and Evaluation Conference. Marseille, France: European Language Resources Association: 6018–6023.
T.Kocmi, R.Bawden, O.Bojar, A.Dvorkovich, Ch.Federmann, M.Fishel, T.Gowda, Y.Graham, R.Grundkiewicz, B.Haddow, R.Knowles, P.Koehn, Ch.Monz, M.Morishita, M.Nagata, L.Barrault, O.Bojar, F.Bougares, R.Chatterjee, M.Costa-jussa, Ch.Federmann, M.Fishel, A.Fraser. Findings of the 2022 Conference on Machine Translation (WMT22). Proceedings of the Seventh Conference on Machine Translation (WMT). Abu Dhabi, United Arab Emirates (Hybrid): Association for Computational Linguistics. pp. 1–45.
T.Kocmi, E.Avramidis, R.Bawden, O.Bojar, A.Dvorkovich, Ch.Federmann, M.Fishel, M.Freitag, T.Gowda, R.Grundkiewicz, B.Haddow, P.Koehn, B.Marie, Ch.Monz, M.Morishita. Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet. Proceedings of the Eighth Conference on Machine Translation. Singapore: Association for Computational Linguistics. pp. 1–42.
Pavan Belagatti. Understanding the Softmax Activation Function: A Comprehensive Guide/ Intelligent applications, now at a lakehouse near you. Watch the product launch on demand. March, 2024
Hunter Philips. A Simple Introduction to Softmax. / Medium.com. May 2023.