For citation:
Devyatkin D. A., Salimovsky V. A., Chudova N. V., Ryzhova A. A., Grigoriev O. G. Large language models and speech genre systematicity. Speech Genres, 2025, vol. 20, iss. 1 (45), pp. 6-23. DOI: 10.18500/2311-0740-2025-20-1-45-6-23, EDN: FISZEX
Large language models and speech genre systematicity
The paper examines a large language model (LLM) to recognize speech genres. Although artificial neural networks are effectively utilized in many important fields, they, however, have a serious drawback. The mechanism of their functioning is hidden from researchers; therefore, the results of their use do not get explanation. The purpose of the study is to reveal the basic mechanisms of functioning of the linguistic model LLM (Transformer) and thereby ensure the interpretability of the data it provides. The research is based on two genres of academic text: “Description of a new scientific phenomenon” and “Explication of a scientific concept.” We verified a hypothesis according to which the LLM feature set is based on the speech systematicity of the recognized genres. It is also shown that since genre-speech systematicity is determined by extralinguistic factors, primarily the characteristics of human consciousness, its manifestations, reflected in the hidden state of the LLM, can be used to model cognitive processes embodied in speech. We also analyze existing approaches to the interpretation of LLMs and describe the applied method to do it. The paper provides the following linguistic interpretation of LLM training and fine-tuning: preliminary training on large text corpora allows a model to display language resources (a system of linguistic units and general principles of their use) relatively completely, while fine-tuning on samples of a certain genre-speech organization restructures the linguistic systematicity into speech systematicity. During the experiments we decoded the hidden state of the LLM and accurately reproduced the composition and frequency of lexis from the training dataset. The classification score for each of the considered genres by the LLM is F1 0.99, we believe this is because of their speech consistency.
- Abhilasha R., Belinkov Y., Hovy E. Probing the probing paradigm: Does probing accuracy entail task relevance? Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, 2021, main vol., pp. 3363–3377.
- Amini A., Pimentel T., Meister C., Cotterell R. Naturalistic Causal Probing for Morpho-Syntax. Transactions of the ACL, 2023, vol. 11, pp. 384–403. https://doi.org/10.1162/tacl_a_00554
- Arutyunova N. D. Genres of communication. In: Chelovecheskij faktor v jazyke: kommunikatsija, modal’nost’, dejksis: kol. monografiya. Otv. red. T. V. Bulygina [Bulygina T. V., ed. The human factor in language: Communication, modality, deixis: Collective monograph]. Moscow, Nauka, 1992, pp. 52–56 (in Russian).
- Bakhtin M. M. Speech genre problem. In: Bakhtin M. M. Estetika slovesnogo tvorchestva [Aesthetics of verbal creativity]. Moscow, Iskusstvo, 1979, pp. 237– 280 (in Russian).
- Balashova L. V., Dementyev V. V. Russkie rechevye zhanry [Russian speech genres]. Moscow, Publishing House YaSK, 2022. 832 p. (in Russian).
- D’yachenko S. V., Iomdin L. L., Mityushin L. G., Lazurskii A. V., Podlesskaya O. Yu., Sizov V. G., Frolova T. I., Tsinman L. L. A deeply annotated corpus of Russian texts: Contemporary state of affairs (SinTagRus). Proceedings of the V. V. Vinogradov Russian Language Institute, 2015, no. 3 (6), pp. 272–300 (in Russian).
- Dementyev V. V. Teoriya rechevykh zhanrov [Theory of speech genres]. Moscow, Znak, 2010. 600 p. (in Russian).
- Devlin J., Chang M.-W., Lee K., Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, volume 1 (Long and Short Papers), pp. 4171–4186. https://doi.org/10.18653/v1/N19-1423
- Duskayeva L. R. Dialogicheskaya priroda gazetnykh rechevykh zhanrov [The dialogic nature of newspaper speech genres]. Saint Petersburg, Saint Petersburg State University Publ., 2012. 274 p. (in Russian).
- Enikolopov S. N., Medvedeva T. I., Vorontsova O. Yu. Linguistic text characteristics in depression and schizophrenia. Medical Psychology in Russia, 2019, vol. 11, no. 5 (58). Available at: http://mprj.ru/archiv_global/2019_5_58/nomer02.php (accessed February 20, 2024) (in Russian).
- Gausenblas K. Is there a “neutral style”? In: Funktsional’naja stilistika: teorija stilej i ikh jazykovaja realizatsija [Functional stylistics: Theory of styles and their linguistic implementation]. Perm, Perm University Publ., 1986, pp. 19–22 (in Russian).
- Golovin B. N. Osnovy kultury rechi [The basics of speech culture]. Moscow, Vysshaya skola, 1988. 320 p. (in Russian).
- Hewitt J., Ethayarajh K., Liang P., Manning C. D. Conditional probing: Measuring usable information beyond a baseline. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, November, pp. 1626–1639. https://doi.org/10.18653/v1/2021.emnlp-main.122
- Hewitt J., Manning C. D. A structural probe for finding syntax in word representations. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, vol. 1 (Long and Short Papers), pp. 4129–4138.
- Khalizev V. E. Teorija literatury [Literary theory]. Moscow, Vysshaya skola, 2002. 437 p. (in Russian).
- Kostomarov V. G. Nash jazyk v dejstvii [Our language in action]. Moscow, Gardariki, 2005. 287 p. (in Russian).
- Kozhina M. N. O rechevoy sistemnosti nauchnogo stilya sravnitelno s nekotorymi drugimi [On speech system of the scientific style in comparison with some others]. Perm, Perm University Publ., 1972. 396 p. (in Russian).
- Kozhina M. N. Rechevedenie: teoriya funktsional’noj stilistiki: izbrannye trudy [Speech studies: Theory of functional stylistics: Selected works]. Moscow, Flinta, Nauka, 2020. 624 p. (in Russian).
- Krogh A., Hertz J. A. Generalization in a linear perceptron in the presence of noise. Journal of Physics A: Mathematical and General, 1992, vol. 25, no. 5, pp. 1135– 1147.
- Kuznetsova Y., Chudova N., Salimovsky V., Sharypina D., Devyatkin D. Possibilities of Automatic Detection of Reactions to Frustration in Social Networks. CEUR Workshop Proceedings. IMS 2021. Proceedings of the International Conference “Internet and Modern Society”. Saint Petersburg, 2021, pp. 159–168.
- Kuznetsov I., Gurevych I. A matter of framing: The impact of linguistic formalism on probing results. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, November, pp. 171–182. https://doi.org/10.18653/v1/2020.emnlpmain.13
- Lagutina K. V., Boychuk E. I., Lagutina N. S. Automatic Classification of Russian-Language Internet Texts by Genre. Artificial Intelligence and Decision Making. 2023, no. 4, pp. 103–114 (in Russian). https://doi.org/10.14357/20718594230410
- Ljashevskaja O. N., Plungjan V. A., Sichinava D. V. About the morphological standard of the National Corpus of the Russian Language. In: Natsional’nyj korpus russkogo jazyka: 2003–2005 [National Corpus of the Russian Language: 2003–2005]. Moscow, Indrik, 2010, рр. 111–135 (in Russian).
- Matveeva T. V. Stat’i po russkoj stilistike [Articles on Russian stylistics]. Moscow, Flinta, 2024. 392 p. (in Russian).
- Matveyeva T. V. Funktsionalnyye stili v aspekte tekstovykh kategoriy [Functional styles in the aspect of text categories]. Sverdlovsk, Ural University Publ., 1990. 172 p. (in Russian).
- Novikov D. A. Interview dated July 26, 2022. Available at: https://new.ras.ru/mir-nauky/news/vokrug-iskusstvennogo-intellekta-sklad... (accessed February 20, 2024) (in Russian).
- Osipov G. S. Metody iskusstvennogo intellekta [Methods of artificial intelligence]. Moscow, Physmatlit, 2011. 296 p. (in Russian).
- Pavlick E. Semantic structure in deep learning. Annual Review of Linguistics, 2022, vol. 8, pp. 447–471.
- Ravfogel S., Prasad G., Linzen T., Goldberg Y. Counterfactual interventions reveal the causal effect of relative clause representations on agreement prediction. Proceedings of the 25th Conference on Computational Natural Language Learning, 2021, November. https://doi.org/10.18653/v1/2021.conll-1.15
- Salimovsky V. A. Zhanry rechi v funktsional’nostylisticheskom osveshchenii (nauchnij akademicheskij text) [Speech genres in functional stylistic perspective (scientific text)]. Perm, Perm University Publ., 2002. 236 p. (in Russian).
- Sedov K. F. Obshchaya i antropotsentricheskaya lingvistika [General and anthropocentric linguistics]. Moscow, Yazyki slavyanskoi kul’tury, 2016. 440 p. (in Russian).
- Solganik G. Ya. Modern journalistic picture of the world. In: Publitsistika i informatsiya v sovremennom obshchestve: sbornik statei [Journalism and information in modern society: Coll. of articles]. Moscow, Moscow University Press, 2000, pp. 9–23 (in Russian).
- Suvorov R. E., Sochenkov I. V. Method for detecting relationships between sci-tech documents based on topic importance characteristic. Artificial Intelligence and Decision Making, 2013, no. 1, pp. 33–40 (in Russian).
- Tenney I., Xia P., Chen B., Wang A., Poliak A., McCoy R. T., Kim N., Durme B. Van, Bowman S., Das D., and Pavlick E. What do you learn from context? Probing for sentence structure in contextualized word representations. International Conference on Learning Representations, 2019. https://doi.org/10.48550/arXiv.1905.06316
- Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser Ł., Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems, 2017, vol. 30, pp. 5998–6008.
- Vygotsky L. S. Myshlenie i rech’ [Thinking and speech]. Moscow, Leningrad, Gos. sots.-ekonom. izd-vo, 1934. 324 p. (in Russian).
- Yan X., Han J. Graph-based substructure pattern mining. IEEE International Conference on Data Mining, 2002, Proceedings, pp. 721–724. https://doi.org/10.1109/ICDM.2002.1184038
- Zhu Z., Pan C., Abdalla M., Rudzicz F. Examining the rhetorical capacities of neural language models. Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2020, pp. 16– 32. https://doi.org/10.1865.v1.2020blackboxnlp-1.3
- Zmitrovich D., Abramov A., Kalmykov A., Tikhonova M., Taktasheva E., Astafurov D., Baushenko M., Snegirev A., Kadulin V., Markov S., Shavrina T., Mikhailov V., Fenogenova A. A Family of Pretrained Transformer Language Models for Russian. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LRECCOLING 2024). Torino, Italia, 2024, pp. 507–524.