Generation of segmented isiZulu text (Journal of the Digital Humanities Association of Southern Africa, 2024)
The complex morphology, conjunctive orthography and widespread occurrence of morphophonological alternation in the Nguni languages havegiven rise to several efforts towards morphologicalsegmentation of tokens of Nguni languages. Forsupervised methods, annotated data is required,which currently exists as canonically segmenteddata in the NCHLT corpus and surface segmenteddata in the Ukwabelana corpus. In this paper, wepresent a method and segmentation strategy basedon a computational grammar for isiZulu. Thegrammar, which itself has some limitations in processing speed and robustness to unexpected input,is used to create a new set of segmentations for thetokens of the Ukwabelana corpus.By training various models with the same architecture but on different datasets, we first show that ourapproach enables us to match the performance of amodel trained on pre-existing data. We also showthat our approach provides the flexibility to determine a suitable segmentation strategy and to generate data that reflects this strategy.
Keywords: Nguni languages; agglutinative languages; morphological segmentation; language models