POS Tagging and NER System for Kannada Using Conditional Random Fields

POS Tagging and NER System for Kannada Using Conditional Random Fields

Arpitha Swamy, Srinath S.
Copyright: © 2021 |Pages: 13
DOI: 10.4018/IJIRR.2021100101
OnDemand:
(Individual Articles)
Available
$37.50
No Current Special Offers
TOTAL SAVINGS: $37.50

Abstract

Parts-of-speech (POS) tagging is a method used to assign the POS tag for every word present in the text, and named entity recognition (NER) is a process to identify the proper nouns in the text and to classify the identified nouns into certain predefined categories. A POS tagger and a NER system for Kannada text have been proposed utilizing conditional random fields (CRFs). The dataset used for POS tagging consists of 147K tokens, where 103K tokens are used for training and the remaining tokens are used for testing. The proposed CRF model for POS tagging of Kannada text obtained 91.3% of precision, 91.6% of recall, and 91.4% of f-score values, respectively. To develop the NER system for Kannada, the data required is created manually using the modified tag-set containing 40 labels. The dataset used for NER system consists of 16.5K tokens, where 70% of the total words are used for training the model, and the remaining 30% of total words are used for model testing. The developed NER model obtained the 94% of precision, 93.9% of recall, and 93.9% of F1-measure values, respectively.
Article Preview
Top

1. Introduction

“Part-of-speech (POS)” tagging is a method to identify the morphosyntactic class (noun, adjective, verb, preposition etc.) of each word in a sentence using the lexical and contextual information. A tag is syntactic label which represents the lexical category of a word. For example, tag NN indicates a common noun class, and a finite verb class is represented with tag VMF.A POS tagger is a tool which labels suitable POS for every word in a given text. Many different methods like transformation based approaches, rule based approaches, Machine Learning (ML) methods, and example based methods and so on can be used to create a POS tagger. Rule based systems have been composed utilizing syntactic guidelines, though Machine Learning methods utilize probabilistic models and the stochastic sentence structure. In this paper, one of the supervised machine learning approaches -Conditional Random Fields is utilized to create a Kannada POS tagger.

“Named Entity Recognition and Classification (NERC)” is a method to identify the named entities and group them into certain predefined classes such as Person, Organization, Location, designation, measurement etc. NER is an important task in the Natural language processing which will be useful in applications - Information Extraction, Automatic Text Summarization etc. The proposed NER model for Kannada language in this study has been trained depending on features like word, POS tags of word and the next word, word prefixes, word suffixes, beginning of the sentence etc. The Conditional Random Fields (CRFs), one of the supervised machine learning algorithms is used to build the NER model in this work.

Kannada is the local language in the state Karnataka and the individuals in the southern pieces of India communicate in this language for the most part. It is one of the Dravidian languages in India and the administrative language of Karnataka state. Despite the fact that there are around 50.8 million speakers who speak Kannada everywhere throughout the globe, research work on computational linguistics for Kannada language is yet slacking. One of the primary explanations behind research to slack in Kannada language in computational semantics for is because of its agglutinative nature and rich morphology.

The Kannada language utilizes 49 phonetic letters, separated into three categories:

  • 1.

    Thirteen letters - vowels (swaragalu);

  • 2.

    Thirty four letters - consonants (vyanjanagalu); and

  • 3.

    Two letters - the anusvara and the visarga (yogavaahakagalu).

The character set of Kannada language is practically indistinguishable from that of other Indian dialects.

The paper is presented as follows: Section 2 carries out a survey on Kannada POS tagger and NER systems. The proposed methodology for the Kannada POS tagging and NER are discussed and the experimental results are illustrated in section 3. In the end, the paper is concluded with future work in section 4.

Complete Article List

Search this Journal:
Reset
Volume 14: 1 Issue (2024)
Volume 13: 1 Issue (2023)
Volume 12: 4 Issues (2022): 3 Released, 1 Forthcoming
Volume 11: 4 Issues (2021)
Volume 10: 4 Issues (2020)
Volume 9: 4 Issues (2019)
Volume 8: 4 Issues (2018)
Volume 7: 4 Issues (2017)
Volume 6: 4 Issues (2016)
Volume 5: 4 Issues (2015)
Volume 4: 4 Issues (2014)
Volume 3: 4 Issues (2013)
Volume 2: 4 Issues (2012)
Volume 1: 4 Issues (2011)
View Complete Journal Contents Listing