Transcription of CHAPTER Regular Expressions, Text Normalization, Edit …
{{id}} {{{paragraph}}}
Speech and Language Processing. Daniel Jurafsky & James H. Martin. Copyright 2021. Allrights reserved. Draft of December 29, ,TextNormalization, Edit DistanceUser: I am : DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPYUser: I need some help, that much seems : WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELPUser: Perhaps I could learn to get along with my : TELL ME MORE ABOUT YOUR FAMILYUser: My mother takes care of : WHO ELSE IN YOU FAMILY TAKES CARE OF YOUUser: My : YOUR FATHERUser: You are like my father in some (1966)The dialogue above is fromELIZA, an early natural language processing systemELIZA that could carry on a limited conversation with a user by imitating the responses ofa Rogerian psychotherapist (Weizenbaum, 1966).
lemmatization Another part of text normalization is lemmatization, the task of determining that two words have the same root, despite their surface differences. For example, the words sang, sung, and sings are forms of the verb sing. The word sing is the common lemma of these words, and a lemmatizer maps from all of these to sing.
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
{{id}} {{{paragraph}}}