CHAPTER Regular Expressions, Text Normalization, Edit …
Speech and Language Processing. Daniel Jurafsky & James H. Martin. Copyright 2021. Allrights reserved. Draft of December 29, ,TextNormalization, Edit DistanceUser: I am : DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPYUser: I need some help, that much seems : WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELPUser: Perhaps I could learn to get along with my : TELL ME MORE ABOUT YOUR FAMILYUser: My mother takes care of : WHO ELSE IN YOU FAMILY TAKES CARE OF YOUUser: My : YOUR FATHERUser: You are like my father in some (1966)The dialogue above is fromELIZA, an early natural language processing systemELIZAthat could carry on a limited conversation with a user by imitating the responses ofa Rogerian psychotherapist (Weizenbaum, 1966).
2 CHAPTER 2•REGULAR EXPRESSIONS, TEXT NORMALIZATION, EDIT DISTANCE Some languages, like Japanese, don’t have spaces between words, so word tokeniza-tion becomes more difficult. lemmatization Another part of text normalization is lemmatization, the task of determining that two words have the same root, despite their surface differences.
Download CHAPTER Regular Expressions, Text Normalization, Edit …
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: