CHAPTER 12 Constituency Grammars - Stanford University

1 Speech and Language Processing. Daniel Jurafsky & James H. Martin. Copyright 2021. Allrights reserved. Draft of December 29, GrammarsBecause the Nightby Bruce Springsteen and Patty SmithThe Fire Next Timeby James BaldwinIf on a winter s night a travelerby Italo CalvinoLove Actuallyby Richard CurtisSuddenly Last Summerby Tennessee WilliamsA Scanner Darklyby Philip K. DickSix titles that are not constituents, from Geoffrey K. Pullum onLanguage Log (who was pointing out their incredible rarity).The study of grammar has an ancient pedigree. The grammar of Sanskrit wasdescribed by the Indian grammarian P sometime between the 7th and 4th cen-turies BCE, in his famous treatise the adhy ay ( 8 books ).

2 And our wordsyn-taxcomes from the Greeks yntaxis, meaning setting out together or arrangement ,syntaxand refers to the way words are arranged together. We have seen various syntacticnotions in previous chapters: ordering of sequences of words ( CHAPTER 2), probabil-ities for these word sequences ( CHAPTER 3), and the use of part-of-speech categoriesas a grammatical equivalence class for words ( CHAPTER 8). In this CHAPTER and thenext three we introduce a variety of syntactic phenomena that go well beyond thesesimpler approaches, together with formal models for capturing them in a computa-tionally useful bulk of this CHAPTER is devoted to context-free Grammars .

3 Context-free gram-mars are the backbone of many formal models of the syntax of natural language (and,for that matter, of computer languages). As such, they play a role in many computa-tional applications, including grammar checking, semantic interpretation, dialogueunderstanding, and machine translation. They are powerful enough to express so-phisticated relations among the words in a sentence, yet computationally tractableenough that efficient algorithms exist for parsing sentences with them (as we showin CHAPTER 13). And in CHAPTER 16 we show how they provide a systematic frame-work for semantic interpretation.

4 Here we also introduce the concept of lexicalizedgrammars, focusing on one example,combinatory categorial grammar, CHAPTER 14 we introduce a formal model of grammar calledsyntactic depen-denciesthat is an alternative to these Constituency Grammars , and we ll give algo-rithms fordependency parsing. Both Constituency and dependency formalisms areimportant for language , we provide a brief overview of the grammar of English, illustrated froma domain with relatively simple sentences called ATIS (Air Traffic Information Sys-tem) (Hemphill et al., 1990). ATIS systems were an early spoken language systemfor users to book flights, by expressing sentences likeI d like to fly to ConstituencySyntactic Constituency is the idea that groups of words can behave as single units,or constituents.

5 Part of developing a grammar involves building an inventory of theconstituents in the language. How do words group together in English? Considerthenoun phrase, a sequence of words surrounding at least one noun. Here are somenoun phraseexamples of noun phrases (thanks to Damon Runyon):Harry the Horsea high-class spot such as Mindy sthe Broadway coppersthe reason he comes into the Hot Boxtheythree parties from BrooklynWhat evidence do we have that these words group together (or form constituents )?One piece of evidence is that they can all appear in similar syntactic environments,for example, before a parties from Brooklynarrive.

6 A high-class spot such as Mindy sattracts..the Broadway copperslove..theysitBut while the whole noun phrase can occur before a verb, this is not true of eachof the individual words that make up a noun phrase. The following are not grammat-ical sentences of English (recall that we use an asterisk (*) to mark fragments thatare not grammatical English sentences):*fromarrive..*asattracts..*th eis..*spotsat..Thus, to correctly describe facts about the ordering of these words in English, wemust be able to say things like Noun phrases can occur before verbs .Other kinds of evidence for Constituency come from what are calledpreposedorpreposedpostposedconstru ctions.

7 For example, the prepositional phraseon September sev-postposedenteenthcan be placed in a number of different locations in the following examples,including at the beginning (preposed) or at the end (postposed):On September seventeenth, I d like to fly from Atlanta to DenverI d like to flyon September seventeenthfrom Atlanta to DenverI d like to fly from Atlanta to Denveron September seventeenthBut again, while the entire phrase can be placed differently, the individual wordsmaking up the phrase cannot be:*On September, I d like to fly seventeenthfrom Atlanta to Denver*OnI d like to fly September seventeenthfrom Atlanta to Denver*I d like to fly on Septemberfrom Atlanta to Denver Context-Free GrammarsThe most widely used formal system for modeling constituent structure in Englishand other natural languages is theContext-Free Grammar, orCFG.

8 CONTEXT-FREEGRAMMARS3free Grammars are also calledPhrase-Structure Grammars , and the formalismis equivalent toBackus-Naur Form, orBNF. The idea of basing a grammar onconstituent structure dates back to the psychologist Wilhelm Wundt 1900 but wasnot formalized until Chomsky (1956) and, independently, Backus (1959).A context-free grammar consists of a set ofrulesorproductions, each of whichrulesexpresses the ways that symbols of the language can be grouped and ordered to-gether, and alexiconof words and symbols. For example, the following productionslexiconexpress that anNP(ornoun phrase) can be composed of either aProperNounorNPa determiner (Det) followed by aNominal; aNominalin turn can consist of one Det NominalNP ProperNounNominal Noun|Nominal NounContext-free rules can be hierarchically embedded, so we can combine the previousrules with others, like the following, that express facts about the lexicon:Det aDet theNoun flightThe symbols that are used in a CFG are divided into two classes.

9 The symbolsthat correspond to words in the language ( the , nightclub ) are calledterminalterminalsymbols; the lexicon is the set of rules that introduce these terminal symbols. Thesymbols that express abstractions over these terminals are callednon-terminals. Innon-terminaleach context-free rule, the item to the right of the arrow ( ) is an ordered list of oneor more terminals and non-terminals; to the left of the arrow is a single non-terminalsymbol expressing some cluster or generalization. The non-terminal associated witheach word in the lexicon is its lexical category, or part of CFG can be thought of in two ways: as a device for generating sentencesand as a device for assigning a structure to a given sentence.

10 Viewing a CFG as agenerator, we can read the arrow as rewrite the symbol on the left with the stringof symbols on the right .So starting from the symbol:NPwe can use our first rule to rewriteNPas:Det Nominaland then rewriteNominalas:Det Nounand finally rewrite these parts-of-speech as:a flightWe say the stringa flightcan be derived from the non-terminalNP. Thus, a CFGcan be used to generate a set of strings. This sequence of rule expansions is called aderivationof the string of words. It is common to represent a derivation by aparsederivationtree(commonly shown inverted with the root at the top).