Speech Recognition: Technology & Patent Landscape

1 Speech recognition : Technology & Patent Landscape iRunway 2015 Page 2 of 36 Public Contents 1 Executive Summary .. 3 2 Introduction .. 5 Technology Overview .. 5 3 Patent Landscape and Analysis .. 7 Leading Patent Owners .. 9 Technology -wise Patent Distribution of Top Assignees .. 11 Seminal Patent Landscape .. 13 4 Application in Mobile Devices & Automobiles ..16 Speech recognition Technology in Mobile Devices .. 16 Patent Landscape of Mobile Device-Related Applications .. 16 Key Players in the Mobile Devices Sector .. 17 Seminal Patents in Mobile Device Applications .. 19 Speech recognition Technology in Automobiles .. 21 Patent Landscape of Automobile-Related Applications .. 21 Key Players in the Automobile Sector .. 22 Seminal Patents in Automobile Applications .. 24 5 Litigation Trend in Patent Landscape ..26 6 Conclusion ..30 7 Glossary ..31 Technology Category.

2 31 Mobile Device Application Categories .. 32 Automobile Application Categories .. 33 8 Authors ..35 Speech recognition : Technology & Patent Landscape iRunway 2015 Page 3 of 36 Public 1 Executive Summary The ability to interface with a machine using natural human language has fascinated the scientific world for many decades. Recent virtual assistants such as Apple s Siri have demonstrated the promise of a comfortable future with Speech recognition and voice-enabled processes penetrating household and industrial applications. While the success rate so far in the world market has been negligible, this may soon change. Microsoft s Cortana, available in the Windows 10 for mobile phones, tablets and, importantly, desktop units may dramatically advance the virtual assistant experience, providing Microsoft with a significant edge over competitors.

3 Unlike previous virtual assistants, Cortana is now tailored to local languages, customs and cultures, and to the corresponding nuances of Speech . Researchers have struggled to build a platform that interprets and responds to voice commands with accuracy and efficiency. While Technology developers such as Nuance Communications have developed a large Speech recognition Technology Patent portfolio in recent years, Microsoft and others have focused on linguistics, building massive dictionaries of vocabulary through neural networks and cloud-based architecture. It appears that linguistics may be the key to reducing processing time and providing a more seamless user experience. This research report examines Speech recognition Technology and its Patent Landscape in the market, providing an overview of key audio signal processing techniques and identifying the IP strengths and weaknesses of top companies.

4 The report also provides in-depth analysis of two widely used Speech recognition applications mobile devices and automobiles. iRunway s analysis found 21,281 granted patents and major industry players such as Microsoft, Nuance Communications, AT&T and IBM lead the list of top Patent holders. While Microsoft owns the largest Patent portfolio in linguistics Technology with 15% of the seminal, or strong, patents in this space, Nuance Communications dominates the recognition category with seminal patents. Sony owns of seminal patents in storage and transmission Technology . Another force of change will soon arrive as at least 172 seminal patents belonging to the leading 10 seminal Patent owners expire in 2016, bringing them into public domain. This will likely prompt a new wave of development in the Speech recognition domain with a dramatic impact on the application of this Technology .

5 This is likely to reduce licensing costs and make Speech recognition Technology more easily available to the larger market. Speech recognition : Technology & Patent Landscape iRunway 2015 Page 4 of 36 Public Mobile devices are emerging as commercially successful ubiquitous tools to perform multiple human activities through voice commands. iRunway s analysis found 3,209 patents related to application of Speech recognition Technology for mobile devices. AT&T, Nuance Communications, Microsoft, IBM and Google are the key Patent holders in this space. Apple does not have a large Speech recognition Patent portfolio and it licenses patents from other industry leaders including Nuance that powers Siri. A growing market for smart vehicles has also bolstered the automobile industry into a key applicant of Speech recognition Technology .

6 The analysis found 648 Speech recognition patents that were applicable to automotive and vehicular systems. Until the turn of the century, automobile manufacturers were relying heavily on Technology companies such as Nuance and Microsoft to implement Speech recognition functionalities in vehicles. In recent times, Apple and Google have emerged as two major players vying for a large market share with their CarPlay and Android Auto Speech -controlled infotainment systems respectively. However, many auto manufacturers have begun developing these technologies in-house. Denso, General Motors and Honda are three leading Patent owners in this space. Other leading owners of patents in Speech -enabled communication, navigation, maneuver, data presentation and techniques to cancel environmental noise include Nuance, Microsoft, Alpine Electronics and AT&T. Speech recognition : Technology & Patent Landscape iRunway 2015 Page 5 of 36 Public 2 Introduction Designing a system that mimics human behavior, particularly the capability of speaking naturally and responding interactively to spoken commands, has intrigued engineers and scientists for centuries.

7 Today, Speech recognition has stepped into every realm, including mobile phones, telecommunications, healthcare, banking, Speech -controlled automobile maneuvering, Speech -based web browsing, robotics, virtual personal assistant, aviation, military, education, handicap aid, security, and media and entertainment to name a few. Technology Overview Speech is technically defined as a sequence of basic units called phonemes. Automated Speech recognition (ASR) systems convert analog Speech signals received through microphones to digital signals that are segmented to retrieve phonemes. Using the phoneme sequence, the ASR system refers to the vocabulary and grammar rules to decipher words or phrases. Processing Speech signals includes removing noise, reducing errors in recognizing phoneme patterns and resolving ambiguity arising from variations in Speech accent, pitch and speed.

8 Statistical models and grammatical rules require exemplary training data and a large volume of vocabulary, words and phrases. With growing vocabulary and advances in semantic analyses, Speech recognition is striving to achieve more accuracy. Today's Speech recognition systems use powerful and complicated statistical modeling techniques to determine the most likely outcome. Each system implements grammatical rules based on a specific dictionary to deduce Speech from phonemes. Acoustic modeling and language modeling are important parts of modern statistically-based Speech recognition algorithms. Hidden Markov Models (HMMs) and neural network analyses are widely used methods to recognize Speech . Language modeling is also used in natural language processing applications such as document classification or statistical machine translation. Performance and accuracy of Speech recognition systems is very crucial in industrial and commercial applications.

9 With the evolution of distributed computing and cloud-based data storage, handling huge dictionaries and complex models to improve accuracy and performance is now a reality. Advancements in Speech recognition techniques and their applications have manifested in Speech recognition : Technology & Patent Landscape iRunway 2015 Page 6 of 36 Public increased patenting activity over the years. iRunway has analyzed patenting activity in this Technology sector for the past 20 years. The Speech recognition Landscape can be classified into four major Technology domains and multiple sub-categories as shown in Figure 1. Please refer to the Glossary for a detailed understanding of the taxonomy. Figure 1: Speech recognition Technology Taxonomy Speech recognition : Technology & Patent Landscape iRunway 2015 Page 7 of 36 Public 3 Patent Landscape and Analysis Since 1994, the USPTO has granted 21,281 patents across categories of this Technology as shown in Table 11.

10 Level 1 Categories Level 2 Categories Number of Patents recognition (8,579) Word recognition 2,044 Speech to text conversion 1,521 Correction and pattern matching 2,082 Models and algorithms 2,817 Speech recognition processes 909 Voice recognition 1,340 Speech recognition application 2,479 Storage and transmission (7,815) Analysis and prediction 2,653 Data storage and distributed computing 1,094 Audio signal compression and expansion 2,176 Speech transformation 2,352 Enhancement and correction 3,102 Storage and transmission application 633 Linguistics (3,519) Translation 1,721 Natural language 1,002 Dictionary building 343 Theoretical interpretation 595 Multilingual support 417 Linguistics application 122 Synthesis (1,368) Text to Speech conversion 1,055 Models and mathematical computations 802 Speech synthesis processes 483 Speech synthesis application 189 Table 1: Technology -wise Distribution of Speech recognition Patents (Source: iRunway analysis based on Patent data from USPTO) The complexity of Speech recognition technologies and their implementation, coupled with innovation needed to evolve efficient Speech assisted systems for commercial or research purposes, has resulted in aggressive patenting activity over the years.