The last post described future ear-level devices as conduits for highly personalized, exquisitely targeted, near and far-field individual communication systems that work in real time. Such a vision combines past and ongoing R&D efforts in Hearables, Hearing Aids, and Augmented Reality (AR). As the media hypes it, these will not be your grandfather’s hearing aids and this will not be our traditional market.
Audiologists and hearing aid manufacturers were working the territory before technology turned visionary and expanded the market. Now, by happenstance or strategy, we find ourselves riding a major disruptive trend, hype included. But traditional hearing healthcare suppliers and clinically licensed providers will no longer be your grandfather’s only choice for augmented hearing devices. Market power entices and Big Data beckons new organizations into our midst.
The Economic View: Market Power Up For Grabs
Fig 1. Patents on wearable devices integrated with augmented reality technology. Source: IFI Claims Global Database.
Who would have thought that Audiologists and hearing aid manufacturers were in AR from the beginning and remain front and center?
Front and center may mean cannon fodder, not market power. Companies are competing fiercely through R&D investments and M&As to gain dominance in a chaotic, highly attractive, fast-growing augmented reality device market. This rapid-fire emergence of AR, including Wearables, is mainly outside the hearing aid industry, according to patent activity (Fig 1). In the last decade (2004-2014), 2029 patents were filed on “augmented reality” by large multinational technology companies.
Our nook in this field is AR integrated into “hearable” devices, which comprises a tiny part of AR activity (in the red section of Venn diagram, Fig 1). Within that nook, hearing aid and Hearable manufacturers are not players of note at present. As of July 2014, 98 companies held Wearables AR patents, chief among them Google (51 patents), Samsung, Santa Fe Science and Technology (a NASA-funded small business), Sony, and Zugara. 1
Oh No, Mr Bill!
Elwha LLC, which has only six patents to its name, was mentioned last July in a post as a “sleeper” of special interest to the hearing device industry. Indeed. Elwha LLC is a patent portfolio holding company owned by Intellectual Ventures (IV). IV is a gigantic patent troll by some accounts, with over $6B in raised capital, 70,000 acquired patents, no products, and all revenues generated through licensing fees and litigation. Microsoft has invested in IV funds since it was co-founded in 2000 by Nathan Myhrvold, formerMicrosoft CTO and a Bill Gates protégé.
Gates and Myhrvold are inventors of record for US patents #8811638 (Audible Assistance) and #8934652 (Visual presentation of Speaker-Related Information), which were issued to Elwha on 8/19/14 and 1/13/15, respectively. The titles sound pretty humdrum, but the visions and images take hearing devices to new territory as Audible Assistance Facilitory and Ability Enhancement Facilitator Systems (AAFS and AEFS), both of which:
- receive data that represents an utterance of a speaker received by a hearing device of the ear, such as a hearing aid, smart phone, media device, or the like….
- identify the speaker based on the received data, such as by performing speaker recognition…
- determine speaker-related information … by locating an information item (e.g., an email message document) associated with the speaker …
- inform the user of the … information, such as by causing an audio representation of the … information to be output via the hearing device… or some other device accessible to the user.
Two patent pictures are worth several thousand words, making this post shorter and also making it clear that hearing hearing devices (item 120) are front and center in the patent concept. Gates and friends may not make products, but they clearly have hearing aid technology directly in their patent-trolling cross hairs, as they imagine comprehensive supply lines of hardware and software built on hearing aid and communications technology platforms. The complete facilitory systems depicted in the images are exciting and will doubtless offer more utility to consumers han the piecemeal offerings available from different industries and vendors in today’s market. But that utopian consumer future does beg the question of how hearing devices in such systems will be designed, manufactured, distributed, and sold in retail environments.
Image for USPTO 8811638 (AAFS)
Image for USPTO 8934652 (AEFS)
Keeping Our Toehold in Hardware
In these David and Goliath times, our little industry has something that could help us succeed where the behemoths may fail. We have hardware. Moreover, our hardware is really good, proven hardware that receives, integrates, and sends. It not only does the job of enhancing/augmenting audibility, it’s got wireless connectivity, comfortable fit, and — most importantly — the real estate to accommodate technology as it moves off of phones in search of a place to be (e.g., watches, pendants, wrist bands, and household appliances). Why put the tech brains into such items (many of which remain in prototype form) when there’s already a proven piece of hardware that people wear, sometimes eagerly, on a full-time basis?
On the other hand, companies like IV, which make huge revenues off of patent licensing, can see the sense in putting a generic hearing device, aka item 120, into as many “facilitatory systems” patents as possible, then extract licensing fees from all who make, sell, or even use such devices. It sounds scary but so is patent law and patent enforcement, not to mention market power.
It also makes sense for companies like Apple and Samsung to gain competitive advantage through optimized supply chains that “lock up key product and component supplies” and license app makers. MFi hearing aids were a first foray into this market solution, possibly representing what futurist and Hearable master Nick Hunn refers to as Apple’s “unerring ability to make another industry segment its slut.” Our little industry needs to be quick and clever to avoid the role of hardware/app slut to Microsoft, Apple, trolls, and other companies seeking to package technological solutions for growing consumer markets.
But for now, we hold the hardware/firmware key to success if we’re bold enough to use it. Maybe it’s time for our industry to come up with a transparent/opaque, RIC-styled, binaural hearing system with licensed streaming directed-audio services honed by continuous monitoring of listener preferences. Designing our own “facilitatory” system would put us squarely in the red (Fig 1) and the black (financial statements). It would also catapult us into the gray area of Big Data and public policy scrutiny. The economic view of Big Data, privacy and security is the subject of the next post in this series.
References and Footnotes
1Paraphrased from Cruz E (July 2014). The IP Landscape for Wearables with Augmented Reality. Patent Intelligence and Technology Report. ificlaims.com
Thank goodness for Apple bringing some sort of standards discipline to the hearing aid & CI industry with their “Made for iPhone” Bluetooth 4.0 Low Energy (“BLE”) logo program.
There is a desperate need for a common industry standard for wireless direct-to-hearing aid and CI audio transmission, especially for public venues such as theaters, arenas, and houses of worship. On this point, the leadership of the Hearing Industries Ass’n trade group has been a miserable #FAIL, as although four of the “Big Six” manufacturers use the 10.6 mHz ISM band for their HIBAN networks between their dongles and hearing devices, each of these four have their own proprietary — hence incompatible — encoding and transmission protocols (and Starkey’s 900 mHz platform on some of their hearing aids is a fifth). Given that in 2014 fully 80% of the hearing aids dispensed in the US have wireless capabilities built in, this presents a miserable situation for the hearing impaired community. Post-Jobs Apple tried to bring the industry together with their “Made for iPhone” BLE logo program; but so far only GN ReSound, Starkey and Cochlear have fully signed on, with others only paying lip service with iOS apps connecting to proprietary streamer dongles, not the actual aids & processors themselves.