Ideas from Microsoft & Co. could change our future

There is a lot of research, hardly any other market is as fast-paced as the tech industry. Last year, Amazon, Alphabet, Meta, Apple, and Microsoft spent a combined $137 billion on research and development.

A look at the latest patent applications reveals some trends. Many revolve around the interface between man and machine, interaction is becoming more natural and smarter. The functions are also expanding, especially in health, artificial intelligence is developing into a family doctor.

What all patents have in common is that they need masses of information: “For many of the emerging technologies to work at all, different types of data are needed,” explains futurist Amy Webb, founder of the Future Today Institute in New York. And every application needs different data. “This potentially exposes users to much more surveillance.”

What exactly is planned? Here are three of the most exciting patents of the first quarter of 2022:

1. Virtual assistants see everything

In a newly granted patent from Microsoft, users can contact their virtual assistant without any words. A direct address feels unnatural and is cumbersome, say the Microsoft researchers. They rely on looks, gestures, face or body movements. Data on “head posture or head position and the like” should also be recorded. The assistant could follow the customer with cameras or infrared sensors. The customer can then activate it by blinking or pointing in the direction of the assistant.

Machine learning makes this possible. Algorithms help the assistant to evaluate the data collected with the camera. “Furthermore, the machine learning algorithms can use facial recognition technology to collect user-specific data and adapt to user-specific preferences, actions, and/or gestures,” the patent reads.

Such non-verbal assistance systems have great market potential, says Dietmar Harhoff. He is Director at the Max Planck Institute for Innovation and Competition in Munich and former Chairman of the Federal Government’s Research and Innovation Expert Commission. “This is a very exciting area for a company like Microsoft, which is actually better known for other products.”

The idea of ​​wordlessly activating a virtual assistant isn’t a new thing, says Harhoff. However, the implementation works much better due to technical advances in computer vision and greater computing capacity. “With sufficient training, machine learning can be used to build algorithms that enable this type of control. It didn’t go so well before.”

Cortana was introduced in 2014 and launched on the market a year later, and the system has already been asked around 18 billion questions. Compared to Amazon’s Alexa or Apple’s Siri, however, Microsoft’s Cortana plays a subordinate role.

Julia Powles, professor of law at the University of Western Australia in Perth, criticizes the new method: “This Microsoft patent illustrates the sheer extent of the company’s surveillance efforts,” she says. “At a certain point we have to ask ourselves: does the usefulness of a virtual assistant justify exposing every aspect of ourselves to machines? What do we risk losing in this process?”

2. Identify pandemics early

The corona pandemic has shown how overwhelmed governments or health authorities can be when faced with a highly contagious virus. It sometimes took governments days to locate outbreaks and hotspots. A lot of time in which the spread of the virus could have been reduced.

Google has an idea how to change that: In January, the Alphabet subsidiary received confirmation of a new patent – ​​a device in the ear can detect infections at an early stage based on an increased body temperature.

The technology could be in Google Pixel Buds. This is not stated in the patent, but the inventors suggest that users could listen to audio content through the device. When the headphones are worn repeatedly, the system is able to measure temperature differences tailored to the user and to indicate a possible infection.

Why not just measure the fever yourself with the thermometer? Not effective, says Google. There are too many disadvantages: “Lack of compliance, bad data and failure to detect the infection before others are infected,” says the patent description.

The inventors explain that the devices could be distributed to thousands or tens of thousands of people if, for example, a contagious disease such as Ebola or swine flu were to break out. “Basically, measuring the temperature in the ear is a very old technology,” says Harhoff from the Max Planck Institute. But the data processing is new, the fever curves are coupled with intelligent devices such as smartphones or watches.

>>> Read here: Breakthrough or sham innovation? So it is with the new Alzheimer’s therapy

The technology could save lives. The patent description states that early detection of the disease not only increases the infected person’s chances of survival, but can also significantly stem or reduce the spread of the contagious disease.

A passage in the patent reads like a comment on the current lockdowns in China – although the patent was written beforehand. Precautionary enforced quarantine “can be expensive and illegal when there is no imminent threat, and discourages quarantined individuals from productive activities,” the description said.

The patent fits into Alphabet’s strategy: in 2019, the group bought Fitbit, which produces fitness trackers and measures health data. Health is a key area that’s facing a major wave of digitization and could help Alphabet offer more diversified data-based services. Companies like Apple and Samsung are headed in the same direction.

Researcher Powles criticizes Google’s idea: “Is this the future we want?” she asks. A company has too much power. Joachim Henkel, Professor of Technology and Innovation Management at the Technical University of Munich, takes a different view: “If something is in the hands of just one company, then its use is limited.” However, state supervision and requirements are necessary.

3. Saving lives with artificial intelligence

A newly granted patent from IBM shows how helpful artificial intelligence can be. Using deep learning, IBM analyzes a person’s physiological data to predict epileptic seizures. “Rapid detection is very important because treatment at the right moment can prevent serious damage to health,” confirms Harhoff. A big market beckons, health insurance companies, for example, are very interested.

Epilepsy is a disease that presents with sudden episodes of loss of consciousness or seizures. The whole body can be affected, but the trigger is in the brain. These neurological signals can be detected via sensors in something called electroencephalography (EEG), which records brain activity in waveforms.

The devices required for this are now so small that they can be worn on the body. The inventors are designing a system that samples brain activity and then matches that information to classification models. These models are created by training neural networks on certain characteristics that indicate a seizure. However, since epileptic seizures occur very rarely in a patient, health data from other patients should also be used to improve the neural networks.

IBM has not only applied for the patent in the USA, but also in China, Germany, Great Britain and Japan. Big Blue seems to have big plans: “Because applying for a patent in so many countries is very expensive,” says Harhoff from the Max Planck Institute.

IBM has been active in the healthcare sector for a long time. Not always with success, says Harhoff. Nevertheless, the company still has many orders in the healthcare system and wants to place services accordingly. “IBM is building a large portfolio in this area,” he says. “They have already filed about 200 patent families, i.e. groupings of patents with similar content.”

A granted patent is not a panacea for success

Tech companies like Google, Microsoft and Co. are constantly trying to outdo each other with new ideas and innovations. All together received around 375,000 patents in the US in fiscal year 2021, with IBM alone registering 8,600 of them and leading the list.

In the field of computer, information and communication technologies, the number of applications has risen sharply over the past thirty years, says scientist Henkel. “The industry is on the move and also patents strategically.”

It is not possible to reliably derive the goals of a company from the patents: “You can see from the patents which markets a company is interested in,” says Henkel. “However, sometimes companies apply for patents just to have a foot in the door in the respective area.”

>> Be there: At the Handelsblatt Innovation Summit we show the latest ideas and products from the research laboratories and companies

Incidentally, the possible effects of a patent are not taken into account when it is awarded by the authorities. “The grant only looks relatively narrowly at novelty and inventiveness against the background of previous scientific advances,” explains scientist Powles. “The fact that a patent is granted says nothing about the social or commercial value of the invention.” Both of these factors must be assessed outside the patent system.

A patent that has been granted does not automatically mean that an innovation will result from it: “There are studies that show that patents make sense in order to earn money with innovations. In many industries, however, a quick route to the market and so-called complementary goods such as marketing, sales and production are more important,” explains Henkel. “Patents are sometimes overrated. They are practically not a panacea.”

More: Germany falls behind when it comes to patents for future technologies – China and South Korea are pushing ahead