Xueping Lin | The Future Trend of Sensors

Note: Partially sourced from Singapore-based consultancy Twimbit’s Sensors report “Transforming Trends: A Paradigm Shift 2021-2023”. This article has restructured the viewpoint and content.

Let the smart world go back to its origin, where sensors are waiting. The basis of intelligence is perception, and sensors are the entrance to perception. Sensors are developing in the direction of intelligence, thinking, analysis and diagnosis. As a set of increasingly obvious intelligent microsystems, sensors are more and more independent and have the ability of self-correction. So, what capabilities will sensors have to detonate the future and drive digital transformation?

Figure 1 Twenty trends in sensor development (Source: Twimbit, a Singapore-based startup consulting firm)

Where are 3D sensors heading?

3D depth sensor technology can create three-dimensional images, which fully meet the needs of human vision. It can use time-of-flight ToF (time-of-flight), structured light, 3D interference, etc. to obtain 3D visual data. Among them, the ToF sensor has attracted much attention in the field of mobile phones, providing excitement for mobile phones that have been busy with constantly improving lens functions. It uses the laser pulse of each pixel on the infrared sensor, the time difference between the external emission and the reflection, to obtain a three-dimensional depth of field and form an imaging technology of a three-dimensional 3D model.

However, the ToF lens in the mobile phone industry has gone through a roller coaster experience, “come on the excitement and go away”. In 2018, its application reached a peak, and everyone who saw it had a share. For example, mid-to-high-end models such as Samsung, Huawei, OPPO, and Xiaomi are equipped with ToF lenses. When everyone thinks that this is the development direction of mobile phone imaging in the future, this kind of mobile phone suddenly disappears quickly. T is like the figure of a courier, the oF lens comes and goes quickly.

The reason is very simple. ToF technology lacks just-needed application support, and there is no widely used killer application that can power the further development of ToF lenses. ToF can be used to scan the shape of an object and then automatically build a 3D model. But for ordinary consumers, establishing a model has no practical effect. Mobile phones with ToF lenses can be used as rulers, but they are not accurate enough. Besides, who would take the matter of using a mobile phone as a tape measure to measure it seriously.

In the current intelligent world, any hard technology depends on the synchronization of software. Without software applications, it is difficult to support the iterative progress of hardware technology. Unless the rise of the metaverse, that is, AR/VR applications, may be able to save the application of TOF lenses on mobile phones.

On the contrary, the sweeping robot can replace the single-line mechanical scanning lidar with a wide-angle TOF camera, which makes it easier to form a “combat sand table” for the room, better plan the path, and make the sweeping robot look less stupid. Otherwise, the sweeping robot will either hit the legs of the table or roll up socks and cables, and the obstacle avoidance effect will be too poor. But the premise is still, not too expensive.

The distance measurement and perception of the driving environment in autonomous driving, human-machine collaboration of industrial collaborative robots, and intelligent logistics vehicles can also make ToF sensors really play a role. But for industry, ToF sensor lenses are still too expensive. From the perspective of the industry chain, the current infrared sensors of ToF lenses are mainly controlled by Sony, Infineon, ON Semiconductor, Texas Instruments, Panasonic , etc. The optical lenses are Largan, Zhejiang Sunny Optical, and AAC Technology. The CMOS image sensor is the core part, mainly from three rivals for many years: Sony, Samsung and Ware’s OmniVision .

3D depth sensors have bigger ambitions in the military. The U.S. Defense Advanced Research Projects Agency DARPA is developing 3D sensing technology for the military to facilitate covert operations at night.

Generally speaking, any self-driving system usually requires some form of active lighting to enable autonomous navigation at night. But turn on the headlights or the launch system of the LiDAR, and the radiation signal will appear. In military applications, it would enable enemy forces to detect the presence of these vehicles from a distance. DARPA is trying to develop 3D vision sensors by exploiting the weak heat signatures of various animate and inanimate objects in the wild. This “invisible headlight” project will explore the information contained in the various thermal radiation in the environment, because all objects emit thermal energy. DARPA’s goal is to explore the development of passive sensors that capture information from even a very small amount of thermal radiation to generate 3D maps for navigation. It would greatly expand self-driving systems that can operate stealthily.

Acoustic sensors have a new face

The biggest feature of acoustic technology is that it is relatively inexpensive compared to sensors of other technologies, and various applications can be explored. And SAW technology, widely used in signal processing of filters, shines brightly on smartphone speakers. Globally, the SAW filter market is mainly occupied by Japanese companies, including Murata, TDK, and Taiyo Yuden as representative manufacturers, with a total market share of about 82% . The Murata family can account for nearly half of the global SAW filter market. Among the domestic RF filter companies, Shenzhen Maijie Microelectronics Technology is regarded as the vanguard of domestic replacement of filters and integrated inductors, and it has also entered Huawei’s supply chain, with revenue of 1.3 billion in 2021.

Surface acoustic wave SAW technology is relatively used at low frequencies and is also more sensitive to temperature, so it will take full advantage of the opportunity in the 4G era. However, in the high frequency field and in the case of multi-signal processing to avoid interference, the BAW resonator technology has a wider application, and it comes for the 5G era and the Internet of Things era. In this regard, American technology is more advanced. Both Qorvo and Texas Instruments have the upper hand. In this area, it is also a pain in the neck that is difficult for China to break through. Texas Instruments also used this technology in the integrated clock function in 2019. With the increasing speed of big data transmission, there are strict requirements for clock signals. The data processing system DPS that requires 18G capacity per second has become an urgent problem for many chip manufacturers to solve. The bulk acoustic wave BAW can well realize the clock technology under the high frequency unified information. Although surface acoustic waves are relatively cheap, from the perspective of the overall technology development trend, bulk acoustic wave BAW is replacing the position of surface acoustic waves, and it has been used in high-end mobile devices such as iPhones.

The sensor is a power station

The sensor has two orientations. One is an integrated sensor, which is integrated with other devices to share the energy input; the other is an independent sensor. The latter is like the Robinson of a desert island survival, it’s best to survive on its own without care. Self-incoming calls are the first challenge.

Self-powered sensors are gaining traction and are ideal for remote monitoring, wireless connectivity, and continuous monitoring.

This often requires the deployment of sensor-based energy harvesters , these micro-energy recovery systems capable of generating micro-currents for their own use from solar, vibration, and thermal energy, among others. In other words, a sensor is an electrical energy generating device and an energy storage device. Since an electric car can be an energy storage system, why can’t a small sensor?

Parker Hannifin Parker, a US supplier of motion and power control systems, acquired an adhesive and vibration management equipment company, Lord, for $3.7 billion in 2019. The latter has been providing accurate measurement of wireless sensors and pressure for aerospace and petrochemicals. sensor. Hank is diversifying away from its traditional powertrain business, especially to strengthen the technological strengths of its engineered materials division in order to fully embrace emerging trends such as electrification and lightweighting. Annual sales of $1.1 billion LORD’s accumulation of coatings, springs, sensor hardware and sensor clouds is a perfect fit for Parker’s needs.

Figure 2 The new favorite of the old power sensor (Source: Parker Lord official website)

And LORD’s MicroStrain sensing business already uses piezoelectric materials to convert the material’s strain energy into electrical energy storage. The era of a sensor independent power station has begun.

From smart robots to smart sensors

Machine learning is everywhere. If the algorithm is not only placed in the machine equipment, but can also be placed in the smallest sensing unit – the sensor, then embedded artificial intelligence will vigorously promote the development of intelligent sensors.

Of course, machine learning is only one of them. Most sensors have moved from interactive to predictive, shifting the initiative of machine intelligence in part to sensor intelligence. Sensors with intelligent real-time data analysis and process correction functions will greatly improve the interaction capabilities of equipment. This also means that edge computing will go a step further at the edge of the machine.

However, the parameters that one sensor can measure are limited, so why not integrate multiple sensors together? As a result, various combination sensors have come one after another. Sensors such as temperature + humidity, pressure + flow, vibration + acceleration + deceleration have become the most used combinations . The sensor fleet is mixed to realize the “multi-purpose” of multi-parameter detection, and the application of closed-loop automation is formed by detecting various parameters. In the field of intelligent manufacturing, there is room for extensive development.

Another direction is the fusion sensor Fusion. Smart sensors are accelerating the development of driverless cars, and the multi-functional fusion of sensors will take advantage of different sensors to provide data analysis and control capabilities, so as to have embedded intelligence. This is especially important in military aircraft. The US F35 fighter jet has been connecting and analyzing multi-domain data. The core is to use fusion sensors to achieve high-speed analysis of multi-dimensional data, and to use data from different platforms, whether it is sea, air, sea and land sensor data. , a variety of heterogeneous data concurrent processing.

On the basis of the existing image sensor CIS (CMOS Image Sensor), Howe, a subsidiary of Weir Semiconductor, has just launched a composite sensor that combines CIS and event vision sensor EVS. EVS is a kind of sensor that does not require exposure time limit. of biomimetic sensors. This will be a super benefit for AR helmet players, and the camera images of mobile phones will be further improved. This feature of integrating image and vision sensors on one chip belongs to pixel-level sensor fusion, which is undoubtedly an important direction in the future.

Pulse flow, blood flow, heartbeat flow, where are they going?

Big health has become a hot spot for future development. Health prevention and diagnosis can be achieved through widely used sensors. Implantable devices for life support, long-term monitoring of critically ill patients, and robotic surgery. The Da Vinci surgical robot, born in 2000, is currently the most successful surgical robot in the world. At first, it was just a stabilizer to assist doctors in surgery, but its ability has become stronger and stronger, and it has become the best companion for surgeons on the operating table in one fell swoop. The Da Vinci robot single-handedly detonated robotic surgery.

According to a report in the Journal of the American Medical Association (JAMA), from 2016 to 2021, the proportion of robotic surgery in surgical procedures has increased from 2% to 15% in the past five years, and it is still accelerating . At present, there should be more than 150 installed machines in China, and 40,000 operations are performed by Da Vinci surgical robots every year.

It has huge advantages in 3D imaging and precise control. With its assistance, doctors can create a 3D image with a magnification of 10-15 times of the lesion without difficulty (traditional imaging systems can only provide 2-3 times the two-dimensional image), and then operate the robot to perform precise surgery. The hero behind this is nearly 500 sensors . Da Vinci robots are expensive, and a machine is tens of millions of yuan. In the cost composition of surgical robots, optical torque sensors account for about 5%.

For medical and home diagnostics, this is just the beginning. The popularity of the Internet of Things, and wearable sensors in health applications have flourished. These sensors can be used for elderly care monitoring and daily health monitoring in a non-invasive manner. Sleep monitoring on the watch is still only for pediatrics, and the prevention of diseases such as diabetes is becoming a hot spot.

There is no doubt that innovations in wearable sensors have brought about changes in the way health monitoring is done . Wearable and implantable sensors transmit health data in real-time, providing quantified exercise data and various physiological data, enabling accurate diagnosis. Different sensors are used in these key devices, including image sensor CMOS, vibration, blood sugar, and optical sensors used in mobile phone imaging. The four ring sensors behind the Apple Watch iWatch use LED light to reflect on the skin to determine the movement and pulse of blood vessels.

Where did the pulse flow, blood flow, heartbeat flow go? All human health data flows into a digital channel in the form of data flow. All human data is stored on an IoT digital health platform. Big data analysis automatically gives a comprehensive picture of people’s health: the existing health prevention model is about to undergo profound changes.

According to data from CB Insights’ 2020 report, a total of $80.6 billion in financing and 55,000 venture capital investments have occurred in this field. Nearly 200 healthcare financings of more than $100 million in 2020 set a new record. The medical giant has more than $550 billion in cash, waiting to be deeply cultivated in this digital medical field. Sensors, on the other hand, bear the brunt of this type of medical investment.

Without sensor research, the Industrial Internet will always be a supporting role

The development of the Industrial Internet puts forward higher demands on sensors. The performance of these sensors, as well as the controller system, largely determines the ability of managers to implement key control functions remotely using the Industrial Internet platform. For an automation company like Honeywell, while consolidating its dominance of control systems, it has an obsessive pursuit of sensors – which is why it has been buying sensors frantically in recent years, including temperature and humidity sensors. Most typically, in 2019, it acquired the originator of gas sensors, the British Citeytech sensor. Only by strengthening the hard-core technologies of control and sensors can the industrial Internet platform be truly implemented.

By 2023, the number of devices connected to the Industrial Internet is expected to reach 21.5 billion. The Industrial Internet needs to solve six problems: connection, perception, control, analysis, communication and application. Connection is just physical work, analysis is not yet useful, communication will become a general technology, and its application will only be a gimmick before analysis and mechanism models are established. Who is controlling the device, and who is sensing the data, are critical gates. This is the crux of the Industrial Internet platform. It is a pity that in China, many industrial Internets have been far away from this core battlefield, and they are playing with all their guns on the application layer, and they only make small waves and make small articles on the massive garbage data.

Soft sensor, also sensor

Soft sensors are an interesting thing. Different from the soft body of the soft robot, the soft sensor is actually a virtual sensor, which is software. It can process multiple measurements simultaneously.

Soft sensors are based on control theory and, through indirect use, may process dozens or even hundreds of measurements simultaneously. Soft sensors play a huge role in data fusion. It combines data from different static and dynamic measurements so that it can be used for fault diagnosis and control applications. The most classic soft sensor can start with the Kalman filter. It is a data processing technology that removes noise and restores real data, which can be regarded as data filtering calculated by software. This is also seen as a kind of soft sensor. Of course, the latest soft sensors use neural networks or fuzzy computing . A soft sensor is a software program that uses information from other sensors to estimate physical quantities rather than measure them directly.

This trend is most pronounced in process automation, where many control functions are software-activated and computer-assisted. High reliability and precision are the hallmarks of soft sensors. For example, pH-based soft sensors can easily perform load triggering for water treatment and peak detection.

Soft sensors can also be seen as integrators of digital technologies, which are integrated from advanced automation, the Internet of Things, real-time analysis of big data, and sensors.

More sensor technology

The combination of optical technology and sensors is benefiting from the emerging optoelectronic technology . The development of the chip is mainly made of silicon, and both the optical and electronic parts can be integrated on the silicon wafer, thus forming optoelectronics technology (photonics). It superimposes the rapid upgrading capabilities of electronics on optical instruments that have been slow for many years, thereby stimulating new vitality, and thus ushering in a moving moment for optical sensors, which will also greatly benefit the solar industry and the industrial Internet.

IO-Link enables digital connectivity to transmit data directly from sensors to IoT interfaces and programmable logic controllers (PLCs). IO-Link technology offers outstanding cost-effectiveness and technical improvements compared to traditional stand-alone module-connected sensor technology. In traditional independent modules, a module is often an independent network node. Every time a node is deployed, a set of chips is required. When the number of control points is large, the cost of the system solution will increase significantly. And IO-Link adopts the master station and slave station method, which means that a master station can expand up to 128 control points, reducing network burden and improving efficiency. At the same time, thanks to the standardization of IO-link communication, it is more convenient to configure such as RFID, valve island, and sensor, not only for status monitoring, but also for parameter configuration and maintenance.

Tianjin Yike’s IO/Link is being widely used in power lithium battery factories. The factory production line in the lithium battery industry is relatively long, the detection points are many and dense, and there are certain requirements for dust protection. At the same time, the degree of automation is high, requiring the cooperation of various components, and hot-swap continuous production and a certain cost performance are also necessary. . Based on the above requirements, the IO-Link solution is very suitable for the characteristics of the lithium battery industry. The great development of the domestic power battery industry has also brought huge opportunities to domestic sensors.

The wireless sensor network is also developing rapidly, and it has very high requirements for low energy consumption. As you can imagine, there are many development scenarios.

Figure 3 Different wireless standards (ISM industrial, scientific, medical bands)

The temperature of thousands of cows needs to be monitored on a daily basis to prevent animal diseases such as foot-and-mouth disease. If the technology of wireless network is used, as long as a temperature sensor with a wireless transmitter is installed on each cow to read the body temperature at any time and transmit it to a main terminal, it can be easily realized. This kind of low-energy sensor cluster can greatly improve the efficiency and speed of wireless networks, but such sensor networks often have to ensure low power consumption before they can be widely used. The two most crucial characteristics of WSNs are reliability and low power, and cost is the third most important.

Non-contact sensing technology, whether it is light, wave, magnetism, laser, sound, etc., is developing rapidly. Infrared temperature sensors have been shining brightly in the temperature detection of the epidemic in the past two years. With higher accuracy and more application scenarios, non-contact sensors will also grow substantially. Similarly, rapid detection of biosensors is also developing rapidly in order to facilitate timely diagnosis, in vitro diagnosis.

Sensors in drones are also a focal point. In fact, a drone is a collection of sensors, which can be thought of as a flying sensor swarm. Drones make extensive use of lidars, tilt sensors, inertial measurement units, and more. Shenzhen DJI is undoubtedly the leader, and US arms suppliers Lockheed Martin and Boeing are also actively participating. The drone market has undoubtedly exploded from the photo entertainment, and the use of the military has greatly increased its attention. This market is still in its infancy, and there are huge opportunities for various sensors.

Lidar, as a non-contact sensor, is being promoted. It captures millions of data points in real-time for the most immediate application in self-driving cars. At present, the domestic progress is not bad. Huawei, Sagitar Juchuang, and Hesai Technology all have very good products. In the field of handling robot AGV, intelligent warehouse management has also promoted the application of lidar, including Germany’s largest sensor manufacturer Sick Optoelectronics Sick, Japan’s largest lidar manufacturer Beiyang Hokuyo, etc., have been widely used. Drones and robots are also good application areas.

There is also a sensor direction, which is a friendly natural interface, which is developed around the capture ability of human senses. Since the birth of the computer, the keyboard is the first for the interactive interface, and the mouse is the second round of the graphical interface. But since then, there has been no decisive progress in human-computer interaction. Who can become the third place in human-computer interface? Voice, touch, and gestures, these highly biometric natural interfaces are all possible. However, the performance of intelligent voice in speaker control was disappointing, and voice control did not detonate the family as expected. The era of smart speakers opened by Amazon’s Echo, as well as Google’s Nest, as well as domestic Xiaomi and other speakers, have all proved to be nothing but a vain carnival of interface interaction. The interface of the smart home is still waiting for a new master. Sensors with natural interfaces still have huge room for development.

Notes: Dark Pit

The role of sensors cannot be overemphasized. Sensors are the five senses of the automation system and the vanguard of digital technology. It is also a super shortcoming that China does not notice, one of the reasons is that its applications are too scattered. There are more than 30,000 kinds of sensors in the world, and the applications are small and narrow. However, the pits it left in the intelligent world are all silent. Just looking at the sky and chasing the stars, it is easy to fall in these dark pits.