Welcome to the WeChat subscription number of “Sina Technology”: techsina
Text/Zhang Zhiwei, Ma Bingying, Ye Lei
Source / Yuanchuan Technology Review (ID: kechuangych)
For AR/VR, the technology circle can be described as “love-hate”, looking for hope in disappointment again and again.
The depth of love is evident, and since 1968, the exploration has not stopped. This year, Ivan Sutherland, the father of computer graphics in the United States, led a team to develop a “head-mounted three-dimensional display”, which became the pioneer of AR glasses. The device was so bulky that it needed to be suspended from the ceiling, and the user had to be strapped to the device to view the cube image steadily, so much so that it was dubbed the “Sword of Damocles”, which could be viewed from a distance.
Subsequently, overseas giants such as Google, Microsoft, Facebook, and Apple, domestic bigwigs such as Byte, Huawei, and Xiaomi, as well as batches of entrepreneurs, rushed in one after another to promote AR/VR into daily life. Even Lao Luo, who is “true”, announced in a high-profile manner that he would start his own business again and enter the AR field, and he has also received investment and optimism from companies such as Meituan.
Not only entrepreneurs are optimistic, but the government is also frequently sending spring breezes. On November 1, the Ministry of Industry and Information Technology, the Ministry of Education and other five departments also issued a document “By 2026, the virtual reality industry will exceed 350 billion yuan, and the terminal sales will exceed 25 million units.”
But AR/VR is also really “hate it all”. More than 50 years later, there are still no attractive consumer products. There are many closed companies and frustrated entrepreneurs.
Even though Facebook changed its name to Meta and said in a high-profile manner that it is all in the “Metaverse”, the sales of AR/VR products were quite bleak, and the Metaverse division lost nearly $20 billion in two years. Investors were dissatisfied with this development, and a long-term shareholder expressed his desire to limit investment in Reality Labs. Dissatisfied investors voted with their feet. On October 27, the stock price plummeted 20%, hitting a six-year low, and the market value was only 1/4 of the high.
What is the magic of AR that makes the giants obsessed? What has changed in the current AR field, is there a window for explosion? Why is it worthy of strong policy encouragement, and what will it bring?
Why do big factories do AR?
In 2021, Meta almost single-handedly shakes the “Metaverse” banner all over the world, and also brings the AR/VR equipment that has been silent for many years back to the public eye. It has been ten years since the first consumer-grade AR device, Google glass, was released. During this period, giants such as Microsoft and Apple also kept trying. Different poses, different purposes.
For Meta, AR/VR is not so much the next-generation terminal and the entrance to the metaverse; it is rather an opportunity to get rid of the “stuck” between Apple and Android platforms.
When Meta was still called Facebook, the advertising business was the lifeline, contributing more than 90% of its revenue. But in 2021 encounters “betrayal”.
This year, Apple updated the IDFA (Identifier for Advertisers) privacy policy, which made it impossible for advertisers to track user IDs for portrait analysis, which would also reduce the accuracy of ad delivery. Without data, the foundations of the digital advertising industry are greatly shaken. Misfortunes do not come singly, and then Google also announced that it will adjust its privacy policy in the next few years. Former partners have put up fences.
Meanwhile, in the social media space, Facebook also faces huge competition.
The sudden emergence of Tik Tok overseas has taken away a large amount of traffic from Meta’s established social platforms Ins and Facebook; and Snapchat is also grabbing Facebook’s territory by relying on its unique “burning after reading”.
In the social media service business, there is “betrayal” later, and “grabbing rice” before, so Zuckerberg had to consider “changing the track”, and the all in Metaverse officially opened. A name change was necessary to show resolve, and Meta followed.
As far as the entrance to the metaverse is concerned, whether it is VR with relatively lower technical difficulty and more stable application scenarios, or AR with more possibilities, Meta has a layout. First, it invested heavily to promote VR, and just entered 2021, it spent more than 10 billion US dollars on its virtual reality division, Reality Labs. Then there are two paths to cut into AR: first improve the function and first make the appearance.
The former is the project code-named Nazare, which will be promoted as a demo machine in the latest release strategy, mainly for developers, aiming to accumulate AR content ecology [1]. The latter is a cooperation project with EssilorLuxottica, the European glasses giant and Ray-Ban’s parent company. It has launched Ray-Ban Stories, smart glasses that can shoot, play music, and answer calls, and will further expand in the direction of AR [2].
Regarding more specific ideas for future products, Meta scientists believe that AR terminals should achieve all-day battery life and better visual effects in the size of current glasses. But the skinny reality is that the rough animations on the Meta metaverse platform have been ridiculed by the crowd, and investors are also skeptical and opposed. Even the relatively mature VR is not optimistic, and the distant AR business is even more worrying.
Google’s exploration of AR continues the “geek style”.
In 2012, Google X Labs released Google Glass, which shocked everyone. Babak Parviz, the initiator of the project, put forward two visions for this product when he participated in the Stanford University Hot Chips Engineering Conference in 2013: a visual communication device , and can access various types of information more quickly [3].
As a pioneer, Google Glass lived up to many expectations. But unfortunately, after this product was officially released, it was also criticized because of its high price, ordinary experience, and user privacy issues. However, Google still relies on the open source advantages of the Android system to silently build the AR industry outside the hardware.
Why is Google still holding on? In a sense, it is also “have to”: if Google as a platform is stuck in the neck of Meta, then the hardware as a terminal is Google’s neck stuck by Apple.
According to foreign media reports, since 2014, Google has begun to pay a huge “protection fee” to Apple to ensure its default search engine status and thus protect advertising revenue. This fee has risen from 1 billion to 8 billion to 12 billion US dollars in just six years, contributing 14% to 21% of Apple’s annual profit[4].
Although the two giants have “cooperated” in the search engine business for many years, and even faced monopoly doubts together, in the face of the huge temptation of “next-generation terminals”, it is a reasonable choice to seek development separately.
In 2017, Google benchmarked Apple’s ARKit and released the AR application development platform ARCore, which enriched the AR content ecosystem in the Android system by providing tool support for developers. The platform supports developers to rapidly develop related AR applications through technologies such as motion tracking, environment understanding, and light estimation. With the advantages of the Android system, ARCore has quickly reached a large number of developers and users. By the end of 2020, the cumulative number of installations of the framework has reached 1 billion [5].
And this year, Google is “getting up wherever you fall”. At the Google Global I/O Developer Conference five months ago, Google once again released new AR glasses, featuring real-time speech-to-text and translation functions, which can “subtitle” conversations in the real world. In addition, it is also preparing to test the actual effect of AR navigation function in Canada [6].
The combination of software and hardware strategy, Google not only wants to continue the glory of Android in the mobile phone era, but also wants to grab the hardware back and be on an equal footing with Apple.
As a veteran operating system giant, Microsoft has always wanted to get rid of the “old” question and keep trying new things: acquiring Nokia, entering AR, transforming the cloud, acquiring Linkedin, etc. AR is just one of many new attempts.
Talking about Microsoft’s story in the AR industry, it is impossible to bypass the product HoloLens released in 2015, and it is impossible to bypass the “soul man” in the team: Alex Kipman. Kipman joined Microsoft after graduating in 2001. For 20 years, he worked in Visual Studio, Windows and Xbox departments, leading the research and development of the somatosensory peripheral product Kinect and the AR product HoloLens.
It is not difficult to see from the department that Microsoft’s AR road originally had a strong game gene, and the overall positioning of the device was also biased towards consumer products.
But contradictions are also emerging here. Other executives, more pragmatic, believe that to B scenarios and the U.S. military are where the bets should continue. After all, the cost of money and labor required to develop a real AR product far exceeds Microsoft’s expectations, and the product form that is too advanced cannot be realized in the consumer market in the short term.
In May 2022, Microsoft also announced that Kawasaki became a customer of the company’s “Industrial Metaverse” business, and employees would wear HoloLens augmented reality to work; as a result, in June, the soul of HoloLens, Kipman, announced his departure from Microsoft, and the HoloLens team will also be “” Software and hardware decoupling”, the hardware is merged into the Windows+Devices department, and the software group joins the Experiences+Devices department, completely turning to enterprise applications that can drive growth [7].
There are other reasons for Kipman’s departure. But it has to be said that Microsoft’s AR business has lost its ideal to reality. In particular, the rapid development of the company’s cloud business has freed Microsoft from its “old” appearance. The exploration of AR can also be slowed down again, and rely on a more familiar path to make money by relying on horizontal technical capabilities.
The giants have their own goals and keep throwing money, so what has changed in the AR field?
How is the present different from the past?
How has AR changed over the years? Let’s talk numbers first.
In the consumer-level field, domestic start-ups such as Tairuo Technology, Lingban Technology, and Yingmu Technology have held frequent press conferences in recent years. We can use Nreal’s new products to compare with the consumer-level originator Google glass; in the commercial/industrial level, we use foreign unicorns. The new beast Magic Leap is compared with the original Hololens.
In the price-sensitive consumer market, the price of AR has dropped significantly. Domestic companies generally target mid-to-low-end mobile phones and choose the price range of 2,000-3,000 yuan for “early adopters” to increase their audience, and generally use Micro OLED with Birdbath optical solutions, striving to provide higher contrast and high resolution at this price range. , Excellent image quality with wide color gamut.
But in terms of effect, although Nreal Air uses a quarter of the price to achieve several times the display effect of Google glass, it still cannot achieve the interactive experience like the industrial-grade HoloLens.
The reason is naturally the balance between performance, weight and cost of consumer grades. Limited by the energy consumption ratio of chips such as SoCs and sensors, and the energy density of lithium batteries, these lightweight glasses under 100g cannot have both. Therefore, in order to ensure wearing comfort, manufacturers have agreed to focus on weight and display, and let the glasses act as near-eye display screens for mobile phones and computers, avoiding the problems of computing power and battery life.
In the commercial market where function is greater than experience, experience is greatly improved. For example, Magic Leap 2, which is free from price constraints, can continue to make efforts in terms of full functionality. The first is to optimize the optical waveguide solution to greatly increase the field of view and brightness, which can adapt to more complex and diverse production environments. Secondly, a large number of sensor units and customized computing power chips are added to provide a wider and more accurate environment perception and human-machine. Interaction ability.
But unfortunately, the industrial grade still faces the dilemma between battery life and performance. The more functions, the more energy consumption, and the increase in weight becomes an inevitable choice when the energy density of the battery is not greatly improved. If it is too heavy, it will reduce the experience.
Finally, whether it is consumer-grade Nreal or industrial-grade Magic Leap, the application ecology is still poor. Content on interacting with the physical world and augmenting information is particularly sparse. At this stage, the consumer market mainly uses the content migration of mobile phones and computer software, while the commercial market relies on the ability of eyewear companies to customize and develop for specific business scenarios.
In summary, technology is developing, but demand is also increasing. AR glasses, whether in consumer or industrial grades, still have insufficient batteries and poor functions.
What to expect in the future?
When it comes to AR, the most anticipated is Apple.
Apple’s entry into AR can be counted from the application for related patents in the field of AR/VR in 2006. The long-rumored MR head-mounted display project code-named N301 has also been established as early as 2015. Although there have been no major achievements in the past few years, considering Apple’s past innovation capabilities and its strong appeal to consumers, the future of the AR field can still be expected to bring changes from Apple. For example, it brings breakthroughs in perception, interaction, software, and computing power.
Perception: The premise that AR glasses provide relevant information. At present, the real-time positioning and map construction (SLAM) algorithm is generally used in the industry to give machines the perception ability. The implementation of the SLAM algorithm relies on sensors such as cameras or lidars. Lidar is a hidden stumbling block for the application of high-precision SLAM in AR glasses at this stage.
Lidar is a key role in providing depth information for SLAM algorithms, but limited by size and weight, it is widely used in robots, smart cars, military equipment and other fields.
In 2017, Apple began to use the US Advanced Manufacturing Fund to invest 800 million US dollars in Finisar, a VCSEL optical chip company, in two batches, occupying a place in the key field of LiDAR, and in the single photon credit diode (SPAD), time management circuit (picosecond) Continuous research and development and patent application in sub-fields such as grades).
Finally, starting with the iPhone 12, each generation of iPhone is equipped with a highly integrated Lidar module, providing users with faster focusing when taking pictures, providing adaptable hardware devices for AR content developers, and achieving high-precision SLAM for future AR glasses. The algorithm lays a good hardware foundation.
Interaction: Brand new problems facing end companies today. Long-term wear and efficient interaction to meet privacy needs are a set of contradictions. At present, the spatial gestures, gloves, mobile phones and other remote controls, voice and other solutions found in the industry do not seem to be excellent answers recognized by consumers.
In 2017, Apple submitted a patent for a finger-controlled device, which was applied to the Apple Watch S6. This patent allows the watch to use the position of the blood vessels in the arm to calculate the movement of the wrist and fingers, and then confirm the gesture action and give corresponding feedback. Kevin Lynch, VP of the Apple Watch project, said that the Apple Watch has been considering richer forms of interaction since the first generation (2015), and the overlap of the timeline seems to verify his statement.
From the perspective of glasses, muscle manipulation can get rid of the dilemma of “seeing the hand” to interact, and micro-motion control also saves time and effort for interaction, adding more usage scenarios. However, from the user’s point of view, the accuracy of muscle control such as gestures, the integration of various interaction methods, and the protection of interaction privacy are the areas of pain that need to be improved urgently.
Software: Unlike flat displays such as mobile phones and computers, AR glasses require more acquisition and display of three-dimensional information, which brings changes to current software development tools and application platforms.
In the field of AR software ecosystem, Apple exerts the advantages of vertical integration of its software as much as possible to build a full-process development system for AR content.
For developers, you can call all perception information of Apple devices through ARKit, and use its auxiliary computing functions in terms of spatial position, ambient light, occlusion, etc.; for professional creators, you can use RealityKit’s 3D modeling and simulation functions to render interactively For ordinary creators, you can create or find suitable preset models in Object Capture and Reality Composer for simple model making.
Apple unifies all the above AR content files into the original “USDZ” format. On the one hand, by preemptively formulating standards, it solidifies the existing development and creation ecological advantages and reduces the loss of development resources to other platforms; on the other hand, the “USDZ” format The files are unobstructed in the Apple system and can be viewed through a variety of browsers, memos, emails, messages and other interfaces, providing users with great convenience.
With a huge base of developers and hardware device users, before the launch of the glasses, Apple gradually built a virtuous cycle of software development and application feedback on the mobile phone side, avoiding the “chicken or egg” problem faced by future AR glasses. .
Computing power: AR glasses are often benchmarked against mobile phones and positioned as the next super terminal. In addition to higher-definition display, more dimensional information, and lighter form, AR glasses that can be worn by us for a long time in the future will affect computing power and computing power. The power consumption ratio has put forward extremely high requirements.
In the field of high-end SoC chips on the mobile terminal, the world has basically formed a duopoly situation between Apple and Qualcomm, but friends who know a little about mobile phones know that starting from Qualcomm’s “Snapdragon” (also dubbed Fire Dragon) 888, Apple’s A-series chips are in The performance and power consumption ratio are far ahead.
However, in the face of possible real-time perceptual computing, multi-dimensional image rendering and analysis in the future, the performance improvement brought by the miniaturization of the CPU process is not what Apple is most concerned about. Starting from the A12 chip, Apple began to focus on the improvement of deep learning capabilities, developed the Core ML framework to intelligently allocate artificial intelligence tasks, added machine learning accelerators to the CPU, and completed the neural network engine (NE) in the next three years. The speed is more than 20 times faster. promote.
The power consumption ratio is the traditional advantage of Apple’s chips, but once the M-series chips for the desktop side were launched, it still refreshed the industry’s perception of Apple’s chip design capabilities, because it created a low-power and high-integrated ARM-based SoC to challenge high power consumption. The feat of the X86 architecture CPU, GPU discrete system.
It is undeniable that Apple’s algorithms and applications for perception and interaction are still being explored, and its software modeling capabilities have just started. The area of the M1 chip is too large for mobile phones and glasses. However, the efforts of Apple and other industrial chain companies will pave the way for the AR world in the next ten years. Added some certainty.
end
People’s imagination about AR can be traced back to 1901. The author of The Wizard of Oz, in his other sci-fi novel, The Master Key, describes a pair of magical glasses that allow the wearer to see through the glasses the personality markers of everyone in front of them—good people show G (representing good), and the bad guy is E (representing evil).
Clearly, literature gives AR the ultimate philosophical question. But the real problem is to make AR products that users say “Good” first. Compared with “stacking, running points and showing off skills”, returning to the ultimate needs of users is the key: fun, interesting, and perhaps not enough to improve efficiency.
Just like computers replacing paper, pens, etc., greatly improving office efficiency; from 2G to 5G, not only the numbers have become larger, but also the richness of information transmission and the efficiency of communication; electric vehicles have replaced fuel vehicles, and production efficiency has improved, The improvement of energy efficiency is also a driving force. The popularity of sweeping robots also stems from the fact that people can make better use of their time and are too lazy to be more efficient.
Unfortunately, AR may still need to be explored for a long time; fortunately, in addition to AR/VR, it seems that there are no other products that can play the important role of “next-generation consumer products” and the entrance to the metaverse. Although there is a popular saying in the investment circle that “this is a bad era”, in the field of AR, it is still the future that pursues “good”.
This article is reproduced from: https://finance.sina.com.cn/tech/csj/2022-11-07/doc-imqqsmrp5282270.shtml
This site is for inclusion only, and the copyright belongs to the original author.