Original link: https://www.latepost.com/news/dj_detail?id=1654
One afternoon in October last year, Liu Du (pseudonym), the original artist of the game, was editing a picture. A game planner came to the seat next to him, pointed to his colleague’s screen and said, “This is a picture made by AI. If you fix it, we will use it. gone.”
He saw three game scene pictures on the screen. The style of painting was realistic, which didn’t match the company’s usual style, but the details were rich enough. The colleague next door used Photoshop to add filters to the three pictures to turn them into a cartoon style, and the changes were completed in an afternoon and sent to the planner.
Liu Du realized that the career growth path he envisioned would not work. He has just graduated for more than a year, and plans to hone his hand-painting skills in this medium-sized game company that focuses on the overseas market, and then jump to a larger platform.
“It’s hard to go on learning hand-painting after working hard.” Liu Du drew a similar picture by himself, and it took three days from conception to painting to polishing the details.
A month later, ChatGPT was released, which quickly raised concerns around the world about AI replacing human creative work. The game industry, where the production cycle of a single work generally exceeds one year and requires hundreds of people inside and outside the company to draw at least thousands of pictures, is particularly concerned about this.
From the heads of game outsourcing companies to art producers and team leaders of game companies of different sizes, the dozen or so practitioners we interviewed were all shocked by the level of AI production in the past six months.
After the shock comes anxiety. After anxiety, what some of them feel is, excitement and fulfillment.
Half a year later, Liu Du no longer worked in the game company. He voluntarily left his job to join a cultural tourism creative company and became a “single-person creative team”. On average, he used Midjourney to generate hundreds of pictures every day, selected a few of them, and used them in the company’s marketing posters after modification.
Liu Du mainly relies on self-study. He first spent more than 30 yuan to buy tutorials from Taobao, and then followed the video experiment of the UP master of station B to keep up with the progress of technological updates as much as possible. Recently, he set up an “AI painting exchange and sharing” group to follow the latest developments with interested friends.
The new job is easier, but the pay is higher. Liu Du said that he got the job because he learned AI. Now he has found the joy of painting again, without using AI, not for work, and painting what he wants to paint.
Tencent Games and Netease Games have already promoted AI drawing tools internally. A large number of game companies put pictures and text generated by artificial intelligence into games and promotional posters.
An artist from a game company in Shanghai said that the company does not encourage employees to use AI drawing tools because “Michelin three-star chefs do not use MSG in cooking.” He still saw many colleagues secretly using related tools to make work faster.
The manpower-heavy outsourcing industry is the hardest hit. The person in charge of a game art outsourcing company in Chengdu has lost many clients in the past two months, all because the client company started to use AI for drawing. He has already suggested to several familiar old employees to plan for the future.
Just like every productivity revolution, some people lose their jobs and some people get development opportunities. Ultimately, the job is taken away by someone who is better at using the new tool.
In half a year, artificial intelligence has changed from a toy to a tool in the game industry
In October last year, Xindong CEO Huang Yimeng said on social media that AI painting is very popular, “It’s time to find a reason for myself and reconfigure a 4090 (graphics card) computer.”
Xindong Company invests in making games while operating a mobile game distribution platform, with an annual income of 3.4 billion yuan. Huang Yimeng has been in business for 20 years since he co-founded the resource sharing community VeryCD with his classmate Dai Yunjie in college. He’s always been keen to experience the latest tech gadgets and games for himself.
However, after the new computer was delivered home, Huang Yimeng mainly used it to play games. “At that time, I didn’t expect that the influence of AI painting would be so great.”
Making a modern game is a huge project, and the main steps require the participation of the art team:
- Concept and planning: According to the preliminary discussion of game planning, make concept maps, so that the art, planning, and development members of the entire team can intuitively feel the atmosphere of the game, target audience, etc.
- Original painting: draw the main characters and scenes in the game, and confirm whether it can be realized in the game with the model, development and other parties.
- Production stage: According to the original painting, make detailed pictures of all characters and scenes from different perspectives, and then hand them over to the model team to make 3D models, which will become characters and objects in the game. Every prop and skill icon in the game must also be drawn by the art team – there will be thousands of them in a complex RPG game.
The scene concept map of The Legend of Zelda: Breath of the Wild, original character paintings, and action 3D model design drawings. Source: Nintendo
The actual scene of “The Legend of Zelda: Breath of the Wild”. Source: Nintendo
“This is a labor-intensive industry.” A person who was in charge of graphics in an international 3A game company said. The core team of game production he was in was about 40 people, and only 3 people were in charge of art. But there is a huge art production system behind these 3 people: there is a 150-person art middle-stage team serving them in the team’s location; there is an in-house art team in Shanghai to serve these 150 people; in other cities in China, such as Nanjing, In Chengdu, there is also a team working for the insourcing team in Shanghai. Many people in the game industry estimate that 50% of the cost of developing a game is spent on art.
More than half a year ago, AI mapping applications could not precisely control the style or content of generated images. All AI tools can do is generate concept maps at the first stage. This is the job that the art team invests the least in the game production process. “Actually, it’s not slow to find pictures on the Internet, just to give everyone a feeling.” A person in charge of the game art center told “LatePost”.
Liu Du’s game company was able to use AI tools for drawing first, mainly because they did not develop new games at that time, and the art work mainly drew concept maps and marketing maps. The most energy-intensive part of production cannot yet be accelerated by AI.
The thinking of many people in game companies is similar to that of Huang Yimeng. At that time, they did not think that AI would greatly increase productivity. At an industry event, someone from the Tencent game team said that “outsourcing is our artificial intelligence.”
By the end of 2022, when the world pays attention to ChatGPT, there will be a major breakthrough in AI mapping applications. A developer from the Korea Institute of Science and Technology used LoRA in Stable Diffusion. Developers only need to upload dozens of pictures of the same style into Stable Diffusion, and then they can train a model that specifically generates pictures of the same style.
LoRA was first invented by Microsoft for low-cost fine-tuning of large language models, which makes AI mapping controllable and cheap. To train a LoRA mapping model based on Stable Diffusion, a computer equipped with an NVIDIA 4090 game graphics card can complete it overnight.
The graph generated based on the LoRA fine-tuning model, the left is a hand-drawn image, and the right is an AI-generated image. Source: Game animator Otani Spitzer
Two months later, Lvmin Zhang, a doctoral student at Stanford University, developed the Stable Diffusion plug-in ControlNet, which allows people to precisely direct the AI model to generate image details. For example, users can ask AI to turn a white cat into a black cat, and the whole process will not change the appearance of the cat; they can also ask AI to color it according to the hand-drawn line draft.
Color your sketches with ControlNet control models. Source: ControlNet’s GitHub page
From Human Commanding Human to Human Commanding AI
In the office area of Xindong Art’s middle stage team, there are two screens on many artificial positions, one displays the game screen, and the other displays the code. After several months of exploration, Xindong gradually tried to embed Stable Diffusion into the game production process.
Apart from concept maps, the most mature process at present is to draw icons. Icons are pictures that represent elements such as props, skills, and functions in the game, and are the most common art assets.
“A good icon can improve the quality of the game, and it is also a part that consumes a lot of resources.” Zeng Zifeng, the person in charge of Xindong Art Center, said. If it is required to be well-made, each icon needs at least one artist to paint for a day. Now with the aid of artificial intelligence, the efficiency of Xindong’s icon production has increased by nearly 4 times.
Artists no longer need to color over and over again according to the prototype, and modify the details, which takes a lot of time. The artist can first draw a dozen complete pictures, adjust Stable Diffusion with LoRA, and train a model with a certain style. Then, they only need to draw a sketch, input it into the model, use ControlNet to control the generation details, and then suggest changes to the diagram generated by the model, let the model continue to generate, and finally finalize it by themselves.
Zeng Zifeng began to learn computer painting in 2000. After graduation, he first did user experience design, and later entered the game industry. He has experienced the impact of many technological innovations on commercial painting and design.
After the New Year this year, Zeng Zifeng introduced artificial intelligence to the head of the art team of a game. He asked the other party to draw a sketch of the backpack icon, specify the desired style, and enter the adjusted Stable Diffusion. In less than a minute, 50 backpack icons with the same style but not the same style appeared on the screen, “If you are not satisfied, just click again to have 50 more”. He saw the uncontrollable excitement on the other’s face. Within 5 minutes, the leader decided to adopt new technology and solve the backlog.
The drawing of icons was mostly done by outsourced teams before. After Zeng Zifeng determined that artificial intelligence tools can improve efficiency, he suspended some art outsourcing, “We need to understand how much research on AI technology will bring to the art process.” He plans to find out how much AI can improve production efficiency internally, and then discuss the new production rhythm and cost with the outsourcing team.
Stop the outsourcing together and the exciting game translation team, the replacement is ChatGPT. “Originally, there were people in-house responsible for the follow-up review and polishing. (Outsourcing) replaced by AI, no matter what, the efficiency will be improved.” Huang Yimeng said. ChatGPT can also name and write descriptions for scenes, characters, and skills in the game according to the settings, “planning must be useful.”
Ubisoft, which is more advanced in technology, has developed a text generation tool Ghostwriter (shadow writer) similar to ChatGPT to assist game developers in writing secondary dialogues of game characters. According to Ubisoft’s demonstration, the developer enters background information into Ghostwriter, and the application will generate several texts, allowing the developer to choose the more suitable one – this selection process is the human feedback to the AI, which can improve the ability of the model.
In industries with a high degree of division of labor, it is common for some people to direct others to perform more elementary tasks. The intervention of AI will not completely replace a type of work, but the manpower required for primary work will be greatly reduced.
The distribution and recommendation of games also requires the intervention of AI large language models. Xindong’s TapTap is the world’s largest mobile game player community, and the Chinese version is used by more than 40 million people every month. In the past year, by improving the recommendation algorithm and other measures, TapTap’s revenue increased by more than 40% to nearly 1 billion yuan.
Li Yunze, head of the TapTap algorithm team, was one of the first people at Xindong to realize the potential of ChatGPT. He left Alimama in 2020 to join Xindong, and led the team to rewrite the TapTap advertising recommendation algorithm. Less than a week after ChatGPT was released, he gave a technical sharing to the company’s executives. After that, most of his energy was devoted to studying how to improve the efficiency of TapTap with AI.
Li Yunze’s most important task now is to lead the team to summarize game-related text data and develop a new search function with an open-source large language model, so that users can find games and find strategies on TapTap more accurately.
“The game is also a learning product. Whether a game is fun or not requires in-depth experience and mastery of skills before you can have a clear feeling.” Li Yunze said that technologies similar to ChatGPT can reduce the threshold for users to learn a game, make the player audience wider, and thus improve the efficiency of the entire industry.
Technical problems that could not be avoided yesterday may be solved today
Technology is iterating rapidly. In the previous six months, Li Yunze and Zeng Zifeng had a common feeling.
After confirming with Huang Yimeng and other company executives that they would develop new TapTap functions, Li Yunze made a plan to use an open source large language model and train with proprietary data accumulated by Xindong. “At that time, I felt that calling ChatGPT’s API was too expensive.”
By the beginning of March, OpenAI reduced the API price of ChatGPT to one-tenth of the original price, and it was less than RMB 20 to process a million words. The open source plan he formulated was shaken, the biggest price problem has been solved, and Xindong began to use the ChatGPT service of Microsoft in China.
Now, the exciting solution is back to the open source model. Since Meta open-sourced LLaMA, a large number of projects are born in the open source community every day, and the effect continues to improve. Although there is still a gap in the effect, the open source community already has all the components to reproduce a ChatGPT.
“The most difficult thing is the learning process, because it progresses very fast.” Li Yunze said. He established a WeChat group with several large-scale model entrepreneurs to discuss the latest technological progress and papers. He didn’t read it for half a day, and there were 99+ unread messages.
“The CPU (brain) has not been running at full capacity for a long time.” Zeng Zifeng also said, “In the past few weeks, it has not stopped at all, because the information is updated too fast.”
When he exchanges technology with his colleagues now, he always emphasizes in advance that “what you say today may not be useful tomorrow.” He has experienced it.
In February of this year, Zeng Zifeng encountered a bottleneck not long after he wrote “use artificial intelligence to improve art efficiency” into the annual OKR. LoRA can control the style of the pictures generated by the model, but it cannot control the details, so it is difficult to use in the actual art production process. He and his colleagues tried various methods – experimenting with different prompt words, adjusting the way of training LoRA, trying to make the model more obedient, but the effect was not good enough.
Then ControlNet came along. “The effects and picture control that I racked my brains to achieve before are easily realized through it.” He said.
In order to allow team members to keep up with the speed of technological iteration faster, Xindong Art’s middle office department organizes AI technology exchange meetings every Friday to encourage colleagues to share skills and new technology information on using AI. The company has also established a Slack channel dedicated to discussing AI .
The scene where Xindong organizes the AI exchange meeting. Source: Xindong Company
Before AI drawing tools and ChatGPT improve the efficiency of game asset production, AI can already imitate voice actors to help game companies improve the efficiency of dubbing game characters, and play a role in 3D model building, animation production, testing and other links.
Now, the impact of AI is far from over. Heartbeat employees are researching how to use AI to generate pictures of game characters from different angles, and find a way to make 3D models at a lower cost. This is the shortcoming of current AI drawing tools-it can understand text and color, but it is difficult to understand space.
Companies with stronger technology have joined in. In the AI tool promotional video released by the game engine company Unity, the first sentence is “generate a 3D female character”. OpenAI also recently published a paper to study how AI can better generate 3D assets.
At the Global Game Developers Conference (GDC) held in March this year, AI was the hottest topic—from game environment materials, dubbing, NPC script writing, etc., all links that artificial intelligence can participate in were discussed. Every discussion about AI was packed, and hundreds of people braved the wind and rain to attend AI events sponsored by organizations such as a16z.
Liam Deane, chief analyst of game technology at Omdia, a market research firm, felt after visiting the Global Game Developers Conference that the industry generally believes that AI will have a major impact on the game industry, but few companies have yet done so. Come out with new technology-driven products such as ChatGPT or Stable Diffusion. According to Omdia, as of March of this year, only 5% of gaming tech vendors had made new tools. This means that there will be huge room for development in the future.
Building a product will be easier, but building a product won’t be easier
AI technology will simplify the process of making art, save time and money, and lower the threshold for developing games. But making a successful game couldn’t be easier. Even making a game probably requires the same number of people.
In the 1980s, Jobs bought Pixar. He believed that as computers became more capable of producing 3D images, the cost of producing special effects for movies would be greatly reduced. He was right and wrong.
Today, the production cost of movie special effects has been greatly reduced, and some pictures that were once unimaginable have been realized. But big-budget movies use more and more realistic computer special effects. Disney’s “Avengers 4” in 2019 cost more than $350 million in computer special effects alone, more than the total expenditure of any film before 2000.
Games are similar. 30 years ago, when a game company developed a game, the developer had to write all the code from scratch and deal with the details of the interaction between the software and various underlying hardware.
Thirty years later, game engines such as Unity and Unreal became popular, simplifying the underlying development work of games. Now game developers want to implement difficult technologies, such as real-time rendering of 3D scenes, which can be completed by various programming interfaces. The production cost of “Cyberpunk 2077” released in 2020 was as high as 174 million U.S. dollars, and the marketing cost another 140 million U.S. dollars. Before the popularization of game engines, $5 million to make a game was considered a huge sum.
As Midjourney founder David Holz (David Holz) summed up, after the emergence of new technologies, some people will try to cut costs, and some people will try to expand the scale of business. “People who are trying to scale their business, they will still spend the same amount of money. And those who are trying to cut costs, I think they will fail.”
A person in charge of the new product incubation business of a medium-sized game company in Beijing said that the game team he is in charge of has been using Midjourney to assist in the original painting for more than half a year, and now he has recruited a junior artist for Midjourney, who is mainly responsible for modifying the pictures. Next, he plans to spend more money to hire a more senior artist.
He feels that advanced artists have become familiar with different styles through a lot of hand-painting training, and can cooperate with artificial intelligence to efficiently handle various styles of image requirements, “the time saved can be used to do more valuable things.”
Hiring needs have also changed. A game original artist who is looking for a job said that he has interviewed four companies in the past month, and each business interviewer will ask him AI-related questions. The attitude of the boss of the outsourcing company in Chengdu is even more determined, “If the team enters again, it will definitely tend to be AI-savvy.”
Most people in this industry have come to accept that AI is the future of game production. A former Tencent game artist said, “Using AI to improve efficiency cannot be avoided.”
They expect to use senior personnel who cannot be replaced by AI to direct AI to make more complex and better products, and to seize the opportunity in this round of technological evolution.
As for how after AI takes over the primary work, how does a young painter find a job opportunity after graduation to practice his skills, improve his basic skills, “from primary to advanced”, and be competent for more complex jobs. This is a question that no one has yet answered.
The games industry has given a vague picture of how AI will transform creative work. This technological transformation will not stop at the gaming industry.
Topic picture: The drawing scene before the advent of computer drawing software. Source: Rare Historical Photos
This article is transferred from: https://www.latepost.com/news/dj_detail?id=1654
This site is only for collection, and the copyright belongs to the original author.