Failed to take over an innovative project, turned around and questioned research fraud? Google’s research in the journal “Nature” sparked a big discussion

Google employees allege that a senior researcher “digged holes” for two other junior AI researchers earlier this year—suggesting that their research was wrong or even falsified.

Attacking project creators for not taking over innovative projects?

In late 2018, Google AI researchers Anna Goldie and Azalia Mirhoseini finally got approval to test their sparked idea. The idea is mainly to use AI to improve the research and development efficiency of the computer chip TPU invented by Google, which is the later Morpheus project, which was strongly supported by Jeff Dean, the head of Google AI, and attracted the company’s chip manufacturing team. interest of.

When designing chip circuits, engineers need to consider many requirements and constraints, and the entire work cycle is often months-long and extremely complex. In June 2021, Goldie and Mirhoseini published a paper in the journal Nature as lead authors, saying that reinforcement learning techniques can be used to achieve chip design capabilities beyond Google engineers and reduce the entire work cycle to a few hours.

Such a result naturally won the attention of the semiconductor industry and many technology media. Andrew Kahng, a professor at the University of California, San Diego, predicted in a review of the paper that this technology will soon land in the field of chip manufacturing. He wrote, “For those who have been working on chip design for many years, the results of Mirhoseini’s team look really magical.” In fact, many of the TPU chips currently used in Google’s data centers were designed with the assistance of the Morpheus project. Samsung and Nvidia have also separately said they are using reinforcement learning techniques to optimize chip designs.

However, Mirhoseini and Goldie, two blockbuster researchers, never imagined that their brilliant success would cause a major trouble that would last for several years.

Google senior researcher Satrajit Chatterjee personally attacked Mirhoseini and Goldie, citing scientific discussions, employees confirmed. The Google employees spoke anonymously because they were not authorized to discuss company affairs. Some employees mentioned that Google’s personnel department has received multiple complaints about Chatterjee’s malicious attacks and issued written warnings to him, but this guy still goes his own way.

The conflict culminated in March when Chatterjee obtained permission from the research manager to openly fire at Mirhoseini and Goldie’s Nature paper. Google has set up a special executive paper review committee for this purpose, and members believe that Chatterjee’s conclusions are untenable and cannot refute the early research results in the paper. Chatterjee was fired from Google that same month.

Google hired Satrajit Chatterjee as a senior machine learning researcher in 2018. He previously served as senior vice president at hedge fund Two Sigma and also at Intel Corporation. When Chatterjee started, Mirhoseini and Goldie were already working at Google Brain, Google’s most famous machine learning lab.

The two employees have no direct work intersection with Chatterjee. But Goldie mentioned in an internal document that Chatterjee had asked to take over the Morpheus project in 2019. After being politely declined, Chatterjee began “breathing” other senior researchers involved in project collaboration and support, questioning the duo’s work, implying errors and even falsification, employees testified.

Chatterjee’s remarks do have an impact because of the seniority. Staff recall that other senior researchers did question Goldie and Miroseini from time to time.

That made Miroseini and Goldie’s jobs at Google increasingly stressful and difficult, insiders say. While Google’s team of chip designers has been staunchly supportive, they have had to expend a lot of extra effort to respond to these accusations of mistakes, and even counterfeiting.

Non-stop discussions within Google

During this time, Goldie posted a document on an internal Google discussion list describing the review board’s refutation of Chatterjee, while accusing him of an unsubstantiated series of attacks on the Morpheus project co-leader and work. Goldie wrote, “Sat Chatterjee launched a so-called ‘fake outcry’ against me and Azalia for over two years. Not only is this a discrediting of our work, it is a baseless claim that Azalia’s and my research results are falsified. “

At the time, Google employees were responding to a post in The New York Times in which Goldie’s documents were published. News of Chatterjee’s firing was first reported by The New York Times, and his lawyers determined that the Google research team was attacking him personally and preventing him from participating in scientific discussions. But it turns out that most of the Googlers who joined the conversation expressed support for the two women researchers and their work. Some former and current Google researchers even made their stance public on social media.

Chatterjee’s attorney, Laurie M. Burgess, declined to make her client available for an interview, denying Chatterjee wrongdoing, and citing Chatterjee’s evidence that Google was unfairly suppressing him. But Burgess said it did not want to share the evidence and did not respond to emails asking for details about the incident.

Asked about Chatterjee, Google spokesman Jason Freidenfelds pulled out a company statement confirming that the person was “fired for cause.” Friedenfelds also provided a statement made by Google Research vice president Zoubin Ghahramani, saying that “we firmly uphold the right to free discussion among researchers, premised on respect.” But Ghahramani did not mention Chatterjee by name in the statement.

Chip design teams are inherently cautious because nanoscale manufacturing is so expensive that any errors in a chip once etched into it can never be fixed. Google has said that TPU has made major breakthroughs in AI research and services, and leases this part of computing resources through Google Cloud Platform cloud services. But even as Google’s hardware leaders put their full trust in the Morpheus project and decide to use it to help design the company’s next-generation TPU, Chatterjee’s criticism of Morpheus hasn’t stopped for a moment.

In May 2021, a Google employee posted on an internal mailing list asking if anyone had tried machine learning to aid circuit board design. Mirhoseini replied that the Morpheus project should help. But Chatterjee immediately jumped in, claiming that traditional technology outperformed machine learning tools, and it was best to directly choose commercial chip design tools on the market.

Google AI chief Jeff Dean quickly joined the discussion, saying that Google was already using Morpheus to design its next-generation TPU chips. Dean also mentioned that the Morpheus technology has already outperformed human chip experts and commercial chip design tools in extensive testing, along with a presentation of the results.

Dean also posted the just-published, peer-reviewed Nature paper by Mirhoseini’s team. The report pointed out that the Morpheus team’s results are superior to Google engineers using commercial chip design tools in the arrangement of TPU circuit blocks. The paper does not disclose the details of these chips, because they belong to Google’s internal confidential information, but attaches the open-source processor design results that can be accessed. The conclusions of the paper were subsequently successfully reproduced by another research team within Google, and the experimental code has also been released as open source ( https://github.com/google-research/circuit_training).

The source mentioned that Chatterjee then targeted the paper in the journal Nature and wanted to team up with several other engineers inside and outside Google to fight it with a new paper. In late 2021, Chatterjee finished writing and wanted to publish outside the company. As a result, Google formed a special committee of five senior chip and AI experts, pledging to weed out managers Chatterjee claimed to be biased, and to review the rebuttal paper impartially. During the period, the special committee also personally tried to reproduce the research results of the Morpheus team.

An anonymous draft of Chatterjee’s paper was subsequently leaked online, showing actual results from different experiments and tests, claiming that the Morpheus algorithm was actually inferior to traditional chip circuit layout software.

After a three-month review, Google’s special committee said Chatterjee’s rebuttal paper could not be published outside the company. The reason for this is that the new paper does not completely reconstruct the entire experiment, so the experiments and data provided do not refute the conclusions of the Nature paper. But the committee gave Chatterjee and the co-authors the opportunity to make further changes.

In March of this year, the special committee decided that the revised paper could only be said to be slightly improved and still fell short of publication standards. The committee explained in an email that “papers written by Google Research have to meet very strict standards, which is why the special committee put a lot of effort into reviewing this article.” Chatterjee was fired from the company in late March. Goldie referred to the special committee’s decision in a document posted to an internal discussion list in early May. She also said that multiple Google managers had investigated and reproduced her team’s results and determined that Chatterjee’s claims were “completely unfounded.”

Zoubin Ghahramani, vice president of research at Google, also joined the discussion and thanked Goldie. He also issued a statement to The New York Times and Wired, saying that Google supported the Nature report and recognized the authors of the paper for designing more practical and efficient AI hardware for the company. Project researchers have published peer-reviewed papers and filed related patent applications.

But another person familiar with the matter said differently, arguing that Google executives clearly supported the original paper, making it difficult for opponents to actually discuss specific issues about chip design in the rebuttal paper. The source also mentioned that they have never seen Chatterjee act inappropriately.

The conflict between Google executives restricting what researchers can publish has long gone public. Google said last year that it would tighten its pre-review process for published content following an outcry over the forced departures of Timnit Gebru and Margaret Mitchell, co-heads of Google Brain’s ethical AI team. Gebru and Mitchell co-authored a research paper criticizing the AI ​​techniques used in Google Search and other products. Faced with internal accusations, Google not only kicked the duo out, but also asked the paper to remove their names.

A memo from Dean, Google’s AI chief, also drew unanimous derision from researchers inside and outside the company, writing that Gebru and Mitchell’s paper “doesn’t meet our publication standards.” Afterwards, thousands of Google employees and outside AI experts signed an open letter criticizing the company for fooling itself. The controversial paper was subsequently accepted by a leading peer-review conference, leaving Google out of control.

It’s unclear whether Chatterjee will publish the paper outside Google’s control or submit it for peer review. But at least inside the company, the algorithms Goldie and Mirhoseini devised are still running, helping Google design future generations of chips.

Reference link:

https://www.wired.com/story/google-brain-ai-researcher-fired-tension/

The text and pictures in this article are from InfoQ