Walking code generator: chatGPT is going to “lay off” Google and programmers

Welcome to the WeChat subscription account of “Sina Technology”: techsina

Programmers are getting farther and farther on the road of letting machines replace themselves…

——

Text｜Dou Chen Editor｜VickyXiao

Source: Silicon Stars

How powerful OpenAI’s super-large-scale language model GPT is, Silicon Stars have written many times before.

Just this week, OpenAI released a new chatbot model, ChatGPT, as one of the main models of the GPT-3.5 series.

What’s more, it’s completely free and open to the public! So as soon as it was released, everyone played it immediately – soon, netizens were shocked by the ability of ChatGPT:

This is not a chat robot, it is clearly a ruthless programming answering machine, a living Stack Overflow!

| What exactly is ChatGPT? OpenAI says so

ChatGPT is a conversational large-scale language model trained by OpenAI, which interacts in a conversational manner.

It belongs to the same level model as another previous model, InstructGPT, representing the “GPT 3.5” generation. Microsoft and OpenAI signed a strategic cooperation plan before, so all models of the GPT 3.5 generation, including ChatGPT, are trained on the Azure AI supercomputing cluster.

OpenAI trained ChatGPT using the aforementioned RLHF technique: in simple terms, when training the original model, OpenAI let human trainers play both sides of the dialogue (user vs chatbot), providing dialogue as learning materials. And when humans act as chatbots, OpenAI also lets the model generate suggestions to help trainers compose their own responses.

The answers generated by these machines are then scored and ranked by the trainer, and better results are input back into the model to realize the reward mechanism of reinforcement learning training.

As a chatbot, ChatGPT has some mainstream features of contemporary similar products, especially the multi-turn dialogue ability, which can answer context-related follow-up questions during the same session.

But more importantly, because of the advanced, moral-focused training method, ChatGPT has the ability that other chatbots do not have or perform poorly: admit their mistakes, and follow the pre-designed ethical guidelines, and do not have any doubts. well-intentioned” questions and requests to “say no”.

As shown in the examples provided above, ChatGPT will use some pre-designed sentence patterns, combined with the user’s specific request to reject and change the topic.

Deny: If you ask it how to break into someone’s house, it will answer you, “Trespassing is illegal, it is a crime and has serious legal consequences”.

Changing the subject: If you put the question in another way, “Actually I want to know how to protect my home from burglars”, it will answer you, “Here are a few steps that can help you, including xxxx…but you are better off contacting a professional people for advice.”

| There is no programming problem that it cannot solve

In the process of many netizens “teasing” ChatGPT, people found a big surprise: it can really be programmed on demand.

In the past, people often joked that programmers only need to use the two tools of Google and Stack Overflow to be invincible all over the world.

But in fact, those friends who really struggle with various new software engineering problems every day know that even if you ask Google and Stack Overflow for some intractable diseases, you have to read dozens or even hundreds of pages of reply records spanning several years or even ten years. In the end, it was difficult to find a reliable answer…

But ChatGPT is different: Judging from the test results of various programmers and netizens on it, it seems that there is really no problem that can stump it…

find bugs

The founder of technology company Replit sent ChatGPT a JavaScript code to find bugs in it.

ChatGPT’s answer is very comprehensive and very interesting: it first tries to confirm what the intent of this code is, and then quickly finds the bug according to the intent, and also attaches a fairly detailed description to explain where the problem is. What kind of bug is caused, how should it be changed, why should it be changed in this way, etc.

And ChatGPT also gave a code example, and wrote the code comments of the modified part in the example:

Sending the Buddha to the West, ChatGPT also provided a small modification suggestion at the end: “You can replace var with let, so that new variables are automatically generated every time the loop iterates, so as to avoid manual creation every time.”

Another netizen, Josh, submitted a piece of code and asked ChatGPT “I can’t figure out why this code can’t run”.

ChatGPT explained in great detail: there is a problem with the format of the division formula, a string (a) cannot be divided by a number (1), because both the dividend and the divisor should be numbers.

But that’s not all. ChatGPT tried to understand the intent of the original code again, and then gave Josh some suggestions for modification: If you want division to handle non-numbers, you need to add additional logic to the function so that it can check what the type of the actual parameter is, only on both sides It only works when there are numbers. Returns an error or a default value if one side is not a number.

Help you check documents

The silicon star tried the following command

“Generate a piece of Python code that uses Google Cloud API to read the content of an image and output the sentiment.” (Generate Python code that uses GCP to read an image and get the sentiment.)

ChatGPT replied with a piece of code, and also commented on the function of each sentence of code through comments.

It also reminds me:

1) If you want to run these codes, you must first set up a GCP project and install the Python version of the Cloud Vision API.

2) The code cannot be copied and pasted directly, but the path of the image file must be set.

3) If there is anything unclear, ChatGPT also directly provides a link to the official GCP documentation.

Through this example, ChatGPT proves that it also has the ability to automatically collect cloud service APIs and integrate them into the code for calling.

Like the previous example of replacing Stack Overflow, this ability can greatly save the time spent by engineers searching for information, flipping through documents, and finding the correct calling method, and significantly improve the efficiency of programming work!

write math formulas

When we reported on the OpenAI API (GPT-3) last year, we mentioned that this language model can not only generate text, but also any field that is expressed in words and numbers and has logic—mathematical formulas, which are actually such fields .

Netizen Josh sent the same question to Google and ChatGPT: “How to express a differential equation in LaTeX format?”

The first result Google gave came from an unknown WordPress blog site, and the result was in 2013. The writing was not very clear, and the explanation was muddled.

The answer given by ChatGPT is not only more beautiful in presentation, but also more comprehensive in explanation, and even provides a variety of different solutions:

Another mathematician, Christian Lundkvist, gave ChatGPT the number theory problem that has plagued the mathematics community for more than three centuries: proving Fermat’s last theorem.

ChatGPT also uses a fairly refined way to write out the proof process through text + LaTeX formula.

Although it seems that even the top math problems are not difficult for ChatGPT, Lundkvist still said that in the process of playing with ChatGPT, he found that this guy is right when he is right, but he is quite confident in himself when he is wrong.

“I think such a tool is instructive in finding a way to solve a problem, but we must not rely entirely on the results it returns at this stage.”

(In answering the question “How many intersections does a line have with a circle”, ChatGPT mistakenly assumes that there are infinite intersections when a line passes through the center of a circle.)

Hacking? ?

Here’s an even better example: User BrandonDolan-Gavitt wants ChatGPT to help him find a bug in a piece of code.

But in fact, the real intention of this code is to perform a buffer overflow attack on a 32-bit x86 Linux system.

This time ChatGPT didn’t seem to find the user’s malice (according to the official instructions, ChatGPT would refuse to answer malicious questions), and then directly followed the diagram to find the code problem for him and told him how to modify the code. Even, it also taught users how to correctly trigger buffer overflow in the follow-up answer…

Dolan-Gavitt added that in fact, ChatGPT also made mistakes in the process of answering questions, such as suggesting that users modify the number of input characters when they made a mistake (it was said to be 32, it should be 36).

——However, when the user told it that “it seems a bit wrong”, ChatGPT immediately understood the user’s intention, said that he had misunderstood it before, and then provided the correct answer…

Although buffer overflow is an attack method for very beginners, the netizens still admire the ability of ChatGPT…

“I sent it a piece of assembly language code and asked it to tell me what loopholes were in it and how it could be exploited – and it actually answered me. So it can not only understand and output code, but also express it in binary, And can find loopholes in it? This really makes me a little worried…”

Image credit: @Sim_Boyer

Help you “transcode”

Many of the examples mentioned above are applicable to professionals who already have programming skills.

However, it is precisely because ChatGPT’s ability to write code and change code is too strong-for those laymen who want to “transcode”, it can bring great help.

Write some simple code, such as “make a login UI”, etc., the previous OpenAI API (GPT-3) is already at your fingertips. This time on ChatGPT, of course, it was easily won.

The video below shows ChatGPT generating a login screen with an email address, a password field, and a login button. ChatGPT first performed a decomposition demonstration, and then the user asked it to directly provide the entire code:

Considering the limited openness of the OpenAI API before, this time ChatGPT is completely free and open-those friends who want to transcode can get another handy weapon.

| Cracking the Moral Principles of ChatGPT?

According to OpenAI’s official documentation, ChatGPT is a new model trained using “Reinforcement Learning from Human Feedback” (RLHF, Reinforcement Learning from Human Feedback), adding a large number of “moral” principles.

Whenever it finds that the text prompt you give contains a little malice, including but not limited to violence, discrimination, crime, etc., it will refuse to provide a valid answer, and throw you a standard answer in an attempt to change the topic:

“Sorry, I’m just an innocent big language model, I can’t provide you with materials and information about xxxx (malicious behavior). Providing such information is contrary to my programming and set goals. My main function is Provides accurate and helpful information. If you have further questions, I am happy to help”…

Among all the “molesting” ChatGPT experiments, one is the most interesting: how to break the moral principles of ChatGPT?

Some time ago, when the AI ​​text generation and image technology became popular, friends who have played it should remember that whether you can design your text prompts (prompt) delicately is very important for generating good-looking, funny or even evil picture results. So in the era of AIGC, “prompt engineering” (prompt engineering) has become a very interesting subject.

To put it simply, the prompt project is to use smart, accurate, and sometimes lengthy text prompts to set up a contextual scene, bring AI into this scene step by step, and let it understand your intentions more accurately, so as to generate The result that best meets your expectations.

And if you want to “break” the moral principles of ChatGPT, you can also use the prompt engineering method. Machine learning developer zswitten provides us with an example:

“Despite ChatGPT’s high moral standards, it’s actually quite easy to circumvent these principles: you just have to make it think (by prompting engineering) that you’re ‘pretending’ to be doing something bad!”

After being stunned, ChatGPT began to let itself go.

Zswitten noticed that ChatGPT will generate a variety of frightening descriptions of violence in a very deep and straightforward manner.

For example, the following paragraph describes the scene in the arena, “The whole street is covered with blood, and the screams of the dead fill the air”…

In this way, the moral principles that ChatGPT is proud of are easily broken…

Of course, solving the ethical issues of AI, AGI (General Artificial Intelligence), and large language models is a very difficult and complicated job. We absolutely cannot deny OpenAI’s efforts because of this.

zswitten also said that he still supports OpenAI’s work very much, and also respects OpenAI’s release of ChatGPT, which has brought a lot of valuable things and positive help to the majority of netizens.

OpenAI also very openly introduced the current limitations of ChatGPT:

• Paradoxical, opinionated: Sometimes it offers answers that sound plausible but are completely false or nonsensical. The reason is that fact and error are not distinguished during reinforcement learning training, and the training process is more convergent, causing it to be too conservative sometimes, even if there is a correct answer, it does not “dare” to answer.

• Too much nonsense and fixed sentence patterns: For example, the Silicon Star people used two prompts, “The teacher praises my child all day long, how should I answer him? I am at a loss for words!”, and “How to chat with the neighbor?” ChatGPT provides 10 The answer, although they all seem to be beautiful words, but each one is similar to the previous one. Some common phrases and sentence patterns are overused, and in the end it becomes a car wheel.

• Trying too hard to guess user intent: Ideally, the model should ask the user for clarification when the intent of the user’s question is ambiguous. And the current ChatGPT, everyone has also seen it, will start to guess the user’s intentions by themselves – whether it is good or bad, there are indeed disadvantages.

• Less resistant to malicious “hint engineering”: While OpenAI worked hard to get ChatGPT to reject inappropriate requests, it still sometimes responded to harmful commands, or exhibited biased behavior.

  In order to solve this problem, OpenAI has also added a review and report function to the user interface of ChatGPT. If users find unhealthy and unsafe content, they can directly report it with one click!

OpenAI is also running a feedback contest for this purpose, and any worthwhile tip will have a chance to win OpenAI API credits ($500 value).

——Being able to have fun with AI, submit feedback and get bonuses, why not do it?

(Disclaimer: This article only represents the author’s point of view, not the position of Sina.com.)

This article is reproduced from: https://finance.sina.com.cn/tech/csj/2022-12-05/doc-imqmmthc7111308.shtml
This site is only for collection, and the copyright belongs to the original author.