Also talk about ChatGPT

I am a little addicted to ChatGPT these two days.

Of course, on the one hand, it has been impressed.

But on the other hand, it is more of a curiosity or suspicion.

After all, in principle, it is not easy to have a good impression of the current AI that gives statistical probability.

Although from the perspective of engineering or pragmatism, it is indeed a direction that is easy to compare promising results.

But at least in terms of the dignity or ego of human reason/logic, it is still hoped that there will be a solution that can meet the direct reasoning process.

Back to ChatGPT itself.

The official selling point was also code generation at the beginning.

As for the others, it may still be an inherent skill of the GPT series.

So I’ve been focusing on that for the past few days.

At the beginning, spell described a script for crawling web content.

It is natural to generate a script based on beautifulsoup.

Then some instruments are added to support script parsing and result variable extraction, as well as subsequent structured data processing and storage.

Finally there is also support for script parameterized command lines.

From the results point of view, it is quite amazing.

What’s more surprising is that if the instruct ported the python version to javascript, the result was quite satisfying.

This involves not only equivalent translation between languages, but also replacement and rewriting of corresponding library functions.

After all, the style of the two languages ​​and the usage of the support library are different.

So if it is a translation process that conforms to human intuition, in addition to the equivalent transformation of the procedure, there is also the selection of a suitable library to ensure the approximate symmetry of the style/structure.

Of course, there is another way to replace a contextual language constraint with the original spell and regenerate.

This way it is relatively simple.

After thinking about it, it is true that this kind of generator is more reasonable.

But the understanding of the problem and the process of generating the appropriate code for this task are still impressive.

The general model of guessing is that semantic annotations form a knowledge graph embed structure, and then some basic infer rules are matched.

An example is someone else’s simple calculus problems such as the day of the week.

The logical representation of the formalized structure after encode may be similar to an attribute structure, and the calculation becomes a pattern matching problem between adjacent subtrees/levels.

For example, the question of what day of the week is after working seven days in a row starting from Wednesday.

The parallel structure is the homomorphic pattern matching/infer problem of the number of days and the corresponding quality (days).

However, when it comes to this specific question, there is another surprising thing, which is to consider the special situation of Saturday and Sunday.

Come to think of it, it might be normal.

These two nodes may have corresponding negative signal/attribute.

Another way to achieve it may be to use the code generator.

Translate natural language into an instrument structure for code generator, and then generate and translate it into natural language.

This approach opens up a new way of thinking in the development of this type of probability model.

After all, what was criticized before is the problem of interpretability.

If you convert to this type of IR and do reasoning calculations in the way of a program/Turing machine, then the part after the IR is acceptable/comfortable/pleasant.

In this way, there is no doubt about the interpretability problem in the second half.

And it can solve quite a few problems.

As long as the corresponding mathematical constraints are given.

The rest is a conventional program calculus problem.

If it is in this direction, I have some good feelings for ChatGPT.

Then try to use this idea to set its own implementation mechanism.

But it is estimated that some ethical and moral restrictions are set.

Just like the meaning of those questions about destroying the world, some answers to themselves are relatively fixed templates.

Especially the part involving context content reasoning.

Although the conversation itself already represents a context, the generated answer clearly denies this.

Therefore, it may not be easy to make a breakthrough in terms of speaking skills.

Then go back to the code generator section to try.

The spell used is a recent description about kafka proxy.

After giving the initial sketch description, a model of consumer poll topic and producer send record comes out.

The step-by-step correction did generate a server accept client request and forward code that seems to work.

After the optimize request is made, some conventional rewriting such as asynchronous/multithreading/reuse can be done.

The refactor and inline directives can also be applied correctly.

Even the part specifying to use kafka library to replace the original socket selector to handle IO can be reconstructed correctly.

It’s pretty shocking to get here.

It really looks like a mentoring/pair programming feel.

But if you think about it carefully, it may not be incomprehensible.

Because of optimization and inline/refactor, it is a common function of modern language server/ide/linter.

In essence, it is also a pattern/rule matching.

For a model with tens of billions of parameters, this may not be a big deal.

After all, it is just a fine tune about the training set.

However, from the perspective of application scenarios, it is quite suitable for the optimize suggestion of the compiler.

Especially in pattern recognition.

It may bring some new breakthroughs, especially in offline profile.

If it is online/jit, it may not be realistic. One is the performance loss/collection of the program itself, and the other is the computing power demand of the model itself.

It is unlikely to go into the runtime like gc.

At least with the current technical architecture, it is quite unrealistic.

The rest are old problems.

How library-related equivalent replacement implementations/refactorings are done.

According to the idea of ​​IR, another problem is the problem of kickstart/bootstrap.

That is how the IR itself generates the appropriate code.

Re-spell an example of generating socks5 proxy.

Then unexpectedly saw some familiar code patterns.

Then there was a sense of relief.

Intuitively, it is similar to the way of copilot, and the code base is used as the corpus.

Then, like text2image and the like, combine related natural language descriptions, such as comments and readme, and related project knowledge graph entities, etc. to make embeds from training.

In this case, the generation will return to the probability model of GPT.

The part that needs fine tune may be the attention of the code block or the like.

Because from experience, ChatGPT can specify specific local code rewriting.

The premise is that you can guide /recall to point to the intended section.

In this case, if ChatGPT says that it does not have logical reasoning ability, it is not necessarily a safe guard.

The essence may still be the fine tune of the GPT model in a specific field.

However, what is more interesting here is the impression aspect at the beginning.

If it is only a probabilistic model, then the results that seem to have a logical reasoning process at first reflect the fact that the current degree of information abundance needs to be viewed from another perspective.

That said, there may not be any ideas/things that are new.

All the things that ordinary people can think/ask, already exist in the corpus in this world.

So GPT can generate these seemingly logical content.

For the idea of ​​indexing everything like Google’s original idea, what needs to be revised is the definition of indexing.

In other words, what is the current definition of the search requirement.

is a match for specific content keywords.

Or the identification match of a specific intension.

The former can’t escape the sentence structure itself at most.

The latter is a pseudo-logic inference made by Shakespeare written by monkeys.

It just so happened that the three-body animation version started broadcasting these two days.

In Sophon’s words, human beings may actually have nothing new.