Fu Qiang said that an architect ‘s thinking and precipitation Menu

The essence of the problem to be solved by the LLM framework and application

Wang Fuqiang

---

Made three programs on LLM Orchestration framework and application design and development:

1. LangChain Minimalist Secret (https://youtu.be/H5lSlWjOePo)
2. Semantic Kernel Quickly Revealed (https://youtu.be/OvNey1tAGJE)
3. ChatGPT with Scala (https://youtu.be/3km1LznvqvY)

If we do a brief review, we can actually find that these frameworks and application design and development models cannot be carried out in accordance with the three-stage progressive thinking. I simply divide these three stages as follows:

1. LLM Externalized Portfolio Coordination
2. LLM Internalization Decomposition and Combination Coordination
3. Integration and coordination of LLM-enhanced externalized capabilities

Moreover, these three stages have a unique main line, that is Prompt Engineering. All ability “summoning” is actually carried out through Prompt, but how to tap the potential of Prompt is the main battlefield of Prompt Engineering. The ideas and methods explored by Prompt Engineering eventually become some prompt skills (tips&tricks) for daily use. ).

LLM Externalized Portfolio Coordination

Call the LLM through the program (prompt is passed in as the input), and then pass the returned result to the next LLM as the input. This kind of combination coordination is carried out outside the LLM, so I call this method “outside the LLM Chain in LangChain, Functions in Semantic Kernel, and even Scala as a generalization method to call ChatGPT through API are actually “LLM external combination coordination”.

LLM Internalization Decomposition and Combination Coordination

For complex tasks, we need to use some prompt techniques to allow LLM to retreat (reasoning) inside the model. Common methods or techniques include CoT (Chain of thought), ToT (Tree of Thought) and even ReAct (Reasoning + Actions).

But this method has a drawback, that is, no matter how automated and “intelligent” it is, it is carried out in a relatively closed environment, that is to say, the version and parameters of the model define its boundaries, no matter how How to decompose a complex system (Task Decomposition) is just an “inner loop” in a relatively closed environment. Sometimes, playing like this is actually unsolvable.

Therefore, we urgently need to find other ways to solve this problem, that is…

Integration and coordination of LLM-enhanced externalized capabilities

This method seeks the intervention of capabilities other than LLM to enhance the capabilities of LLM. The commonly used methods are:

1. Program-aided LM
2. Tool argued LM

The plugins mechanism of ChatGPT is actually such a way of playing. LangChain’s Tool/Toolkit and SemanticKernel’s Native Functions also belong to this category.

By introducing external programs and tools to enhance the original large language model, the application based on the large language model has more flexible scheduling and coordination capabilities. Of course, more importantly, it can communicate with the external environment in real time, so that It seamlessly integrates into the existing environment and ecology.

LangChain’s Agent, Semantic’s Planner, and projects and functions such as AutoGPT all belong to this category, and are usually built using an architecture called MRKL. Essentially, they have two types of任务分解+外部能力调用The blessing of ability.

OK, the above is a summary of the previous three programs, I hope it will be helpful to everyone.

If you want to know more about AI, AGI and AIGC, you are welcome to visit https://edu.afoo.me or join the “crazy for AI” knowledge planet (https://ift.tt/eDcGq0o) .

---

---

---

---

© Fuqiang Wang Personal Copyright, All Rights Reserved.