Let’s talk about how Redis handles requests

Leave a Comment / / luozhiyun | 我的技术分享

Original link: https://www.luozhiyun.com/archives/674

Please declare the source for reprinting~ This article was published on luozhiyun’s blog: https://www.luozhiyun.com/archives/674

Redis 5.0 source code used in this article

I feel that this part of the code is quite interesting, I try to explain it in a more popular way

Overview

I remember that I explained how to create a server-side program for Go in an article that explained the Go language HTTP standard library :

  • The first is to register the processor;
  • Open the loop listening port, and create a Goroutine every time a connection is monitored;
  • Then the Goroutine will wait for the request data in a loop, and then match the corresponding processor in the processor routing table according to the requested address, and then hand the request to the processor for processing;

In code it’s like this:

 func (srv *Server) Serve(l net.Listener) error { ... baseCtx := context.Background() ctx := context.WithValue(baseCtx, ServerContextKey, srv) for { // 接收listener 过来的网络连接rw, err := l.Accept() ... tempDelay = 0 c := srv.newConn(rw) c.setState(c.rwc, StateNew) // 创建协程处理连接go c.serve(connCtx) } }

For Redis, it is a little different, because it is single-threaded and cannot use multi-threaded processing connections, so Redis chooses to use an event driver based on the Reactor pattern to implement concurrent processing of events.

reactor

The so-called Reactor mode in Redis is to monitor multiple fds through epoll. Whenever these fds respond, they will notify epoll to call back in the form of events. Each event has a corresponding event handler.

For example, accept corresponds to the acceptTCPHandler event handler, read & write corresponds to the readQueryFromClient event handler, etc., and then assigns the event to the event handler for processing through the cyclic dispatch of events.

Therefore, the above Reactor mode is implemented through epoll. For epoll, there are mainly three methods:

 //创建一个epoll的句柄，size用来告诉内核这个监听的数目一共有多大int epoll_create(int size)； /* * 可以理解为，增删改fd 需要监听的事件* epfd 是epoll_create() 创建的句柄。 * op 表示增删改* epoll_event 表示需要监听的事件，Redis 只用到了可读，可写，错误，挂断四个状态*/ int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event)； /* * 可以理解为查询符合条件的事件* epfd 是epoll_create() 创建的句柄。 * epoll_event 用来存放从内核得到事件的集合* maxevents 获取的最大事件数* timeout 等待超时时间*/ int epoll_wait(int epfd, struct epoll_event * events, int maxevents, int timeout);

So we can implement a simple server according to these three methods:

 // 创建监听int listenfd = ::socket(); // 绑定ip和端口int r = ::bind(); // 创建epoll 实例int epollfd = epoll_create(xxx); // 添加epoll要监听的事件类型int r = epoll_ctl(..., listenfd, ...); struct epoll_event* alive_events = static_cast<epoll_event*>(calloc(kMaxEvents, sizeof(epoll_event))); while (true) { // 等待事件int num = epoll_wait(epollfd, alive_events, kMaxEvents, kEpollWaitTime); // 遍历事件，并进行事件处理for (int i = 0; i < num; ++i) { int fd = alive_events[i].data.fd; // 获取事件int events = alive_events[i].events; // 进行事件的分发if ( (events & EPOLLERR) || (events & EPOLLHUP) ) { ... } else if (events & EPOLLRDHUP) { ... } ... } }

call process

So according to the above introduction, you can know that for Redis, an event loop is nothing more than these few steps:

  1. Register event listeners and callback functions;
  2. Loop waiting to get events and process them;
  3. Call the callback function to process the data logic;
  4. Write data back to Client;

reactor2

  1. Register fd to epoll, and set the callback function acceptTcpHandler, if there is a new connection, the callback function will be called;
  2. Start an infinite loop and call epoll_wait to wait and continue to process events. Later, we will return to the aeMain function to loop the aeProcessEvents function;
  3. When there is a network event, it will call the callback function acceptTcpHandler all the way to readQueryFromClient for data processing, readQueryFromClient will parse the client data, find the corresponding cmd function to execute;
  4. After receiving the client request, the Redis instance will write the data to be returned into the client output buffer after processing the client command instead of returning it immediately;
  5. Then every time the aeMain function loops, the beforeSleep function is called to write the data in the buffer back to the client;

The whole process of the above event loop is actually very clear in the code steps, and there are many articles on the Internet, so I won’t talk about it more.

Command execution process & writeback client

command execution

Let’s talk about what many articles on the Internet have not mentioned, let’s see how Redis executes commands, then stores them in the cache, and writes the data from the cache back to the client.

reactor3

We also mentioned in the previous section that if there is a network event, the readQueryFromClient function will be called, which is where the command is actually executed. Let’s follow this method and look down:

  1. In readQueryFromClient, the processInputBufferAndReplicate function will be called to process the requested command;
  2. In the processInputBufferAndReplicate function, processInputBuffer will be called and it will be judged whether the command needs to be copied to other nodes if it is in cluster mode;
  3. The processInputBuffer function will process the requested command in a loop, and call the processInlineBuffer function according to the requested protocol, and call processCommand after the redisObject object to execute the command;
  4. When processCommand executes a command, it will go to the server.commands table through lookupCommand to find the corresponding execution function according to the command, and then after a series of verifications, call the corresponding function to execute the command, and call addReply to write the returned data to the client output. buffer;

server.commands will register all Redis commands in the populateCommandTable function as a table to get command functions based on command names.

For example, to execute the get command, the getCommand function will be called:

 void getCommand(client *c) { getGenericCommand(c); } int getGenericCommand(client *c) { robj *o; // 查找数据if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.nullbulk)) == NULL) return C_OK; ... } robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply) { //到db中查找数据robj *o = lookupKeyRead(c->db, key); // 写入到缓存中if (!o) addReply(c,reply); return o; }

Find the data in the getCommand function, and then call addReply to write the returned data to the client output buffer.

data write-back client

After the above command is executed and written to the buffer, the data needs to be retrieved from the buffer and returned to the Client. For the process of writing data back to the client, it is actually completed in the event loop of the server.

reactor4

  1. First, Redis will call the aeSetBeforeSleepProc function in the main function to register the function beforeSleep of the write-back package into eventLoop;
  2. Then Redis will judge whether beforesleep has been set when calling the aeMain function for the event loop, and if so, it will be called;
  3. The handleClientsWithPendingWrites function will be called in the beforesleep function, which will call writeToClient to write the data back from the buffer to the client;

Summarize

This article describes how the entire Redis request processing model looks like. From registering and monitoring fd events to executing commands, and finally writing data back to the client, a general analysis has been made. Of course, this article is also a bit different from my previous articles. There is no long-winded post code, mainly I don’t think it is necessary. If you are interested, you can follow the flow chart to see the code.

Reference

http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf

https://time.geekbang.org/column/article/408491

http://remcarpediem.net/article/1aa2da89/

https://github.com/Junnplus/blog/issues/37

https://www.cnblogs.com/neooelric/p/9629948.html

Scan code_Search joint communication style-white version 1

Talking about how Redis processes requests first appeared on luozhiyun`s Blog .

This article is reproduced from: https://www.luozhiyun.com/archives/674
This site is for inclusion only, and the copyright belongs to the original author.

Leave a Comment