Original link: https://blog.frytea.com/archives/678/
OpenResty Complete Development Guide: Building Million-Level Concurrent Web Applications Luo Jianfeng
131 notes
◆ Chapter 1 Introduction
[illustration]
OpenResty uses four digits as the version number, in the form: abcx, where the first three digits are the internal Nginx version, which is the major version number, and the fourth digit is the release version number of OpenResty itself, which is the minor version number
/usr/local/openresty/ #Installation main directory  ├── bin #Store executable files  ├── luajit #LuaJIT runtime library  ├── lualib #Lua component  ├── nginx #Nginx core operating platform  ├── pod #Data used by the reference manual (restydoc) └── site #Data used by the package management tool (opm)
Using the “Shebang” (#! ) of UNIX, specify resty as the interpreter in the first line of the script file
[illustration]
◆ Chapter 2 Nginx Platform
Nginx is a high-performance, high-stable lightweight HTTP, TCP, UDP and reverse proxy server
To configure HTTP-related functions, you need to use the command http{} to define the HTTP service provided by OpenResty. The usual form is:
http { #http block starts, all HTTP related functions server { #server block, the first web service listen 80; #monitor port 80 location uri { #location block, URI needs to be specified… #Definition when accessing this URI Specific behavior} #location block end} #server block end server { #server block, the second web service listen xxx; #monitor xxx port… #other location definitions} #server block end} #http block end
Since there are too many http blocks, if they are all written in one file, the configuration file may be too large and difficult to maintain.Reverse proxy (Reverse Proxy) is a very important technology in today’s network. It is located between the client and the real server (the so-called back-end), accepts the client’s request and forwards it to the back-end, and then sends the back-end The processing result is returned to the client
◆ Chapter 3 Lua Language
The word “Sol” means sun in Portuguese (the official language of Brazil), and “Lua” means moon.
Well-known applications include Adobe Lightroom, Fire-fox, Redis, etc., while games include “World of Warcraft”, “Angry Birds”, “My World” and so on.
Lua language provides six basic data types
Lua’s string form is very flexible, single quotes or double quotes are acceptable, and escape characters are also allowed in strings
Lua also supports raw strings in the form of “[[…]]”, the characters in brackets will not be escaped, which is very convenient when writing regular expressions or strings with quotation marks and slashes
Similar to multi-line comments, the form of “[[…]]” also supports inserting “=” between brackets, and if “[[” is followed by a newline, then Lua will automatically ignore this newline, which is a problem when writing a large amount of text. very convenient feature
Another feature of Lua strings is that they are read-only
Lua language internally uses a global hash table to manage all strings, so multiple identical strings will not occupy multiple copies of memory
Variables in the Lua language have the concept of scope, which are divided into local variables and global variables, and the names are case-sensitive
Local variables need to be declared with the keyword “local”, and the scope is limited to this code block (in the file or statement block). Variables declared without the keyword “local” are global variables, and can be used directly without declaration
In Lua, global variables should be used as little as possible, and local variables should be used more
Interpreter lookups are also faster because of “localization”
A more commonly used global variable is: “_” (underscore, also a legal variable name)
There is no “constant” in the Lua language. In practice, we usually use variables with all uppercase names to represent constants
Operations in Lua include arithmetic operations, relational operations, logical operations, string operations, etc.
Use “~=” for unequal comparisons
Lua will check the type of the variable when performing a greater than or less than comparison operation, and an error will occur if the type is different
But the behavior of “==” and “~=” is different. If the types are different, false will be returned directly
In order to avoid accidents, you must use the function tonumber()/tostring() to explicitly convert numbers or strings before doing comparison operations
Lua’s logical operators are and, or, and not
nil and false are considered false, others are true, including the number 0
x and y, if x is true, return y, otherwise return x
x or y, just the opposite of the and operation, if x is true, return x, otherwise return y
not x, only return true/false, negate x
Lua provides a special operator “..” for string concatenation operations
The concatenation operation can also automatically convert numbers to strings without explicitly calling the tostring() function:
String concatenation operations should be used sparingly, because each string concatenation creates a new string object
If you operate an ultra-long string (such as a large block of tens of MB) multiple times, it may cause LuaVM memory to run out and an error to occur.
To calculate the length of a string, you can use another special operator “#”
The relational operation of the string is checked one by one based on the character sequence (such as the most commonly used ASCII code table), but the equality comparison directly calculates the internally saved hash value
Many times the operation on nil will cause an error
If a variable may be nil, it is best to use the or operator to give it a default value
Statements in Lua include assignment statements, branch statements, and loop statements
The format of Lua statements is very free, and indentation is not mandatory. You can use “;” at the end of the statement to indicate the end, but it is not required
A statement (code) block is declared using the form “do…end”
Lua uses “=” to store a value of a certain type in a variable
Lua also allows comma-separated declaration or assignment of multiple variables in one statement
If a variable is assigned a value of nil, it means that the variable is deleted.
There is only one branch statement in Lua, which is if-else
Multiple branches need to use “elseif”
There are three kinds of loop statements in Lua language: while, repeat-until and for
The repeat-until loop is similar to while, but the meaning of conditional judgment and while is opposite
Lua’s for loop statement has two forms: numerical loop and range loop
The value loop of for is similar to the standard for statement in other languages, but the form is more concise
The variable var moves forward (or backward) from m to n, and executes the statement in the loop body. The parameter step is used to control the step size of var forward or backward. It can be omitted, and the default is 1
The variable var in the for loop is automatically declared as a local variable
You can use break or return to directly jump out of the loop. The usage is the same as other languages, but it must be at the end of the statement block—that is, followed by the end/until keyword
If you want to end the loop at any position, you can use the “do break end” form
The LuaJIT extension used by OpenResty supports goto, which can be used to implement continue
In the Lua language, a function is a special type of variable that holds a statement block, executes the statement block (that is, “call”) with parameters, and then returns the result
Lua functions are variables, and can (preferably) be localized using local.
Any number of actual parameters can be passed in, the default value of less is nil, and more is ignored
The return value of the function uses the return statement, which can return multiple values separated by commas, but only one value will be returned if it is surrounded by parentheses when called
The parameters of Lua functions are all passed by value (except tables)
If there is only one incoming parameter when calling, and this parameter is a string or a table, then Lua allows to omit the “()” of the function and write the parameter directly after the function name
But Lua’s table is more flexible, and can simulate common data structures such as array, list, dict, set, map, or other arbitrarily complex structures
The key used as an index in a Lua table can be any non-nil value, so when the key type is an integer, the table is equivalent to an array, and when the key type is a string, the table is equivalent to a dictionary or an associative array
The Lua table does not have any restrictions on the type of value, and of course it can also be another table, so as to realize the nesting of multiple tables.
The definition table in Lua uses curly braces “{}”
Direct use of “{}” is an empty table. Simply list the elements in the table to declare a table in the form of an array. Use the form of “key=value” to declare a table in the form of a dictionary.
When using the form of “key=value”, the key does not need to use single quotes or double quotes. If it must be used (for example, there are spaces or other special symbols in the key), use the form of “[key]=value”
The comma “,” when defining the table can also be replaced with a semicolon “;”, there is no difference between the two, but some formal distinctions can be made
Lua’s table is a dynamic data structure, which can not only access existing elements, but also add or delete elements to the table at any time
Elements in the operation table need to use square brackets “[]”
Integer subscript indices must start counting from 1 when the table is used as an array
If the key is a string, we can also directly use the dot “.” to operate
The operator “#” can calculate the number of array elements in the form table, and cooperate with the for loop to realize the traversal of the array
For tables in the form of dictionaries, there is no way to directly obtain the number of elements, using “#” will return 0
The second form of the for loop statement – the range loop is mainly used to traverse the elements in the table, but requires the cooperation of two standard library functions: ipairs() and pairs().
A module is a collection of functions, usually expressed as a Lua table, which contains various functions provided by the module author, and can be accessed by using the dot “.”
Use the require function to load the module, the parameter is the file name where the module is located (omit the suffix)
Usually we need to use variables to save the return result of the require function
require will execute the code in the file while loading the module
If a string is used as the key, the table itself is an object, and variables and functions can be stored arbitrarily
In terms of “encapsulation”, Lua does not provide modifiers such as private and public, and all members in the table are public
If you want to implement private members, you can use local modification in the module file
The “polymorphism” feature is very simple for Lua. Since the table is dynamic, the members inside can be replaced at runtime, without the trouble of static binding in compiled languages.
Inheritance is not recommended in the Lua language. The alternative is to use the “prototype” mode to “clone” a new object from a “prototype” object, and then dynamically change its properties, so as to achieve the same as “inheritance”. ” similar effect.
The “prototype” mode requires the use of Lua’s advanced feature “metatable” (metatable) and the function setmetatable().
The metatable describes the basic behavior of the table, which is somewhat similar to operator overloading in C++ or Python. What we need to use is the “__index” metamethod, which overloads the operation of finding the key in Lua, that is, table.key.
The function setmetatable(t, meta) sets the metatable of table t to meta and returns t.
If the “__index” method is set in meta, then the operation t.key on t will also be applied to meta, that is, meta.key. In this way, table t “clones” all members of table meta, and table meta becomes the “prototype” of table t.
You can further understand Lua’s “prototype” operation through the following example:
local proto = {} — first declare a prototype object, temporarily an empty table function proto.go() — add a method to the table, that is, the member function print(“go pikachu”) end local mt = { __index = proto } — Define the metatable, pay attention to overloading “__index” local obj = setmetatable({}, mt) — Call setmetatable to set the metatable and return a new table obj.go() — The new object is a “clone” of the prototype, and the prototype can be executed operation
The key operation in the code is to define the metatable mt, which only needs to set the “__index” method, and then use the function setmetatable to clone a new object from mt.
These two steps can also be combined into one operation:Lua provides a special operator “:” for object-oriented use of member functions in the table. Its function is basically the same as “.”, but a “self” parameter is implicitly passed in when calling the function
“:” and self can be used not only in function calls, but also in function definitions
“:” is actually a kind of “grammatical sugar”, which is a simplified “.”
It is recommended to use “:” as much as possible, it is more concise
You should use OpenResty’s own ngx.re series of functions
The efficiency of passing table.insert is not high, we can add elements at the end more efficiently in the following way:
a[#a + 1] = ‘nginx’ — use the “#” operator to get the length to add elements
The io library is a function for operating files. Since files are usually stored on disk and are blocking operations, the speed is very slow, so they should be used as little as possible in OpenResty.
When reading data, you can use the parameter ” a” (that is, all) to read the entire file, or “ l” (that is, line) to read a line, and use a number to read bytes of the specified length
The os library contains functions related to the operating system and time and date
Visually (but not very accurately), a “closure” is a “living function” that exists in the “high-dimensional space” of the program and can manipulate data outside the function arbitrarily
If we want Lua code to be more robust, we can use protected mode to execute functions that may fail
pcall (protected call) is a special function in the base library, it “protects calling” a function, it will never make an error, and returns the result of the call with true/false
◆ Chapter 4 LuaJIT Environment
LuaJIT is another implementation of the Lua language, including an interpreter written in assembly language and a JIT compiler
Use the form of “::label::” to define the label, and then you can use goto to change the flow of the program at any time
LuaJIT has enhanced the table library by adding some new functions, the more useful ones are table.new, table.clear and table.clone.
The function table.clear sets the table as an empty table, but retains the previously allocated memory
The function table.clone is a unique feature of the LuaJIT branch of OpenResty, which can efficiently “shallow” copy the table (shallow clone)
The ffi library can not only call system functions and C functions inside OpenResty, but also load a dynamic library in the form of so and call functions in the dynamic library
LuaJIT always uses the interpreter to run the compiled bytecode first, and does “hot spot analysis” at runtime. If a certain piece of code is “hot” enough, it will automatically trigger the JIT compiler and try to recompile the bytecode. The local machine code allows the program to run at the fastest speed.
Some Lua functions will not be compiled because of the high cost of implementation, and can only be run in the form of bytecode. These are called NYI (Not Yet Implemented).
◆ Chapter 5 Development Overview
OpenResty provides a special command “content_by_lua_block”, which can write Lua code in the configuration file to generate response content
To start the application, you need to use the “-c” parameter to let OpenResty run with the specified configuration file: /usr/local/openresty/bin/openresty -c ”
pwd/hello.conf”The example in section 5.1 is the simplest OpenResty application, with only one configuration file, and the application code is written in the configuration file. But the actual project is much more complicated than that. It is best to manage and maintain configuration files and application codes separately. In addition, there will be other monitoring scripts, log files, data files, etc., and they must be organized with a good directory hierarchy.
Usually the directory structure of an OpenResty application is as follows: [illustration]
path/to/application #The main directory of the application ├── bin #Script directory, storing various script files ├── conf #Configuration directory, storing Nginx configuration files │ ├── http #Store HTTP service configuration files│ ├── stream #Store the configuration file of TCP/UDP serviceIn OpenResty, ngx_lua and stream_lua belong to two different subsystems, but the functions and formats of the instructions are basically the same
OpenResty currently focuses on the two stages of initing and running, and has made a more detailed division.
Developers must have a better understanding of the meaning and functions of these stages, and then combine their actual business needs to choose the appropriate stage to write code to implement functions
OpenResty uses “timers” to periodically (one or more times) execute “background tasks”.
Most of these interfaces are located in the global table ngx and can be accessed without require (but there are some exceptions)
Different from the standard library functions that come with Lua, they are based on Nginx’s event mechanism and Lua’s coroutine feature, which are “100% nonblocking”, allowing us to easily write synchronous, non-blocking and efficient code
OpenResty comes with many Lua libraries (located in lualib in the installation directory), and lua-resty-core is the most important one. It uses ffi to reimplement most of the original functions in OpenResty and adds some new functions. .
◆ Chapter 6 Basic Functions
ngx.sleep is a synchronous non-blocking sleep function provided by OpenResty, which can “sleep” for any length of time without blocking the entire service. At this time, OpenResty will process other requests based on the coroutine mechanism, and wait for the sleep time to ” Go back” and continue to execute the subsequent code of ngx.sleep.
MessagePack is a binary data encoding format, which is smaller and more compact than JSON, and is suitable for serializing and transmitting large quantities of data.
OpenResty provides six regular expression-related functions in the table ngx.re. Their underlying implementation is the PCRE library, which is extremely fast and can completely replace the string matching functions of the Lua standard library.
Regular replacement also has two functions: ngx.re.sub and ngx.re.gsub, we’d better not add the “o” option when using it (see Section 6.5.2 for the reason
ngx.re.gsub is an enhanced version of ngx.re.sub, which can perform multiple regular replacements
The capacity of the Cache is usually limited, and some algorithm needs to be used to update and eliminate data. The more common ones are FIFO, LFU, LRU, etc.
OpenResty provides a convenient and easy-to-use Cache library lua-resty-lru-cache based on the LRU algorithm, and supports the expiration time function (expire)
The functional interface of the cache object is very simple and easy to use. It provides basic set/get/delete operations. It is like a Key-Value hash table. The elements in the cache can also be any Lua data (numbers, strings, functions, tables, etc.) without serialization or deserialization
◆ Chapter 7 HTTP Service
The status code indicates the processing status of the HTTP request. There are more than one hundred in the current RFC specification, and only a few of the most common ones are defined in OpenResty.
◆ Chapter 8 Accessing the Backend
It is necessary to use database services such as Redis and MySQL to store cache, sessions and other data, use message queue services such as Kafka and RabbitMQ to send messages asynchronously, and access business services such as Tomcat and PHP, access configuration services such as ZooKeeper and Consul, and coordinate these backends comprehensively In order to present a fully functional application service to end users.
This article is transferred from: https://blog.frytea.com/archives/678/
This site is only for collection, and the copyright belongs to the original author.