Proxy TCP or UDP with Nginx

/ jason | 张小凯

Original link: https://jasonkayzk.github.io/2022/10/24/%E4%BD%BF%E7%94%A8Nginx%E4%BB%A3%E7%90%86TCP%E6%88%96UDP/

Using Nginx for proxy forwarding is a very common function;

For example: sometimes a service is started using Docker on the development machine, and the port of the container is not exposed when it is started. If remote access is required later, the configuration needs to be modified, which is troublesome;

At this point, you can use Nginx, set the proxy of the container on the development machine (as opposed to our manual Ingress), and then you can connect!

Proxy TCP or UDP with Nginx

The part of installing Nginx will not be introduced here. I can find a lot on the Internet;

It is recommended to use the source code method to compile and install, because the source code package includes some common modules, which can be installed by yourself!

Here is mainly to introduce the configuration of Nginx proxy TCP or UDP;

Install the ngx_stream_module module

Since the proxy TCP/UDP relies on the ngx_stream_module module, we need to install it first;

Those who have installed this module can be ignored;

First enter the directory after nginx decompression:

 $ lltotal 832drwxr-xr-x 3 root root 4096 Oct 24 13:46 objs-rw-r--r-- 1 root root 438 Oct 24 13:45 Makefiledrwxr-xr-x 14 root root 4096 Aug 2 17:56 ..drwxr-xr-x 10 1001 1001 4096 Nov 27 2021 .drwxr-xr-x 3 root root 4096 Nov 27 2021 nginx-backupdrwxr-xr-x 6 1001 1001 4096 Nov 27 2021 autodrwxr-xr-x 2 1001 1001 4096 Nov 27 2021 confdrwxr-xr-x 4 1001 1001 4096 Nov 27 2021 contribdrwxr-xr-x 2 1001 1001 4096 Nov 27 2021 htmldrwxr-xr-x 2 1001 1001 4096 Nov 27 2021 mandrwxr-xr-x 9 1001 1001 4096 Nov 27 2021 src-rw-r--r-- 1 1001 1001 311503 May 25 2021 CHANGES-rw-r--r-- 1 1001 1001 1397 May 25 2021 LICENSE

Declare our build configuration:

 ./configure --prefix=/usr/share/nginx --with-compat --with-http_stub_status_module --with-http_ssl_module --with-stream=dynamic

Note: The --with-compat here must be brought, otherwise the compiled link library may not be available!

See:

Finally compile:

 make

After compilation, there will be a compiled ngx_stream_module.so module in the objs/ directory:

 $ ll objs/*.so-rwxr-xr-x 1 root root 1360112 Oct 24 13:46 objs/ngx_stream_module.so-rwxr-xr-x 1 root root 970168 Nov 27 2021 objs/ngx_mail_module.so-rwxr-xr-x 1 root root 216256 Nov 27 2021 objs/ngx_http_perl_module.so-rwxr-xr-x 1 root root 186904 Nov 27 2021 objs/ngx_http_image_filter_module.so-rwxr-xr-x 1 root root 197728 Nov 27 2021 objs/ngx_http_xslt_filter_module.so

At this point, our module is ready!

Modify Nginx configuration

After the ngx_stream_module module is ready, first we modify the entry configuration file of Nginx:

Reference this module on the first line and add the submodule configuration:

/etc/nginx/nginx.conf

 + load_module /opt/nginx-1.20.1/objs/ngx_stream_module.so;+ stream {+ include /etc/nginx/conf.d/*.stream;+ }

It is strongly recommended to configure nginx in a modular way, namely:

In the entry file, just include other configuration files without writing other configuration logic!

Then add sub-configurations in conf.d , here is the connection proxy configuration of clickhouse as an example:

conf.d/click-house.stream

 upstream CLICKHOUSE { server 172.19.0.2:8123;}server { listen 18123; proxy_connect_timeout 30s; proxy_timeout 600s; proxy_pass CLICKHOUSE;}

The above 172.19.0.2:8123 is the ip:port in the container. If K8S is deployed, it can be obtained by the following command:

 $ kubectl -n my-ch get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEclickhouse-demo-01 LoadBalancer 10.43.93.132 172.19.0.2,172.19.0.3,172.19.0.4,172.19.0.5 8123:30842/TCP,9000:31655/TCP 15hchi-demo-01-demo-01-0-0 ClusterIP None <none> 8123/TCP,9000/TCP,9009/TCP 15hchi-demo-01-demo-01-1-0 ClusterIP None <none> 8123/TCP,9000/TCP,9009/TCP 15h

And the 18123 in the server is the port of the external proxy of the server defined by itself;

When connecting externally, use server:18123 18123 directly to connect to ClickHouse in the container!

Finally, restart Nginx:

 systemctl restart nginx

to take effect!

appendix

Article reference:

This article is reproduced from: https://jasonkayzk.github.io/2022/10/24/%E4%BD%BF%E7%94%A8Nginx%E4%BB%A3%E7%90%86TCP%E6%88%96UDP/
This site is for inclusion only, and the copyright belongs to the original author.