Original link: https://jasonkayzk.github.io/2023/06/27/%E5%A4%A7%E9%A1%B5%E5%86%85%E5%AD%98%E7%AE%80% E4%BB%8B/

This article briefly introduces large page memory;

Introduction to Huge Pages

(Traditional) Huge Pages

What is Huge Page Memory

Huge pages (HugePages), sometimes called “huge memory pages”, “memory huge pages”, “standard huge pages”. The operating system manages memory in units of memory pages, and the size of memory pages has an impact on system performance:

If the memory page is set too small, there will be many memory pages, and the array for managing memory pages will be relatively large, which consumes memory. At the same time, the size of TLB (Translation Lookaside Buffer, page table register buffer, which can be understood as page table buffer) is fixed, resulting in TLB MISS increased. In different applications, the optimal value of the memory page size is different. Therefore, the general system supports the value of various memory pages.

Advantages of Huge Pages

To sum up, “huge pages” help Linux systems manage virtual memory. As the name suggests, in addition to the standard 4KB page size, it can also help manage huge pages (usually 2MB) in memory. Using “huge memory pages”, you can define a page size of up to 1GB.

For those businesses with very frequent memory operations, it can effectively improve performance. In short, by enabling huge page memory, the system only needs to deal with fewer page mapping tables, thus reducing the overhead of accessing/maintaining them!

Huge page memory usage: hugetlbfs

(Traditional) Huge page memory needs to reserve huge pages in advance in the system. Users must explicitly specify the use of huge pages through the mmap underlying interface implemented by the hugetlbfs file system or through the libhugetlbfs.so library!

See: https://blog.csdn.net/weixin_48101150/article/details/117824214

Huge page memory principle

Huge page memory is similar to dedicated memory, which will extract a large memory from the system, and to use this memory, the application must modify the program.

Secondly, if the program uses less memory but applies for a large page, it will cause memory waste.

At this point, you can use Transparent Huge Pages (THP);

See:

https://zhuanlan.zhihu.com/p/542132384

Transparent Huge Pages (THP)

What are Transparent Huge Pages

In addition to the standard huge pages above, there are actually two types of huge pages under Linux:

Standard Huge Pages
Transparent Huge Pages

Standard Huge Pages: It was introduced after Linux Kernel 2.6. The purpose is to use larger memory pages (memory page size) to adapt to larger and larger system memory, so that the operating system can support the large page capacity function of modern hardware architecture.

Transparent Huge Pages: Abbreviated as THP, Transparent Huge Pages (THP) are enabled by default in RHEL 6 for all applications. The kernel tries to allocate huge pages where possible, and the main kernel address space itself is mapped as huge pages, reducing TLB pressure on the kernel code. The kernel will always try to use huge pages to satisfy memory allocations. If no huge pages are available (eg because physically contiguous memory is not available), the kernel will fall back to normal 4KB pages. THP is also swappable (unlike hugetlbfs). This is accomplished by breaking large pages into smaller 4KB pages, which are then swapped out gracefully.

The difference between the two is:

For the allocation mechanism of huge pages, the standard huge page management is a pre-allocation method, while the transparent huge page management is a dynamic allocation method, and there is no need to modify the program source code.

Reasons to turn off transparent huge pages

Reference from: https://blog.csdn.net/qq_20408423/article/details/121606540

In the preparatory work before Oracle installation, there is a necessary work: disable transparent huge pages (Disabling Transparent HugePages).

This is due to the fact that there are some problems in the mixed use of transparent huge pages and traditional huge pages, which lead to performance problems and system restarts. Therefore, ORACLE officially does not recommend enabling transparent huge pages (THP) when using RedHat 6, OEL 6, SLES 11 and UEK2 kernels, because there are some problems with transparent huge pages:

In a RAC environment, Transparent Huge Pages (THP) can cause abnormal node restarts and performance issues;
In a stand-alone environment, transparent huge pages (THP) can also cause some abnormal performance problems;

On platforms after Linux6.x, it is recommended to close transparent huge pages when installing Oracle.

By default, Red Hat Enterprise Linux 6, SUSE Linux Enterprise Server 11, and Oracle Linux 6 enable transparent HugePages memory with earlier versions of the Oracle Linux Unbreakable Enterprise Kernel 2 (UEK2) kernel. Transparent HugePages memory is disabled in later versions of the Oracle Linux UEK2 kernel, Transparent HugePages can cause runtime memory allocation delays.

To avoid performance issues, Oracle recommends disabling Transparent Huge Pages on all Oracle Database servers. Oracle recommends using standard HugePages to improve performance. Transparent HugePages memory differs from standard HugePages memory because the kernel khugepaged threads dynamically allocate memory at runtime. Standard HugePages memory is pre-allocated at startup and does not change at runtime.

Use transparent huge pages

View the status of transparent huge pages

Linux 7 has the transparent huge page function enabled by default, you can check whether it is enabled:

 $ cat /sys/kernel/mm/transparent_hugepage/enabled[always] madvise never

By default, the status is always;

Three modes are distinguished:

never, prohibit the use of transparent huge pages;
always, always use transparent huge pages;
madvise, use transparent huge pages through madvise;

If “always” is enabled, the system defaults to allocate huge pages when the required memory is greater than 2MiB, and will allocate huge pages by default!

To emphasize:

The allocated memory is not necessarily all huge pages. Huge pages are “pseudo-aligned” set according to a certain configuration item. For example, the virtual machine defaults to 2MiB alignment (pmd intermediate page table entry), and 3MiB is allocated through malloc, of which 2MiB is a huge page. page, the remaining 1MiB is the ordinary page of buddy;

Close Transparent Huge Pages

You can change the above always to never to close the transparent huge page;

temporarily closed

Temporarily disable transparent huge pages:

 echo never > /sys/kernel/mm/transparent_hugepage/enabled

permanently closed

If you want to close permanently:

Edit the /etc/default/grub file as the root user and add the following GRUB startup parameters after the GRUB_CMDLINE_LINUX_DEFAULT variable:

 transparent_hugepage=never

For example:

 GRUB_CMDLINE_LINUX="net.ifnames=0 biosdevname=0 resume=/dev/mapper/bigcloud--enterprise--linux--for--euler-swap rd.lvm.lv=bigcloud-enterprise-linux-for-euler/root rd.lvm.lv=bigcloud-enterprise-linux-for-euler/swap crashkernel=512M transparent_hugepage=never"

Update the grub configuration:

 grub2-mkconfig -o /boot/grub2/grub.cfg

Finally, restart the system for the configuration to take effect.

Note: After modification, you need to restart the host to take effect!

After restarting, use the following command to check whether the shutdown is successful:

 cat /sys/kernel/mm/transparent_hugepage/enabledalways madvise [never]

other commands

Check the huge page memory method:

 #查看大页内存使用情况grep Huge /proc/meminfo#查看各个numa节点的大页内存情况cat /sys/devices/system/node/node*/meminfo | fgrep Huge#查看大页内存挂载情况cat /proc/mounts

appendix

article reference

This article is reproduced from: https://jasonkayzk.github.io/2023/06/27/%E5%A4%A7%E9%A1%B5%E5%86%85%E5%AD%98%E7%AE%80% E4%BB%8B/
This site is only for collection, and the copyright belongs to the original author.