[20260503]21c下测试pre_page_sga=false时的疑问.txt
--//21c下测试pre_page_sga=false时遇到的问题，发现ipcs -m显示中nattch列显示的信息与11g遇到的情况不同，做一个记录与分析。
1.问题提出：
SYS@book> @ ver2
==============================
PORT_STRING : x86_64/Linux 2.4.xx
VERSION : 21.0.0.0.0
BANNER : Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
BANNER_FULL : Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0
BANNER_LEGACY : Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
CON_ID : 0
PL/SQL procedure successfully completed.
$ cat /u01/app/oracle/dbs/initbook.ora
SPFILE='/u01/app/oracle/dbs/spfilebook.ora'
use_large_pages=false
# sysctl -w vm.nr_hugepages=0
vm.nr_hugepages = 0
--//启动数据库：
SYS@book> startup pfile='/u01/app/oracle/dbs/initbook.ora'
ORACLE instance started.
Total System Global Area 1107294056 bytes
Fixed Size 9684840 bytes
Variable Size 654311424 bytes
Database Buffers 436207616 bytes
Redo Buffers 7090176 bytes
Database mounted.
Database opened.
# ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 0 oracle 600 9687040 116
0x00000000 32769 oracle 600 1090519040 58
0x00000000 65538 oracle 600 7090176 58
0xafa94c20 98307 oracle 600 16384 58
--//看到第1个共享内存段的nattch =116，基本都是其他共享内存段的2倍，为什么？
--//建立1个新会话.
# ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 0 oracle 600 9687040 118
0x00000000 32769 oracle 600 1090519040 59
0x00000000 65538 oracle 600 7090176 59
0xafa94c20 98307 oracle 600 16384 59
--//看到第1个共享内存段的nattch增加2,不理解为什么其他仅仅增加1.
--//以前我可以简单的执行ipcs -m，通过nattch就可以大致确定连接数据库的进程数量。
--//如果use_large_pages设置其他参数，就看不到这样的现象呢？而且以前的11g也不会看到这样的现象
--//如何解析看到的现象?
$ cat /proc/$(pgrep pmon)/maps | grep rw-s
60001000-6093d000 rw-s 00001000 00:04 0 /SYSV00000000 (deleted)
61000000-a2000000 rw-s 00000000 00:04 32769 /SYSV00000000 (deleted)
a2000000-a26c3000 rw-s 00000000 00:04 65538 /SYSV00000000 (deleted)
a3000000-a3004000 rw-s 00000000 00:04 98307 /SYSVafa94c20 (deleted)
7f44a7f2f000-7f44a7f30000 rw-s 00000000 08:11 18861347 /u01/app/oracle/dbs/hc_book.dat
2.分析：
--//使用strace跟踪看看:
$ cat test1.txt
3611 shmget(0xafa94c20, 0, 0) = 98307
3611 shmctl(98307, IPC_STAT, 0x7ffde564cc40) = 0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3611 shmdt(0x7fd9c109b000) = 0
3611 shmget(0xafa94c21, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xafa94c22, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xafa94c23, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmat(98307, 0xa3000000, 0) = 0xa3000000
3611 shmat(32769, 0x61000000, 0) = 0x61000000
3611 shmat(65538, 0xa2000000, 0) = 0xa2000000
3611 shmat(0, 0x60000000, 0) = 0x60000000
3611 shmget(0xa93a2388, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xa93a2389, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xa93a238a, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xa93a238b, 0, 0) = -1 ENOENT (No such file or directory)
3610 +++ exited with 0 +++
3611 shmdt(0x61000000) = 0
3611 shmdt(0xa2000000) = 0
3611 shmdt(0x60000000) = 0
3611 shmdt(0xa3000000) = 0
3611 +++ exited with 0 +++
--//看man shmat文档：
--//shmat() attaches the System V shared memory segment identified by shmid to the address space of the calling process.
--//The attaching address is specified by shmaddr with one of the following criteria:
--//shmdt() detaches the shared memory segment located at the address specified by shmaddr from the address space of
--//the calling process. The to-be-detached segment must be currently attached with shmaddr equal to the value returned
--//by the attaching shmat() call.
--//开始第1次调用shmat，第2个参数是0.
--//3611 shmat(98307, 0, 0) = 0x7fd9c109b000,而且对应的是第4个共享内存段。
void *shmat(int shmid, const void *shmaddr, int shmflg);
int shmdt(const void *shmaddr);
DESCRIPTION
shmat() attaches the System V shared memory segment identified by shmid to the address space of the calling process.
The attaching address is specified by shmaddr with one of the following criteria:
If shmaddr is NULL, the system chooses a suitable (unused) address at which to attach the segment.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--//如果shmaddr是NULL，系统选择合适（不用）地址绑定该段，什么意思不理解。不过后面跟着调用shmdt函数，已经detaches.
If shmaddr isn't NULL and SHM_RND is specified in shmflg, the attach occurs at the address equal to shmaddr rounded down
to the nearest multiple of SHMLBA. Otherwise shmaddr must be a page-aligned address at which the attach occurs.
--//再执行一次：
$ strace -f -y -e ipc -o test1a.txt sqlplus -s -l / as sysdba <<<quit
$ sdiff -w 168 test1.txt test1a.txt
3611 shmget(0xafa94c20, 0, 0) = 98307 | 3729 shmget(0xafa94c20, 0, 0) = 98307
3611 shmctl(98307, IPC_STAT, 0x7ffde564cc40) = 0 | 3729 shmctl(98307, IPC_STAT, 0x7ffe19b52f40) = 0
3611 shmat(98307, 0, 0) = 0x7fd9c109b000 | 3729 shmat(98307, 0, 0) = 0x7f421db9c000
3611 shmdt(0x7fd9c109b000) = 0 | 3729 shmdt(0x7f421db9c000) = 0
3611 shmget(0xafa94c21, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xafa94c21, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xafa94c22, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xafa94c22, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xafa94c23, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xafa94c23, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmat(98307, 0xa3000000, 0) = 0xa3000000 | 3729 shmat(98307, 0xa3000000, 0) = 0xa3000000
3611 shmat(32769, 0x61000000, 0) = 0x61000000 | 3729 shmat(32769, 0x61000000, 0) = 0x61000000
3611 shmat(65538, 0xa2000000, 0) = 0xa2000000 | 3729 shmat(65538, 0xa2000000, 0) = 0xa2000000
3611 shmat(0, 0x60000000, 0) = 0x60000000 | 3729 shmat(0, 0x60000000, 0) = 0x60000000
3611 shmget(0xa93a2388, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xa93a2388, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xa93a2389, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xa93a2389, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xa93a238a, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xa93a238a, 0, 0) = -1 ENOENT (No such file or directory)
3611 shmget(0xa93a238b, 0, 0) = -1 ENOENT (No such file or directory) | 3729 shmget(0xa93a238b, 0, 0) = -1 ENOENT (No such file or directory)
3610 +++ exited with 0 +++ | 3729 shmdt(0x61000000) = 0
3611 shmdt(0x61000000) = 0 | 3729 shmdt(0xa2000000) = 0
3611 shmdt(0xa2000000) = 0 | 3729 shmdt(0x60000000) = 0
3611 shmdt(0x60000000) = 0 | 3729 shmdt(0xa3000000) = 0
3611 shmdt(0xa3000000) = 0 | 3729 +++ exited with 0 +++
3611 +++ exited with 0 +++ | 3728 --- SIGCHLD {si_signo=SIGCHLD,si_code=CLD_EXITED, si_pid=3729, si_status=0, si}
> 3728 +++ exited with 0 +++
--//如果换一个方式启动数据库use_large_page=only，不使用pfile参数文件，仅仅贴出测试结果,具体过程略.
$ strace -f -y -Ttt -o test6.txt sqlplus -s -l / as sysdba <<<quit
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 262144 oracle 600 10485760 58
0x00000000 294913 oracle 600 1090519040 58
0x00000000 327682 oracle 600 8388608 58
0xafa94c20 360451 oracle 600 2097152 58
$ cat test6.txt
5451 16:30:44.491186 shmget(0xafa94c20, 0, 0) = 360451 <0.000011>
5451 16:30:44.491295 shmctl(360451, IPC_STAT, 0x7fffb00f5440) = 0 <0.000007>
5451 16:30:44.491329 shmat(360451, 0, 0) = 0x2aaaaac00000 <0.000014>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5451 16:30:44.491371 shmdt(0x2aaaaac00000) = 0 <0.000012>
5451 16:30:44.491400 shmget(0xafa94c21, 0, 0) = -1 ENOENT (No such file or directory) <0.000006>
5451 16:30:44.491432 shmget(0xafa94c22, 0, 0) = -1 ENOENT (No such file or directory) <0.000004>
5451 16:30:44.491452 shmget(0xafa94c23, 0, 0) = -1 ENOENT (No such file or directory) <0.000004>
5451 16:30:44.491471 shmat(360451, 0xa3000000, 0) = 0xa3000000 <0.000008>
5451 16:30:44.491505 shmat(294913, 0x61000000, 0) = 0x61000000 <0.000010>
5451 16:30:44.491538 shmat(327682, 0xa2000000, 0) = 0xa2000000 <0.000009>
5451 16:30:44.491564 shmat(262144, 0x60000000, 0) = 0x60000000 <0.000009>
5451 16:30:44.497938 shmget(0xa93a2388, 0, 0) = -1 ENOENT (No such file or directory) <0.000008>
5451 16:30:44.497984 shmget(0xa93a2389, 0, 0) = -1 ENOENT (No such file or directory) <0.000005>
5451 16:30:44.498007 shmget(0xa93a238a, 0, 0) = -1 ENOENT (No such file or directory) <0.000005>
5451 16:30:44.498026 shmget(0xa93a238b, 0, 0) = -1 ENOENT (No such file or directory) <0.000005>
5450 16:30:44.815494 +++ exited with 0 +++
5451 16:30:44.817079 shmdt(0x61000000) = 0 <0.000035>
5451 16:30:44.817140 shmdt(0xa2000000) = 0 <0.000006>
5451 16:30:44.817160 shmdt(0x60000000) = 0 <0.000006>
5451 16:30:44.817178 shmdt(0xa3000000) = 0 <0.000006>
5451 16:30:44.819265 +++ exited with 0 +++
--//感觉也没有什么不同。
--//不过我发现1个特点。use_large_pages<>false的shmat(360451, 0, 0) = 0x2aaaaac00000 <0.000014>的地址是不变的。
3.换一个跟踪方式，使用gdb：
$ cat test1.gdb
set pagination off
set logging overwrite on
set logging on
break shmdt
commands
silent
shell date ; sleep 1
c
end
--//注：仅仅跟踪执行exit的shmdt函数调用。
--//session 1:
SYS@book> @ spid
==============================
SID : 2
SERIAL# : 53957
PROCESS : 6427
SERVER : DEDICATED
SPID : 6428
PID : 8
P_SERIAL# : 17
KILL_COMMAND : alter system kill session '2,53957' immediate;
PL/SQL procedure successfully completed.
--//window 1:
$ gdb -f -p 6211 -x test1.gdb
--//window 2:
$ rm test1.ipcs -f; watch -n 1 "ipcs -m|ts.awk | tee -a test1.ipcs"
--//session 1:
SYS@book> quit
Disconnected from Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0
--//window 1:
$ gdb -f -p 6211 -x test1.gdb
...
Breakpoint 1 at 0x7f5a9e8a5020: file ../sysdeps/unix/syscall-template.S, line 81.
(gdb) c
Continuing.
Sun May 3 10:42:00 CST 2026
Sun May 3 10:42:01 CST 2026
Sun May 3 10:42:02 CST 2026
Sun May 3 10:42:03 CST 2026
[Inferior 1 (process 6428) exited normally]
--//window 2:
--//按ctrl+c中断执行.
$ cat /proc/$(pgrep pmon)/maps | grep rw-s
60001000-6093d000 rw-s 00001000 00:04 0 /SYSV00000000 (deleted)
61000000-a2000000 rw-s 00000000 00:04 32769 /SYSV00000000 (deleted)
a2000000-a26c3000 rw-s 00000000 00:04 65538 /SYSV00000000 (deleted)
a3000000-a3004000 rw-s 00000000 00:04 98307 /SYSVafa94c20 (deleted)
7f44a7f2f000-7f44a7f30000 rw-s 00000000 08:11 18861347 /u01/app/oracle/dbs/hc_book.dat
5451 16:30:44.817079 shmdt(0x61000000) = 0 <0.000035> --//第2共享内存段
5451 16:30:44.817140 shmdt(0xa2000000) = 0 <0.000006> --//第3共享内存段
5451 16:30:44.817160 shmdt(0x60000000) = 0 <0.000006> --//第1共享内存段
5451 16:30:44.817178 shmdt(0xa3000000) = 0 <0.000006> --//第4共享内存段
--//查看test1.ipcs的输出：
[2026-05-03 10:42:01]
[2026-05-03 10:42:01] ------ Shared Memory Segments --------
[2026-05-03 10:42:01] key shmid owner perms bytes nattch status
[2026-05-03 10:42:01] 0x00000000 0 oracle 600 9687040 112
[2026-05-03 10:42:01] 0x00000000 32769 oracle 600 1090519040 56
[2026-05-03 10:42:01] 0x00000000 65538 oracle 600 7090176 56
[2026-05-03 10:42:01] 0xafa94c20 98307 oracle 600 16384 56
[2026-05-03 10:42:01]
[2026-05-03 10:42:02]
[2026-05-03 10:42:02] ------ Shared Memory Segments --------
[2026-05-03 10:42:02] key shmid owner perms bytes nattch status
[2026-05-03 10:42:02] 0x00000000 0 oracle 600 9687040 112
[2026-05-03 10:42:02] 0x00000000 32769 oracle 600 1090519040 55
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-05-03 10:42:02] 0x00000000 65538 oracle 600 7090176 56
[2026-05-03 10:42:02] 0xafa94c20 98307 oracle 600 16384 56
[2026-05-03 10:42:02]
[2026-05-03 10:42:03]
[2026-05-03 10:42:03] ------ Shared Memory Segments --------
[2026-05-03 10:42:03] key shmid owner perms bytes nattch status
[2026-05-03 10:42:03] 0x00000000 0 oracle 600 9687040 110
[2026-05-03 10:42:03] 0x00000000 32769 oracle 600 1090519040 55
[2026-05-03 10:42:03] 0x00000000 65538 oracle 600 7090176 55
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-05-03 10:42:03] 0xafa94c20 98307 oracle 600 16384 56
[2026-05-03 10:42:03]
[2026-05-03 10:42:04]
[2026-05-03 10:42:04] ------ Shared Memory Segments --------
[2026-05-03 10:42:04] key shmid owner perms bytes nattch status
[2026-05-03 10:42:04] 0x00000000 0 oracle 600 9687040 110
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~--//问题在这里为什么调用1次shmdt(0x60000000)减少2。
[2026-05-03 10:42:04] 0x00000000 32769 oracle 600 1090519040 55
[2026-05-03 10:42:04] 0x00000000 65538 oracle 600 7090176 55
[2026-05-03 10:42:04] 0xafa94c20 98307 oracle 600 16384 55
[2026-05-03 10:42:04]
[2026-05-03 10:42:05]
[2026-05-03 10:42:05] ------ Shared Memory Segments --------
[2026-05-03 10:42:05] key shmid owner perms bytes nattch status
[2026-05-03 10:42:05] 0x00000000 0 oracle 600 9687040 110
[2026-05-03 10:42:05] 0x00000000 32769 oracle 600 1090519040 55
[2026-05-03 10:42:05] 0x00000000 65538 oracle 600 7090176 55
[2026-05-03 10:42:05] 0xafa94c20 98307 oracle 600 16384 55
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[2026-05-03 10:42:05]
[2026-05-03 10:42:06]
--//问题看到这里，已经超出自己的分析能力。突然觉得是否与shmid=0有关呢。
4.重启数据库看看：
--//第1次重启数据库。
--//startup pfile=/u01/app/oracle/dbs/initbook.ora
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 131072 oracle 600 9687040 108
0x00000000 163841 oracle 600 1090519040 54
0x00000000 196610 oracle 600 7090176 54
0xafa94c20 229379 oracle 600 16384 54
--//163841-131072 = 32769
--//问题照旧.
--//第2次重启数据库。
--//startup pfile=/u01/app/oracle/dbs/initbook.ora
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 262144 oracle 600 9687040 94
0x00000000 294913 oracle 600 1090519040 47
0x00000000 327682 oracle 600 7090176 47
0xafa94c20 360451 oracle 600 16384 47
--//第3次重启数据库。
--//startup pfile=/u01/app/oracle/dbs/initbook.ora
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 393216 oracle 600 9687040 94
0x00000000 425985 oracle 600 1090519040 47
0x00000000 458754 oracle 600 7090176 47
0xafa94c20 491523 oracle 600 16384 47
--//第4次重启数据库。
--//startup pfile=/u01/app/oracle/dbs/initbook.ora
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 524288 oracle 600 9687040 94
0x00000000 557057 oracle 600 1090519040 47
0x00000000 589826 oracle 600 7090176 47
0xafa94c20 622595 oracle 600 16384 47
--//第5次重启数据库。
--//startup pfile=/u01/app/oracle/dbs/initbook.ora
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 655360 oracle 600 9687040 106
0x00000000 688129 oracle 600 1090519040 53
0x00000000 720898 oracle 600 7090176 53
0xafa94c20 753667 oracle 600 16384 53
--//每次重启shmid数量会增加。不过我还是发现1个特点第1个共享内存段shmid有一个规律。
$ echo 131072 262144 393216 524288 655360 | tr " " "\n"| xargs -IQ factor Q
131072: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
262144: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
393216: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3
524288: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
655360: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 5
$ factor 131072 | grep -o " 2"|wc
17 17 51
--//17个2。
--//也就是总是2^17*N.(除了第1次开机shmid=0),可以理解N=0开始.
--//第6次重启数据库。
--//startup pfile=/u01/app/oracle/dbs/initbook.ora,use_large_pages=only
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x00000000 786432 oracle 600 10485760 54
0x00000000 819201 oracle 600 1090519040 54
0x00000000 851970 oracle 600 8388608 54
0xafa94c20 884739 oracle 600 2097152 54
--//use_large_pages=only,ipcs -m的输出nattch就不存在这个问题.
$ factor 786432
786432: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3
--//相当于2^17*6
--//如果你数学功底足够,开一个玩笑,还是可以看出shmid的一个规律:
819201-786432 = 32769
851970-819201 = 32769
884739-851970 = 32769
--//每个相差32769,32768+1 = 0x8001
$ ipcs -m | awk '/[0=9]/{print $2}'| awk 'NR==1{a=$1} NR>1{print $1-a;a=$1}'
32769
32769
32769
786432 = 0xc0000
819201 = 0xc8001
851970 = 0xd0002
884739 = 0xd8003
--//16进制的最后3位数可以看出属于那个段.
--//补充21c,多个共享内存段的情况.
--// kernel.shmmax = 268435456,机器重启的测试结果.
# ipcs -m | awk '/[0=9]/{print $2}'| awk 'NR==1{printf "0x%08x %d\n",$1,$1;a=$1} NR>1{printf "0x%08x %d %d\n", $1,$1,$1-a;a=$1}'
0x00000000 0
0x00008001 32769 32769
0x00010002 65538 32769
0x00018003 98307 32769
0x00020004 131076 32769
0x00028005 163845 32769
0x00030006 196614 32769
0x00038007 229383 32769
4.小结:
--//总之还是没有定位21c下pre_page_sga=false的情况下ipcs -m输出nattch的问题,放弃该问题探究.
--//另外关于ipcs -m 的shmid值,另外再写一些blog分析.
