When increasing logical volumes or file systems, the error “0516-404 allocp: This system cannot fulfill the allocation request” often occurs. Many AIX administrators are then a little at a loss as to what the cause of the problem is: is it max LPs or upper bound, the chosen strictness, or does the problem have completely different causes. Unfortunately, the error message is only generic and does not say what the real problem is. A good way to get to the cause of the problem is the alog lvmt, in which all LVM actions, especially the low-level commands, including error messages are logged. The following is a series of examples which show how useful this log is to find the cause of the problem quickly in many cases.
1. Successful extension
First, let’s take a look at the alog lvmt for a successful filesystem extension. We are expanding a small file system by only 8 MB, which in our case corresponds exactly to the PP (Physical Partition) size:
# chfs -a size=+8M /fs01 Filesystem size changed to 2097152 #
The mentioned lvmt log can be shown with the aid of the alog command and the “-o” option:
# alog –t lvmt –o … [S 12255362 16908296 03/22/21-19:10:28:736 extendlv.sh 789] extendlv fslv01 1 … [S 13500646 12255362 03/22/21-19:10:28:900 allocp.c 1425] allocp -i 00ca0a6000004b00000001785af0a43e.1 -t e -c 1 -s 1 -u 1024 -e m -a m [3 13500646 0:047 allocp.c 1226] get_pv_mirror_pool: LV not using mirror pools. [E 13500646 0:047 allocp.c 1558] allocp: exited with rc=0 … [E 12255362 0:763 extendlv.sh 33] extendlv: exited with rc=0 #
You can see that the extendlv command has been started, with only one LP (Logical Partition) being added here. The intermediate command /usr/sbin/allocp takes over the allocation of the required PPs. It is called with a number of arguments:
-i LVID of the logical volume -t logical volume type -c number of LV copies -s number of logical partitions -u maximum number of physical volumes (upper bound) -e inter physical volume allocation policy -a intra physical volume allocation policy
The message “LV not using mirror pools” clearly indicates that no mirror pools are used by the logical volume. If a logical volume uses mirror pools, this line is missing from the output. The absence of the message is an indication of the use of mirror pools!
The allocp command was successful: “exited with rc = 0“. Ultimately, extendlv was also successful.
2. Max LPs too small
In our first case, it is not necessary to look at the lvmt log, as the error message here points directly to the problem. Since this case shows up frequently in practice, we have included it here, for the sake of completeness:
# chfs -a size=+2G /fs01 0516-787 extendlv: Maximum allocation for logical volume fslv01 is 512. #
The error message clearly indicates that the parameter Max LPs is too small:
# lslv fslv01 … MAX LPs: 512 PP SIZE: 8 megabyte(s) … #
The problem can be eliminated very easily with the help of chlv:
# chlv -x 1024 fslv01 # chfs -a size=+2G /fs01 Filesystem size changed to 10305536 Inlinelog size changed to 20 MB. #
3. Not enough free PPs in the VG
When trying to expand an existing file system by 10 GB, we get the following error message:
# chfs -a size=+10G /fs01 0516-404 allocp: This system cannot fulfill the allocation request. There are not enough free partitions or not enough physical volumes to keep strictness and satisfy allocation requests. The command should be retried with different allocation characteristics. #
Here is the corresponding extract from the lvmt log:
# alog –t lvmt –o … [S 21889060 4128932 03/22/21-20:04:19:854 extendlv.sh 789] extendlv fslv01 1280 … [S 19398754 21889060 03/22/21-20:04:20:040 allocp.c 1425] allocp -i 00ca0a6000004b00000001785af0a43e.1 -t e -c 1 -s 1280 -u 1024 -e m -a m [3 19398754 0:055 allocp.c 1226] get_pv_mirror_pool: LV not using mirror pools. [1 19398754 0:055 allocp.c 999] allocp: FAIL: total_avail_pps(1012)<(size(1280)*requests_unfilled(1) [E 19398754 0:055 allocp.c 1558] allocp: exited with rc=-1 … [E 21889060 0:346 extendlv.sh 33] extendlv: exited with rc=1 #
The logical volume fslv01 is expanded by 1280 LPs and there are no mirror pools in the game. The error message from allocp “FAIL: total_avail_pps (1012) <(size (1280) * requests_unfilled (1)” means that currently only 1012 PPs are free and therefore 1280 PPs cannot be allocated.
A look at the VG shows that in fact only 1012 PPs are free:
# lsvg vg00 … MAX LVs: 256 FREE PPs: 1012 (8096 megabytes) … #
The desired expansion is not possible without additional disk capacity. A maximum of 8 GB can currently be added to the file system.
4. Upper bound is too low
In the following example extending a filesystem also fails:
# chfs -a size=+10G /fs01 0516-404 allocp: This system cannot fulfill the allocation request. There are not enough free partitions or not enough physical volumes to keep strictness and satisfy allocation requests. The command should be retried with different allocation characteristics. #
We look at the lvmt log again:
# alog –t lvmt –o … [S 12386466 15007772 03/22/21-20:16:36:273 extendlv.sh 789] extendlv fslv01 1280 … [S 7274510 12386466 03/22/21-20:16:36:507 allocp.c 1425] allocp -i 00ca0a6000004b00000001785af0a43e.1 -t e -c 1 -s 1280 -u 1 -e m -a m [3 7274510 0:103 allocp.c 1226] get_pv_mirror_pool: LV not using mirror pools. [1 7274510 0:103 allocp.c 999] allocp: FAIL: total_avail_pps(1012)<(size(1280)*requests_unfilled(1) [E 7274510 0:103 allocp.c 1558] allocp: exited with rc=-1 … [E 12386466 0:431 extendlv.sh 33] extendlv: exited with rc=1 #
The messages in the log look exactly like in the previous example. A look at the number of free PPs in the VG shows, however, that there are enough free PPs:
# lsvg vg00 … MAX LVs: 256 FREE PPs: 3454 (27632 megabytes) … #
There are 3454 free PPs in the VG. In the message “FAIL: total_avail_pps (1012) <(size (1280) * requests_unfilled (1)” from the lvmt log, however, only 1012 available PPs are assumed! On closer inspection you can see that the intermediate command allocp was called with the option “-u 1“, ie a maximum of 1 physical volume can only be used for the logical volume. The LV may currently only use additional space on the physical volume already used by the LV (here hdisk2).
We increase the upper bound to 2:
# chlv -u 2 fslv01 #
And then we try the extension again:
# chfs -a size=+10G /fs01 Filesystem size changed to 25214976 Inlinelog size changed to 49 MB. #
And this time it is successful.
5. Problem because of strictness=yes
Extending a mirrored file system fails:
# chfs -a size=+10G /fs01 0516-404 allocp: This system cannot fulfill the allocation request. There are not enough free partitions or not enough physical volumes to keep strictness and satisfy allocation requests. The command should be retried with different allocation characteristics. #
The following messages can be found in the lvmt log:
# alog –t lvmt –o … [S 13762618 19857542 03/23/21-10:53:06:616 extendlv.sh 789] extendlv fslv01 1280 … [S 14352436 13762618 03/23/21-10:53:06:837 allocp.c 1425] allocp -i 00ca0a6000004b00000001785af0a43e.1 -t e -c 2 -s 1280 -u 3 -e m -a m [3 14352436 0:098 allocp.c 1226] get_pv_mirror_pool: LV not using mirror pools. [2 14352436 0:115 map_alloc.c 2057] figure_min_alloc: WARN: Could not fulfill allocation, num_lps_to_fill=109 [E 14352436 0:115 allocp.c 1558] allocp: exited with rc=-303 … [E 13762618 0:431 extendlv.sh 33] extendlv: exited with rc=1 #
There are still free capacities on the physical volumes in the VG:
# lsvg -p vg00 vg00: PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION hdisk2 active 1271 1143 255..126..254..254..254 hdisk3 active 1271 1171 255..154..254..254..254 hdisk4 active 1271 1143 255..126..254..254..254 hdisk5 active 1271 1171 255..154..254..254..254 #
In total, this should actually be sufficient, but according to the error message “WARN: Could not fulfill allocation, num_lps_to_fill = 109“, 109 PPs cannot be allocated. The intermediate command allocp was started with “-u 3“, so a total of up to 3 physical volumes can be used by the LV. The LV currently uses the two physical volumes hdisk2 and hdisk4:
# lslp fslv01 fslv01:/test LPs #LPs COPY1 COPY2 COPY3 0001-0128 128 hdisk2 (None) hdisk4 (None) - #
(The lslp command is a shell script written by us to clearly display the LPs of logical volumes. It is ultimately based on “lslv -m“.)
That means that in addition to hdisk2 and hdisk4 for the allocation, either hdisk3 or hdisk5 could be used. In total you would get 2 * 1143 + 1171 = 3457 free PPs. Only 2 * 1280 = 2560 PPs are required for the extension. However, now the strictness and the allocation policies come into play. The strictness is yes here and the Inter Physical Volume Allocation Policy is min (“-e m”). I.e.: of the 3 physical volumes hdisk2, hdisk3 and hdisk4, space is initially allocated from the first physical volume (hdisk2) for mirror copy 1, 1143 PPs are available on hdisk2 (this means that 137 PPs are still missing for the first mirror copy). Then space is allocated from the second physical volume (hdisk3) for mirror copy 2; there, too, 1143 PPs are available. Then it continues with the hdisk4 and the mirror copy 1. The still missing 137 PPs are available on hdisk4 and are assigned to mirror copy 1. Then it continues with mirror copy 2. There is no more free space on hdisk2, therefore hdisk3 is used next, there are still 28 PPs available, which are then assigned to mirror copy 2. This means that 109 PPs are still missing in mirror copy 2. These can not be allocated, however, since the only physical volume with free capacity is hdisk4, which, however, is already being used by mirror copy 1. Using hdisk4 for the missing 109 PPs in mirror copy 2 would violate strictness=yes, since then both mirror copies for the last 109 LPs would be on hdisk4.
The message “WARN: Could not fulfill allocation, num_lps_to_fill = 109” is an indication that the strictness cannot be adhered to.
6. Insufficient free capacity in the used mirror pools
The mirror pools DC01 and DC02 are used on one system. A file system extension fails:
bash-4.3# chfs -a size=+512M /fs01 0516-404 allocp: This system cannot fulfill the allocation request. There are not enough free partitions or not enough physical volumes to keep strictness and satisfy allocation requests. The command should be retried with different allocation characteristics. bash-4.3#
Here are the related messages from the lvmt log:
# alog –t lvmt –o … [S 7929950 9502800 03/23/21-11:56:56:506 extendlv.sh 789] extendlv fslv01 64 … [S 10354926 7929950 03/23/21-11:56:56:785 allocp.c 1425] allocp -i 00ca0a6000004b00000001785af0a43e.1 -t e -c 2 -s 64 -K 2 -u 4 -e m -a m [1 10354926 0:189 map_alloc.c 1963] figure_min_alloc: FAIL: num_lps_to_file=9 [S 10354942 7929950 03/23/21-11:56:57:004 putlvodm.c 1370] putlvodm -K 00ca0a6000004b00000001785af0a43e -X 0 … [E 7929950 0:565 extendlv.sh 33] extendlv: exited with rc=1 #
The notice that no mirror pools are used is missing! So there are mirror pools in use!
We therefore list the assignment of the physical volumes to the mirror pools to give us an overview:
# lsvg -P vg00 Physical Volume Mirror Pool hdisk2 DC01 hdisk3 None hdisk4 DC02 hdisk5 None #
Obviously, 2 physical volumes in the VG are not assigned to a mirror pool. The free capacities are as follows:
# lsvg -p vg00 vg00: PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION hdisk2 active 1271 55 00..00..00..00..55 hdisk3 active 1271 1171 255..154..254..254..254 hdisk4 active 1271 55 00..00..00..00..55 hdisk5 active 1271 71 00..00..00..00..71 #
55 PPs are still available in both mirror pools (DC01 hdisk2 and DC02 hdisk4). However, 64 PPs (“-s 64“) were requested. This means that 9 PPs are missing in each of the two mirror pools for the extension, which is also indicated in the message “figure_min_alloc: FAIL: num_lps_to_file = 9“.
After the two “new” physical volumes have been assigned to mirror pools, the file system extension is successful:
# chpv -p DC01 hdisk3 # chpv -p DC02 hdisk5 # # # chfs -a size=+512M /fs01 Filesystem size changed to 20971520 Inlinelog size changed to 40 MB. #
There are certainly a number of other situations in which file systems cannot be enlarged. It is crucial that a look at the lvmt log can hopefully, in many cases, provide an indication of the possible cause.