Tag: Filesystem

Posted on

Path to the executable of a running AIX process

If a process was not started with an absolute path, it is surprisingly difficult to find out the absolute path for the associated executable.

We demonstrate this using Splunk as an example:

$ ps -ef |grep splun[k]d
    root  7143802  5702116   0   Apr 23      - 23:26 splunkd --nodaemon -p 8089 _internal_exec_splunkd
    root 31916484  7143802   0   Apr 23      -  0:00 [splunkd pid=7143802] splunkd --nodaemon -p 8089 _internal_exec_splunkd [process-runner]
$

When starting the process with the PID 31916484, the argument 0 was also changed.

Part of the information about a process is available via the process file system /proc. Since Splunk runs as root, root privileges are required to access the information in /proc about the process.

Below /proc there is a subdirectory for each running process, with the PID as the directory name.

# ls -l /proc/31916484
total 32
-rw-------    1 root     system            0 Apr 28 15:17 as
-r--------    1 root     system          128 Apr 28 15:17 cred
--w-------    1 root     system            0 Apr 28 15:17 ctl
lr-x------   38 root     system            0 Apr 28 13:31 cwd -> /root/
dr-x------    1 root     system            0 Apr 28 15:17 fd
dr-xr-xr-x    1 root     system            0 Apr 28 15:17 lwp
-r--------    1 root     system            0 Apr 28 15:17 map
-r--------    1 root     system            0 Apr 28 15:17 mmap
dr-x------    1 root     system            0 Apr 28 15:17 object
-r--r--r--    1 root     system          448 Apr 28 15:17 psinfo
lr-x------   48 root     system            0 Apr 28 09:02 root -> /
-r--------    1 root     system        12288 Apr 28 15:17 sigact
-r--------    1 root     system         1520 Apr 28 15:17 status
-r--r--r--    1 root     system            0 Apr 28 15:17 sysent
#

In the subdirectory object, in addition to the open files, the executable is also available:

# ls -l /proc/31916484/object
total 854216
-r-xr-xr-x    1 root     system    194980846 Nov 16 2019  a.out
-r-xr-xr-x    1 bin      bin           10749 Sep 21 2015  jfs2.10.5.103039
-r--r--r--    1 bin      bin        12858422 May 28 2019  jfs2.10.5.225767
-r-xr-xr-x    1 bin      bin           77411 Mar 25 2021  jfs2.10.5.4157
-r-xr-xr-x    1 bin      bin        13438344 Jul 27 2021  jfs2.10.5.4205
-r--r--r--    1 bin      bin         1351386 Mar 09 2021  jfs2.10.5.4209
-r--r--r--    1 bin      bin           80450 Jul 27 2021  jfs2.10.5.4220
-r-xr-xr-x    1 root     system    194980846 Nov 16 2019  jfs2.10.9.139342
-r-xr-xr-x    1 root     system      3487967 Nov 12 2019  jfs2.10.9.155672
-r-xr-xr-x    1 root     system       228087 Nov 12 2019  jfs2.10.9.155673
-r-xr-xr-x    1 root     system      2688333 Nov 16 2019  jfs2.10.9.155675
-r-xr-xr-x    1 root     system       901800 Nov 12 2019  jfs2.10.9.155677
-r-xr-xr-x    1 root     system      4256593 Nov 12 2019  jfs2.10.9.155679
-r-xr-xr-x    1 root     system       568791 Nov 16 2019  jfs2.10.9.155684
-r-xr-xr-x    1 root     system      6166802 Nov 12 2019  jfs2.10.9.155685
-r-xr-xr-x    1 root     system      1239656 Nov 12 2019  jfs2.10.9.155686
-r-xr-xr-x    1 root     system       124218 Nov 16 2019  jfs2.10.9.155690
#

The file a.out represents the executable. Access to this file is redirected to the executable’s file. Information such as the inode number and device (file system) of the executable can be found with the istat command:

# istat /proc/31916484/object/a.out
Inode 139342 on device 10/9     File
Protection: r-xr-xr-x  
Owner: 0(root)          Group: 0(system)
Link count:   1         Length 194980846 bytes

Last updated:   Tue Jul 27 07:15:40 CEST 2021
Last modified:  Sat Nov 16 01:23:05 CET 2019
Last accessed:  Thu Apr 28 10:38:13 CEST 2022

#

The inode number of the executable is 139342. The file system is the file system on the device with major number 10 and minor number 9 (“device 10/9“).

You could now first find out the device by searching under /dev:

$ ls -l /dev | grep "10,  9"
brw-rw----    1 root     system       10,  9 Jul 27 2021  hd10opt
crw-rw----    1 root     system       10,  9 Apr 05 13:36 rhd10opt
$

and then determine the associated file system via df:

$ df | grep hd10opt
/dev/hd10opt    12582912   7527920   41%    33349     4% /opt
$

However, this is even easier. You can simply specify the path for the a.out file in df:

# df /proc/31916484/object/a.out
Filesystem    512-blocks      Free %Used    Iused %Iused Mounted on
/dev/hd10opt    12582912   7527912   41%    33349     4% /opt
#

The file system we are looking for is the /opt file system. Now you can find out the absolute path to the executable by searching for the inode number 139342 in /opt:

# find /opt -inum 139342
/opt/splunkforwarder/bin/splunkd
#

The process with the PID 31916484 runs the executable /opt/splunkforwarder/bin/splunkd.

With a little trick you can also shorten the search significantly. For this you need a shell as user root. In this shell you open the a.out explicitly with exec and a (free) descriptor number:

# exec 5</proc/31916484/object/a.out
#

Our current shell has thus opened the executable (via the file descriptor 5)! Now you can use the procfiles command to display the open files in this shell. Using the “-n” option, which displays the absolute paths of files that belong to a file descriptor:

# procfiles -n $$
19136808 : ksh
  Current rlimit: 10000 file descriptors
   0: S_IFCHR mode:00 dev:10,4 ino:4463 uid:0 gid:0 rdev:21,3
      O_RDWR | O_NOCTTY  name://dev/pts/3
   1: S_IFCHR mode:00 dev:10,4 ino:4463 uid:0 gid:0 rdev:21,3
      O_RDWR | O_NOCTTY  name://dev/pts/3
   2: S_IFCHR mode:00 dev:10,4 ino:4463 uid:0 gid:0 rdev:21,3
      O_RDWR | O_NOCTTY  name://dev/pts/3
   5: S_IFREG mode:0555 dev:10,9 ino:139342 uid:0 gid:0 rdev:0,0
      O_RDONLY size:194980846  name:/opt/splunkforwarder/bin/splunkd
   10: S_IFREG mode:0444 dev:10,5 ino:124151 uid:0 gid:0 rdev:0,0
      O_RDONLY size:5875  name:/usr/lib/nls/msg/EN_US/ksh.cat
   63: S_IFREG mode:0600 dev:10,4 ino:41933 uid:0 gid:0 rdev:0,0
      O_RDWR | O_APPEND size:5494  name://root/.sh_history
#

Note: The special variable $$ is replaced by the shell with the shell’s PID.

For file descriptor 5, the absolute path /opt/splunkforwarder/bin/splunkd is displayed.

The file descriptor should of course be closed again:

# exec 5<&-
#
Posted on

What is the size of the internal log in JFS2

inline log size

A trivial question we stumbled across recently:

How big is the internal JFS2 log currently?

The size of the internal JFS2 log must meet the following two conditions:

    1. The log cannot be larger than 10% of the file system size.
    2. The maximum size cannot exceed 2047 MB.

When creating a JFS2 filesystem with internal log, if no size is specified for the log (-a logsize=value), 0.4% of the filesystem size is used by default. The value 0.4% is documented in the crfs manual page.

But how big is the internal JFS2 log right now?

This information is provided by the dumpfs command. It expects either the mount point of a JFS2 file system or the device file of the underlying logical volume as an argument. The command lists the superblock and additional control information. The output can be very long for larger file systems. Since we are only interested in the JFS2 log, it is advisable to filter the output using the grep command:

# dumpfs /data | grep -i log
aggregate block size    4096            log2 of aggregate block size    12
LVM I/O Transfer size   512             log2 of LVM transfer  size      9
log2 of block size/transfer size        3
Aggregate attributes    J2_GROUPCOMMIT J2_INLINELOG
log device      0x8000002700000001 log serial number    0x26
Inline Log: 541065216 (132096); 1024
fsck Service Log number of blocks: 50
Extendfs Inline Log Working Space: 541065216 (132096); 1024
#

The last value in the line “Inline Log:” indicates the size of the internal log in blocks. The block size of the file system can be found in the line “aggregate block size“. In our case, the internal log has a size of 1024 blocks, each with 4096 bytes. This gives a size of 4 MB (1024 * 4 KB).

If an external log is used, the output looks like this:

# dumpfs / | grep -i log
aggregate block size    4096            log2 of aggregate block size    12
LVM I/O Transfer size   512             log2 of LVM transfer  size      9
log2 of block size/transfer size        3
log device      0x8000000a00000003 log serial number    0xb
Inline Log: 0 (0); 0
fsck Service Log number of blocks: 50
Extendfs Inline Log Working Space: 0 (0); 0
#

The internal log has a size of 0 blocks.

However, this is not the easiest way. Chris Gibson points out the “-q” option of the lsfs command, which displays additional information for JFS and JFS2 file systems:

# lsfs -q /filesystem
Name            Nodename   Mount Pt               VFS   Size    Options    Auto Accounting
/dev/fslv01     --         /filesystem            jfs2  1048576 --         no   no
  (lv size: 1048576, fs size: 1048576, block size: 4096, sparse files: yes, inline log: yes, inline log size: 4, EAformat: v1, Quota: no, DMAPI: no, VIX: yes, EFS: no, ISNAPSHOT: no, MAXEXT: 0, MountGuard: no)
#

The size of the inline log is specified there directly in MB (inline log size: 4).

Determining the size of the internal JFS2 log is therefore no problem with the right command (dumpfs lsfs)!

Posted on

AIX: Applications of the namefs-Filesystem

Occasionally one needs a directory (or a file system) somewhere else in the file system or maybe even at several different places in the file system. Instead of solving the problem with symbolic links, you can elegantly use the namefs file system.

In the following example /data/in is required elsewhere:

# ls -l /data/in
total 16
-rw-r--r--    1 root     system          554 May 14 16:10 file1
-rw-r--r--    1 root     system          381 May 14 16:10 file2
# ls -l /other/place
total 0
#

Mounting the directory to the desired location /other/place:

# mount -v namefs /data/in /other/place
# ls -l /other/place
total 16
-rw-r--r--    1 root     system          554 May 14 16:10 file1
-rw-r--r--    1 root     system          381 May 14 16:10 file2
#

The mount with the namefs file system additionally offers the possibility to specify mount options, which then only apply to the directory. One can do so to mount a directory with Direct-I/O, even though the original directory was not mounted with Direct-I/O:

# mount -v namefs -o dio /data/in /other/place
# mount
  node       mounted        mounted over    vfs       date        options     
-------- ---------------  ---------------  ------ ------------ ---------------
         /dev/hd4         /                jfs2   May 02 11:30 rw,log=/dev/hd8
...
         /data/in         /other/place     namefs May 14 16:14 rw,dio         
#

When accessing the files below /other/place, Direct-I / O will be used. When accessing via the “originals” under /data/in, no Direct-I / O will be used!

However, access to files is limited to the underlying physical file system, as with NFS. This can easily be demonstrated by the file system /. We mount / using namefs onto /mnt and look at /mnt/usr and /mnt/var:

# mount -v namefs / /mnt
# ls -l /mnt/usr /mnt/var
/mnt/usr:
total 0
lrwxrwxrwx    1 root     system           11 Apr 17 07:49 lib -> /../usr/lib

/mnt/var:
total 0
#

The directories are empty or contain a symbolic link, /usr and /var clearly look different!

Of course, this can also be exploited, e.g. in cases where interesting data has been over-mounted. We dropped a file below /home before /dev/hd1 was mounted onto /home. The root file system currently mounted on /mnt allows access to this over-mounted data:

# ls -l /mnt/home
total 0
-rw-r--r--    1 root     system            0 May 14 17:48 overmounted_file
#

Another application is to protect a directory against overwriting. We demonstrate this for the directory /data with 2 test files:

# ls -l /data
total 16
-rw-r--r--    1 root     system          554 May 14 17:52 file1
-rw-r--r--    1 root     system          381 May 14 17:52 file2
# cp /etc/hosts /data
# ls -l /data
total 24
-rw-r--r--    1 root     system          554 May 14 17:52 file1
-rw-r--r--    1 root     system          381 May 14 17:52 file2
-rw-r--r--    1 root     system         2075 May 14 17:54 hosts
#

Overwriting or changing data is currently still possible, as shown by the successful cp command. Now we protect the data by doing a mount with the namefs file system and the option ro (read-only):

# mount -v namefs -o ro /data /data
# cp /etc/services /data
cp: /data/services: Read-only file system
#

The data obviously can not be changed anymore. Here we have over-mounted /data with a read-only version of itself!

Mounts with the namefs pseudo-file system can not only be done on jfs2 file systems, but also for NFS file systems or the procfs file system.

Finally we show the mounting of a file to some other place in the file system. We want to make the file /etc/hosts available via the name /hosts. To do this, we first create an empty file /hosts and then mount the file /etc/hosts onto this empty file:

# touch /hosts
# ls -l /hosts
-rw-r--r--    1 root     system            0 May 14 17:59 /hosts
# mount -v namefs /etc/hosts /hosts
# ls -l /hosts
-rw-rw-r--    1 root     system         2075 Apr 26 10:47 /hosts
#

Before the mount, /hosts was 0 bytes in size, after the mount 2075 bytes!

The namefs file system thus offers some interesting possibilities, that can be useful for some problems.