Typically, in most environments, repositories on the Internet can only be accessed from a few AIX systems. In order to still make repositories such as the AIX toolbox available for all AIX systems, it is possible to set up a mirror of a remote repository on a central system with a direct or indirect (via proxy) Internet connection. This is shown below using the example of the AIX Toolbox repositories.

In principle, the files (RPM packages) of the repositories can be downloaded to a local server with any tool (e.g. wget or curl). However, this is particularly easy with the reposync command from the yum-utils RPM package.

By default, reposync synchronizes all (enabled) repositories from the configuration file /opt/freeware/etc/yum/yum.conf into the current directory. A separate subdirectory is created for each repository, the directory name is the repository ID. The way reposync works can be configured using a number of options.

We use the standard configuration of YUM on our central web server aixweb from the installation:

aixweb # cat /opt/freeware/etc/yum/yum.conf
[main]
cachedir=/var/cache/yum
keepcache=1
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
plugins=1

[AIX_Toolbox]
name=AIX generic repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/
enabled=1
gpgcheck=0

[AIX_Toolbox_noarch]
name=AIX noarch repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/noarch/
enabled=1
gpgcheck=0

[AIX_Toolbox_72]
name=AIX 7.2 specific repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc-7.2/
enabled=1
gpgcheck=0

aixweb #

The repositories should be saved on the web server under the directory /var/www/htdocs/aixtoolbox. With the option “-p” the command reposync can be given a target directory for the repositories:

aixweb # reposync -p /var/www/htdocs/aixtoolbox
[AIX_Toolbox: 1     of 2743  ] Downloading CUnit/CUnit-2.1.3-1.aix6.1.ppc.rpm
CUnit-2.1.3-1.aix6.1.ppc.rpm                                              | 129 kB  00:00:00    
[AIX_Toolbox: 2     of 2743  ] Downloading CUnit/CUnit-devel-2.1.3-1.aix6.1.ppc.rpm
CUnit-devel-2.1.3-1.aix6.1.ppc.rpm                                        |  83 kB  00:00:00    
[AIX_Toolbox: 3     of 2743  ] Downloading GeoIP/GeoIP-1.6.12-1.aix6.1.ppc.rpm
GeoIP-1.6.12-1.aix6.1.ppc.rpm                                             | 1.8 MB  00:00:00    
…
[AIX_Toolbox_noarch: 299   of 301   ] Downloading yum-utils/yum-plugin-versionlock-1.1.31-2.aix6.1.noarch.rpm
yum-plugin-versionlock-1.1.31-2.aix6.1.noarch.rpm                         |  17 kB  00:00:00    
[AIX_Toolbox_noarch: 300   of 301   ] Downloading yum-utils/yum-utils-1.1.31-1.aix6.1.noarch.rpm
yum-utils-1.1.31-1.aix6.1.noarch.rpm                                      |  89 kB  00:00:00    
[AIX_Toolbox_noarch: 301   of 301   ] Downloading yum-utils/yum-utils-1.1.31-2.aix6.1.noarch.rpm
yum-utils-1.1.31-2.aix6.1.noarch.rpm                                      |  88 kB  00:00:00    
aixweb #

The download of the approx. 15 GB of data can take some time. A separate subdirectory was created for each of the 3 repositories:

aixweb # ls -l /var/www/htdocs/aixtoolbox
total 48
drwxr-xr-x  260 root     system        16384 Dec 05 12:25 AIX_Toolbox
drwxr-xr-x   19 root     system         4096 Dec 05 12:39 AIX_Toolbox_72
drwxr-xr-x   50 root     system         4096 Dec 05 12:40 AIX_Toolbox_noarch
aixweb #

After the data has been downloaded, the metadata for the repositories can be generated using the createrepo command. The directory of the repository is specified as an argument. The command must be started once for each repository:

aixweb # createrepo /var/www/htdocs/aixtoolbox/AIX_Toolbox
Spawning worker 0 with 342 pkgs
Spawning worker 1 with 342 pkgs
Spawning worker 2 with 342 pkgs
Spawning worker 3 with 342 pkgs
Spawning worker 4 with 342 pkgs
Spawning worker 5 with 342 pkgs
Spawning worker 6 with 342 pkgs
Spawning worker 7 with 341 pkgs
…
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
aixweb #  
aixweb # createrepo /var/www/htdocs/aixtoolbox/AIX_Toolbox_72
…
aixweb #
aixweb # createrepo /var/www/htdocs/aixtoolbox/AIX_Toolbox_noarch
…
aixweb #

The mirrors of the AIX toolbox repositories are now ready for use. Since these are already under the document root (/var/www/htdocs) of the Apache web server on our server aixweb, they can be used immediately from other systems via HTTP or HTTPS.

The local mirrors should be regularly synchronized with the official repositories. To do this, the commands reposync and createrepo from above, can be executed regularly (e.g. via cron).

On the web server aixweb itself, YUM would, at the moment, according to the standard configuration, still access the IBM web server directly, although there are now local copies available. If you change the entries in /opt/freeware/etc/yum/yum.conf to the local web server, the synchronization using reposync no longer works, since reposync then accesses the local web server. Another open point is the download of the repositories for AIX 7.1 and 7.3, currently these are not downloaded because the AIX web server aixweb runs under AIX 7.2 and therefore does not need the repositories for AIX 7.1 and 7.3 and has therefore not configured it.

A simple solution to both of these problems is to use a separate configuration file for synchronizing the repositories. We copy the current standard configuration /opt/freeware/etc/yum/yum.conf into the file /opt/freeware/etc/yum/reposync.conf (name can be freely choosen):

aixweb # cp /opt/freeware/etc/yum/yum.conf /opt/freeware/etc/yum/reposync.conf
aixweb #

We take this opportunity to expand the configuration to include the repositories for AIX 7.1 and AIX 7.3:

aixweb # cat /opt/freeware/etc/yum/reposync.conf
[main]
cachedir=/var/cache/yum
keepcache=1
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
plugins=1

[AIX_Toolbox]
name=AIX generic repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc/
enabled=1
gpgcheck=0

[AIX_Toolbox_noarch]
name=AIX noarch repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/noarch/
enabled=1
gpgcheck=0

[AIX_Toolbox_71]
name=AIX 7.1 specific repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc-7.1/
enabled=1
gpgcheck=0

[AIX_Toolbox_72]
name=AIX 7.2 specific repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc-7.2/
enabled=1
gpgcheck=0

[AIX_Toolbox_73]
name=AIX 7.3 specific repository
baseurl=https://anonymous:anonymous@public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc-7.3/
enabled=1
gpgcheck=0

aixweb #

When starting reposync, the “-c” (or “—config”) option can be used to specify the name of a configuration file that is to be used instead of the default /opt/freeware/etc/yum/yum.conf.

aixweb # reposync -c /opt/freeware/etc/yum/reposync.conf -p /var/www/htdocs/aixtoolbox
AIX_Toolbox                                                               | 2.6 kB  00:00:00    
AIX_Toolbox_71                                                            | 2.5 kB  00:00:00    
AIX_Toolbox_72                                                            | 2.5 kB  00:00:00    
AIX_Toolbox_73                                                            | 2.5 kB  00:00:00    
AIX_Toolbox_73/primary_db                                                 |  12 kB  00:00:00    
AIX_Toolbox_noarch                                                        | 2.5 kB  00:00:00    
[AIX_Toolbox: 1     of 2743  ] Skipping existing CUnit/CUnit-2.1.3-1.aix6.1.ppc.rpm
[AIX_Toolbox: 2     of 2743  ] Skipping existing CUnit/CUnit-devel-2.1.3-1.aix6.1.ppc.rpm
[AIX_Toolbox: 3     of 2743  ] Skipping existing GeoIP/GeoIP-1.6.12-1.aix6.1.ppc.rpm
…
[AIX_Toolbox_noarch: 300   of 301   ] Skipping existing yum-utils/yum-utils-1.1.31-1.aix6.1.noarch.rpm
[AIX_Toolbox_noarch: 301   of 301   ] Skipping existing yum-utils/yum-utils-1.1.31-2.aix6.1.noarch.rpm
aixweb #

For the two new repositories (AIX 7.1 and 7.3), the metadata must first be generated, again using the createrepo command:

aixweb # createrepo /var/www/htdocs/aixtoolbox/AIX_Toolbox_71
…
aixweb # createrepo /var/www/htdocs/aixtoolbox/AIX_Toolbox_73
…
aixweb #

However, the metadata must also be updated for the existing repositories, since new RPM packages may have been added. In order not to have to regenerate the metadata for the existing RPM packages, the option “—update” can be specified with the createrepo command. This option ensures that the existing metadata is adopted and metadata is only generated and added for new RPM packages. For repositories with many (thousands) of RPM packages this shortens the runtime of createrepo considerably:

aixweb # createrepo --update /var/www/htdocs/aixtoolbox/AIX_Toolbox
…
aixweb #
aixweb # createrepo --update /var/www/htdocs/aixtoolbox/AIX_Toolbox_noarch
…
aixweb #
aixweb # createrepo --update /var/www/htdocs/aixtoolbox/AIX_Toolbox_72
…
aixweb #

The reposync command and the “createrepo –update” commands for the individual repositories should be executed regularly via cron (e.g. once a week) to keep the repository mirror up to date.

On the AIX web server aixweb itself, YUM can then access the repositories locally (file:///var/www/htdocs/aixtoolbox/AIX_Toolbox etc.) or via the web (http://localhost/aixtoolbox/AIX_Toolbox etc.). Of course, direct access to the IBM repositories would still be possible.