Saturday, August 2, 2014

Speeding Up Oracle Linux 7 Updates with "Delta RPMs"

Linux configuration management and updates can be time-consuming when taken seriously. If your servers are spread across different areas, a nice alternative to deploying several yum repositories consists in building delta RPMs or .drpm to reduce the network traffic and speed up updates. Those files are treated by yum to rebuild rpm from their previous version. This blog shows the steps associated with relying on delta RPMs to update Oracle Linux 7. Because it relies on RPMs and it distributes them for free too, those steps should also work as is with CentOS7.

In Oracle Linux 5 and 6, delta RPMs required the yum-presto plug-in. Starting with version 7, it comes with the base release. It does not really change the game, but it make the setup simpler. This article shows how to synchronize and create a repository. It explains how to setup an Apache HTTP server and how to setup yum clients to use with .drpm files.

Building a Repository with Delta RPMs

Oracle and CentOS publish their RPMs in free Internet repositories. Building a your own repository consists in synchronizing files from them with a directory on a filesystem of one of your server or virtual machine. The yum-utils package contains the reposync command that can be use for that purpose. Start by registering the repository you want to synchronize in /etc/yum.repos.d. The reposync command below synchronizes the registered ol7_latest and ol7_UEKR3 repositories into /u01/app/oracle/distribs/oraclelinux:
yum -y install yum-utils

mkdir -p /u01/app/oracle/distribs/oraclelinux

reposync --repoid=ol7_latest \
         --repoid=ol7_UEKR3 \
         -p /u01/app/oracle/distribs/oraclelinux
Note:
If you need to synchronize with non-public repositories, including the ones containing KSplice or DTrace RPMs, Oracle provides the uln-yum-mirror script. for more details about that script and how to use it, refer to Oracle® Linux - Administrator's Guide for Release 7 - 2.8 Creating and Using a Local ULN Mirror
Once the RPMs downloaded on your server, you can index and create the delta RPMs with the createrepo command. createrepo comes in its own package and to create .drpm files, the deltarpms package must be installed. createrepo --deltas and --oldpackagedirs options generate the .drpm files in the drpms directory. They also create the prestodelta.xml index file that is used by yum:
yum -y install createrepo deltarpms

cd  /u01/app/oracle/distribs/oraclelinux/ol7_latest

sudo createrepo --deltas --oldpackagedirs . .

cd  /u01/app/oracle/distribs/oraclelinux/ol7_UEKR3

sudo createrepo --deltas --oldpackagedirs . .

Installing and configuring Apache 2.4

Oracle Linux 7 provides an Apache 2.4 HTTP server:
yum -y install httpd

httpd -v
Server version: Apache/2.4.6 (Red Hat)
Server built:   Jul 23 2014 04:34:53
The Directory configuration syntax has changed with Apache 2.4. To allow access to the repositories you've created, add the directive below to your Apache HTTP configuration file:
<Directory "/u01/app/oracle/distribs/oraclelinux">
    Options +Indexes +FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>
Create an Alias to that directory with the directive below:
Alias /yum/ /u01/app/oracle/distribs/oraclelinux/
You can then add access to port 80 in the firewall and start Apache:
firewall-cmd --permanent --add-service http
firewall-cmd --add-service http
success

firewall-cmd --list-all
public (default, active)
  interfaces: enp0s3 enp0s8
  sources: 
  services: dhcpv6-client http ssh
  ports: 
  masquerade: no
  forward-ports: 
  icmp-blocks: 
  rich rules: 

systemctl start httpd
systemctl enable httpd
systemctl status httpd
httpd.service - The Apache HTTP Server
   Loaded: loaded (/usr/lib/systemd/system/httpd.service; disabled)
   Active: active (running) since Mon 2014-07-28 23:00:10 CEST; 5s ago
 Main PID: 2238 (httpd)
   Status: "Processing requests..."
   CGroup: /system.slice/httpd.service
           ├─2238 /usr/sbin/httpd -DFOREGROUND
           ├─2239 /usr/sbin/httpd -DFOREGROUND
           ├─2240 /usr/sbin/httpd -DFOREGROUND
           ├─2241 /usr/sbin/httpd -DFOREGROUND
           ├─2242 /usr/sbin/httpd -DFOREGROUND
           └─2243 /usr/sbin/httpd -DFOREGROUND

Jul 28 23:00:10 blue.resetlogs.com systemd[1]: Started The Apache HTTP Server.
Hint: Some lines were ellipsized, use -l to show in full.
You should be able to access the repositories from your browser:

Configuring and using yum

For your servers to use deltas RPMs with yum, you must register your repositories:
cat /etc/yum.repos.d/blue.repo

[ol7_latest]
name=Oracle Linux $releasever Latest ($basearch)
baseurl=http://blue/yum/ol7_latest/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol7_UEKR3]
name=Latest UEK for Oracle Linux $releasever ($basearch)
baseurl=http://blue/yum/ol7_UEKR3
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
Install the deltarpms package so that yum can recreate RPMs from Delta RPMS:
yum install deltarpms
Delta RPMs are then managed automatically and transparently from the client as you can checkout for yourself in the command below:
yum -y update
ol7_UEKR3                    | 3.6 kB  00:00:00     
ol7_latest                   | 3.2 kB  00:00:00     
(1/2): ol7_UEKR3/primary_db  | 1.1 MB  00:00:00     
(2/2): ol7_latest/primary_db | 5.1 MB  00:00:01     
Resolving Dependencies
--> Running transaction check
---> Package kernel-uek.x86_64 0:3.8.13-35.3.2.el7uek will be installed
---> Package kernel-uek-firmware.noarch 0:3.8.13-35.3.2.el7uek will be installed
--> Finished Dependency Resolution

Dependencies Resolved
=======================================================================
 Package             Arch    Version              Repository     Size
=======================================================================
Installing:
 kernel-uek          x86_64  3.8.13-35.3.2.el7uek  ol7_UEKR3      32 M
 kernel-uek-firmware noarch  3.8.13-35.3.2.el7uek  ol7_UEKR3     1.8 M

Transaction Summary
=======================================================================
Install  2 Packages

Total download size: 34 M
Installed size: 116 M
Downloading packages:
ol7_UEKR3/prestodelta                                               |  830 B  
Delta RPMs reduced 34 M of updates to 7.9 M (76% saved)
kernel-uek-3.8.13-35.3.1.el7uek_3.8.13-35.3.2.el7uek.x86_64.drpm    | 7.6 MB
kernel-uek-firmware-3.8.13-35.3.1..._3.8.13-35.3.2....noarch.drpm   | 329 kB
Finishing delta rebuilds of 2 package(s) (34 M)
kernel-uek-3.8.13-35.3.2.el7uek.x86_64.rpm                          |  32 MB 
----------------------------------------------------------------------------
Total                                                      1.8 MB/s |  40 MB
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : kernel-uek-firmware-3.8.13-35.3.2.el7uek.noarch   1/2 
  Installing : kernel-uek-3.8.13-35.3.2.el7uek.x86_64            2/2 
  Verifying  : kernel-uek-firmware-3.8.13-35.3.2.el7uek.noarch   1/2 
  Verifying  : kernel-uek-3.8.13-35.3.2.el7uek.x86_64            2/2 

Installed:
  kernel-uek.x86_64 0:3.8.13-35.3.2.el7uek 
  kernel-uek-firmware.noarch 0:3.8.13-35.3.2.el7uek                    

Complete!
Obviously more CPU and I/Os should be consumed on your guests and hosts but it nicely speed up upgrades...
Bibliography:
For more information about Delta RPMs, visit Fedora Presto Page.

No comments:

Post a Comment