Wednesday, December 4, 2013

Querying an Oracle VM Server Xenstore from a Linux Guest

Accessing the Oracle VM Xenstore from a Linux Guest is the most efficient and sometimes most reliable way to collect data about your Oracle virtualized environment. Among the various things you can get from Xen internal database are (1) the DomU name, (2) the Dom0 VNC access point or (3) the mapping between the VM disk devices and the underlying storage.

For instance, if you want to trigger a storage snapshot from your guest, you can directly and automatically discover the LUN serial number with the xenstore-ls command and request a snapshot without connecting to the Oracle VM Manager.

This article explains how to install and use the various xenstore tools/utilities to get those values.

Building Xenstore Tools

Oracle doesn’t provide the xenstore tools for your guest as part of its RPM repository. Building your own is not a big deal. Download the latest Oracle VM Server Source ISO from My Oracle Support; Oracle keeps it in the 16410428 placeholder. The file you are looking for is named xen-4.x.x-yy.el5.zz.src.rpm and is available from the src.iso file.

Then install xen source file on one of your Oracle Linux guest. Below is an example of how to install the source code from Oracle VM 3.2.6 :
rpm -ivh xen-4.1.3-25.el5.53.src.rpm
cd rpmbuild/SOURCES
tar -zxvf xen-4.1.3-ovs.tar.gz
cd xen-4.1.3-ovs

Once you’ve installed the source, make sure you have all the necessary RPMs to build it. below is an example that works with Oracle Linux 6:
yum install oracle-rdbms-server-11gR2-preinstall \
     libuuid-devel openssl-devel ncurses-devel   \
     dev86 iasl python-devel SDL-devel

And build the tools with a simple make command:
cd ~/rpmbuild/SOURCES/xen-4.1.3-ovs/tools
make

Installing xen-detect and the Xenstore Tools

Building a RPM to install the tools you need for your guest is straightforward. All you need are the xen-detect, libxenstore.so.3.0, xenstore and xenstore-control files. Assuming you’ve installed and built them from the source on your guest, the script below performs a typical installation:
cd ~/rpmbuild/SOURCES/xen-4.1.3-ovs/tools/misc
install xen-detect /usr/local/bin
cd ~/rpmbuild/SOURCES/xen-4.1.3-ovs/tools/xenstore
install libxenstore.so.3.0 /usr/local/lib
install xenstore xenstore-control /usr/local/bin
cd /usr/local/bin
ln -f xenstore xenstore-chmod
ln -f xenstore xenstore-exists
ln -f xenstore xenstore-list
ln -f xenstore xenstore-ls
ln -f xenstore xenstore-read
ln -f xenstore xenstore-rm
ln -f xenstore xenstore-write

Using xen-detect and Xenstore Tools

xen-detect allows to quickly figure out, from your guest, the type of virtualization that is currently in use. This is a sample output taken from a paravirtualized DomU:
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
xen-detect
Running in PV context on Xen v4.1.

To access the server xenstore from the guest, you must first allow access to the xenbus to the xenstore tools running in userspace. Mount the xenfs systems to /proc/xen to allow the access:
mount -t xenfs none /proc/xen

You can then use the various xenstore commands to get the DomU Name or Id:
xenstore-read name
0004fb000006000042165a8746757cac

xenstore-read domid
3

You can list backend devices from the store:
xenstore-ls /local/domain/`xenstore-read domid` |grep "backend ="

backend = "/local/domain/0/backend/vkbd/3/0"
backend = "/local/domain/0/backend/vfb/3/0"
backend = "/local/domain/0/backend/vbd/3/51712"
backend = "/local/domain/0/backend/vif/3/0"
backend = "/local/domain/0/backend/console/3/0"

Check out the storage details; below is a command that allows to match the storage device to its virtualized storage source:
xenstore-ls -f /local/domain/0/backend/vbd/3/51712

/local/domain/0/backend/vbd/3/51712/domain = "0004fb000006000042165a8746757cac"
/local/domain/0/backend/vbd/3/51712/frontend = "/local/domain/3/device/vbd/51712"
/local/domain/0/backend/vbd/3/51712/uuid = "383157e9-4204-3c43-325f-df6d38f2f1ff"
/local/domain/0/backend/vbd/3/51712/bootable = "1"
/local/domain/0/backend/vbd/3/51712/dev = "xvda"
/local/domain/0/backend/vbd/3/51712/state = "4"
/local/domain/0/backend/vbd/3/51712/params = "/OVS/Repositories/0004fb0000030000b85220e0f581c9d4/VirtualDisks/0004fb0000120000237098d4453a0ae7.img"
/local/domain/0/backend/vbd/3/51712/mode = "w"
/local/domain/0/backend/vbd/3/51712/online = "1"
/local/domain/0/backend/vbd/3/51712/frontend-id = "3"
/local/domain/0/backend/vbd/3/51712/type = "file"
/local/domain/0/backend/vbd/3/51712/node = "/dev/loop0"
/local/domain/0/backend/vbd/3/51712/physical-device = "7:0"
/local/domain/0/backend/vbd/3/51712/hotplug-status = "connected"
/local/domain/0/backend/vbd/3/51712/feature-flush-cache = "1"
/local/domain/0/backend/vbd/3/51712/feature-discard = "0"
/local/domain/0/backend/vbd/3/51712/feature-barrier = "1"
/local/domain/0/backend/vbd/3/51712/sectors = "25165824"
/local/domain/0/backend/vbd/3/51712/info = "0"
/local/domain/0/backend/vbd/3/51712/sector-size = "512"

You can discover how a network device relates to a Dom0 network bridge:
xenstore-ls -f /local/domain/0/backend/vif/3/0

/local/domain/0/backend/vif/3/0/bridge = "c0a83900"
/local/domain/0/backend/vif/3/0/domain = "0004fb000006000042165a8746757cac"
/local/domain/0/backend/vif/3/0/handle = "0"
/local/domain/0/backend/vif/3/0/uuid = "874c2912-38a6-a27c-3728-7858ca82f0eb"
/local/domain/0/backend/vif/3/0/script = "/etc/xen/scripts/vif-bridge"
/local/domain/0/backend/vif/3/0/state = "4"
/local/domain/0/backend/vif/3/0/frontend = "/local/domain/3/device/vif/0"
/local/domain/0/backend/vif/3/0/mac = "00:21:f6:00:00:00"
/local/domain/0/backend/vif/3/0/online = "1"
/local/domain/0/backend/vif/3/0/frontend-id = "3"
/local/domain/0/backend/vif/3/0/feature-sg = "1"
/local/domain/0/backend/vif/3/0/feature-gso-tcpv4 = "1"
/local/domain/0/backend/vif/3/0/feature-rx-copy = "1"
/local/domain/0/backend/vif/3/0/feature-rx-flip = "0"
/local/domain/0/backend/vif/3/0/hotplug-status = "connected"

You can also find the VNC address and port associated with your guest:
xenstore-ls -f /local/domain/0/backend/vfb/3/0

/local/domain/0/backend/vfb/3/0/vncunused = "1"
/local/domain/0/backend/vfb/3/0/domain = "0004fb000006000042165a8746757cac"
/local/domain/0/backend/vfb/3/0/vnc = "1"
/local/domain/0/backend/vfb/3/0/uuid = "2d259c00-dcf8-bfec-0290-0a056e9a3769"
/local/domain/0/backend/vfb/3/0/vnclisten = "127.0.0.1"
/local/domain/0/backend/vfb/3/0/frontend = "/local/domain/3/device/vfb/0"
/local/domain/0/backend/vfb/3/0/state = "4"
/local/domain/0/backend/vfb/3/0/keymap = "fr"
/local/domain/0/backend/vfb/3/0/online = "1"
/local/domain/0/backend/vfb/3/0/frontend-id = "3"
/local/domain/0/backend/vfb/3/0/xauthority = "/root/.Xauthority"
/local/domain/0/backend/vfb/3/0/feature-resize = "1"
/local/domain/0/backend/vfb/3/0/hotplug-status = "connected"
/local/domain/0/backend/vfb/3/0/location = "127.0.0.1:5900"
/local/domain/0/backend/vfb/3/0/request-update = "1"

What’s next...

Using Xenstore tools from guests is very convenient if you use Oracle VM with databases on a SAN and want to perform snapshots as sources for backups. This is because Oracle VM dom0 name their devices with Linux device-mapper and use the “scsi_id -g -u -s /block/” command behind the scene. Those same serial numbers can also be discovered in storage arrays to control you are not missing any LUN for your snapshot. Obviously you could also query Oracle VM manager but that’s not always as obvious since you cannot even be sure from your DomU name. 

One more word for those who are using Netapp Data Ontap... Once you have the scsi serial number, you can check for the LUN with the “lun serial -x /vol/name/lun” command. It allows to perfectly match LUN and filesystems or ASM diskgroups, the same way NetApp SnapDrive would do on a Linux physical host! Hopefully, Netapp will code this integration for us soon in their tool to ease our workload.

No comments:

Post a Comment