Introduction While working on Exadata Database Machines (X2 and above), I started seeing that root file system is filling fast. By default root file system on Exadata Computed nodes X2 and above are configured with 30GB space by Oracle ACS. This space may not be sufficient to store operating system, Exadata software, log file and diagnostic files. Also 30GB space may not be sufficient if you want to store patches during platinum patching. Exadata Manuals provides a procedure that you can use on Exadata X2 and above to extend the logical volume space on which the root file system is mounted. On Exadata V2, by default root files system is configured as 60GB non-lvm file system. Exadata X4: [root@dbm01db01 ~]# df -h / Filesystem Size Used Avail Use% Mounted on /dev/mapper/VGExaDb-LVDbSys1 30G 24G 4.8G 84% / On Exadata X2 and above, by default root file system is configured as 30GB and it uses logical volume. Exadata V2: [root@oraclouddbadm01 ~]# df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda1 60G 27G 30G 47% / On Exadata V2, by default root file system is configured as 60GB non-lvm file system In this article, I will demonstrate how you can extend the size of root file system on Exadata Compute nodes online (without downtime). Why Extend root system partition size? The default size of 30GB may not be sufficient and we may run out of space quickly. Root partition is used for: Operating system, trace files, log files, diagnostic files and other file Exawatcher logs Onecommand files Store Exadata patches Support Exadata software upgrades on Compute nodes. About root System Partitions root file system is installed on two system partitions LVDbSys1 and LVDbSys2. At a give time one partition is active and mounted (or inactive). The inactive partition is used as a backup location during upgrade. Bothe system partitions must be sized equally. Display System Partitions Use the following command to display root System Patitions. [root@dbm01db01 ~]# lvdisplay | grep LVDbSys LV Path /dev/VGExaDb/LVDbSys1 LV Name LVDbSys1 LV Path /dev/VGExaDb/LVDbSys2 LV Name LVDbSys2 How to check which System Partition is Active? As root user connect to one of compute nodes and execute the “imageinfo” command as follows: [root@dbm01db01 ~]# imageinfo Kernel version: 2.6.39-400.276.1.el6uek.x86_64 #1 SMP Wed Jan 27 15:36:25 PST 2016 x86_64 Image kernel version: 2.6.39-400.276.1.el6uek Image version: 12.1.2.3.0.160207.3 Image activated: 2016-02-23 12:27:11 -0600 Image status: success System partition on device: /dev/mapper/VGExaDb-LVDbSys1 From the above output the active system partition is installed on /dev/mapper/VGExaDb-LVDbSys1. Environment Details Exadata Model X5-2 Quarter Rack HC 8TB Exadata Components Storage Cell (14), Compute node (8) & Infiniband Switch (1) Exadata Storage cells DBM01CEL01 – DBM01CEL14 Exadata Compute nodes DBM01DB01 – DBM01DB08 Exadata Software Version 12.1.2.3.0 Exadata DB Version 11.2.0.4 BP20 Prerequisites root user access Exadata Storage Software Version: 11.2.3.2.1 and above This procedure does not require an outage on the server. Procedure to Extend root File System on Exadata Compute Nodes File system root is built on volume group, so it makes it easy to extend the size of logical volume (LVDbSys1 and LVDbSys2) on which root file system is mounted. Here we are going to increase the size of root file system one compute node at a time (rolling). root file can be resized online without downtime. My recommendation is to run with the commands first on any one node to make sure there are no issues before doing it in parallel. Display the current allocated, free and used space in root partition [root@dbm01db01 ~]# df -h / Filesystem Size Used Avail Use% Mounted on /dev/mapper/VGExaDb-LVDbSys1 30G 24G 4.8G 84% / Here the root file system is allocated 30GB (default), used space is 24GB and Available is 4.8GB. The current available size is clearly very small. Display the current volume configuration [root@dbm01db01 ~]# lvs -o lv_name,lv_path,vg_name,lv_size LV Path VG LSize LVDbOra1 /dev/VGExaDb/LVDbOra1 VGExaDb 200.00g LVDbSwap1 /dev/VGExaDb/LVDbSwap1 VGExaDb 24.00g LVDbSys1 /dev/VGExaDb/LVDbSys1 VGExaDb 30.00g LVDbSys2 /dev/VGExaDb/LVDbSys2 VGExaDb 30.00g Check that root partition can be resized online. [root@dbm01db01 ~]# tune2fs -l /dev/mapper/VGExaDb-LVDbSys1 | grep resize_inode Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file The tune2fs command should display “resize_inode” option in the output. If this option is not displayed in the output, you must unmount the file system before resizing the partition. Ensure that free space is available in the volume group “VGExaDb” [root@dbm01db01 ~]# vgdisplay -s "VGExaDb" 1.63 TiB [284.00 GiB used / 1.36 TiB free] From the above output we can see that there is 1.36TB free space available in the Volume Group. Note: If there is no sufficient space in the Volume group, run the reclaimdisks.sh utility to reclaim disk space. # /opt/oracle.SupportTools/reclaimdisks.sh -free –reclaim If the utility has been run and there is not enough free space, then the LVM cannot be resized. Using the lvextend command, resize both LVDbSys1 and LVDbSys2 logical volumes. In this scenario we are extending the Logical volume by 30GB. This will make the root partition 60GB. [root@dbm01db01 ~]# lvextend -L +30G /dev/VGExaDb/LVDbSys1 Size of logical volume VGExaDb/LVDbSys1 changed from 30.00 GiB (7680 extents) to 60.00 GiB (15360 extents). Logical volume LVDbSys1 successfully resized [root@dbm01db01 ~]# lvextend -L +30G /dev/VGExaDb/LVDbSys2 Size of logical volume VGExaDb/LVDbSys2 changed from 30.00 GiB (7680 extents) to 60.00 GiB (15360 extents). Logical volume LVDbSys2 successfully resized Using the resize2fs command, resize the file system [root@dbm01db01 ~]# resize2fs /dev/VGExaDb/LVDbSys1 resize2fs 1.43-WIP (20-Jun-2013) Filesystem at /dev/VGExaDb/LVDbSys1 is mounted on /; on-line resizing required old_desc_blocks = 2, new_desc_blocks = 4 Performing an on-line resize of /dev/VGExaDb/LVDbSys1 to 15728640 (4k) blocks. The filesystem on /dev/VGExaDb/LVDbSys1 is now 15728640 blocks long. Since LVDbSys2 is inactive you must execute e2fsck before resizing the file system [root@dbm01db01 ~]# e2fsck -f /dev/VGExaDb/LVDbSys2 e2fsck 1.43-WIP (20-Jun-2013) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/VGExaDb/LVDbSys2: 122199/3932160 files (0.3% non-contiguous), 5496667/7864320 blocks [root@dbm01db01 ~]# resize2fs /dev/VGExaDb/LVDbSys2 resize2fs 1.43-WIP (20-Jun-2013) Resizing the filesystem on /dev/VGExaDb/LVDbSys2 to 15728640 (4k) blocks. The filesystem on /dev/VGExaDb/LVDbSys2 is now 15728640 blocks long. Verify that the space is increased for the active system partition [root@dbm01db01 ~]# df -h / Filesystem Size Used Avail Use% Mounted on /dev/mapper/VGExaDb-LVDbSys1 60G 24G 33G 42% / root partition has been extended to 60GB. Conclusion In this article we have learned how to extend root file system on Exadata Compute node without an outage. Also we have learned that root file system is installed on two system partitions for redundancy. When we resize the root file system we must ensure that both the system partitions are resized to the same value to avoid issues at a later time.
↧