Written by KH Wissem Introduction Typical IT projects has to go through a long process of definition and acquisition of the infrastructure (hardware software and licenses). Cloud Computing comes to address the difficulties in Infrastructure acquisition and definition. Cloud Computing make easier, fast and at lower cost the infrastructure acquisition with a great ability to scale down, scale up, and scale out in no time. In this article, we will talk about the leader Cloud provider; Amazon Web Services (AWS); we will explain a bit of the components of the AWS, the AWS client installation and commands. Later, for the Oracle database administrators, I will explain how to backup the Oracle databases using volume snapshots. Amazon Web Services has an extremely easy to use web console called the AWS Management Console ; this is a commonly used method to manage access and work with the AWS services. But as a Database administrator and Sysadmin, we need a more flexible way to manage the AWS services (the instances, the volumes, the snapshots etc ...). Here comes the AWS Cli, to help us scripting and automate the management tasks. In this document, I’ll provide some working examples of how to use AWS Cli to provision AWS services, especially create snapshots on Oracle database volumes. AWS components It is worth to explain some of the AWS components before jump into the command lines and scripting. We will start explaining what is an EC2 service? , what is an EC2 instance and EBS volumes? Later we will explain Instance and Volume tagging. EC2 Introduction Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides a scalable compute capacity on demand and pay per use in the cloud. It is designed to make web-scale cloud computing easier for developers. A high scalable means you can scale out from a few of virtual servers to thousands in no time. Amazon offers to the Oracle customers the ability to license Database 12c, Oracle Fusion Middleware, and Oracle Enterprise Manager to run in the AWS. However, Oracle customers can also use their software licenses on Amazon EC2 with no additional license fees. As we are going to manage our Oracle database environments by ourselves, we run the production oracle databases on EC2 instances. So what is an EC2 instance? EC2 Instance EC2 instances are nothing but virtual servers. Each virtual server comes with its own server resources such as CPU, Memory, Storage and network. Amazon EC2 provides many instance types to address different use cases and application workloads. The instances types include combinations of CPU, memory, storage and networking capacity to give the users the flexibility to choose the appropriate resources of your applications. For a complete list of instance type, please visit the following link: https://aws.amazon.com/ec2/instance-types/ Each EC2 instance has a set of volumes attached to it. So what is an EBS volume? EBS Volumes EBS Volumes are nothing but block level storage devices attached to EC2 instances. Oracle database administrators have to create the EBS volumes prior to attach them to an EC2 instance. Those volumes will be presented to the OS typically a Linux system, then partitioned and later used by the Oracle instance as an ASM disk or typical File system. Now that we had a brief introduction of what are some of the AWS components we are going to use in this document. It is time to show some practical examples and start with the AWS client utility. The AWS Client utility As we mentioned before in this document, the AWS Cli make simpler the scripting and the management tasks of the AWS services. Let s get started with the AWS Cli installation and configuration. AWS Cli Installation In this section we will go through the installation of the AWS Cli in a Redhat Linux system. Later in this article all the scripts have been tested in Redhat Linux system. Python versions supported are 2.6.5 and above: python --version If you don't have Pip installed then download the installation script from pypa.io: curl -O https://bootstrap.pypa.io/get-pip.py Run the script with Python: $ sudo python get-pip.py If you get the following error: /tmp/tmpYOIx07/pip.zip/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning . Then upgrade python as root: Download Python 3.3.5 using the wget command and run the following commands in order: $wget http://python.org/ftp/python/3.3.5/Python-3.3.5.tar.xz $yum install install xz $tar xf Python-3.3.5.tar.xz $cd Python-3.3.5 $. /configure --prefix=/usr/local --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib" $make && make altinstall Install again the Pip: $sudo /root/Python-3.3.5/python get-pip.py Install the CLI Install awscli using pip. If you’d like to have awscli installed in an isolated Python environment, sudo pip install awscli Configure awscli to create the required ~/.aws/config file. aws configure –profile PVolSnapshot Once executed, you will be prompted to enter the user’s ACCESS Key ID and the AWS Secret Access Key (supplied by your AWS administrator) along with the region name and the default output format. AWS Access Key ID [None]: xxxxxx AWS Secret Access Key [None]: xxxx Default region name [None]: eu-west-1 Default output format [None]: json You can choose table, text of JSON as the Default output format. If you have multiple users and each user needs to access to the Linux Box, then you can use profile. You can configure the AWS Command Line Interface to use a role by creating a profile for the role in the~/.aws/config file. In our environment, we use the profile name “PVolSnapshot” for AWS volume snapshots. AWS Cli commands Let’s perform our first command on AWS. The following JSON shows that AWS currently has 11 regions. Every region runs completely isolated from the others. Network latency, connectivity, support and so on are the keys to choose your region prior building the servers and demand the services. Each region is split up to availability zones (AZ) which are engineered to be isolated from failures in other Availability Zones, and to provide inexpensive, low-latency network connectivity to other zones in the same region. For more details about regions and AZ please read the following page: https://aws.amazon.com/about-aws/global-infrastructure/ aws ec2 describe-regions --region eu-west-1 --profile PVolSnapshot { "Regions": [ { "Endpoint": "ec2.ap-south-1.amazonaws.com", "RegionName": "ap-south-1" }, { "Endpoint": "ec2.eu-west-1.amazonaws.com", "RegionName": "eu-west-1" }, { "Endpoint": "ec2.ap-southeast-1.amazonaws.com", "RegionName": "ap-southeast-1" }, { "Endpoint": "ec2.ap-southeast-2.amazonaws.com", "RegionName": "ap-southeast-2" }, { "Endpoint": "ec2.eu-central-1.amazonaws.com", "RegionName": "eu-central-1" }, { "Endpoint": "ec2.ap-northeast-2.amazonaws.com", "RegionName": "ap-northeast-2" }, { "Endpoint": "ec2.ap-northeast-1.amazonaws.com", "RegionName": "ap-northeast-1" }, { "Endpoint": "ec2.us-east-1.amazonaws.com", "RegionName": "us-east-1" }, { "Endpoint": "ec2.sa-east-1.amazonaws.com", "RegionName": "sa-east-1" }, { "Endpoint": "ec2.us-west-1.amazonaws.com", "RegionName": "us-west-1" }, { "Endpoint": "ec2.us-west-2.amazonaws.com", "RegionName": "us-west-2" } ] } The output from describe-availability-zones should be that of the AWS Availability Zones for our configured region. aws ec2 describe-availability-zones --region eu-west-1 --profile PVolSnapshot { "AvailabilityZones": [ { "State": "available", "RegionName": "eu-west-1", "Messages": [], "ZoneName": "eu-west-1a" }, { "State": "available", "RegionName": "eu-west-1", "Messages": [], "ZoneName": "eu-west-1b" }, { "State": "available", "RegionName": "eu-west-1", "Messages": [], "ZoneName": "eu-west-1c" } ] } You want to get the instance ID from the Linux box, by running the following command. This command gets the instance metadata from the special IP 169.254.169.254. This is an IP available in AWS to get instance metadata like instance ID, IP address and so on. $ curl -s http://169.254.169.254/latest/meta-data/instance-id Now that we have run some of AWS Cli commands. Let’s create an EBS volume, attach it to the EC2 instance and later create our Oracle database. At the end we will perform a volume snapshot to backup our Oracle database. Oracle database 12c volume Snapshot backup Create an EBS Volume Let’s first create and then attach a 20 GB EBS volume to our Ec2 Instance. We want to create the volume in eu-west-1 region (Ireland) and more specifically in the AZ eu-west-1b. aws ec2 create-volume \ --size 20 \ --availability-zone eu-west-1b \ --profile PVolSnapshot A JSON returned looks like below: Make note of the Volume ID created: vol-9fgdaf34 { "AvailabilityZone": "eu-west-1b", "Encrypted": false, "VolumeType": "standard", "VolumeId": "vol-9fgdaf34", "State": "creating", "SnapshotId": "", "CreateTime": "2016-05-25T08:50:48.671Z", "Size": 20 } Attach the Volume Now it is time to attach the volume to the instance: We must supply the command with the volume ID, the instance ID (we got previously from AWS metadata IP) and the device name to be attached. aws ec2 attach-volume \ --volume-id vol-9fgdaf34\ --instance-id i-d45f976e \ --device /dev/xvdj --profile PVolSnapshot Run the lsblk command to list all devices attached to the host: We confirm the /dev/xvdj device got attached. From within our Ec2 instance, we should now see the new EBS volume as /dev/xvdj. [root@xxxx ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 15G 0 disk └─xvda1 202:1 0 15G 0 part / xvdf 202:80 0 200G 0 disk ├─vg01-jv01 (dm-2) 252:2 0 30G 0 lvm /jv01 xvdh 202:112 0 100G 0 disk xvdi 202:128 0 20G 0 disk └─vg_swap-swap (dm-0) 252:0 0 20G 0 lvm [SWAP] xvdj 202:144 0 20G 0 disk [root@xxx ~]# Partition the disk We can now use this block device as a normal disk within Linux. Let’s go ahead and partition the disk, apply a file system and then mount it on /datatest. Run again the lsblk command to list all devices attached to the host: Run the following command providing the inputs when prompt: fdisk /dev/xvdj n p 1 enter enter enter p w Later run the following commands and Add the entry to the fstab file. parted -s -a optimal /dev/xvdj mklabel msdos parted -s -a optimal /dev/xvdj mkpart primary ext2 0% 100% mkfs.ext4 /dev/xvdj1 mkdir /datatest mount /dev/xvdj1 /datatest Create a Tag for the Volume The AWS Cli and management console allow you add tags to your AWS instances and volumes. Tags are a key value pairs of text that allow you to add description, label to easy identify your AWS resources. As an example you tag your instances whatever it is a Linux or a Windows running. Remember this is a volume belonging to the database server and files. We need to tag this volume to ensure including it in the snapshots jobs. We use the Tag key: tag4snap- and a value database name concatenated to the mount name. In this example, test is the name of our database and oradata is the mount name. Use the following AWS Cli command to tag the volume: aws ec2 create-tags --resources vol-9fgdaf34 --profile PVolSnapshot --tags Key=tag4snap-test, Value='"{\"test_oradata\"}"' Create the test database instance We create an 11g database instance called test using the silent option of the DBCA. All data files, archive logs go to /datatest the volume we created previously. chown -R oracle:dba /datatest chmod -R 775 /datatest /jv01/app/oracle/product/12.1.0/dbhome_1_12R1/bin/dbca -silent \ -createDatabase \ -templateName General_Purpose.dbc \ -gdbName test \ -sid test \ -SysPassword h3ll0 \ -SystemPassword h3ll0 \ -emConfiguration NONE \ -datafileDestination /datatest \ -storageType FS \ -characterSet AL32UTF8 \ -memoryPercentage 40 alter system set log_archive_dest_1='location=/datatest'; shutdown immediate; startup mount; alter database archivelog; alter database open; Backup the database using EBS volume snapshot Now that we have a test instance running in archive log mode we either perform a clean shutdown of the database during snapshot creation or issue the command ALTER DATABASE BEGIN BACKUP; This command brings the online database in backup mode. Once the database is in online backup mode, we run the following AWS Cli command to create a volume snapshot: aws ec2 create-snapshot --region eu-west-1 --volume-id vol-9fgdaf34 --description "This is my first volume snapshot." Now it is time to either start up the database or ends the database backup mode: ALTER DATABASE END BACKUP; Conclusion In this article we have introduced the AWS cloud world. We have seen some of the AWS components like the EC2 instance and EBS volumes, the AWS Cli. We have installed and configured the AWS utility. Later, we have created an EBS volume and attached it to an EC2 instance. We have seen the steps to create an Oracle 12c instance using the already created EBS volume. At the end we have backed up the Oracle 12 instance using the EBS volume snapshots. In the next article, we will see all the steps to restore an Oracle 12c instance using the EBS volume snapshots.
↧