En/celerra data migration service
Welcome to Simone Giustetti's wiki pages.
Languages: English - Italiano
Celerra Data Migration Service
The CDMS service allows to migrate shared disks and file systems from heterogeneous sources to a Celerra Disk Array through the NFS or CIFS protocol. Using CDMS the data unavailability gap can be minimized while shares read or write access can be guaranteed throughout the whole migration process. Many file attributes: user permissions, last update date, last access date and such are preserved after the migration.
This paper contains a brief description of each step needed to migrate a NFS shared file system using the NFS protocol from a generic file server to a Celerra. As a case study, we'll pretend to move some shared disk space configured as follows:
- Name: app_doc
- Size: 200 Gb
- Operating System: Linux
- Exported as: nfs001.pro.nas:/SHARE/app_doc
- Exported to hosts db01acvs and db01bcvs through the pro.nas domain
- Mounted on mountpoint: /mnt/SHARE/app
It will be migrated to:
- Name: cvs_app_doc
- Size: 300 Gb
- Located on: the Celerra cl001nas second Data Mover
- Esported through interface: cl001nas.fe.pro.nas
- Exported to hosts db01acvs and db01bcvs
Migration Prerequisites
Prior to migrating the share, some checks need performing: mainly network infrastructure and data source checks.
Network Connectivity
One among the main check concerns should be network connectivity. Both the data source and the Celerra should communicate with each other and the network infrastructure should be able to stand the to be migrated data transfer load. The throughput check becomes fundamental in a scenario where you plan to use an all purpose network instead of a dedicate one. A choked network could result in a very long migration period and hosts located in the very same network could suffer from low throughput and consequently bad performances as well. It'll be wise to evaluate even alternative data move paths such as the standard back-up / restore sequence.
Another infrastructure check to perform regards firewall presence. It would be better not to have traffic filtering apparatuses between the migration involved servers. The moving data could generate a high throughput and crash firewalls with obvious repercussions to network security.
Once the low level checks are over, it's time to move to higher ones. The Celerra should be able to read the NFS shared data therefore the migration targeted Data Mover Fully Qualified Domain Name or the corresponding IP address is to be added to the share access permissions. Moreover data shall not be written and modified during the copy phase if not through the Celerra export; never from the source side. The source share is to be exported in read-only mode. The export configuration update can be postponed in order to reduce downtime for the shared file system, but the configuration update is bound to be resolved before the actual data copy process starts otherwise it'll be too late.
Were data modified from source side, the data move will reach completion anyway, but following congruency checks will fail and the file system will be unusable. The migration will result in a failure and you'll have to start over from scratch.
In our example we will assume that:
- Network connectivity is in place and network services / daemons are correctly configured and running.
- A DNS server is running and both forward and reverse look-up work flawlessly.
- The network is sized well enough to support the migration effort.
- Writing on the shared source disk will be permitted up until the last minute.
File System Sizing
The target file system should be sized large enough to contain the source one. If disk space were to run out during data transfer, the migration will fail. The Celerra Network Server family of products provides a Perl script: diskUsage.pl to calculate the exact file system size and avoid bad planning originated issues. Script diskUsage.pl is to be run on the same host where the source file system is located. For a detailed description of the script usage and its options, please refer to the official documentation.
In our example we will assume that the target file system size is more than enough to contain the source one.
Block Size Check
Another size related check you have better execute is the block size one. If not instructed otherwise, all new file systems on a Celerra use a default block size of 8 Kb. The source file system could possibly use a lesser block size. The difference could result in excessive fragmentation on the target file system and performance issues. To avoid any inconvenience you are suggested to force a block size equal to the source file system one.
Creating the Target File System
Checks successfully completed; it's now time to create a new file system that we will later export through the Celerra. In our example we will log in to the Control Station via a terminal window and use the Command Line Interface provided commands. For a brief description of the creation procedure please refer to a previous article of mine celerra share nfs .
Connect to the Control Station using user nasadmin and the proper password. To create the file system issue command:
nas_fs -name cvs_app_doc -type mgfs -create size=300G pool=clar_r5_performance -option nbpi=4096
Option -type mgfs impose a file system type meant for data migration. When data migration starts, the file system will index the source one contents and will then drive the data copy service in order to give priority to accessed files. Downtime will be minimal as a result.
Option -option nbpi=4096, impose a 4 Kb block size. We will now create a mount point and will mount there the file system:
server_mountpoint server_2 -c /cvs_app_doc server_mount server_2 -o rw cvs_app_doc /cvs_app_doc
Migration
Checks and pre-migration activities are over; we'll now copy data from the source to the target file system.
Early Phase
Service CDMS should be running in order for the copy to start. To ensure that the service is running issue command:
server_setup server_2 -Protocol cdms -option start
Check for network connectivity one last time:
server_ping server_2 nfs001.pro.nas
The target file system should be mounted on the Data Mover. To check mount status run command:
server_cdms server_2 -info cvs_app_doc
A positive feedback output should look like:
server_2 : cvs_app_doc:
All listed commands can be run by user nasadmin from a terminal window logged in to the Celerra Control Station.
Source Configuration Update
If the source share configuration were not updated yet, it's now time to proceed. Enable the read-only option for any client host having access rights to the shared file system in order to block write operations. The configuration change requires a service stop that will affect all clients using the shared resource therefore should be planned together with application support groups. For a Red Hat Linux file server, the update is limited to file /etc/exports. The share dedicated line should look somewhat like:
/SHARE/app_doc -rw db01acvs.pro.nas db01bcvs.pro.nas
Update it in:
/SHARE/app_doc -ro db01acvs.pro.nas db01bcvs.pro.nas cl001nas.be.pro.nas
Then restart the NFS server daemon to confirm the new settings:
service nfs restart
All commands are to be run either by user root or some other high privileged one.
The shared file system should be unmounted from client hosts db01acvs.pro.nas and db01bcvs.pro.nas prior to the nfs001.pro.nas service configuration update in order to avoid them any issue.
File System Association
Everything is now in place to link the source and target file system. Run command:
server_cdms server_2 -c cvs_app_doc -t nfsv3 -p /cvs_app_doc -s nfs001.pro.nas:/SHARE/app_doc
Checking the service state, the resulting output should look something like:
server_cdms server_2 -info cvs_app_doc server_2 : cvs_app_doc: path = /cvs_app_doc cid = 0 type = nfs source = nfs001.pro.nas:/SHARE/app_doc options =
More evidence about the link status can be found in the log file:
server_log server_2
Search the last few lines of the output for something like:
2011-07-05 17:58:04: ADMIN: 4: Command succeeded: connect fsid=02 type=nfsv3 path=/cvs_app_doc nfs=nfs001.pro.nas:/SHARE/app_doc proto=TCP
Data Filter
When performing a migration, not all data need moving. The CDMS service can filter source shared content and exclude part of it while copying data. The filter works reading directives from an exclusion file which contains a listing of all the files and directories to ignore. Every line in the list is an item to be ignored. The first character from each line is used to determine whether a single file or a whole directory is the filter target. A file formatting example follows:
f /<path>/<file> d /<path>/<directory>
The exclusion file must be located in the migration target Data Mover directory structure, in a directory the CDMS service can read. In our example the log directory is cdms_log_2. Any text editor can be used to write and update the file; a text editor such as vi:
vi /nas/quota/slot_2/cdms_log_2/exclude_cvs_20110706.txt
We are going to exclude a directory containing some old database dumps. The exclusion list follows:
d /cvs_app_doc/cvs_app_doc/backup
Where "backup" is the directory containing old useless data.
You need high privileges to create the exclusion file; user nasadmin ones are not enough. Issue the su - command and enter the correct password to became root and gain administrator privileges or simply open a second connection to the Control Station using the root user account.
The directory path origins from the root of the Celerra created file system. The file system name is repeated twice because the CDMS service creates an empty directory named after the file system, inside the same file system, where to copy data. To check for the exclusion path correctness use the specialCmd utility provided by the Celerra Network Server. To perform the check in our "real life" example run command:
/nas/tools/cdms/specialCmd getOnline server_2 /cvs_app_doc/cvs_app_doc/backup
The command produces no output and its outcome shall be obtained by an analysis of the log more recent lines:
server_log server_2
Excluding Snapshots
Sometimes you'll need to migrate data located on storage using snapshots to perform back-ups. Snapshots have to be excluded from the migration otherwise you'll risk to saturate the target file system and cause the entire process to fail. Let's assume to migrate a file system between two Celerras as an example. The source is configured to daily execute a checkpoint (that's a snapshot in Celerra terminology) with a weekly retention. The total amount of data read will almost add up to eight times the real file system size: the existing data summed to the seven daily copies of the previous week checkpoints. The workaround consists of adding the following filter to the esclusion file:
d /<file system>/<file system>/.ckpt
NetApp storage use snapshots too. To block the snapshots add the following line to the exclude file:
d /<file system>/<file system>/.snapshot
Migration Start
To start the data copy procedure issue command:
server_cdms server_2 -start cvs_app_doc -p /cvs_app_doc -l /cdms_log_2 -e /cdms_log_2/exclude_cvs_20110706.txt
To monitor the migration status run command:
server_cdms server_2 -info cvs_app_doc
server_2 :
cvs_app_doc:
path = /cvs_app_doc
cid = 0
type = nfs
source = nfs001.pro.nas:/SHARE/app_doc
options =
threads:
path =
state = ON_GOING
log = /cdms_log_2
An ON_GOING status means the data copy was correctly started and is in progress.
To obtain a rough esteem of data transfer speed and migration duration, check for the disk space utilization of the target file system with:
while true; do server_df server_2 /cvs_app_doc;sleep 60;done
That will output disk space utilization once every minute. Use keyboard combination CTRL+c to kill the process.
New File System Export
Once the data copy procedure is running, the Celerra file system can be exported to the client hosts and services needing the shared resource can be started again. Issue the following command sequence to export the file system:
server_export server_2 -Protocol nfs -option rw=db01acvs.pro.nas:db01bcvs.pro.nas /cvs_app_doc/cvs_app_doc server_export server_2 -Protocol nfs -option root=db01acvs.pro.nas:db01bcvs.pro.nas /cvs_app_doc/cvs_app_doc server_export server_2 -Protocol nfs -option access=db01acvs.pro.nas:db01bcvs.pro.nas /cvs_app_doc/cvs_app_doc
To check the export status from the Control Station run:
server_export server_2 | grep cvs_app_doc
To perform the check from clients instead use:
showmount -e cl001nas.fe.pro.nas | grep cvs_app_doc
To mount the file system on clients db01acvs.pro.nas and db01bcvs.pro.nas connect to both hosts using the root account and run command:
mount -t nfs -o rw,bg,hard,intr,nfsvers=3,rsize=8192,wsize=8192,proto=tcp cl001nas.fe.pro.nas:/cvs_app_doc/cvs_app_doc /mnt/SHARE/app
The mount operation can be automated at boot time by updating the /etc/fstab configuration file. Modify line
nfs001.pro.nas:/SHARE/app_doc /mnt/SHARE/app nfs rw,bg,hard,intr,nfsvers=3,rsize=8192,wsize=8192,proto=tcp 0 0
into
cl001nas.fe.pro.nas:/cvs_app_doc/cvs_app_doc /mnt/SHARE/app nfs rw,bg,hard,intr,nfsvers=3,rsize=8192,wsize=8192,proto=tcp 0 0
The service downtime period stretches from the instant the source share configuration is updated to read-only mode to the subsequent mount of the new share on the client hosts.
Data Copy End
Command
server_cdms server_2 -info cvs_app_doc
provides a feedback about the file and directory copy status. When the status changes to SUCCEED the data copy phase is successfully over and it's time to move on to post migration configuration.
Post Migration Configuration
Once the data copy is over, file system type is to be changed from mgfs to standard Celerra format. Connect to the Celerra Control Station using the nasadmin user account and issue command:
server_cdms server_2 -C cvs_app_doc server_2 : done
The conversion does not interfere with file system exports and can be performed without any drawback to users or processes connected to the shared resource.
As a last step we will ensure data security by updating the back-up configuration or by scheduling a checkpoint policy for the new file system. The following command schedules a daily checkpoint at 02.30 a.m. with a one week retention.
nas_ckpt_schedule -create cvs_app_doc_daily -filesystem cvs_app_doc -recurrence daily -every 1 -runtimes 02:30 -keep 7
Conclusion
This paper listed and described some of the many steps needed to perform a NFS shared file system migration. The shared disk was moved from a generic file server to an Emc² Celerra family Network Server. The CDMS service was used to perform data migration.
Please refer to the official Emc² documentation and whitepapers to further delve into the subject.
For any feedback, questions, errors and such, please e-mail me at studiosg [at] giustetti [dot] net
External links
NFS protocol (FreeBsd Handbook)
Languages: English - Italiano