Migrate File Server

Hey folks,

This month [July], I had two major projects to lead and complete, to perform migration of old file storage to the new one.

The source of one of storage were [1] NetApp and [2] EMC which is pretty common in our IT World,the purpose is to migrate them to Reduxio Storage [It really doesn’t matter what is the destination or what is the  type of storage though, could be Windows FS, QNAPM EMC, NETAPP or whatever…)

One of the customers has more than 15 TB and the second one had quite small storage which is around 700 GB, I tried to think about all relevant concerns I might experience, how can I do it without interrupting, without losing data, and without losing NTFS, ACL permissions.

EMC has the domestic tool which calls “EMCOPY.EXE” that we could use, according to some reviews it know to do the work, regarding NetApp, perhaps, there is some domestic tool, not something that I know. One of my opinions were to purchase a tools, which knows to copy in parallel, copying NTFS, ACL and so on, there were some concerns about share and quotes settings that were configured on source side, really didn’t know how could I migrate them as well,unfortunately didn’t find a tool which meets my personal requirements. Therefore I decided that make use in RoboCopy which known as a traditional tool, RoboCopy doesn’t know copying the share and quotes settings as well but have other amazing capabilities that caused me to give up about purchasing a third party tool.

By the way, in the beginning, I wanted to use in DFS, Microsoft feature which is a good option, it has replication machinist that could provide me the same result, despite that, after a deep investigation over the internet decided to give respect to RobCopy for making this project.

If you have Veeam, and the volume would be assigned as LUN your life could be easier, but in my case, both are volumes which mean that Veeam couldn’t see them and not backup [no agent].

Please let me share with you some important point I wrote down before migration process has initiated, just to realize the current status and futures steps:

  • Check the Full Name length of the current folders, Windows has limitation up to 260 characters, you can use this script in order to get the current length:
  • Get-ChildItem -Recurse| Where-Object {$_.FullName.Length -gt 256} | select-object FullName, @{n=”FullNameLength”;e={ $_.fullname.length}} |fl | Out-File C:\Characters\SHARE1.txt | Format-List
  • The current volume is based NETAPP which contains roughly 700 GB.
  • NETAPP connected to AD via LDAP and groups and users are assigned to folders
  • There is Folder Redirection GPO of RDSH that 100 users keep their files on NETAPP Volume.
  • Post copying files we have to reconfigure the Quotes and Shares settings in the new server, probably manually.
  • The copy process will be done by “RoboCopy.exe” tool.
    -Re-Check Permission changes during file transfer.
    – Re-Check File changes during transferring.
  • Make sure there are no opened files. Therefore, I would suggest starting this project on Thursday night.
  • Please disconnect all opened sessions on Terminal Service during the weekend, we would like to prevent users from opening files.
  • We are going to use in “B” switch which gives us the “Backup mode”, it knows to override files and folder permissions settings.

Tasks:

  • Export Share and quotes settings from an old storage.
  • Creating a new Windows Server 2012R2 and installing File Server.
  • Creating a new volume on Reduxio.
  • Initiate copy process from NETAPP to Reduxio using ROBOCOPY.
  • Assign Shared Permission and Quotes settings.
  • Re-run deltas process from the old NetApp again toward Reduxio in order to complete gaps.
  • Change IP, DNS name.
  • Review all settings configured properly.
  • Wait at least a month, ensuring we are aligned, that all data sync successfully and then we can start the decommissioning process.
  • Configure VSS on OS. Backups, alerts, A/V.

So what command I have used?:

  • We have to run the command below to initiate the transfer process from NetApp to Reduxio.
Robocopy "\\NetApp\OldVolume" "\\Reduxio\NewVolume" /MIR /SEC /B /R:3 >> Logfile.txt
/MIR = Mirror a directory tree with subfolders, and including empty folders

/SEC = Allow us to copy the NTDS permissions.

/B = Backup Mode

/R:3 = to set retry on failed attempts to 3, by default if you are note specified a value it will try to copy opened file till 1 million times, 
therefore 3 times it is enough, after 3 times it will skip on opened files. (you will see opened files that weren't copied on log).

>> = dropping info into the TXT file.
  • As soon as we finish we the intensive transferring we will need to run the following command in order to reduce the deltas which made during the time of file transferred.

Deltas:

 Robocopy "\\NetApp\OldVolume" "\\Reduxio\NewVolume" /R:3 /E /XO /B >> Logfile.txt

Deltas Switches explanations:

/R:3 = to set retry on failed attempts to 3

/XD = to exclude directories from source.

/E = copy all contents including empty directories of Source Folder to Destination Folder

/B = Backup Mode

>> = Dropping info into the TXT file.

In a case that you already copied some files but permission transferred incorrectly or missed, you can run the following command to fix SECURITY settings:

“/MIR /SECFIX /SEC”

What about some limitations with RoboCopy?

The main weird thing I encountered is that one of my customers has 1 Volume with the following path:

\\PelegIT

Under PelegIT path there were 30 folders, I couldn’t find a way to set in RoboCopy a parameter which knows to copy main path “\\PelegIT”, namely if I had a folder after “\\PelegIT\”folder” all would good! but I didn’t have.

It so frustrates me, (I will be glad if someone knows how to solve it)

Therefore I had to find a workaround, Export folder list to TXT using PowerShell:

  1. Export folder list to TXT using PowerShell:
Get-WmiObject -class Win32_Share -computer FileServer >> Folders.txt

2.Open you Folders.txt and edit the file to contains only folder list names,  remove unnecessary text, space, and so on, then save the file again

3. Create new folder according to TXT in the new Volume:

$GetFolders = Get-Content -Path C:\Folders.txt foreach ($folder in $GetFolders) { $NewPath = Join-Path "\\NewVolume\d$" -ChildPath $folder New-Item $NewPath -type Directory } Get-Content -Path C:\FolderList1.txt

then start with copying process, it will copy file folder per folder, matching names,

Robocopy "\\OldStorage\1" "\\NewStorage\1" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\2" "\\NewStorage\2" /MIR /SEC /B  /R:3
Robocopy "\\OldStorage\3" "\\NewStorage\3" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\4" "\\NewStorage\4" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\5" "\\NewStorage\5" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\6" "\\NewStorage\6" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\7" "\\NewStorage\7" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\8" "\\NewStorage\8" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\9" "\\NewStorage\9" /MIR /SEC /B /R:3
Robocopy "\\OldStorage\10" "\\NewStorage\10" /MIR /SEC /B /R:3