Date/Time stamps and Azure file shares

My good friend Bill Wilder pinged me a while back and asked what date/time stamp is placed on the files in an Azure File share, and where does it come from? I thought that was a good question, and figured the only way to find out is to do a bit of trial & error. So this blog entry is for Bill, who can read it as soon as he digs out from under the 7 feet feet of snow he’s gotten in the past month.

There are multiple cases I want to look at.

• Use PowerShell to transfer a file.
• Log into a VM, attach the file share, and transfer the files between the VM and the desktop using the Windows Clipboard.
• Write an application that uses the storage client library to transfer files from the desktop to the file share.
• Use AzCopy to transfer files between the desktop and the file share.

While writing this, I found that if I copy a file from one folder to another in Windows 8, it changes the Created Date to the desktop’s current date/time, and leaves the modified date “as is”. It shows the modified date in File Explorer. this seems odd to me, but I guess if you think of Created Date as the date the file was first put in the current location, it could make sense. What’s weird about this is that it is really easy to have a file with a Created Date that comes long after the Modified Date. I mention this because there is a similar behavior in some of the following cases.

For the purposes of these tests, I set the time on my desktop machine to be 5 minutes behind so I could discern between the desktop machine’s date/time and the VM’s date/time (which I am referring to as actual date/time). The tests were run on 2/9/2015 and all of the times are in Pacific Standard Time.

Note: I tried to set the time difference to an hour, but when I did that, I got 403 Forbidden errors with every call to Azure Storage, so I guess they have some kind of check on the time difference of your computer against the real date/time.

I’ve included the PowerShell commands, C# code for using the storage client library, and AzCopy commands, so you can replicate these tests if you want to and aren’t familiar with one of the methods.

Using PowerShell

Note: You may need to install the most recent version of the Azure PowerShell cmdlets from GitHub. At the time of this writing, it is 0.8.11. You can also use the Web Platform Installer, but I’m not a fan because there’s no telling what it will install along with whatever you asked for. Last time I used it to install the Azure cmdlets, it installed the newest version of the Azure Tools/SDK, which I wasn’t ready to install (2.5 has breaking changes). If you go to the link above to GitHub, you can download the Windows Standalone version (it’s an msi) and install it.

To find which version you have, you can run PowerShell and type the following:

> Get-Module Azure

I’m going to use the PowerShell ISE so I can save my commands as a script in case I want to use them later. First I’m going to set up a couple of variables for storage account name and key. This makes it easy to just modify those fields and run the script for a different storage account. Then I’m going to create the context for the storage account. This tells which storage account to use when this variable is used in subsequent commands.

$storageAcctName = “your storage account name”
$storageAcctKey = “your storage account key ending with ==”

$ctx = New-AzureStorageContext $storageAcctName $storageAcctKey

Next, I’m going to create a new file share and retain a reference to it ($s). It knows which storage account to use because I’m supplying the context I just defined. I’m naming my file share “contosofileshare”.

$s = New-AzureStorageShare contosofileshare -Context $ctx

To check and see the files shares set up in a storage account, you can use this command:

Get-AzureStorageShare –Context $ctx

So now I have a file share. I’m going to create a VM in Azure and RDP into it. If you don’t know how to create a VM in Azure, download the Fundamentals of Azure book from Microsoft Press (it’s free!) and go to the chapter on VM’s; it has follow-along instructions that show you how to create a VM.  (Disclaimer: I’m one of the authors of the book. I get no royalties, so please don’t think I’m recommending the book for financial reasons!)

After logging into the VM, I’m going to mount my file share. First I need a command prompt window. (If your VM is running Windows Server 2012, it’s like Windows 8. Click the windows button to go to the start screen, then just start typing “Command Window” to search for it, and click on it when it shows up on the right side.) I can use a NET USE command to attach the file share. The NET USE command looks like this:

NETUSE z: \\storageaccountname.file.core.windows.net\contosofileshare /u:storageaccountname storageaccountkey

After this is done, you can type NET USE and it will show you all of the file shares available, and what drive letter they are mapped to. I mapped mine to z:. Now I can open file explorer and go see what’s on the Z drive.

Back to PowerShell, I’m going to upload a file called Chihuly.jpg. On the desktop, if I right-click on this file and look at its properties (Details):

• Created Date: 11/18/2014 12:21:38 PM.
• Modified Date: 7/5/2007 7:52:04 AM.
• Actual time: 2/9/2015 4:01 PM.
• Desktop time: 2/9/2015 3:56 PM.

I’m going to upload the file and see what it looks like on the other side. The following command in PowerShell uploads the file.

Set-AzureStorageFileContent -Share $s -Source D:\_temp\_AzureFilesToUpload\Chihuly.jpg

Now if I go to the File Explorer on the VM and navigate to the Z: share and look at that file, the properties displayed are these:

• Created Date: 2/9/2015 4:01 PM
• Modified Date: 2/9/2015 4:01 PM

So using PowerShell to upload files to an Azure file share changes both Created Date and Modified Date to the current date/time. The date/time on the desktop machine is not used.

Where does Azure get the Created Date and Modified Date – from a time server somewhere? From the server holding the storage? That’s the question.

Now I’m going to download the file using PowerShell and see what the date/time stamps are on the downloaded file.  The following command in PowerShell downloads the file.

Get-AzureStorageFileContent -Share $s -Path Chihuly.jpg -Destination d:\_temp\_AzureFilesDownloaded\

Desktop time: 6:15 PM
Actual time: 6:20 PM

After the download:

Created Date: 2/9/2015 6:15 PM
Modified Date: 2/9/2015 6:15 PM

From this, we can see that using PowerShell to download files from an Azure file share changes the Modified Date to the date/time on the local machine. In this case, it sets the Created Date to the time on the local machine, but in some test cases, it set it to a few minutes before that. Where does it get the value for created date?

Copy and Paste

Now I’m going to try the same thing with copy (desktop) and paste (vm). I copy the file on the desktop into my Windows Clipboard (Ctrl-C), then click on the VM and go to the share and paste the file (Ctrl-V). This is interesting.

Desktop time: 3:59 PM
Actual time: 4:04 PM
Desktop Created Date: 11/18/2014 12:21 PM PST
Desktop Modified Date: 7/5/2007 7:52 AM

Copy/Paste results:

File Share Created Date: 2/9/2015 4:04 PM 
File Share Modified Date: 7/5/2007 7:52 AM

From this, I’m going to surmise that using copy and paste to upload files to an Azure file share changes the created date to the current date/time, but leaves the modified date as the original value. This is the same behavior as Windows.

Now I’m going to try copying the file from the Azure file share (via the VM) to the local desktop and check the dates.

Desktop time: 6:13 PM
Actual time: 6:18 PM

Created date: 6:13 PM
Modified date: 7/5/2007 7:52 PM

Using copy and paste to download files from an Azure file share changes the created date to the current date/time on the desktop, but leaves the modified date unchanged – it is the same value as that the file on the file share. This is the same behavior as Windows.

Storage Client Library

What if I write some code to upload and download files to/from the file share? Let’s look at what that looks like.

First, I need a method to get a reference to the directory on the file share that I’m going to use. This will be used for both the upload and the download. Comments inline tell how this works. It’s very similar to blob storage.

public CloudFileDirectory SetupQuickCommands(string storageAccountName, string storageAccountKey,
    string shareName, string subFolderPath)
{
    //set up file share
    //get reference to the storage account 
    string connectionString = @"DefaultEndpointsProtocol=https;AccountName=" + storageAccountName +
        ";AccountKey=" + storageAccountKey;
    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.Parse(connectionString);

    //create the cloudfileclient using the storage account object
    CloudFileClient cloudFileClient = cloudStorageAccount.CreateCloudFileClient();

    //get a reference to the share
    CloudFileShare cloudFileShare = cloudFileClient.GetShareReference(shareName);

    //does it go in the root directory? need to set CloudFileDirectory instance accordingly
    //this points to the directory on the file share 

    CloudFileDirectory rootDirectory = cloudFileShare.GetRootDirectoryReference();
    CloudFileDirectory cloudFileDirectory = null;

    if (string.IsNullOrWhiteSpace(subFolderPath)) //write it to the root directory
    {
        cloudFileDirectory = rootDirectory;
    }
    else
    {
        //write it to the requested directory
        cloudFileDirectory = rootDirectory.GetDirectoryReference(subFolderPath);
    }
    return cloudFileDirectory;
}

Next, we need code for doing the upload and download. Each of these will call the code above to get the reference to the directory on the file share.

Here is the code to upload a file. After getting the reference to the directory on the file share, this gets a reference to the file and uploads to that file. SubFolderPath is the path on the share to be used. If this is blank, the file goes in the root of the share.

public string QuickUpload(string storageAccountName, string storageAccountKey,
    string shareName, string subFolderPath, string fileName)
{
    CloudFileDirectory cloudFileDirectory = SetupQuickCommands(storageAccountName, 
        storageAccountKey, shareName, subFolderPath);
    //upload the file 
    string fileNameOnly = Path.GetFileName(fileName);
    CloudFile cloudFile2 = cloudFileDirectory.GetFileReference(fileNameOnly);
    cloudFile2.UploadFromFile(fileName, System.IO.FileMode.Open);
    return string.Empty;
}

And here is the code to download a file. The localFolderPath is the destination on the local machine for the Download directory where the files will be placed.

public string QuickDownload(string fileName, string storageAccountName, string storageAccountKey,
    string shareName, string subFolderPath, string localFolderPath)
{
    CloudFileDirectory cloudFileDirectory = SetupQuickCommands(storageAccountName,
        storageAccountKey, shareName, subFolderPath);

    //put this in the folder they specify, in a subfolder called Downloads
    //if Downloads doesn't exist, create it 
    string localPath = Path.Combine(localFolderPath, @"Downloads");
    if (!Directory.Exists(localPath))
    {
        Directory.CreateDirectory(localPath);
    }

    //download the file
    CloudFile cloudFile = cloudFileDirectory.GetFileReference(fileName);
    string downloadTo = Path.Combine(localPath, fileName);
    cloudFile.DownloadToFile(downloadTo, FileMode.Create);

    return string.Empty;
}

So now let’s run our tests using the code above.

Desktop time: 5:50 PM

Actual time: 5:55 PM

After uploading the file, the results are as follows:

Created Date: 2/9/2015 5:55 PM

Modified Date: 2/9/2015 5:55 PM

From this, I going to surmise that using the storage client library to upload a file, the Date Created and Date Modified are both set to the actual current date/time.

This makes sense. The PowerShell upload calls the REST API, and so does the storage client library.

When I use the storage client library to download the file to the desktop, what do I get?

Desktop time: 6:21 PM

Actual time: 6:26 PM

After downloading the file, the results are as follows:

Created Date: 2/5/2015 5:27 PM

Modified Date: 6:21 PM

It is setting the Modified Date to the date/time on the desktop machine. Where’s that created date coming from?

What about AzCopy?

This should be the same as the storage client library and PowerShell, but I’m going to try it to be thorough. Click to download AzCopy and/or learn more about it. I’m going to list the commands that I used to upload and download files. One thing to note: There is a switch you can add (/mt) that will retain the modified date/time on the blob when you download it. I’ve done this without the /mt switch because I want to see what it gets set to by default.

Desktop time: 4:07 PM

Actual time: 4:12 PM

The command looks like this. Take the space out before storageaccountname – I just put it in so it wouldn’t make it a live hyperlink.

AzCopy d:\_temp\_AzureFilesToUpload\

https:// storageaccountname.file.core.windows.net/sharename/ Chihuly.jpg /DestKey:storageaccountkey

After uploading the file, the results are as follows:

Created Date 4:12 PM

Modified Date: 4:12 PM

When uploading a file using AzCopy, the Created Date and Modified Date are both set to the current date/time.

Desktop time: 6:20 PM

Actual time: 6:25 PM

The download command looks like this. d:\_temp\_AzureFilesDownloaded is the local directory I’m downloading the files to. Again, take the space out before the storage account name.

AzCopy https:// storageaccountname.file.core.windows.net/sharename/

    d:\_temp\_AzureFilesDownloaded Chihuly.jpg /SourceKey:storageaccountkey

Created Date: 6:20 PM

Modified Date: 6:20 PM

When downloading a file using AzCopy, the Created Date and Modified Date are both set to the date/time of the local desktop. (It could be that this Created Date is sometimes earlier than Modified Date; I only ran this test a couple of times.)

RESULTS

Uploading files from the desktop to the file share:

PowerShell, Storage Client Library, AzCopy: Uploading files to an Azure file share changes the Created Date and Modified Date to the current date/time.

Copy/Paste: Uploading files changes Created Date to the current date/time, but retains the original Modified Date. This is the same behavior as Windows.

Downloading files:

PowerShell, Storage Client Library, AzCopy: Sets Created Date and Modified Date to the date/time on the desktop. In some cases, the Created Date was actually set earlier than the modified date, so I’m not sure it’s actually using the desktop date/time for that value.

Copy/Paste: Changes the Created Date to the desktop date/time, leaves the Modified Date unchanged.

How does this compare to blob storage?

The deal with the Created Date is just weird, so I wanted to compare this behavior with that of uploading and downloading files using the storage client library and AzCopy to or from blob storage.

When you upload a blob using either method, it sets the Last Modified Date to the current date/time. (There is no created date property on blobs.) This is consistent with uploading files to the file share using PowerShell, the Storage Client Library, and AzCopy.

When you download a blob from blob storage using AzCopy, it sets the Created Date and Modified Date to the current date/time.

When you download a blob from blob storage using the storage client library, I saw some very strange results:

Desktop time: 6:40 PM

Actual time: 6:45 PM

Last Modified Date on the blob is 6:44 PM

Download blob.

Created Date: 2/5/2015 5:27 PM

Modified Date: 6:22 PM 

Remember that unless specified, all of the dates are 2/9/2015. I have no idea where it’s getting that value for Created Date. Note that the Created Date is the same one given to the file when downloading it from the file share using the storage client library. Coincidence? Probably not.

Overall results

In most cases, the date/time stamps assigned to the files as they are transferred between a desktop and an Azure file share make sense. The one case I have no explanation for is where the Created Date comes from when downloading a file to the desktop. While it is frequently equal to the modified date, it is not always the case. Does Azure get this value from a time server somewhere, or from a machine hosting storage in Azure, or what? I don’t know. I’ve submitted this question to the Azure MVP distribution list, and if I get an answer, I’ll update this blog entry.

In the meantime, if you decide to write your own syncing software to sync file shares or blobs, I would consider using a hash of the file or blob to discover if the files are actually different, and use the Modified Date and not the Created Date.