Hello Everybody! This week in adition of hosting our service on the Azure cloud, we take a closer look at the Azure API itself.
We are going to look closer at two functionalities we need to migrate our Amazon code to Azure:
- Azure Tables
- And Azure blobs
The first one allows us to store information in a database-like way, the second one is to store content.
Before starting, I strongly recommend you take a look at the Azure documentation concerning those topics:
And also this one as the naming for table and blobs has some restrictions that are good to keep in mind in order to avoid some unnecessary troubles…
Finally, I wanted to point out that blog article from where I extracted most of the Azure code I ‘m going to present here. I found that utility code was quite well and smartly written, I didn’t feel the need to rewrite from scratch an Azure toolkit on my own…so I have little credit for this one.
I – Using Azure Table Service
Using Azure tables is rather straightforward. Here an example on how to create a table client. I created a function able to work in both environments: local and cloud.
Oh yes, one cool thing about Azure over Amazon is that you can entirely simulate the cloud environment locally, so technically you can test your app, pretty much 100% offline before deploying on the cloud, something that isn’t doable with Amazon. I find it pretty handy:
public AzureTableHelper(string accountKeyName, bool useLocalStorage)
{
if(useLocalStorage)
{
_account = CloudStorageAccount.DevelopmentStorageAccount;
}
else
{
// Retrieve storage account from connection-string
string accountkey = CloudConfigurationManager.GetSetting(
accountKeyName);
_account = CloudStorageAccount.Parse(accountkey);
}
// Create the table client
_tableClient = _account.CreateCloudTableClient();
_tableClient.RetryPolicy = RetryPolicies.Retry(4, TimeSpan.Zero);
}
In order to insert a custom entity inside a table, it needs to derives from TableServiceEntity. An abstract class with a constructor taking two arguments: a partition key and a row key. Those properties allow to uniquely identify the entity inside our table, kind of intuitive…
Here is how to insert an entity inside a table:
public bool InsertEntity(string tableName, TableServiceEntity entity)
{
try
{
TableServiceContext tableServiceContext =
_tableClient.GetDataServiceContext();
tableServiceContext.AddObject(tableName, entity);
tableServiceContext.SaveChanges();
return true;
}
catch (DataServiceRequestException ex)
{
return false;
}
catch (StorageClientException ex)
{
if (ex.StatusCode == System.Net.HttpStatusCode.NotFound)
return false;
throw;
}
}
Another nice feature of the Azure API, is the ability to use LINQ to perform queries to retrieve entities from our table for example.
Below is the Azure version of my “GetDbModelInfo” server method, pretty slick:
public ModelInfo[] GetDbModelInfo()
{
try
{
List<ModelInfo> dataList = new List<ModelInfo>();
IEnumerable<TableModelInfo> infos =
_azureTableHelper.QueryEntities<TableModelInfo>
(_tableName).Where(e => e.PartitionKey ==
TableModelInfo.InfoPartitionKey).
AsTableServiceQuery<TableModelInfo>();
if (infos != null)
{
foreach (TableModelInfo info in infos)
{
dataList.Add(info.ToModelInfo());
}
}
return dataList.ToArray();
}
catch(Exception ex)
{
return null;
}
}
-
II – Using Azure Blob Service
-
Let’s now take a look at the blob service, this will be the equivalent of Amazon S3.
-
The client creation code is pretty much the same than the one I have for tables:
-
public AzureBlobHelper(string accountKeyName, bool useLocalStorage)
{
if(useLocalStorage)
{
_account = CloudStorageAccount.DevelopmentStorageAccount;
}
else
{
// Retrieve storage account from connection-string
string accountkey = CloudConfigurationManager.GetSetting(accountKeyName);
_account = CloudStorageAccount.Parse(accountkey);
}
// Create the blob client
_blobClient = _account.CreateCloudBlobClient();
_blobClient.Timeout = new TimeSpan(0, 5, 0);
_blobClient.RetryPolicy = RetryPolicies.Retry(4, TimeSpan.Zero);
}
-
The next thing we want to achieve with blobs is obviously uploading content. A blob has a limitation of 4MB, so if we want to upload bigger content, it is the responsibility of the programmer to split the data into 4MB (or less) chunks in order to perform the upload.
-
I borrowed code, (again!) from that blog in order to achieve this, and slightly enhanced it:
-
internal class BlobBlock
{
public static readonly int BlobBlockSize = 1 * 1024 * 1024; // 4 MB
public BlobBlock(string id, byte[] content)
{
Id = id;
Content = content;
}
public string Id
{
get;
set;
}
public byte[] Content
{
get;
set;
}
}
private IEnumerable<BlobBlock> GeDataBlocks(byte[] data)
{
HashSet<BlobBlock> hashSet = new HashSet<BlobBlock>();
if (data.Length == 0)
return hashSet;
int blockId = 0;
int startIdx = 0;
int chunkSize = BlobBlock.BlobBlockSize;
while (chunkSize == BlobBlock.BlobBlockSize)
{
// check if it is the last loop
if ((startIdx + chunkSize) > data.Length)
chunkSize = data.Length - startIdx;
byte[] chunk = new byte[chunkSize];
Array.Copy(data, startIdx, chunk, 0, chunkSize);
BlobBlock block = new BlobBlock(
Convert.ToBase64String(System.BitConverter.GetBytes(blockId)),
chunk);
hashSet.Add(block);
startIdx += chunkSize;
blockId++;
}
return hashSet;
}
// Put (create or update) a block blob.
// Return true on success, false if unable to create, throw exception on error.
public bool PutBlockBlob(string containerName, string blobName, byte[] data)
{
try
{
CloudBlobContainer container =
_blobClient.GetContainerReference(containerName);
CloudBlockBlob blob = container.GetBlockBlobReference(blobName);
HashSet<string> blocklist = new HashSet<string>();
foreach (BlobBlock block in GeDataBlocks(data))
{
using(MemoryStream ms = new MemoryStream(block.Content, true))
{
blob.PutBlock(block.Id, ms, null);
blocklist.Add(block.Id);
}
}
blob.PutBlockList(blocklist);
return true;
}
catch (StorageClientException ex)
{
if (ex.StatusCode == System.Net.HttpStatusCode.NotFound)
return false;
throw;
}
}
-
Concretely my service code using those helper functions is pretty slick:
-
public byte[] GetDbModel(string modelId)
{
try
{
Stream stream = null;
_azureBlobHelper.GetBlob(_containerName, modelId, out stream);
byte[] result = StreamToByteArray(stream);
stream.Close();
stream.Dispose();
return result;
}
catch (Exception ex)
{
return null;
}
}
-
public bool AddDbModel(RemoteModelData modelData)
{
try
{
string blobName = modelData.ModelInfo.ModelId;
_azureBlobHelper.PutBlockBlob(
_containerName,
blobName,
modelData.Data);
bool res = _azureTableHelper.InsertEntity(
_tableName,
new TableModelInfo(modelData.ModelInfo));
//Notify callback
//DbModelAddedNotifySubscribers(modelData.ModelInfo);
return true;
}
catch (Exception ex)
{
return false;
}
}
-
III – Handling timeouts
-
That’s it, we are done with the porting to Azure. The last thing I was facing are timeouts: by default timeouts are set to 1 minute, which in the case of large upload/download was insufficient with my connection.
-
The tricky part is that there are several places where you need to set those timeouts. They are all located in the config files: you will need to set obviously the server and client timeouts, in that case I set it to 5 minutes.
-
Here is a portion of the server config file:
-
<system.web>
<compilation debug="true" targetFramework="4.0" />
<httpRuntime executionTimeout="300"/>
</system.web>
<basicHttpBinding>
<binding
name="AdnCloudServicesBinding"
transferMode="Streamed"
messageEncoding="Mtom"
receiveTimeout="00:05:00"
sendTimeout="00:05:00"
openTimeout="00:05:00"
closeTimeout="00:05:00"
maxBufferSize="2147483647"
maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647">
<readerQuotas
maxDepth="200"
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"/>
</binding>
</basicHttpBinding>
And the client side:
<?xml version="1.0"?>
<configuration>
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding
name="WCFCHttpEndpoint"
transferMode="Streamed"
messageEncoding="Mtom"
receiveTimeout="00:05:00"
sendTimeout="00:05:00"
openTimeout="00:05:00"
closeTimeout="00:05:00"
maxBufferSize="2147483647"
maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647">
<readerQuotas
maxDepth="200"
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"/>
</binding>
</basicHttpBinding>
</bindings>
<client>
<endpoint
address=
"http://adncloudviewer.cloudapp.net/AdnCloudViewerSrv.svc"
binding="basicHttpBinding"
bindingConfiguration="WCFCHttpEndpoint"
contract="AdnCloudViewerSrv.IAdnCloudViewerSrv"
name="WCFCHttpEndpoint" />
<endpoint address="http://127.255.0.0:82/AdnCloudViewerSrv.svc"
binding="basicHttpBinding"
bindingConfiguration="WCFCHttpEndpoint"
contract="AdnCloudViewerSrv.IAdnCloudViewerSrv"
name="WCFCHttpEndpointLocal" />
</client>
</system.serviceModel>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/>
</startup>
</configuration>
-
We now run on Azure! All in all, I found the Azure API a bit more more complex than the Amazon one, however it appears to me much more flexible and powerful, leveraging many advantages of .Net programming that Amazon doesn’t have.
-
If you look for a detailed comparison between both, you may want to check this out, a very good blog on Azure technologies!
-
As usual, the complete sample is right there for you to have a look.
Subscribe
Comments