NewRelic (http://newrelic.com) is a great performance monitoring tool. We use it for nearly all of our clients and it provides a wealth of information on things like page speed, server responsiveness, .NET errors, database connectivity, and more. 

The only problem from our perspective is that it requires the installation of software on the server (the “server agent” and monitoring tools). 

In Azure, this can be a problem, because every time Azure instances disappear they are reimaged from scratch based on the most recent “package” that was deployed to them. This can and does happen, and there is no warning as to when this might occur. You’ll simply arrive one morning to find your monitoring is gone. 

If this deployment package that you sent to the server didn’t include the NewRelic installation, then you can kiss your monitoring goodbye. You’ll have to reinstall it manually.

Installing NewRelic in Visual Studio

NewRelic’s default installation instructions for Azure can be found here: https://docs.newrelic.com/docs/agents/net-agent/azure-installation/azure-cloud-services

Effectively, the crux of this is:

1. Install Nuget package for NewRelic in your VS.NET solution.
2. Enter licence key and application name.
3. Deploy to Azure using VS.NET.

This is great if you’re using Visual Studio.NET to deploy directly to Azure. 

It is also worth noting that in order to do this, you need the Azure SDK on your machine and you need to be using an Azure .NET "Cloud Service Project”, not a regular web application project.

Unfortunately when using Sitecore’s Azure module, this process is abstracted through the module itself. The module handles deployment - you don’t deploy anything at all through Visual Studio. What you do instead is create deployment packages and deploy your code to a particular deployment (on-premises) server, then kick off an “Upgrade Files” from there to push your changes up to Azure.

In fact, if you run the command as per the NewRelic instructions and install the Nuget package, it will download the files correctly into the “packages” folder on your development environment, but will then spit out errors that look like this:

***Updating the Windows Azure ServiceDefinition.csdef with the newrelic.cmd Startup task***
Unable to find the ServiceDefinition.csdef file in your solution, please make sure your solution contains an Azure deployment project and try again.
***Updating the Windows Azure ServiceConfiguration.*.cscfg files with the license key***
Unable to find any ServiceConfiguration.cscfg files in your solution, please make sure your solution contains an Azure deployment project and try again.
***Updating the projects .config file with the NewRelic.AppName***

These errors are occurring because I’m using a regular web application project in Visual Studio, not an Azure SDK “Cloud Service Project” instead.

Picking Up The Slack

NewRelic got us part of the way there. It provided us, as part of the package, with a batch file (newrelic.cmd) and a couple of installer files.

With a bit of digging (thanks to: https://discuss.newrelic.com/t/nuget-doesnt-work-with-cloud-service-project-in-visual-studio-2013/5550/13), I was able to find the snippets of code that NewRelic actually uses to run said startup script. These snippets failed to install during the Nuget package install (see above) because, as I said, I’m using a regular web application and not an Azure SDK-style project.

For those of you who’ve delved a bit deeper into Sitecore Azure module, you’ll know that it generates the ServiceDefinition.csdef and ServiceConfiguration.cscfg files automatically from fields within Sitecore. You’ll find these fields on the relevant “Production” or “Staging” nodes in your content tree on the system -> modules -> Azure -> Environment -> Region -> Farm -> Role -> Deployment node.

For example, if you want to modify the fields for the Australia East production deployment, you’d look for the node:

… Australia East -> Delivery01 -> Role01 -> Production

Step 1 - The Installer & Script

The first step is to get Sitecore to include the batch file (newrelic.cmd) and the two installers in the upload process. This will include them as part of the website itself, so that you can then kick off an install once the system has processed the deployment package itself.

In order to do this, first, copy the files from the packages\NewRelicWindowsAzure.5.4.16.0\content folder on your machine (or the location where your Nuget packages get installed). In my case, these files are:

• newrelic.cmd
• NewRelicAgent_x64_5.4.16.0.msi
• NewRelicServerMonitor_x64_3.3.3.0.msi

Place these files on the deployment (on-premises) server in the folder: %wwwroot%\App_Data\AzureOverrideFiles\bin. This will ensure they are uploaded to the bin folder on any new Azure instance along with your actual website files.

Step 2 - Modify the Config

You’d then modify the two fields to incorporate snippets.

Service Definition

Note that you should include the NewRelic task after the default Sitecore startup task. These snippets tell Azure:

• to run the newrelic.cmd batch file, which installs the two installers that are included with your deployment; and,
• to look for a configuration setting in the ServiceConfiguration.cscfg file for the NewRelic license key
  

<?xml version="1.0" encoding="utf-16"?>
<ServiceDefinition ... >
  <WebRole name="SitecoreWebRole" enableNativeCodeExecution="false" vmsize="ExtraLarge">
    ...
    <Startup>
      ...
      <Task commandLine="newrelic.cmd" executionContext="elevated" taskType="simple">
        <Environment>
          <Variable name="EMULATED">
            <RoleInstanceValue xpath="/RoleEnvironment/Deployment/@emulated" />
          </Variable>
          <Variable name="IsWorkerRole" value="false" />
          <Variable name="LICENSE_KEY">
            <RoleInstanceValue xpath="/RoleEnvironment/CurrentInstance/ConfigurationSettings/ConfigurationSetting[@name='NewRelic.LicenseKey']/@value" />
          </Variable>
        </Environment>
      </Task>
    </Startup>
    <ConfigurationSettings>
      ...
      <Setting name="NewRelic.LicenseKey" />
    </ConfigurationSettings>
    ...
  </WebRole>
</ServiceDefinition>

Service Configuration

This minor change defines the NewRelic licence key (you should have your own or use your client’s key) so that it can be referred to by the installers.

<?xml version="1.0" encoding="utf-16"?>
<ServiceConfiguration ...>
  <Role name="SitecoreWebRole">
    <ConfigurationSettings>
      ...
      <Setting name="NewRelic.LicenseKey" value="xxxxxxxxxxxxx" />
    </ConfigurationSettings>
    ...
  </Role>
</ServiceConfiguration>

Step 3 - Setting the Application Name

Finally, you need to modify the Web.config of your application to include the name of the application that you’d like to appear in NewRelic.

By default, NewRelic’s Nuget installation would have added the name to your Web.config. 

You may want to alter this. If you just want to change it for your entire application, then by all means, just go and change it in the Web.config file.

However, if you want to do it on a per-region basis, for example, or even have a separate application name for Editing, for example, then you can’t just modify the Web.config because Sitecore’s Azure module will deploy the same Web.config to all instances regardless.

In order to deploy a server- or region-specific application name (or otherwise), you’ll need to again make use of the fields Sitecore has available on the “Production” or “Staging” nodes in the Content Editor.

In this case, you update the Custom Web Config Patch field to include the following XSLT transform snippet:

<xsl:template match="/configuration/appSettings">
    <xsl:copy>
      <xsl:apply-templates select="node()|@*" />
      <xsl:element name="add">
        <xsl:attribute name="key">NewRelic.AppName</xsl:attribute>
        <xsl:attribute name="value">My Application Name</xsl:attribute>
      </xsl:element>
    </xsl:copy>
  </xsl:template>

This will add the relevant application key to the generated Web.config. Because this is done on each deployment node, you can have a different name for each deployment if you want.

Deploy It And You’re Away

I’m not going to pretend this will all work perfectly the first time around, but I just tried it and it seemed to work quite seamlessly. Clearly there are some issues, like leaving a batch file and two installers in your website’s bin folder, but presumably you could get rid of these through some other startup mechanism if you really had to.

For the time being, though, this is a small price to pay to have monitoring installed automatically on the server instances.

Thoughts?

If you have any thoughts or comments about this process, or you’d just like to thank us for posting it, feel free to reply with a comment.

Happy developing!

Written by Christian Brenner
Head of Technology
NOW Digital

The Background

NOW Digital deals with many clients whose websites are in the cloud, and many of these are using the Microsoft Azure cloud platform. Sitecore works relatively seamlessly with the cloud and is intrinsically well suited to perform on the Azure platform. To assist with this, Sitecore have provided the Sitecore Azure module, which plugs into Sitecore and connects you to the cloud, allowing you to perform deployment operations via a local on-premises (deployment) environment.

Now, the cloud might be akin to this godlike entity that never fails in most people’s minds, but in practice it still suffers some of the same limitations as regular hosting environments. It also still has the concept of “servers” existing in a particular location, even though it tries to abstract this and call them something else.

Specifically:

• “Cloud services” are similar to web servers, and these exist in a region.
• “SQL Azure” databases exist on a server, which exists in a region.

As such, you also get latency. If you, for example, are located in Australia and the “environment” (whatever it is) is located in the US, then you get latency because of the time it takes the data to travel that far.

For our purposes, the latency difference between Southeast Asia and Australia East (regions that are in relatively close proximity) was significant enough (approx. 1-2 seconds per page load in a delivery environment) that the client wanted to shift their environment from Southeast Asia to Australia East in order to service their primary market (which is Australia), but retain a delivery environment in Southeast Asia to service the Asian market.

Now, Sitecore’s Azure implementation allows three primary configuration options:

• Live Server - Editing & Delivery both hosted on the one server.
• On-Premises Editing - a CM/CD setup, but with Editing in a local (on-premises) environment and Delivery in the cloud (you can have as many Delivery Farms as you like)
• Editing in Azure - a CM/CD setup but with Editing in the cloud as well (i.e. you have one Editing Farm in Azure as well as your Delivery Farms)

Our client had the latter of these three, and in particular, they wanted to move Editing, because it was considerably slower than it should have been based on Sitecore’s typical performance output, and the responsibility for this lay with latency.

Only One Editing Farm

Sitecore’s Azure module allows you to relatively easily add new Delivery Farms with the click of a button in the Sitecore Azure module. It also, however, allows you to customise what it’s doing by manually editing the nodes that are created in the Sitecore content tree under the system -> modules -> Azure section.

(This becomes important later, so write that down.)

In fact, to “shift” a regular Delivery Farm the process is relatively straightforward:

• Create a new Delivery Farm in the desired region.
• Publish all content to this environment.
• Change the IP / Traffic Manager configuration.
• Decommission the old Delivery Farm.

Theoretically, I thought to myself, what I could do is proceed down the same path, except with an Editing Farm.

Unfortunately, Sitecore’s Azure module:

• Doesn’t provide a way to “Move” an Editing Farm.
• Rather studiously enforces the whole “only one Editing Farm allowed” rule.

This rules out the possibility of using Sitecore Azure to create a new Editing Farm for me.

Or does it?

Surely somebody has done this before, we thought. It must be online somewhere. Sitecore must have documentation around it.

So we researched it. And researched it. And researched it.

Time came for us to move the Editing environment and, despite a wealth of information now available online about how to use Sitecore with Azure, the pros and cons and how to do this and that, there was absolutely zero information on how to move an Editing Farm from one region to another. Clearly it wasn’t a common requirement.

So we contacted Sitecore’s support portal and got nothing (though, in fairness, they did eventually come back with a solution slightly different to the one shown here, and one earmarked with the caveat “this isn’t thoroughly tested so be careful”).

What Help Is There?

During this research, I did come across three relatively useful articles on how to “customise” your Azure configuration, one each in regard to cloud services, databases, database services and storage services. 

These provided a handy insight as to “how the Sitecore Azure module” actually works behind the scenes. For those who have used it, you will realise that its black box nature can be somewhat frustrating when trying to do something new and relatively undocumented / untested.

These four articles are:

• https://kb.sitecore.net/articles/726135
• https://kb.sitecore.net/articles/001970
• https://kb.sitecore.net/articles/179306
• https://kb.sitecore.net/articles/728236
  

The important thing about these articles was that they implied that, if Sitecore Azure can find something, it uses it, and if it can’t find something, it creates it. 

This theory also seemed to be confirmed by my own testing when I had been mucking around with creating new Delivery Farms over time.

Another useful link is “how to clean up after a failed Azure deployment”, which gives you good insight into what artefacts Sitecore Azure creates and how you can manually manipulate them:

• https://kb.sitecore.net/articles/771717

As such, these articles all implied the Sitecore Azure module was relatively robust and I “should” be able to wrangle it to work in my favour by potentially fooling it that an Editing environment didn’t already exist.

This one line (above) is effectively what this solution is all about.

The Setup

In any case, it’s important to note that this particular process worked well for us given our current environment. It might not necessarily work well in other environments depending on the configuration.

So take note of this setup:

• Sitecore 7.1 (rev 130926) with Sitecore Azure 3.1
• Azure with Editing in the Cloud
• SQL Azure / PaaS model
• Editing Farm in Southeast Asia
  
• Delivery Farm in Southeast Asia
• Delivery Farm in Australia East
• Analytics, Core & WFM shared between all environments, and residing on a server in Australia
• Each Delivery environment has its own “web” DB
• Editing also has a “web” DB (for previewing published content)
• Traffic Manager in place handling delivery

Note that we are wanting to shift Editing from Southeast Asia to Australia East. Note that there is already a Delivery Farm in Australia East, meaning that a “storage” blob already exists in that region.

So: if your configuration sounds similar to this, press on.

Be Careful

First and foremost, before doing this, you should be very careful to:

• Inform your content editors about a content freeze during which time they won’t be able to update the website
• Backup all your databases (including delivery, just in case

Getting Ready

1. Create a blank cloud service in the new region using your desired name. Remember to use the correct naming convention or you could blow up Sitecore Azure. Specifically, this means using “Ae” instead of “Sa” in a name such as “clientabcprodsace01role01scddc”.
   
   
2. Initiate a content freeze.
   
   
3. Backup all databases using Export feature in Azure portal. Specifically export “master” and (editing) “web” databases to a storage blog container in Azure, as these will be the ones shifted to the new region. Also backed up core, analytics and WFM, as well as all delivery databases, just in case.
   
   
4. Create a new DB server in the new region. Set the password to some randomly generated password that is 12 characters long in keeping with other SQL Azure passwords (https://www.random.org/passwords/).
   
   
5. Create a new database in the Azure portal using the “restore” option. Select your backup from your storage blob container, and do this for “master” and “web”.
   
   Ensure the name of the restored database is in keeping with Sitecore Azure standards (e.g. if you’re shifting from Southeast Asia to Australia East, you should change “SaCe01” to “AeCe01”, in the name of the database.
   
   Also remove the timestamp off the end of the name of the database so it will have a “nice” Azure DB name that is in line with all your other databases.
   
   
6. Once complete, login to your on-premises deployment server using RDP, the one you use Sitecore Azure on to manage your deployments.
   
   This server is the one where you go to run the “Upgrade Files” command.
   
   
7. In the Sitecore Content Editor on this environment, expand system -> modules -> Azure -> [YourEnvironment].
   
   
8. Expand the region your editing environment is currently in (e.g. Southeast Asia).
   
   
9. Drag-and-drop the “Editing01” node into the region you want it to be in. Note all this does is tell Sitecore Azure to “draw” the dot for the editing environment in the new region. It doesn’t actually change any underlying structures in Azure.

Now you’re going to “fool” the Azure module by manually updating the configuration, bearing in mind you’ve already:

• Created a new cloud service that is empty
• Created a new database server
• Restored master and web onto this server.

Fooling Sitecore

So:

1. Update Storages -> Default item fields to match the storage fields already included in the equivalent Delivery01 storage node. In other words, you’re going to use the same storage service as the Delivery01 environment already uses. These fields are:
   
   “Service Name” - set to the same name as the storage already associated with that region (grab from the Delivery Storage node)
   
   “Primary Access Key” - set to the same as the storage already assocaited with that region (grab from the Delivery Storage node)
   
   Based on my testing, if you didn’t already have a delivery environment here (and thus no storage service), you could theoretically set these fields to blank and it will create it for you, but I haven’t tested this behaviour.
   
   
2. On the Sql01 node, set the name of the server you created.
   
   “Server Name” - set to the name of the DB server created in step 4 (Getting Ready)
   
   
3. Expand the Set01 node, and for each (“master” / “web”) database node, update the connection strings for the target database.
   
   “Target Database” - set the full connection string for the new database on the new server you restored in step 5 (Getting Ready)
   
   
4. On the Production node, update fields as follows:
   
   “Service Configuration” - update the settings below so that the connection strings (which are for storage services) have updated endpoint names and account keys matching the storage service details you set in step 1 above (Fooling Sitecore)
   
   These two connection strings are in the XML, and are called:
   
   Microsoft.WindowsAzure.Plugins.Caching.ConfigStoreConnectionString
   Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString
   
   “DNS Host Name” - set to be the name of the empty cloud service you created in step 1 (Getting Ready)
   
   “Connection Strings” - update the master and web connection strings to include new server / catalog name / password combination (if you use this particular field to override connection strings)
   
   “Private ID” - set to blank (this maps to the “deployment ID” of the deployment once created, and since it doesn’t exist yet (cloud service is empty) we want Sitecore to create it for us
   
   
5. Now, open up the “ConnectionStrings.config” file on this on-premises deployment server and add the new connection strings (for master and web) to this file as well (remove the old ones if you want).
   
   IMPORTANT: Connection string names should be in the format: [CloudServiceName]master_[ServerName] (substitute “master” for “web” when doing the other database).

Deploying

At this point we’ve created our empty cloud service, told the content authors not to update anything, moved our master & web databases, and we’ve setup a manual configuration in Sitecore to try to fool the Azure module into believing the Editing environment still needs to be created.

From here:

1. Restart your application (on-premises deployment server) by modifying the web.config file.
   
   
2. Open the Sitecore Azure module (not the Content Editor this time).
   
   
3. Select “Production [Production]”.
   
   
4. Your editing instance should now be visible in the new region you dragged it to, and it will no longer detect the old Editing environment you’d previously connected.
   
   As such, it will show a Blue Dot say “Resume Deployment” when you click on it.
   
   
5. Select “Resume Deployment”.
   
   
6. Select your machine instance size (Extra Large or A4 recommended).
   
   
7. You will be prompted because all databases are already in existence. Select “OK” to use all existing databases as-is.
   
   This will not make any changes to the existing databases and simply connect in place. Do this for master, web, analytics, WFM, core and any other database you’ve shifted.
   
   
8. Deployment will take place. This may take some time (up to 30 mins to 1 hr depending on connection speed, etc.)
   
   You can keep a tabs on it by viewing Sitecore log files, which are updated when Sitecore does something and also the Azure portal itself, which will periodically update.
   
   Your cloud service in Azure will eventually have a single instance created in it within the Production slot. This will be where your files will be uploaded and where your Editing environment will soon live.
   
   
9. Once deployed, you may need to ensure that your publishing targets and connection strings are correct on the Editing environment.
   
   Since you backed up and restored master DB, publishing targets will still be visible in Sitecore as they were previously, but the file “publishTargets.config” is generated by Sitecore Azure during deployment and this tells those publishing targets their database details. For some reason this file seems to only use the publishing targets of the current region’s delivery environments and ignores all others.
   
   So, if your setup is like ours, the Southeast Asia delivery publishing target was omitted. We had to manually add this, which is not ideal give any custom changes are deleted each time an Azure instance is re-imaged (which happens monthly or whenever Azure performs an update).
   
   We are currently investigating a more permanent resolution to this last issue and will update this post when it is found.
   
   
10. Likewise, repeat this process for ConnectionStrings.config. See above notes around the persistence of this update.

Thoughts?

If you have any thoughts or comments about this process, or you’d just like to thank us for posting it, feel free to reply with a comment.

Happy developing!

Written by Christian Brenner
Head of Technology
NOW Digital