Techila Distributed Computing Engine Set-Up Guide for Microsoft Azure

1. Introduction

This document is designed for persons who are planning to set up a Techila Distributed Computing Engine environment in Microsoft Azure. If you are unfamiliar with the Techila terminology used in this document, please refer to Introduction to Techila Distributed Computing Engine. For information about the Techila Distributed Computing Engine IT architecture, please see it-architecture-techila-distributed-computing-engine.html[Techila Distributed Computing Engine IT System Architecture]. More information about network connections and ports can be found here.

Screenshots and dialogs in this document are from Microsoft Windows 7. Depending on your system, the appearance of screens might be different.

The structure of this document is as follows:

Preparation contains a list of prerequisites that must be met before proceeding with setting up a TDCE environment in Microsoft Azure.

Deploying the Techila Server in Microsoft Azure contains instructions for deploying the Techila Server in Microsoft Azure.

Deploying Techila Workers in Microsoft Azure contains instructions for deploying Techila Workers in Microsoft Azure. Instructions are also provided for removing Techila Workers that have already been deployed.

Installing Bundles contains instructions for installing Bundles on the Techila Server. These Bundles are required if you are planning on using R or Python in your computational tasks.

Adding End-Users contains instructions for adding End-Users to the TDCE environment.

Components Created During the Deployment Process contains a description of Microsoft Azure components that are created when deploying the Techila Server and Techila Workers in Microsoft Azure.

Removing Components from Microsoft Azure contains instructions on how to remove all TDCE components from Microsoft Azure. Note that performing the steps described in this Chapter will also remove all data from the Techila Server meaning the state of the Techila Server will be lost.

2. Preparation

Before proceeding, please verify that you have the following components available:

Techila License package
Techila Deployment Tool for Microsoft Azure (delivered in a zip-file named azureinstaller.zip)

If you do not have a Techila License package, please contact your Account Manager or Techila Support at support@techilatechnologies.com.

The Techila Deployment Tool for Microsoft Azure is available in the Techila Extranet.

If you do not have access to Techila Extranet, please contact your Account Manager or Techila Support at support@techilatechnologies.com to request for Techila Extranet access.

2.1. Ensuring that Java is Available

In order to launch the Techila Deployment Tool for Microsoft Azure, Java needs to be installed on the computer you are using. Java Platform, Standard Edition 6 (or newer) Java Development Kit (JDK) or Java Runtime Environment (JRE) are supported.

When Java is installed and available from the command line, Java version can be verified with the following command. Please note that executing the command requires that the Java installation path is listed in the PATH environment variable.

java -version

Figure 1. Verifying the Java version from the command line.

2.2. Extracting the Techila License File

The Techila License file will be needed when deploying the Techila Server in Microsoft Azure. Please follow the steps below to extract the Techila License file to your computer:

Locate the Techila License package. In the figure below, a Techila License package is located in the directory C:\TechilaLicense.

Figure 2. The Techila License is delivered in a ZIP file.
Extract the Techila License package. After extraction, a Techila License file will be available as illustrated below. This file will be imported to the Techila Server later during the deployment process.

Figure 3. A Techila License file can be recognised from the .license suffix.

2.3. Extracting the Techila Deployment Tool for Microsoft Azure

The Techila Deployment Tool for Microsoft Azure is available in the Techila Extranet and is delivered in a zip-file called azureinstaller.zip.

Locate the zip-file containing the Techila Deployment Tool for Microsoft Azure. In the figure below, the zip-file is located in the directory C:\Downloads.

Figure 4. The Techila Deployment Tool for Microsoft Azure is delivered in a ZIP file.
Extract the azureinstaller.zip package to a directory that does not require administrative privileges for creating files. This is required, because the Techila Deployment Tool will create files in the current working directory.

After extraction, the Techila Deployment Tool (azurearminstaller.jar) will be available as illustrated below.

Figure 5. The Techila Deployment Tool for Microsoft Azure

3. Deploying the Techila Server in Microsoft Azure

This Chapter contains instructions for deploying the Techila Server in Microsoft Azure. Instructions for removing the Techila Server can be found in Removing the Techila Server.

Navigate to the directory that contains the Techila Deployment Tool for Microsoft Azure (azurearminstaller.jar) and launch the application. When using a file manager, the application can be launched by double clicking on the azurearminstaller.jar file icon.

When using a command prompt, the application can be launched using command java -jar azurearminstaller.jar. Please see Techila Deployment Tool Command Line Parameters for information on additional command line parameters that can be defined when launching the Techila Deployment Tool from the command line.

Note! If you are unable to launch the application from the command line, please verify that your PATH environment variable contains the location of your Java installation.

Note! The azurearminstaller.jar application will create files in the current working directory, which means that the application must have write access to the directory. Ensure that the directory containing the azurearminstaller.jar file does not require administrative privileges rights for creating files.

Figure 6. In this example, the file azurearminstaller.jar that contains the Techila Deployment Tool for Microsoft Azure is located in directory C:\AzureDeployment. The application can be launched by double clicking the file icon.
After launching the Techila Deployment Tool for Microsoft Azure, the Certificate Management tab will be displayed. If you are launching the Techila Deployment Tool for Microsoft Azure for the first time, files required for deployment process will be automatically generated to the current working directory.

Figure 7. The Techila Deployment Tool for Microsoft Azure. When the application is launched the first time, a Microsoft Azure management certificate and a private key will be generated into the current working directory.

The files required for deployment process include a Microsoft Azure management certificate and the private key. These files are illustrated below.

Figure 8. Files required in the deployment process can be found in the same directory as the Techila Deployment Tool for Microsoft Azure.
During the following steps, the generated certificate will be linked to your Microsoft Azure Subscription. Click the Open Microsoft Azure Management Portal button. This will open the Azure Management Portal in your web browser.

Figure 9. Clicking on the highlighted button will open the Azure Management Portal in your web browser.
If required, sign in using your account.

Figure 10. Signing in.
After signing in, select Subscriptions and click on the subscription you want to use.

Figure 11. Click on the subscription.
After clicking on the subscription, click the Management Certificates item.

Figure 12. Click Management Certificates.
Click the Upload icon.

Figure 13. Certificates can be uploaded by clicking the Upload icon.

A new section will be displayed, which can be used to select the management certificate you wish to upload. If you have multiple subscriptions, you can also choose which subscription you wish to use.
Make sure you are uploading the certificate to the subscription you want to use and click on the file icon to open a file dialog window.

Figure 14. Depending on how many subscriptions you have, the view will might be different. If you have multiple subscriptions, you can use the drop-down menu to select the subscription. Clicking the highlighted file icon will allow you to select the certificate generated by Techila Deployment Tool for Microsoft Azure.
Select the azure_management.cer file and click the Open button. The file can be found in the same directory as the Techila Deployment Tool for Microsoft Azure.

Figure 15. Select the certificate generated by the Techila Deployment Tool for Microsoft Azure. The file will be located in the same directory as the Techila Deployment Tool for Microsoft Azure
After clicking open, the management certificate will be visible as illustrated below. Click the Upload button to upload the management certificate.

Figure 16. Uploading the management certificate.
After starting the upload process, the status of the operation will be displayed at the top of the screen.

Figure 17. For a short duration after starting the upload process, the operation will be displayed.

After the certificate has been uploaded and added to your subscription, it will be visible as illustrated below.

Figure 18. A successfully added management certificate.

The certificate has now been successfully added to your Microsoft Azure subscription. The next steps will show how to link the subscription identifier to the Techila Deployment Tool for Microsoft Azure
Click the Overview item to navigate back to the overview.

Figure 19. Opening the overview.
Copy the subscription identifier to your clipboard. You can do this by selecting the value shown under Subscription ID and by using the right mouse button menu.

Figure 20. After highlighting the subscription identifier, you can copy the value to your clipboard by using the right mouse button menu or respective keyboard shortcut.
Select the Techila Deployment Tool for Microsoft Azure you launched earlier and paste the subscription identifier to the Microsoft Azure Subscription Identifier field.

Figure 21. The subscription identifier can be copied to the field by using the Ctrl-V keyboard shortcut or the right mouse button menu.
Click the Test Microsoft Azure Connection button. This will verify that the Techila Deployment Tool for Microsoft Azure is properly configured for your Microsoft Azure subscription.

Figure 22. After Testing the Management Certificate, you will be able to continue to the Deploy Server tab using the Next button.
Click the Next button to continue to the Deploy Server tab.

Figure 23. Click the Next button to continue to the Deploy Server tab.
On the Deploy Server tab, click the `Import License`button. This will open a file dialog window, which will be used to select the Techila License file.

Figure 24. Importing a License is required before deploying the Techila Server. Techila Licenses are provided by Techila Technologies.
After clicking the Import License button, use the file dialog window to select your Techila License file. The Techila License file can be recognized from the .license suffix. After selecting the file, click the Open button.

Figure 25. The file dialog window can be used to select your Techila License.
After importing your Techila License, the Licensed to field will contain the corresponding information stored in the Techila License file.

Figure 26. The Techila License information will be displayed after a successful import process.
Using the drop-down menu, choose the Microsoft Azure Region where the Microsoft Azure cloud services will be created.

Figure 27. The drop-down menu can be used to select the desired region.
After selecting the location for the services, click the Deploy/Start Server button to begin the Techila Server deployment process. This deployment process will create all necessary cloud services, databases, storage accounts and will also automatically deploy one (1) Large Azure Instance, which will be used to run the Techila Server. Please see Techila Server Costs for information on how to get an estimate for the costs of the Techila Server.

Figure 28. Clicking the Deploy/Start button will start the deployment process. This process may take several minutes.
During the deployment, a computational Project will be automatically created for testing purposes. The progress of the test Project will be indicated by a status window, which will be visible for the duration of the Project. The status of the Project will also be displayed in the text field of the Techila Deployment Tool.

After the deployment process has been completed, a message stating Techila Server Installation completed will be displayed and the Open Techila Server Web Interface button will be enabled. The password for the user admin will be displayed in the Administrator password to Techila Server Web Interface field.

Note! If you are re-deploying the Techila Server, the password will not be displayed.

The Techila Server has now been successfully deployed and is ready for use. You can now continue with deploying Techila Workers in Microsoft Azure as described in Deploying Techila Workers in Microsoft Azure.

Note! If you want, you can change the Techila Web Interface password for the admin user. This is described in the following steps.

Figure 29. A successful Techila Server deployment to Microsoft Azure.
After the Techila Server has been deployed, you can change the Techila Web Interface password for the admin user by entering a new password to the Administrator password to Techila Server Web Interface. After entering the new password, click the Set Password button.

Figure 30. Changing the Techila Web Interface password for the admin user.
After clicking the button, the message Administrator password changed will be displayed. The password has now been successfully reset.

Figure 31. A successfully changed password.

4. Deploying Techila Workers in Microsoft Azure

This Chapter contains instructions for deploying Techila Workers in Microsoft Azure. Instructions for removing Techila Workers can be found in Removing Techila Workers.

Please note that in order to deploy (or remove) Techila Workers, a deployed Techila Server must be available. Instructions for deploying the Techila Server in Microsoft Azure can be found in Deploying the Techila Server in Microsoft Azure.

Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.

Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.

Figure 32. After successfully deploying the Techila Server in Microsoft Azure, several new files have been automatically generated in the directory containing the Techila Deployment Tool for Microsoft Azure. The application can be launched by double clicking the azurearminstaller.jar file icon.
After the Techila Deployment Tool for Microsoft Azure has been launched, click the Test Microsoft Azure Connection button.

Figure 33. Performing a connection test is required before you can proceed to the other tabs in the Techila Deployment Tool for Microsoft Azure
After the connection test has been performed, click on the Deploy Workers tab to open the tab that will be used to deploy Techila Workers in Microsoft Azure.

Figure 34. After a successful connection test, the Deploy Workers tab will be available.
The Deploy Workers tab can be used to deploy Techila Workers on Microsoft Azure instances. Before any Techila Workers have been deployed, the tab should resemble the one illustrated above.

The Total value in the highlighted Total/Max field contains information on how many cores are currently used by existing Techila Worker deployments and how many cores would be used if the specified number of Techila Workers (value in the Selected Worker Count)would be deployed.

The Max value in the highlighted Total/Max field displays the maximum number of cores that can be used for Techila Worker deployments. The maximum value will depend on the core quota of your Microsoft Azure Subscription.

Figure 35. The Deploy Workers tab will contain information on the currently deployed Techila Workers.

When deploying Techila Workers, each Techila Worker deployment will be deployed to a separate cloud service. For the purposes of this Chapter however, it is assumed that you will only deploy Techila Workers to the default cloud service. More information on creating additional cloud services and doing more diverse deployments can be found in Deploying Techila Workers on Different Instance Sizes.

When deploying Techila Workers, you will need to specify three settings. Click on the links for more info.
- Techila Worker Size
- Techila Worker deployment count
- Instance shutdown delay
  
  To deploy Techila Workers, select the desired Worker Size from the drop-down menu and specify the desired number of Techila Workers in the Selected Worker Count field and specify the desired value as the shutdown delay. After specifying the settings, click the Deploy button to begin the deployment process
  
  In the example below, the Worker Size has been set to Small and the Techila Worker count to ten (10). The value of the shutdown delay has been set to 30, meaning that the instances will be automatically shut down after 30 minutes of idle time.
  
  Figure 36. After selecting the desired Worker Size and the number of Techila Workers that should be deployed, click the Deploy button to begin the deployment process.
After the deployment process has been completed, the Current Techila Worker count field will display the number of deployed Techila Workers and will match the value you specified in the previous step. The deployed Techila Workers can now be used in computational Projects.

In the example below, 10 Techila Workers have been deployed on Small Microsoft Azure Instances. After you have deployed Techila Workers, the Worker Instance Size drop-down menu will be disabled and all new Techila Workers will be deployed on the same Microsoft Azure instance size. You can now continue with installing Bundles on the Techila Server as described in Installing Bundles.

Figure 37. After the Techila Workers have been deployed, the Current Worker Count and Status columns will display information on the number of deployed Techila Workers.

4.1. Deploying Techila Workers on Different Instance Sizes

This Chapter contains instructions on how to deploy Techila Workers on different instance sizes using multiple cloud services.

Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.

Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.
Click the Test Microsoft Azure Connection button

Click the Deploy Workers tab. In the example below, there are already 10 Techila Workers deployed on Small Microsoft Azure instances. These Techila Workers are deployed in one (1) cloud service, which is the default cloud service used in the Techila Deployment Tool. Click the Add Service button to add a new cloud service.

Figure 38. In this example, the default cloud service already contains 10 deployed Techila Workers on Small Microsoft Azure instances.
After clicking the Add Services button, a new line will be displayed containing the information of the new cloud service. This cloud service can be used to deploy Techila Workers on different instance sizes (or the same size) than the default cloud service.

Figure 39. After clicking the Add Services button, a new cloud service will be displayed.
Specify the following settings for the new cloud service.
- Techila Worker Size
- Techila Worker deployment count
- Instance shutdown delay
  
  After defining the settings, deploy the Techila Workers by clicking the Deploy button for the new cloud service.
  
  Note! If you have added multiple new cloud services and wish to deploy Techila Workers to all of these cloud services, click the Deploy All button.
  
  In the example below two (2) Techila Workers will be deployed on Medium instances.
  
  Figure 40. Deploying Techila Workers to a new cloud service is done in a similar manner as with the default cloud service.
After the Techila Workers have been deployed to the new cloud service, the information will be updated in the Current Worker Count and Status columns. These Techila Workers can now be used in computational Projects.

Figure 41. After the Techila Workers have been deployed, the information will be automatically updated for the cloud service.

5. Installing Bundles

This Chapter contains instructions for installing Bundles on the Techila Server. Please note that in order to install Bundles, a deployed Techila Server must be available. Instructions for deploying the Techila Server in Microsoft Azure can be found in Deploying the Techila Server in Microsoft Azure.

Installed Bundles will be automatically transferred to Techila Workers if components are required in the computational Job that is assigned to the Techila Worker. The Bundle transfer process from the Techila Server to the Techila Workers may take several minutes depending on the size of the Bundle, but will only need to be performed once.

After the Bundle has been transferred to Techila Workers, the Techila Workers will automatically re-use the Bundle in subsequent computational Projects. Please note that if the Microsoft Azure Instances hosting the Techila Workers are restarted, the Bundles will need to be transferred again from the Techila Server to the Techila Workers. This is due to the stateless nature of Techila Workers in a Microsoft Azure environment.

There are two different methods for installing Bundles:

Installing Runtime Bundles. This is the default method, which is typically used when deploying a Techila Distributed Computing Engine (TDCE) environment in Microsoft Azure.
Installing Custom Bundles. This method is typically used only when you create your own Runtime Bundles or when you receive Bundles from Techila Technologies.

5.1. Installing Runtime Bundles

This Chapter contains instructions for installing Runtime Bundles that are available in the Bundle list. Runtime Bundles contain runtime components that are required when performing computations using the following programming languages.

R
Python
1. Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.
  
  Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.
2. Click the Test Microsoft Azure Connection button
3. Click the Install Bundles tab
  
  The Install Bundles tab contains a list of Bundles that can be installed on the Techila Server. Bundles can be marked for installation by ticking the checkboxes of the Bundles you wish to install. The naming convention of the Bundles is typically of the following format:
  
  <Programming Language> <Version> <Processor Architecture>
  
  Figure 42. The appearance of this tab will vary, depending on which Bundles are currently available. In this example, Bundles containing the runtime components for R and Python can be selected and installed.
  
  Runtime Bundles are version and programming language specific. Please see following links for more information.
  - Programming language specific Bundles
  - Version specific Bundles
4. Select the Bundles you wish to install by ticking the checkboxes and click the Install and Test Bundles button to install the Bundles. In the example below, the Bundle containing the runtime components for R version 2.14.2 will be installed.
  
  Figure 43. Bundles are installed by ticking the checkboxes of the Bundles you wish to install and clicking the Install and Test Bundles button. Multiple Bundles can be installed by ticking several checkboxes.
5. After clicking the Install and Test Bundles button, the installation process will be started. This may take several minutes. After the Bundles you selected have been installed, a message stating that the installation process was successful will be printed and the selections you made will be grayed out.
  
  The Bundles that were installed will now be automatically used in computational Projects created by End-users. You can now continue with adding End-Users to the TDCE environment as described in Adding End-Users.
  
  Figure 44. After successfully installing a Bundle, the selection will be grayed out. The runtime components in the Bundle will now be automatically used in computational Projects that have dependencies to the Bundle. In this example, the support for R version 2.14.2 was added.

5.2. Installing Custom Bundles

This Chapter contains instructions for uploading Bundles from your computer to the Techila Server in Microsoft Azure. The procedure described here can be used to e.g. upload Runtime Bundles that are not included in the Bundle list. Typical usage scenarios include uploading Bundles received from Techila Technologies and uploading Bundles that you have created using the Techila Bundle Creator Tool.

Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.

Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.
Click the Test Microsoft Azure Connection button
Click the Install Bundles tab
Click the Install Bundle File button. This will open a file dialog window which can be used to select the Bundle you wish to upload to the Techila Server.

Figure 45. Clicking the highlighted button will open a file dialog window.
Using the file dialog window, select the Bundle you wish to upload to the Techila Server. Click the Open button to upload the Bundle to the Techila Server.

Figure 46. Select the Bundle you wish to upload. In this example a file called example-bundle.jar has been selected.
After starting the upload process, a Status Window will displayed containing information on the upload process.

Figure 47. Duration of the upload process will depend on the size of the Bundle and available network bandwidth.
After the Bundle has been uploaded, the corresponding message will be displayed in the message area. The Bundle can now be used in computational Projects.

Figure 48. A successfully uploaded Bundle.

6. Adding End-Users

This Chapter contains instructions for adding End-Users to the Techila Distributed Computing Engine (TDCE) environment. All End-Users in the TDCE environment will require a personal keystore containing their End-User Key. This End-User Key will be used for signing Bundles when a computational Project is created.

In addition to the actual keystore file, the End-User will also need the password of the keystore. This password will be defined during the keystore file creation process described in this Chapter.

In addition to the keystore and keystore password, the End-User will also need the hostname of the Techila Server when configuring the techila_settings.ini file as described in Getting Started. The hostname of the Techila Server will be displayed in the Techila Deployment Tool for Microsoft Azure after clicking the Test Microsoft Azure Connection button in the Certificate Management tab.

Please note that the protocol (https://) and trailing forward slash need to be removed as illustrated in the example below.

Parameter	Example Value
Techila Server Web Interface	https://techila-<Your Azure Subscription ID>.cloudapp.net/
Hostname	techila-<Your Azure Subscription ID>.cloudapp.net

Parameter

Example Value

Techila Server Web Interface

https://techila-<Your Azure Subscription ID>.cloudapp.net/

Hostname

techila-<Your Azure Subscription ID>.cloudapp.net

Note! After you have successfully added End-Users to the TDCE environment, please refer to the Getting Started for instructions on how to configure your Techila SDK to connect to the Techila Server. Programming language specific preparation steps are described in the document series "Techila Distributed Computing Engine with". These documents can be found here.

Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.

Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.
Click the Test Microsoft Azure Connection button.
Click the Create Users tab.

Figure 49. The hostname of the Techila Server will be displayed in the message area. If you have performed multiple operations since you launched the Techila Deployment Tool, you might need to scroll back in order to see the highlighted message.

This tab can be used to launch the Techila Keytool, which can be used to create keystore files containing End-User Keys. Click the Open Techila Keytool button to launch the Techila Keytool.

Figure 50. Keystore files containing End-User Keys are generated with Techila Keytool.
After launching the Techila Keytool, the main window will be displayed. Click the Create a new end-user key and sign it button.

Figure 51. The Techila Keytool can be used to create End-User Keys (used for signing Bundles) and Administrator Keys (used for creating End-User Keys).
After clicking the button, a new window will be displayed. Specify the full name of the End-User by filling in the First Name and Last Name fields.

The Alias field will be automatically updated to match the values entered to the First Name and Last Name fields.

Note! The End-User can use the value shown in the Alias field to log in to the Techila Web Interface.

The Validity period field displays how long the End-User Key will be valid. By default, all End-User Keys will be valid for 365 days. A different validity period can be defined by entering the desired time period in the field.

Using the Keystore Password and Verify Password fields, specify the keystore password that will be used to protect the keystore file. Note! This is the password that the End-User will need to enter when creating computational Projects in the TDCE environment.

After you have filled all necessary fields, click the Generate Key button to continue.

Figure 52. After specifying the End-User`s name, the Generate Key button will be enabled. In this example, an End-User with the name Demo User would be added to the Techila Distributed Computing Engine environment.
After clicking the Generate Key button, a new window will be displayed. This window will be used to specify the following settings regarding the keystore file:
- The name of the keystore file
- The destination directory where the keystore file will be stored
  
  Click the Save button to create the keystore file.
  
  Figure 53. This window can be used to specify a name for the file and the location where it will be stored. In this example, the keystore file would be named demouser.jks and stored in a directory called AzureDeployment.
A new window will be displayed that contains information on where the keystore was stored on your computer. The key will also be automatically sent to the Techila Server and a corresponding transfer status message will be displayed in the message area. If the transfer was successfully completed, the following message should be displayed:

Key has been sent to the server successfully.

Figure 54. View after the key has been successfully transferred to the Techila Server

The End-User has now been successfully added to the TDCE environment. The End-User Key that was created can now be used for signing Bundles, which will be required when creating computational Projects.

In order for the End-User to access the TDCE environment they will need the following file:
- The keystore file that was created (demouser.jks in the example)
  
  And the following information:
- The keystore password
- The hostname of the Techila Server. The hostname of the Techila Server will be displayed in the message area of the Techila Deployment Tool each time the Test Microsoft Azure Connection button is clicked.
Click the Close button to close the Techila Keytool or the Back to Main Menu button if you wish to add more End-Users to the TDCE environment.

7. Components Created During the Deployment Process

This Chapter contains a description of Techila Distributed Computing Engine (TDCE) components that are automatically created in Microsoft Azure when deploying the Techila Server and Techila Workers.

These components are:

Techila Server Database
Techila Server Storage Account
Cloud Services for the Techila Server and Techila Workers
Techila Network

An overview of these components is visible under All Items as illustrated below.

Figure 55. An overview of the Techila Distributed Computing Engine components that have been deployed to Microsoft Azure. If you have created several cloud services for Techila Worker deployments, you will have more cloud services visible with a techilaworker prefix

7.1. Techila Server Database

The Techila Server database is used to store information about the Techila Distributed Computing Engine (TDCE) environment (e.g. information about Techila Workers, Projects, Jobs, etc.). The server hosting the Techila Server database can be viewed by clicking on the SQL server in the All resources view. To view the actual database, click on the SQL database.

Figure 56. The Techila Server database components.

7.2. Techila Server Storage Account

The Techila Server storage account contains a 500 GB VHD image, which is used to store data that is generated when the TDCE system is used. This data includes files containing Project results and Bundles containing computational data. The storage account can be viewed by clicking on the Storage account in the All resources view.

Figure 57. The storage account for the Techila Server.

7.3. Cloud Services for the Techila Server and Techila Workers

During deployment, several cloud services will be created; one for the Techila Server instance and one or more for the Techila Worker instances. The number of cloud services created for Techila Workers depends on how many cloud services you have created using the Techila Deployment Tool.

The cloud service of the Techila Server can be identified by the techila- prefix and the cloud services of Techila Workers by the techilaworker- prefix.

Cloud service details can be viewed by clicking on the Cloud service (classic) items in the All resources view.

Figure 58. Cloud services can be viewed by clicking on the Cloud services (classic) items.

7.4. Techila Network

The TechilaNET network contains network configuration settings used by the Techila Distributed Computing Engine environment. The network details can be viewed by clicking on network in the All Resources view.

Figure 59. Opening the network details.

8. Removing Components from Microsoft Azure

This Chapter contains instructions for deleting the various Techila Distributed Computing Engine components from Microsoft Azure that were created when setting up the Techila Distributed Computing Engine environment.

8.1. Removing Techila Workers

This Chapter contains instructions on how to delete Microsoft Azure Instances that are hosting Techila Workers. Terminating all instances will stop hourly costs from being incurred by instances that are hosting the Techila Workers.

Note! After removing Techila Workers, verify that the Microsoft Azure Instances were successfully deleted. The status of Microsoft Azure Instances can be viewed using the Microsoft Azure Management Portal.

Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.

Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.
Click the Test Microsoft Azure Connection button
Click the Deploy Workers tab

When removing Techila Workers, the following approaches can be typically applied:

Removing ALL deployed Techila Workers in ALL cloud services
Removing ALL deployed Techila Workers in one cloud service
Reducing the number of deployed Techila Workers in a cloud service

Please follow the steps applicable to your situation.

8.1.1. Removing ALL deployed Techila Workers in ALL cloud services

To remove ALL currently deployed Techila Workers, click the Shutdown All button.

Figure 60. The Shutdown All button can be used to remove all currently deployed Techila Workers.

After the operation has been completed, the changes will be visible in the Deploy Workers tab as illustrated below.

Figure 61. After the Techila Workers have been removed, the Status column will display the message Not Deployed

After removing all Techila Workers, verify that the instances were deleted using the Azure portal. Instructions can for this can be found in Verifying that instances were deleted.

8.1.2. Removing ALL deployed Techila Workers in one cloud service

To remove ALL currently deployed Techila Workers in one cloud service, click the Shutdown button for the specified cloud service.

In the example below, clicking the highlighted button would remove all Techila Workers deployed to the default cloud service. Techila Workers deployed to the second cloud service would remain unaffected.

Figure 62. The Shutdown buttons can be used to remove Techila Workers deployed to the applicable cloud service. This also means that you can choose a new instance size when deploying Techila Workers to the cloud services.

After the operation has been completed, the changes will be visible in the Deploy Workers tab. In the example below, the Status column displays Not Deployed, indicating that Techila Workers in the cloud service have been removed.

Figure 63. In this example, the default cloud service does not have any Techila Workers deployed. This also means that you can choose a new instance size when deploying Techila Workers to this cloud service.

After removing Techila Workers, verify that the instances were deleted using the Azure portal. Instructions can for this can be found in Verifying that instances were deleted.

8.1.3. Reducing the number of deployed Techila Workers in a cloud service

To reduce the number of deployed Techila Workers in a cloud service, specify the new Techila Worker count in the Selected Worker Count Field. The value entered to the field will determine how many Techila Workers will remain deployed after the operation. After you have entered the value, click the Deploy button for the cloud service.

In the example below, there are currently ten (10) Techila Workers deployed in the first cloud service and the Selected Worker Count field contains the value three (3). These settings could be used to remove seven (7) of the Techila Workers that are currently deployed in the cloud service.

Figure 64. The number of deployed Techila Workers can be reduced by entering a smaller number in the Techila Worker Count field. This operation only affects the specified cloud service, meaning the Techila Worker count in other cloud services remains unchanged.

After reducing the amount of Techila Workers, verify that the instance count has been successfully updated to the Azure portal. Instructions can for this can be found in Verifying that instances were deleted.

8.2. Removing the Techila Server

This Chapter contains instructions on how to delete the Microsoft Azure Instance that is hosting the Techila Server. Terminating the instance will stop hourly costs from being incurred from the instance hosting the Techila Server. The state of the Techila Server is preserved in the database, meaning deleting the Techila Server instance will not delete your Techila user account and/or other settings. You can redeploy the Techila Server later by using the Techila Deployment Tool.

Note! Removing the Techila Server will also remove all Techila Workers that are currently deployed.

Note! After removing the Techila Server, verify that the Microsoft Azure Instances were successfully deleted. The status of Microsoft Azure Instances can be viewed using the Microsoft Azure Management Portal.

Navigate to the directory containing the Techila Deployment Tool for Microsoft Azure and launch the application by double clicking on the azurearminstaller.jar file icon.

Note! If the Techila Deployment Tool is already running and you have performed a connection test, you can continue from Step 3.
Click the Test Microsoft Azure Connection button
Click the Deploy Server tab
Click the Shutdown Server button to remove the Techila Server

Figure 65. The Shutdown Server button can be used to delete the Microsoft Azure Instance hosting the Techila Server
After the Techila Server has been removed, a Shutdown Verification window will be displayed. Click the OK button to open the Microsoft Azure Management Portal.

Figure 66. Clicking the OK button will open the Microsoft Azure Management Portal.
In the Azure portal, open the All resources view and click on the Techila Server’s cloud service to check that the instance hosting the Techila Server has been terminated successfully. The name of this cloud service starts with techila.

Figure 67. Opening the cloud service used to host the Techila Server.

If the instance hosting the Techila Server has been successfully deleted, the view should resemble the one illustrated below.

Note! If the instance was not deleted successfully, delete the instance by clicking the Delete icon at the top of the page and select Delete the production deployment.

Figure 68. View when no instances are deployed.
Return to the All resources view and check the status of all cloud services that were used for deploying Techila Workers. The names of these cloud services start with techilaworker.

Figure 69. Opening a cloud service used to host Techila Workers.

If all instances hosting Techila Workers have been successfully deleted from the cloud service, the view should resemble the one illustrated below.

Note! If the instances were not deleted successfully, delete the instances by clicking the Delete button at the bottom of the page and select Delete the production deployment. Screenshots on how to manually delete instances can be found in Verifying that instances were deleted.

Figure 70. View when no instances are deployed.
If you had created more cloud services for Techila Workers, also check that instances in those cloud services have been deleted.

8.3. Removing all Components Created During Deployment

This Chapter contains instructions on how to remove all Techila Distributed Computing Engine (TDCE) components from Microsoft Azure.

Note! Removing all deployed TDCE components will also remove all data from the Techila Server, meaning the state of the Techila Server will be lost. The steps described in this Chapter are typically only required if you wish to discontinue the use of your current Techila Server or to re-deploy a Techila Server to a clean Microsoft Azure environment.

In the Azure portal, open the `All resources view.

Figure 71. Techila Distributed Computing Engine components.
Tick the checkbox next to each Techila Distributed Computing Engine component and click the Delete icon.

Figure 72. Ticking the checkboxes and clicking delete.
Confirm the deletion by entering yes and click delete to delete selected components.

Figure 73. Confirming the deletion.
After the components have been deleted, the view should resemble the one shown below.

Figure 74. An environment with no Techila Distributed Computing Engine components.

Note! If the view does not update automatically, refresh the view in your browser (typically done with F5)
The next steps describe how to remove the Management Certificate that has been linked to your Microsoft Azure Subscription. Note! If you plan on re-deploying the Techila Server in Microsoft Azure by using the same Microsoft Azure Subscription, removing the Management Certificate is not required. The same Management Certificate can be used in the re-deployment process.

Click on Subscriptions.

Figure 75. Click on Subscriptions
Select the subscription you are using.

Figure 76. Select your subscription.
Select Management Certificates, tick the checkbox next to the Techila management certificate and click the Delete icon.

Figure 77. Select and delete the management certificate.
When prompted, confirm the action.
After the Management Certificate has been deleted, it will no longer be displayed in the list.

Figure 78. In this screenshot, no Management Certificates have been linked to the subscription.
All TDCE components have now been removed from your Microsoft Azure environment.

9. Appendix

9.1. Techila Web Interface Login Credentials

After deploying the Techila Server in Microsoft Azure, the Techila Web Interface can be accessed with the login credentials shown below:

Techila Web Interface

Techila Web Interface
Login	admin
Password	Displayed in the Techila Deployment Tool `Deploy Server` tab in the field `Administrator password to Techila Server Web Interface`. Password can also be reset using the `Set Password` button on the `Deploy Server` tab.

admin

Password

Displayed in the Techila Deployment Tool Deploy Server tab in the field Administrator password to Techila Server Web Interface. Password can also be reset using the Set Password button on the Deploy Server tab.

Note! It is recommended that you change the default password as soon as possible.

9.2. Techila Server Costs

When deploying a Techila Distributed Computing Engine environment using the Techila Deployment Tool, several different components will be automatically created in your Microsoft Azure environment. Please see Components Created During the Deployment Process for a description of the components that are created.

Out of these components, the following components will incur billing for the Techila Server while it remains running:

Techila Server database
Techila Server storage
Techila Server instance

If you shut down the Techila Server instance, you will no longer be billed for the instance it as long as it remains shutdown. Billing for the Techila Server will resume when you restart it. Billing for the database and storage will continue even if you stop the Techila Server.

The following steps describe how you can get a cost estimate for these Techila Server by using the Microsoft Pricing calculator. The values in the cost estimate have been calculated by assuming the Techila Server remains running at all times (730 hours per month).

Using your web browser, open the following link to access the Microsoft Pricing Calculator.

https://azure.com/e/4d15983f025c4501a62096f1cd106d09
When prompted, click Confirm to import the estimate settings.
After importing the settings, click on the Estimates tab to view the estimate.
After navigating to the Estimates tab, you can see the cost estimate for the Techila Server components. If you want, you can change the region of the components to view the cost in the region you prefer. To do this, simply click on the items in the list and use the Region drop down menu to change the location.

9.3. Updating the Techila Deployment Tool

When a new version of the Techila Deployment Tool is released, please follow the steps below to update your Techila Deployment Tool:

If you have the Techila Deployment Tool running, close the tool.
Download the new version of the Techila Deployment Tool from the Techila Extranet
Extract the new azurearminstaller.jar file over your existing azurearminstaller.jar file.

The new version is now ready for use. Previously generated files and configuration settings will be automatically used by the new version of the Techila Deployment Tool.

9.4. Techila Deployment Tool Command Line Parameters

When launching the Techila Deployment Tool from the command line, additional command line parameters can be defined that control the deployment process. The general syntax for defining command line parameters is shown below:

java -jar azurearminstaller.jar --<parameter> <value>

The following parameters can be defined for the Techila Deployment Tool:

Parameter	Description	Example value
port	Defines the port that will be used as the management channel.	80

Parameter

Description

Example value

port

Defines the port that will be used as the management channel.

For example, the following syntax could be used to launch the Techila Deployment Tool and deploy a Techila Server which uses port 80 (instead of default 25001) for the management channel.

java -jar azurearminstaller.jar --port 80

9.5. Techila Worker Size

The Selected Worker Size drop-down menu can be used to select the Microsoft Azure Instance size that will be used when deploying Techila Workers. Selections made in the drop-down menu will determine the maximum number of Techila Workers that can be deployed. This is because different Microsoft Azure Instance sizes will require a different number of cores per instance. The maximum number of Techila Workers you can deploy will be determined by the core quota of your Microsoft Azure Subscription and the selected Techila Worker Size.

When selecting the Techila Worker Size, it is recommended that you choose the Worker Size that best suits the distributed application(s) you plan on computing on the Techila Workers. Most important properties include the network bandwidth, amount of memory, amount of available disk space and the instance price per hour. Information on these properties can be found on the following webpage:

https://azure.microsoft.com/en-us/documentation/articles/cloud-services-sizes-specs/

As a rule of thumb, CPU intensive applications that only require small network transfers and do not require large amounts of memory or IO activity can be performed on smaller instance sizes. As the amount of transferred data, required memory and IO activity increase, so does the recommended instance size.

The example below shows the maximum number of Techila Workers that can be deployed on different instance sizes when using a Microsoft Azure Subscription that has 16 cores available in the core quota. This corresponds to a situation where the Microsoft Azure core quota is 20 cores and the Techila Server has been deployed on a Large Instance (taking up 4 cores).

Figure 79. The selected Worker Size affects the maximum number of Techila Workers that can be deployed. In this example, the Microsoft Azure Subscription core quota has 16 cores available. The CPU Core Count column displays information on how many CPU cores are required by the Techila Worker deployment.

Note! The Selected Worker Size drop-down menu will only be available if there are no currently deployed Techila Workers for the selected cloud service.

Figure 80. The Selected Worker Size drop-down menu can be used to set the Instance size.

After you have deployed Techila Workers to a cloud service, all additional Techila Workers deployd to this cloud service will be automatically deployed on the same Instance size. If you wish to deploy Techila Workers on different Instance sizes at the same, please see instructions in Deploying Techila Workers on Different Instance Sizes.

9.6. Techila Worker deployment count

The Selected Worker Count field can be used to specify the number of deployed Techila Workers. In the example below, the deployment count is set to ten (10) and the Worker Size has been set to Small. These settings could be used to deploy ten (10) Techila Workers on Small Microsoft Azure Instances.

Figure 81. The value defined in the Selected Worker Count field is used to control the number of Microsoft Azure Instances that should be deployed as Techila Workers.

If you have already deployed Techila Workers, specifying a larger value will enable you to increase the number of deployed Techila Workers. In the example below, the current Techila Worker deployment count is ten (10) and the Selected Worker Count field contains the value 15. These settings could be used to deploy five additional Techila Workers, bringing the total number of deployed Techila Workers to 15.

Figure 82. Additional Techila Workers can be deployed by entering a larger value to the field. With the settings shown in this example, the Techila Worker deployment count would be increased from 10 to 15.

If you enter a larger number to the Selected Worker Count field than allowed by your Microsoft Azure core quota, the Total/Max field will be highlighted in red and the Deploy button will be disabled as illustrated below.

Figure 83. When the deployment would exceed your Microsoft Azure core quota, the Deploy button will be disabled.

Instructions for reducing the number of deployed Techila Workers can be found in Removing Techila Workers.

9.7. Instance shutdown delay

The Shutdown Idle Delay (minutes) field can be used to specify a shutdown delay for the Techila Workers that are deployed in the cloud service. Specifying a shutdown delay will automatically shutdown the instances hosting the Techila Workers if they remain idle for the specified time. It is however strongly recommended that the deletion of the instances is always verified using the Microsoft Azure management portal.

The shutdown counter will be activated when computational Jobs are detected on the Techila Workers belonging to the specified cloud service. Computational activity is polled at five second intervals, meaning extremely short test Projects might not trigger the shutdown counter. Please note that this also means that if the Techila Workers have only been deployed, but no Projects have been created, the instances will not be automatically shutdown.

The counter will be automatically reset when a new computational Job is detected on any of the Techila Workers belonging to the specified cloud service. It is also possible to modify the shutdown delay of existing Techila Worker deployments by entering a new value to the field and clicking the Deploy Workers button.

In the example below, the shutdown delay has been set to 30 minutes and the Techila Worker deployment count to 10. This means that all Techila Workers will be automatically terminated if they remain idle for 30 minutes after a Job has been detected on any of the Techila Workers.

Figure 84. Specifying that Techila Workers should be shut down after 30 idle minutes.

Automatic shutdown can be disabled by entering the value 0 field as illustrated below.

Figure 85. Specifying that Techila Workers should not be automatically shutdown

It is also possible to modify the Shutdown Idle Delay (minutes) value of existing Techila Worker deployments. This can be done by entering the new desired value to the field and clicking the Deploy button for the specified cloud service.

9.8. Programming language specific Bundles

Bundles containing the runtime components are programming language specific, meaning you will need to install the Bundles for the programming languages you plan on using. For example, if you wish to perform computations using the R programming language, you will need to select and install the Bundle containing the R runtime components.

In the example below, the Bundle containing the runtime components for R version 2.14.2 has been selected.

Figure 86. Support for programming languages can be added by installing applicable Bundles.

9.9. Version specific Bundles

Bundles containing the runtime components are version specific. It is therefore recommended that you install Bundles that contain the same version of the runtime components as used by the End-User. The versions of the runtime components stored in the Bundles are visible in the list displayed in the Install Bundles tab.

In the example below, the version of the runtime components in the R Bundle has been highlighted. The runtime components correspond to the R version 2.14.2.

Figure 87. The version of runtime components in Bundles should match the development environment versions used by the End-Users.

9.10. Verifying that instances were deleted

This Chapter provides instructions for verifying that Techila Worker instances were successfully deleted from Microsoft Azure.

In the Azure portal, open the All resources view and click on the Techila Worker cloud service that can be identified by the techilaworker prefix.

In the cloud services detail view, verify that there are no instances running. If there are no instances running, a text stating "You have nothing deployed to the current environment will be displayed". Please note that if you only reduced the amount of instances running (meaning you did not shut them all down), then you should see the new, smaller instance count in the view.

If you have multiple Techila Worker cloud services, perform the same check for each cloud service.

Note! If you shut down all Techila Workers but still see Techila Workers running, delete the instances by using the Azure portal. To delete instances, click the Delete icon.

After clicking the Delete icon, an additional section will be displayed. Tick the Production Deployment checkbox and click the Delete button.

Deleting the instances may take several minutes. After the instances have been deleted, the view should resemble the one shown below.