Quick Guide - Techila Distributed Computing Engine in AWS Marketplace

1. Introduction

Techila Distributed Computing Engine is a distributed computing middleware and management solution that brings an on-demand supercomputer to every desk. The solution is designed to save the time of users and IT experts, to solve challenges related to parallel application development and deployment, and to speed up the idea-to-deployment cycle.

This document is a quick start guide that describes how you can easily set up a TDCE environment in AWS Marketplace and run your first computational Project. For more detailed information, please see the full user guide. The full user guide also describes various other topics not covered here, such as using GPU instances and making custom images.

2. Starting the Techila Server

This chapter gives a short description on how to start the Techila Server, for more detailed information please see the this chapter in the user guide.

Using your web browser, navigate to the AWS Marketplace:

https://aws.amazon.com/marketplace
To locate the Techila Server - Enterprise Edition offering, either use the search box at the top of the marketplace view or alternatively, click the link below to open the product page in your browser.

https://aws.amazon.com/marketplace/pp/prodview-rha2rbv7ls6cc
Click the Continue to Subscribe button.
Agree to the terms by clicking the Accept Terms button.
Wait for the subcription to activate. After it has been activated, click the Continue to Configuration button.
Next you will need to select either Techila Distributed Computing Engine or Techila Distributed Computing Engine with Amazon Aurora. If you choose Techila Distributed Computing Engine, Techila Server data will be stored in a PostgreSQL database on the Techila Server instance. If you select the Amazon Aurora option however, your Techila Server will use an Aurora RDS to store Techila Server data.

Note! Aurora RDS is not available in the following regions:
- eu-north-1
- ap-east-1
- sa-east-1
- me-south-1
- us-gov-west-1
- us-gov-east-1
  
  If you want to use TDCE in these regions, you need to select Techila Distributed Computing Engine. You cannot use Amazon Aurora.
  
  More information about Amazon Aurora can be found in the following link:
  
  https://aws.amazon.com/rds/aurora/
After selecting the Delivery Method option, you will be presented with new dropdown menus that allow you to choose the Version and Region. Unless otherwise specified, it is always recommended to use the latest version. The region can be selected so that it matches you geographical location and other usage criteria.

After selecting the version and region, click the Continue to Launch button.
Review your settings and click the Launch button.
The following steps include filling out the CloudFormation template that will be used to deploy the environment. Click the Next button to continue.
The fields in the CloudFormation template will be slightly different depending on if you chose the Techila Distributed Computing Engine or Techila Distributed Computing Engine with Amazon Aurora option earlier. Using the CloudFormation template, you will be have to define a name (1 in the screenshot) for the stack and choose the SSH key (2 in the screenshot) you want to use. You will also need to define a password (3 in the screenshot) that you will use to access the Techila Configuration Wizard.

If you do not have an AWS SSH key, you will need to create (or import) one so you can select it in 2. Instructions for this can be found using the link below:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html

This SSH key will allow you to establish an SSH connection to the Techila Server or Techila Workers that have a Linux operating system. This is typically only needed when performing troubleshooting. Normal usage does not require establishing any SSH connections to either the Techila Server or Techila Workers.

After setting the values, click Next to continue to the next page.
On this page, everything can left at the default values. Click Next to continue.
On the final page, tick the checkboxes at the bottom of the page and click Create Stack to create the stack.
The view should now automatically change to display information about the stack creation process. This process can take several minutes.

Please note that the page may not refresh automatically. You will need to manually refresh the page to update the information displayed.

When the deployment is complete, a message stating CREATE_COMPLETE will be displayed.
Before proceeding, you will need to subscribe also to the Techila Worker images. This can be done using the following links:
- https://aws.amazon.com/marketplace/pp/prodview-47et3r36j7hqa
- https://aws.amazon.com/marketplace/pp/prodview-pwqg4edp4ntos
  
  Subscribe to these product and accept the terms just as above for the Techila Server in steps 1-4. But do not proceed to launching the worker images individually. To launch worker, use the Techila Dashboard on the Techila Server as described below.
The Techila Server deployment is now completed. Please continue with the next chapter which describes Accessing the Techila Dashboard

3. Accessing the Techila Dashboard

This chapter gives a short description on how to access the Techila Dashboard. For more detailed information please see the this chapter in the user guide.

Click on the URL displayed in the Outputs tab.
Opening the Techila Dashboard might display a credential warning. The appearance of the warning will depend on what browser you are using. Accept the certificate and continue.
Enter admin as the username. In the password field, enter the password you defined when creating the stack. Click the Log in button to log in.

After logging in, you will be presented the first page of the Techila Configuration Wizard. This page will be used to download the application plugins (delivered in a file called TechilaSDK.zip) from the Techila Server. This is described in the following chapter.

4. Downloading and Configuring the Application Plugins

This chapter gives a short description on how to download and configure the application plugins, for more detailed information please see the this chapter in the user guide.

The application plugins are delivered in a file called TechilaSDK.zip. In addition to containing the application plugins for e.g. MATLAB, Python and R, the TechilaSDK.zip contains ready-to-run examples for various programming languages. The application plugins package is also referred to as the "Techila SDK".

Click the highlighted link to start downloading the application plugins (TechilaSDK.zip) to your computer.
After downloading the file, click the Continue button.

After clicking the Continue button, the second page of the Techila Configuration Wizard will be displayed. This page contains a link to a help page, which contains instructions on how to configure the application plugins on your computer.

Click the highlighted link to open a help page containing links on how to configure the application plugins on your computer.

Please follow the instructions on the help page to configure the application plugins for the programming language you are using before continuing.

For reference, the below table contains the direct links to programming language specific instructions describing the application plugin configuration steps:

Table 1. Configuration Instructions
Programming Language	Link
Python	https://www.techilatechnologies.com/techila-for-aws-marketplace/install-and-configure-techila-for-python
R	https://www.techilatechnologies.com/techila-for-aws-marketplace/install-and-configure-techila-for-r-programming-language
MATLAB	https://www.techilatechnologies.com/techila-for-aws-marketplace/install-and-configure-techila-for-matlab

After configuring the Techila SDK on your computer, select your web browser and navigate to the third page of the Techila Configuration Wizard using the Continue button.

5. Increasing Computing Throughput

This chapter gives a short description on how to increase the amount of computing capacity available. For more detailed information please see the this chapter in the user guide.

Note As Techila Distributed Computing Engine is purchased via the AWS Marketplace, the Techila Worker software costs are rounded up to the next full hour for each instance. If you deploy 10 Techila Worker instances for 20 minutes, you will be billed for 10 hours of Techila Worker software usage. More details can be found in the following AWS document:

https://docs.aws.amazon.com/marketplace/latest/userguide/pricing-ami-products.html

The third page of the Techila Configuration Wizard is used to adjust the computing throughput by starting and shutting down Techila Worker VM instances. Tip! Add this page to your browser’s bookmarks so you can easily access it later.

To start Techila Workers select the Techila Worker VM instance type, choose the operating system and define how many Techila Worker VM instances you want to start.

Note! You must not exceed your AWS EC2 instance quota. There are no quota checks in the start process, meaning the system will not work as intended if you try to start more Techila Workers than allowed by AWS EC2 quota. Instructions for viewing quotas can be found in the AWS documentation:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html

After defining the properties listed above, click the Start button to start the Techila Workers.

If you already have Techila Workers that are running, you can increase (or decrease) the amount of Techila Workers by entering the desired number in the field and clicking the Update button.

Techila Workers will be started when the Start button is clicked.

After clicking the Start button, the deployment process for the Techila Workers will start. Progress information will be displayed at the bottom of the page, on the Worker Status and Deployment Status lines.

The Worker Status line contains information about the status of Techila Workers. The initial value when no instances have been deployed will say 0 / 0. When you start instances, the values on this line will show how many Techila Workers are ready for computations. For example, if you have started 5 instances and the value says 3 / 5, it means that 3 Techila Workers are currently ready and can be used for computing purposes.

When all Techila Workers are ready, the values should be the same, e.g. 5 / 5 as illustrated below.

6. Running Computations

Verify functionality by executing one of examples included in the Techila SDK. Simple examples for R, Python and MATLAB can be found in the following Techila SDK directories:

Table 2. "Hello World" Examples
Programming languege	"Hello World" Example
Python	https://www.techilatechnologies.com/techila-for-aws-marketplace/python-hello-world
R	https://www.techilatechnologies.com/techila-for-aws-marketplace/r-hello-world
MATLAB	https://www.techilatechnologies.com/techila-for-aws-marketplace/matlab-hello-world

Additional information on how to use the Techila Distributed Computing Engine functions from different programming languages can be found using the following links. These documents can be used as reference material on how to implement different TDCE features when modifying your own code to run in TDCE.

Note! After starting Techila Workers, program specific runtime components will be transferred from the Techila Server to the Techila Workers during the first Project. Depending on the programming language you are using, the transfer process may take several minutes. After the runtime components have been transferred, they will be stored on the Techila Workers' hard disks and will be stored there until the Techila Worker VM instances are deleted. During subsequent Projects, the runtime components will be used from the Techila Worker’s hard disk, meaning there will be no network transfer delay.

After you have completed processing computational workloads, it is recommended to delete ALL Techila Workers VM instances to prevent unnecessary costs from being incurred. Instructions for deleting Techila Worker VM instances can be found in Decreasing Computing Throughput.

7. Decreasing Computing Throughput

This chapter gives a short description on how to decrease the amount of computing capacity available. For more detailed information please see the this chapter in the user guide.

Note! If you are deleting instances, it is strongly recommended that the deletion of the instances is always verified using the AWS Management Console located at:

https://console.aws.amazon.com

Note! Do not delete the Techila Server, as it will be needed again when you want to start new Techila Workers.

To delete ALL Techila Worker VM instances, click the Stop button.

8. Next Steps

Congratulations! When you reach this point, you should have successfully set up a TDCE environment and run your first computational Project.

As mentioned earlier, this document is a quick start version of the full user guide. If you want to have more detailed information about the deployment and management process, please see the document Techila Distributed Computing Engine in AWS Marketplace.

Finally, when modifying your own code to run in TDCE, programming language specific guides linked below can be used as reference material. Among other things, you can find walkthroughs of examples on how to enable various Techila Distributed Computing Engine features.