Techila Distributed Computing Engine Integration Guide

This document is intended for developers who are planning on integrating Techila Distributed Computing Engine (TDCE) to existing applications. If you are unfamiliar with the terminology or the operating principles of the TDCE system, please see Introduction to Techila Distributed Computing Engine.

The structure of this document is as follows.

Techila Distributed Computing Engine Integration contains an overview of the TDCE integration process and lists different approaches that can be used to add TDCE functionality to an application.

Creating a Redistributable Package from Techila SDK files contains a description on how to create a redistributable package by using the files in the Techila SDK. Creating a redistributable package will essentially consist of removing unnecessary files from the Techila SDK and modifying the techila_settings.ini configuration file to match the settings in your TDCE environment.

Including the Redistributable Package with your Application illustrates how including the redistributable TDCE package will change the directory structure of your application when it is installed by an End-User. Including the redistributable TDCE package will enable the End-User to access to the TDCE environment, without having to install the Techila SDK.

Creating a Runtime Bundle from a Locally Installed Application contains a description of Runtime Bundles, which can be used to transfer files required on the Techila Workers. Using a Runtime Bundle is recommended in situations where your application requires a multitude of files or other configuration parameters (e.g. such as setting environment variables) when executed on the Techila Workers.

Appendix contains programming language specific lists of files that need to be included when creating a redistributable TDCE package.

Integrating Techila Distributed Computing Engine (TDCE) to an existing application will enable the computationally intensive part of the application to be performed in the TDCE environment. Typically the integration process will contain similar steps as described in Introduction to Techila Distributed Computing Engine.

These steps typically include the following steps:

For more detailed information on how to distribute computations from applications developed with different programming languages, please see the programming language specific how-to guides located here.

The Techila SDK is software development kit provided by Techila Technologies, which includes API`s for several commonly used programming languages. The files included in the Techila SDK can also be used to create a redistributable package, which can included in applications that you plan on distributing to your End-Users. This redistributable package will enable End-Users to access the Techila Distributed Computing Engine (TDCE) computing environment by using the integrated TDCE API functions that you have added to your application.

Please note that when creating a redistributable package from the Techila SDK files, the relative path of the files and directory names will need to match the ones used the Techila SDK. The image below illustrates directories that will typically need to be included when creating a redistributable TDCE package.

Required files and folders will depend on the programming language used in your application. Please refer to Appendix 1: Programming language specific redistributable Techila packages for lists of files and directories required for several commonly used programming languages.

After creating a redistributable package from the files included in the Techila SDK, you will need to include the redistributable package with your application. This will ensure that End-User can access the Techila Distributed Computing Engine (TDCE) environment without having to install any additional files.

When designing the deployment process for your TDCE enabled application, best practices include storing the techila_settings.ini file in the End-Users home directory or other location that can be used to store both temporary and permanent data. This is because the default settings in the techila_settings.ini file are configured to create directories for log information and temporary files under the directory that contains the techila_settings.ini file.

In order to execute the computationally intensive part of the application (the Techila Worker Code) on Techila Workers, all necessary files required during runtime execution will need to be transferred and made available on the Techila Worker. These files can include dynamically linked libraries (.DLL`s), shared objects (.SO`s), class files and JAR archives.

Files that are required on all of the Techila Workers during runtime execution can either be transferred in Runtime Bundles or as additional data files common to all Jobs.

If the executable application requires multiple libraries and/ or other files during execution on the Techila Workers, the most efficient method will typically be to package all necessary files in a Runtime Bundle. This Runtime Bundle can then be imported to the computational Project to easily construct the runtime environment on each Techila Worker. Having a Runtime Bundle available on the Techila Server will often enable a faster launch of the Techila Distributed Computing Engine (TDCE) processes because the need of data transfer between the End-User’s computer and the TDCE environment can be minimized.

The image below illustrates how files and folders located on a local computer can be transferred to Techila Workers by using a Runtime Bundle. Runtime Bundles can be created using the Bundle Creator Tool or the createBundle command, which is available on the Techila Command Line Interface. Both tools are included in the Techila SDK.

When creating a Runtime Bundle, it is also possible to specify values for environment variables and other resources that might be needed to construct the runtime environment on the Techila Workers. These resources can also be, for example, used to make executable files included in the Bundle available for use during computational Jobs or to modify values of environment variables.

After a Runtime Bundle has been created and it has been transferred to the Techila Server, the Bundle can be imported into Projects. When the Bundle is imported in a Project, the Bundle will be automatically transferred to all participating Techila Workers. After the Bundle has been transferred to a Techila Worker, it will be extracted and the folder structure will be restored, assuming file copying has been disabled.