It is strongly recommended that one is familiar with the following before reading this tutorial.
- The Introduction to VPAC, Linux and HPC Job Submission
- Using MATLAB DCS at VPAC
- PuTTY User Manual
- Pageant - help with windows ssh pass phrase
Step1: Connect to the account using public key and private key using PuTTY key generator:
Step2: Download PSCP and Plink and save in a folder (for example C:\WINDOWS). Then go to Environment tab in the System Control Panel and set the path to the saved folder (e.g., C:\WINDOWS)
Step3: Start the PuTTY application and enter details as below. Then save the session;
After doing this save the session and try log in your account with PuTTY. You should be able to login automatically without having to enter your username and password.
Step4: Run the Pageant application, as follows;
Step5: You will need to establish a c:\matlab directory where you can put your data and results, as scp does not work with spaces in path names in this instance. The entire MATLAB Windows toolbox needs to be copied into this directory. i.e.,
cp c:\$matlabroot\toolbox\distcomp\examples\integration\pbs\nonshared\windows\ c:\matlab\windows
Before using MATLAB PCT you will need to configure the MATLAB path to include the correct set of submit functions for your system. To do this go to the File menu and select the "Set Path" option. Select the "Add Folder" option and then add one of the two options below depending on whether you use Linux or Windows. $MATLAB represents the location where MATLAB is installed on your computer, you will need to navigate to the directory below by hand.
Again in MS-Windows the task is a little trickier. First the existing path has to be removed i.e.,
Then the new path (see above) has to be added.
The concept for this process is that you will write some simple code to send to VPAC cluster and it will run the code there then return back to you the result. You can copy the following code put in an .m file and run it your MATLAB directly (make sure that all the above sections are setup correctly).Running A Parallel Job using MATLAB Distributed Computing
In general, to connect to a VPAC cluster, MATLAB will make use of the running Pagent to connect using the public and private key. A connection is made to prepare the job and submit it to the cluster. After this is done, the result will be send back to the local MATLAB instance.Using FileDependencies property to send code and data to VPAC
Assume, we have a dataset called trainin.mat and an algorithm called test1.m (below).
Now we want to write the code to send the test.m function and trainin.mat to VPAC and run it there: The code for this is below:
For large dataset files it is not best practise to use FileDependencies to send it to the VPAC cluster every time we execute the program. Instead, one can just put the dataset on VPAC and program MATLAB to access it when required. . For example, with the trainin.mat dataset, we can upload it to our account on VPAC first and program MATLAB to access it instead of sending it as filedependencies to VPAC each time.How to run .m file with data set which are already on VPAC
In some case, you have a very big function file (.m file) and a big data set, and it is not sufficient to send that function file to server every time you run it. You may want to upload everything to server at one time and then program the cluster to run that function up there directly.
This tutorial will show you how to do that:
To make it easy to explain the concept, I only use a small function to show that it is running well using this method:
Assume, we have a funtion file : abc.m and the dataset trainin.mat. We want to upload these two file to the cluster, then tell matlab to run it there.
Assuming these these two file on VPAC sever are already in one's home directory, or subdirectory, then another function needs to be written that calls the abc.m function on another. (test1.m)
This is the function that you will send from you computer to VPAC to run it, it will call the function abc.m and run it there.
When we have test1.m ready, we again need to write the code to submit this function.
When you run this file, it will send the test1.m file to VPAC and run it there. Inside the test1.m file it calls the abc() function, so that abc.m file will also run and return the result.
This tutorial was originally written by Quan Vu with assistance from Gang Li. VPAC would like to extend their warmest thanks for their work.