Cluster architecture
Nodes
| Role | Hostname | CPU (Thread x GHz) | RAM (GB) | GPU/OpenGL |
|---|---|---|---|---|
| login | psyclogin | 16 x 2.1 | 64 | Quadro M5000 |
| master | pscmatlab | 4 x 3.06 | 32 | No |
| worker | psycmri01 | 10 x 3.06 | 32 | No |
| worker | psycmri02 | 10 x 3.06 | 32 | No |
Logical architecture
Different nodes have different roles within the cluster:
Login node
As its name suggests, Login node is the entry point to the cluster either via an interactive (VNC) session (How to login) or for file transfer via SFTP (How to access storage). This machine can be used for writing scripts, viewing and editing files, transferring data, etc. You can also submit jobs of your full-fledged pipeline to the scheduler from here.
You can test and debug your scripts in this interactive session locally either on a very small number of subject or running a subset of your pipeline; but 
you should never run the whole full-fledged pipeline on the Login node locally, because it can clog up the node and incapacitate other users.
Master node
The Master node runs the scheduler that manages the workload across the Worker nodes. Normally, you do not access it.
Worker nodes
Worker nodes are the main computing resources of the cluster. By default, you never access them directly; instead jobs are submitted via the scheduler running on the Master node.
Data
Storage spaces
You have two locations to store items and both of them are available from each node. From your desktop, you can copy files to and from it using an SFTP client program (see Requirements).
Home space
/home/[user name], Desktop icon “[user name]'s Home”
The Home space is private and visible only by you by default. Its main role is to store your configuration settings. You are supposed to use it only as an intermediate storage (e.g. Downloads), which should be transferred either to e.g. the Work space. Scripts you do not want to share may be stored here.
You should not carry out analysis here!
Work space
/MRIWork/MRIWork[PI's number] E.g.: /MRIWork/MRIWork02 for Kathy Rastle
The Work space is private and visible for your group by default. Here, you can store all the data, images, scripts, documents, etc you are working on.
You should carry out analysis also here.
- If you are a PI, you can create folders for project(s) and give permission to your group members working on that project(s)
- If you are not a PI, you need to ask your PI to create a folder for you and give you permission for that folder
Archive space
/MRIArchive/MRIWork[PI's number] E.g.: /MRIArchive/MRIWork02 for Kathy Rastle
The Archive space is private and visible for your group by default. Here, you can store all the data, images, scripts, documents, etc you are not working on any more.
You should not carry out analysis here!
File and folder management works in the same way as for Work space.
Software
| Software | Extra Tools | Version(s) | Location | Configuration (see also Customise shell environment) |
|---|---|---|---|---|
| MATLAB | R2015b R2018a R2018b | /usr/local/apps/matlab | source /usr/local/apps/psycapps/config/matlab_bash (use option “help” for more info) |
|
| automaticanalysis | 5.4.0 | /usr/local/apps/psycapps/automaticanalysis | Add required version to MATLAB path | |
| CmapTools | 6.0.3 | /usr/local/apps/psycapps/CmapTools/CmapTools | Not required | |
| conda (python management) | 3 (with python 2.7.14 and 3.6.3) | /usr/local/apps/psycapps/conda | source /usr/local/apps/psycapps/config/conda_bash (use option “help” for more info) |
|
| CONN | 17.f | /usr/local/apps/psycapps/CONN | ||
| CoSMoMVPA | 1.1.0 | /usr/local/apps/psycapps/CoSMoMVPA | ||
| FreeSurfer | 6.0.0 | /usr/local/apps/psycapps/freesurfer | source /usr/local/apps/psycapps/config/freesurfer_bash (use option “help” for more info) |
|
| FSL | 5.0.9 5.0.10 6.0.0 | /usr/local/apps/psycapps/fsl | source /usr/local/apps/psycapps/config/fsl_bash (use option “help” for more info) |
|
| FIX | 1.065 | /usr/local/apps/psycapps/fsl_utils/fix | ||
| FSLNets | 0.6.3 | /usr/local/apps/psycapps/fsl_utils/fslnets | ||
| ICA-AROMA | 0.3 | /usr/local/apps/psycapps/fsl_utils/ica-aroma | ||
| mricron | /usr/local/apps/psycapps/mricron/mricron | Not required | ||
| R | 3.4.0 | /usr/lib64/R | Not required | |
| rstudio | 0.98.1103 | /usr/local/apps/psycapps/rstudio/rstudio | Not required | |
| SPM | SPM8 SPM12 SPM12 git (latest) | /usr/local/apps/psycapps/spm | Add required version to MATLAB path | |
| Anatomy Toolbox | 2.2c | /usr/local/apps/psycapps/spm_toolboxes/Anatomy | ||
| CAT12 | 1148 1355 | /usr/local/apps/psycapps/spm_toolboxes/CAT12 | ||
| GIFT | 4.0b | /usr/local/apps/psycapps/spm_toolboxes/GIFT | ||
| LI toolbox | 1.3 | /usr/local/apps/psycapps/spm_toolboxes/LI_toolbox | ||
| rWLS | 4.0 | /usr/local/apps/psycapps/spm_toolboxes/rWLS | ||
| TFCE | 121 | /usr/local/apps/psycapps/spm_toolboxes/TFCE | ||
| marsbar | 0.44 | /usr/local/apps/psycapps/spm_toolboxes/marsbar |
Installation
For members of psycadm group only
Software not requiring system level changes can be installed/copied in /usr/local/apps/psycapps. If you install something here, please, run this script on the main installation folder to set group ownership and permissions:
/usr/local/apps/psycapps/bin/set_permission.sh <directory>
The script works recursively, so it also takes care of subfolders.
