Connection and file transfer

Connection to frontal node (ruche01 / ruche02) from a local machine with SSH

Using shell command ssh (Linux/Mac)

You can connect to the ruche, using SSH with the following command :

user@mycomputer:~$ ssh username@ruche.mesocentre.universite-paris-saclay.fr
Password:
[username@ruche01 ~]$

When the SSH tunnel is established, you are connected to the ruche front-end, where you can edit your source files and compile your code.

Using a SSH client on Windows (PuTTY)

To access ruche from a Windows machine, a SSH client is mandatory. For example, PuTTY.

When asked to configure your connection, you have to fill the "Host Name (or IP address)" field with "ruche.mesocentre.universite-paris-saclay.fr" and ensure that the "connection type" is set to "SSH".

When Putty is configured correctly, you just have to click the "Open" button and a refer to the previous section in order to log in.

Data management on ruche

Ruche storage uses the GPFS technologie to share data among all the nodes in the cluster. The calculation nodes have acces to your $HOME and $WORKDIR folders. There is no need for copying your files and executables to another location for execution.

Home directory, working directory

As a user, you have access to two separate directories for your own usage :

• the users directory (or home directory or homedir) : /home/username (also accessible via /gpfs/users/username) where you can store your source files (code, light data). This volume is limited to 50 Go per user. You can refer to this directory with the environment variable $HOME.
• the working directory (or workdir) : /workdir/username (also accessible via /gpfs/workdir/username) where you can store your binaries and data. This volume is limited to 500 Go per user. You can refer to this directory with the environment variable $WORKDIR.

As a rule of thumb, all your executables, local libraries, environment configuration files should be located in your $HOME directory and all your simulation datasets and results should be in your $WORKDIR directory.

File quotas

You can quickly check your file quota with the command ruche-quota. If you need more info on your file usage, try the mmlsquota command.

If your computations require more space on your $WORKDIR directory, you can ask the support team for quota extension.

Any request for a quota that exceeds 2 Tos must be justified and sent to ruche.support@universite-paris-saclay.fr. The following persons will be implicated in the decision process and must be in copy of the mail:

• The project referent
• The lab referent.
• The Mesocenter managers: Guillaume Bernadat, Pierre-Alain Boucard and Thomas Schmitt

Data transfer

Shell commands scp and rsync on Linux/Mac

In order to transfer files from the cluster or to the cluster, you can use the shell command scp from your computer.

The following command copies the directory my_dir on the workdir of the user username.

user@mycomputer:~$ ls my_dir/
file01.txt  file02.txt
user@mycomputer:~$ scp -r my_dir username@ruche.mesocentre.universite-paris-saclay.fr:/workdir/username
Password:
file01.txt                                                                       100%  132KB 132.3KB/s   00:00
file01.txt                                                                       100%  132KB 132.3KB/s   00:00
user@mycomputer:~$

The following command makes a local copy of the directory my_dir from the homedir of the user username.

user@mycomputer:~$ scp -r username@ruche.mesocentre.universite-paris-saclay.fr:/gpfs/users/username/my_dir .
Password:
user@mycomputer:~$

To transfer bigger files (when the tranfer is long enough), you should use the shell command rsync with the options --partial --progress (same as -P). The --partial option allows rsync to keep partially transferred files and thus, in case an error occurs, to restart the tranfer from the partial files and not from scratch. The --progress tells rsync to print information showing the progress of the transfer. The following command copies the directory my_dir on the workdir of the user username.

user@mycomputer:~$ ls my_dir/
file01.txt  file02.txt
user@mycomputer:~$ rsync -P -r my_dir username@ruche.mesocentre.universite-paris-saclay.fr:/workdir/username
Password:
sending incremental file list
my_dir/
my_dir/file01.txt
 10,737,418,240 100%  121.84MB/s    0:01:24 (xfr#1, to-chk=1/3)
my_dir/file02.txt
 10,737,418,240 100%  111.19MB/s    0:01:32 (xfr#2, to-chk=0/3)
user@mycomputer:~$

Using a scp client on Windows (WinSCP)

To transfer data from a Windows machine, a SCP client is mandatory. For example, WinSCP or FileZilla.