ARC Compute Node Usage

The CASA sessions and disk usage for the arcp cluster:

Sun Mar 31 01:42:00 CET 2024

(updated every 20min, click reload to get latest version)

ARC Cluster

LUSTRE disk usage:

Filesystem              1M-blocks      Used Available Use% Mounted on
192.168.1.3@o2ib:/esofs 998757344 655130029 343625843  66% /lustre

Free memory and CASA sessions on the cluster nodes:

arcp1

Head Node - no casa sessions should be run here!
              total        used        free      shared  buff/cache   available
Mem:         128543       41544       51995        3481       35003       81961

arcp2

Mem:         108486        7092       45475        1551       55918       98927

arcp3

Mem:          23878        6445        6230        1203       11202       15599

arcp4

Mem:          90342        4844       74506        1243       10991       83425

arcp5

Mem:         515622       32134      228976        3072      254512      477070
dpetry   1337973 96.2  1.7 11061936 8989428 ?    Il   Mar27 4201:56 /diska/tmp/arcproc/casa-from-tarball/casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/bin/python3 -m casashell --pipeline --rcdir /lustre/home/dpetry/.12057.mycasa --nologger -c global DOSPLIT, DOCONTSUB, SPACESAVING; DOSPLIT=True;DOCONTSUB=True;SPACESAVING=0;execfile('scriptForPI.py')

arcp6

Mem:         515616       15240      465444        3032       34930      493996

arcp7

Mem:         515616        4215      492363        3167       19036      504886

arcp8

Mem:         515616        2184      498093        2418       15339      507667

arcp9

Mem:         515616        2350      499779        2982       13485      506936

arcp10

Mem:         515616        9036      475245        3270       31333      499393

arcp11

Mem:         515616       21938      339234        2233      154443      488095
dpetry   3155765  100  1.0 7400456 5336376 ?     Rl   Mar30 258:26 /diska/tmp/arcproc/casa-from-tarball/casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/bin/python3 -m casashell --pipeline --rcdir /lustre/home/dpetry/.2141.mycasa --nologger -c global DOSPLIT, DOCONTSUB, SPACESAVING; DOSPLIT=True;DOCONTSUB=True;SPACESAVING=0;execfile('scriptForPI.py')

arcp12

Mem:         515615       24981      262637        2784      227996      484503
dpetry   2682191 96.8  1.0 7409872 5586300 ?     Rl   Mar30 245:01 /diska/tmp/arcproc/casa-from-tarball/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --pipeline --rcdir /lustre/home/dpetry/.6772.mycasa --nologger -c global DOSPLIT, DOCONTSUB, SPACESAVING; DOSPLIT=True;DOCONTSUB=True;SPACESAVING=0;execfile('scriptForPI.py')

arcp13

Mem:         515615        1515      512352         113        1747      511033

arcp14

Mem:         515615        3703      469333        2710       42577      505854

arcp15

Connection closed by 134.171.14.79 port 22

arcp16

Mem:         515615       41080      390840        2205       83694      468982
pipeproc 1645988 80.2  5.1 29413060 27233208 ?   Rl   Mar28 3010:56 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645989 10.9  0.0 1996292 338348 ?      Sl   Mar28 409:13 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645990  8.8  0.0 1929980 462860 ?      Sl   Mar28 332:56 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645991  8.3  0.1 2389508 653224 ?      Sl   Mar28 313:35 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645992 10.6  0.1 2725808 899140 ?      Sl   Mar28 398:12 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645993 10.2  0.0 2225172 259444 ?      Sl   Mar28 383:31 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645994  9.5  0.1 2685484 865540 ?      Sl   Mar28 358:25 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645995  8.4  0.1 2724240 713648 ?      Sl   Mar28 318:37 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d
pipeproc 1645970  0.0  0.0   9940  2412 ?        S    Mar28   0:00 /bin/sh /bin/xvfb-run -d mpicasa -n 8 casa --nocrashreport --nogui --pipeline --logfile casa-20240328-101357361360725.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u f6fb65a8-d2a2-428c-91fc-dace132becb5 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X362b/X46d

arcp17

Mem:         515615      130075      232555        3047      152984      379146

arcp18

Mem:         515615        2933      502801        2612        9880      506723

arcp19

Mem:         515615       27288      466128        2886       22198      482093
lmaud    3906933 48.7  3.0 23729540 16212160 pts/0 Sl+ Mar06 17140:29 /lustre/opsw/work/lmaud/Pipeline_testing/CASA/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --pipeline

arcp20

Mem:         515615       40538      342785        2746      132291      468984
pipeproc 1122293 97.4  2.7 16848952 14693392 ?   Rl   Mar05 35617:19 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122294 17.2  0.0 2140620 492112 ?      Sl   Mar05 6289:43 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122295 17.2  0.1 2241312 772696 ?      Sl   Mar05 6299:42 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122296 17.2  0.1 2243608 593824 ?      Sl   Mar05 6305:24 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122297 17.2  0.1 2460828 711412 ?      Sl   Mar05 6293:00 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122298 17.2  0.0 2340148 408444 ?      Sl   Mar05 6297:52 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122299 17.2  0.1 2534956 791552 ?      Sl   Mar05 6298:15 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
pipeproc 1122300 17.2  0.0 2485584 509048 ?      Sl   Mar05 6292:57 /lustre/opsw/alma/opt/pipelinedriver/2023AUG/casa/casa-6.5.4-9-pipeline-2023.1.0.124/lib/py/bin/python3 -m casashell --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502
dpetry   3266259 97.0  1.4 9810264 7737804 ?     Rl   Mar30 547:56 /diska/tmp/arcproc/casa-from-tarball/casa-6.4.1-12-pipeline-2022.2.0.68/lib/py/bin/python3 -m casashell --pipeline --rcdir /lustre/home/dpetry/.5998.mycasa --nologger -c global DOSPLIT, DOCONTSUB, SPACESAVING; DOSPLIT=True;DOCONTSUB=True;SPACESAVING=0;execfile('scriptForPI.py')
pipeproc 1122275  0.0  0.0   9940  2348 ?        S    Mar05   0:00 /bin/sh /bin/xvfb-run -d mpicasa -n 8 casa --nocrashreport --nogui --pipeline --logfile casa-20240305-153527433073184.log -c /opt/pipelinedriver/2023AUG/bin/pipeline_driver_build/bin/pipelinedriver.py -u 22eacb5c-0d36-4411-ad46-566691a7f626 -t /opt/pipelinedriver/2023AUG/config/templates/template_hifa_calimage.json -c /opt/pipelinedriver/2023AUG/config/config/config_hifa_calimage.json -m uid://A001/X2f7/X502

arcp21

Mem:         515615        4647      496201        3037       14766      504583

arcp22

Mem:         515615        3498      488314        4072       23802      504697

arcp23

Mem:         515587        6899      481953        3067       26734      502277

arcp24

Mem:         515588        2925      488642        2898       24020      506421

arcp25

Mem:         515588        2769      487587        3018       25231      506457

arcp26

Mem:         515588        5448      484350        3012       25789      503784

arcp27

Mem:         515338        3582      486026        2964       25730      505446

arcp28

Mem:         515338        3330      487365        2995       24642      505667

arcp29

Mem:         515338        2737      488501        2977       24099      506278

arcp30

    arcp >=30 out of service. Expected recovery t.b.d.

arcprhel5

Mem:         48271       5102      43169          0       1136        547

(script version 2 Nov 2023, DP)