Missing documentation on how to use the launcher with GPU programs such that each job runs on a different GPU?

[mauriciogtec]

Is there any documentation/examples on how to use the launcher with a GPU Tensorflow or Pytorch program correctly? Specifically, is it possible to use the launcher such that each job is assigned a different GPU of a multi-GPU node? Something like --gpus-per-task 1.

[siliu-tacc]

I have developed a different application named launcher_gpu (based on launcher) to make it work for GPU resources.

If you are working on the Frontera machine, you can use it with its module as:
module load launcher_gpu

Si

[emsisson]

Greetings, siliu-tacc,

Is your application launcher_gpu available, or has it been incorporated into launcher? We have a TACC user at UT M. D. Anderson who would like to have that capability on our HPC cluster.

Regards,
Eric Sisson

[vtrueTACC]

Hello, Please use the alternate TACC package pylauncher for gpu applications: https://github.com/TACC/pylauncher [https://repository-images.githubusercontent.com/3768564/24ab3997-69df-4933-9647-6683c528368c]<https://github.com/TACC/pylauncher> GitHub - TACC/pylauncher<https://github.com/TACC/pylauncher> Contribute to TACC/pylauncher development by creating an account on GitHub. github.com Thanks, vtrue

…

________________________________ From: emsisson ***@***.***> Sent: Monday, August 26, 2024 11:26 AM To: TACC/launcher ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [TACC/launcher] Missing documentation on how to use the launcher with GPU programs such that each job runs on a different GPU? (#61) Greetings, siliu-tacc, Is your application launcher_gpu available, or has it been incorporated into launcher? We have a TACC user at UT M. D. Anderson who would like to have that capability on our HPC cluster. Regards, Eric Sisson — Reply to this email directly, view it on GitHub<#61 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADJB42N32O4UBBRYDWJLMELZTNJJ3AVCNFSM6AAAAABNEK45U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJQGU4TMMRUGE>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[emsisson]

Thank you! Regards, Eric From: vtrue ***@***.***> Reply-To: TACC/launcher ***@***.***> Date: Monday, August 26, 2024 at 12:04 To: TACC/launcher ***@***.***> Cc: Eric Sisson ***@***.***>, Comment ***@***.***> Subject: [EXTERNAL] Re: [TACC/launcher] Missing documentation on how to use the launcher with GPU programs such that each job runs on a different GPU? (#61) SLOW DOWN! - EXTERNAL SENDER: ***@***.*** Be suspicious of tone, urgency, and formatting. Do not click/open links or attachments on a mobile device. Wait until you are at a computer to confirm you are absolutely certain it is a trusted source. If you are at all uncertain use the Report Phish button and our Cybersecurity team will investigate. Hello, Please use the alternate TACC package pylauncher for gpu applications: https://github.com/TACC/pylauncher<https://urldefense.com/v3/__https:/github.com/TACC/pylauncher__;!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjFCW58oaI$> [https://repository-images.githubusercontent.com/3768564/24ab3997-69df-4933-9647-6683c528368c]<https://github.com/TACC/pylauncher><https://urldefense.com/v3/__https:/repository-images.githubusercontent.com/3768564/24ab3997-69df-4933-9647-6683c528368c*5D*3Chttps:/*github.com/TACC/pylauncher*3E__;JSUvJQ!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjFsboQQRg$> GitHub - TACC/pylauncher<https://github.com/TACC/pylauncher><https://urldefense.com/v3/__https:/github.com/TACC/pylauncher*3E__;JQ!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjFZYtQsNo$> Contribute to TACC/pylauncher development by creating an account on GitHub. github.com Thanks, vtrue

…

________________________________ From: emsisson ***@***.***> Sent: Monday, August 26, 2024 11:26 AM To: TACC/launcher ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [TACC/launcher] Missing documentation on how to use the launcher with GPU programs such that each job runs on a different GPU? (#61) Greetings, siliu-tacc, Is your application launcher_gpu available, or has it been incorporated into launcher? We have a TACC user at UT M. D. Anderson who would like to have that capability on our HPC cluster. Regards, Eric Sisson — Reply to this email directly, view it on GitHub<#61 (comment)><https://urldefense.com/v3/__https:/github.com/TACC/launcher/issues/61*issuecomment-2310596241*3E__;IyU!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjFtQ6WmEg$>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADJB42N32O4UBBRYDWJLMELZTNJJ3AVCNFSM6AAAAABNEK45U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJQGU4TMMRUGE><https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ADJB42N32O4UBBRYDWJLMELZTNJJ3AVCNFSM6AAAAABNEK45U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJQGU4TMMRUGE*3E__;JQ!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjF_Tdf8KQ$>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/TACC/launcher/issues/61*issuecomment-2310662585__;Iw!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjFt-SR13M$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AHNCOMOH6BBNRZPQID65WJ3ZTNNVLAVCNFSM6AAAAABNEK45U6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMJQGY3DENJYGU__;!!PfbeBCCAmug!imceuE3uebsZCTAw2NVvM59ZACNQ5RHT2g7qmecdWglk2gZitv7XUdzVsMj5C6Kre1ZpMaCM3h31CPjFRPMxgcI$>. You are receiving this because you commented.Message ID: ***@***.***> The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems.