From 3c71604529d9a68e689d285300061cc2505a05ff Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Fri, 10 Feb 2023 13:32:30 -0800 Subject: [PATCH 1/7] tutorials: list "flux mini submit" first Problem: "flux mini submit" is probably the most basic command to learn, but is listed second. Lets list it first. Also remove the hyphens in the command listing. --- tutorials/commands/index.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tutorials/commands/index.rst b/tutorials/commands/index.rst index 2c2fbd19..c19a6318 100644 --- a/tutorials/commands/index.rst +++ b/tutorials/commands/index.rst @@ -6,8 +6,8 @@ Command Tutorials Welcome to the Command Tutorials! These tutorials should help you to map specific Flux commands with your use case, and then see detailed usage. + - ``flux mini submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance" - ``flux proxy`` (:ref:`ssh-across-clusters`): "Send commands to a flux instance across clusters using ssh" - - ``flux-mini-submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance" This section is currently 🚧️ under construction 🚧️, so please come back later to see more command tutorials! @@ -16,5 +16,5 @@ This section is currently 🚧️ under construction 🚧️, so please come bac :maxdepth: 2 :caption: Command Tutorials - ssh-across-clusters flux-mini-submit + ssh-across-clusters From 1c4e6843f632bcec7cbc92f6a0b3370bd8aba1cd Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Sat, 11 Feb 2023 06:54:38 -0800 Subject: [PATCH 2/7] tutorials: capitalize Flux when used alone Problem: When flux is used alone in a statement, i.e. "Flux instance", Flux should be capitalized. Capitalize Flux. --- tutorials/commands/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/commands/index.rst b/tutorials/commands/index.rst index c19a6318..57bed74b 100644 --- a/tutorials/commands/index.rst +++ b/tutorials/commands/index.rst @@ -7,7 +7,7 @@ Welcome to the Command Tutorials! These tutorials should help you to map specifi with your use case, and then see detailed usage. - ``flux mini submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance" - - ``flux proxy`` (:ref:`ssh-across-clusters`): "Send commands to a flux instance across clusters using ssh" + - ``flux proxy`` (:ref:`ssh-across-clusters`): "Send commands to a Flux instance across clusters using ssh" This section is currently 🚧️ under construction 🚧️, so please come back later to see more command tutorials! From ee6072bfb5b554b786f729e0733882cb785adffc Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Sat, 11 Feb 2023 06:59:44 -0800 Subject: [PATCH 3/7] tutorials: markup text of flux commands Problem: Flux commands, i.e. flux start, should be marked up. Use double backticks around "flux start" in several locations. --- tutorials/commands/ssh-across-clusters.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tutorials/commands/ssh-across-clusters.rst b/tutorials/commands/ssh-across-clusters.rst index 03b4d65d..c897afbe 100644 --- a/tutorials/commands/ssh-across-clusters.rst +++ b/tutorials/commands/ssh-across-clusters.rst @@ -13,7 +13,7 @@ Create a Flux Instance ---------------------- First, let's create the allocation on the first cluster. We typically want to ask for an allocation, -and run flux start via our job manager. Here we might be on a login node: +and run ``flux start`` via our job manager. Here we might be on a login node: .. code-block:: console @@ -22,7 +22,7 @@ and run flux start via our job manager. Here we might be on a login node: $ srun -N4 -n4 --pty --mpibind=off flux start And then we get our allocation! You might adapt this command to be more specific to your resource manager. E.g., slurm uses srun. -After you run flux start, you are inside of a Flux instance on your allocation! +After you run ``flux start``, you are inside of a Flux instance on your allocation! Let's run a simple job on our allocation. This first example will ask to see the hostnames of your nodes: .. code-block:: console From 987bf0e1dc2e6faf692e4c607715db814d310d2f Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Mon, 13 Feb 2023 07:28:03 -0800 Subject: [PATCH 4/7] tutorials: remove trailing whitespace Problem: There is some trailing whitespace in the flux mini submit tutorial. Remove it. --- tutorials/commands/flux-mini-submit.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tutorials/commands/flux-mini-submit.rst b/tutorials/commands/flux-mini-submit.rst index ebe79070..2e89ce8e 100644 --- a/tutorials/commands/flux-mini-submit.rst +++ b/tutorials/commands/flux-mini-submit.rst @@ -28,7 +28,7 @@ for each task. In the second submission, we are asking for our job to start 1 task on just one node with 2 cores allocated for the task. There are many different options to customize your job submission. For further -details, please see :core:man1:`flux-mini`. +details, please see :core:man1:`flux-mini`. A :ref:`jobID` (e.g., ``ƒSUEFPDH``) is returned for every job submitted. You can view the status of your running jobs with ``flux jobs``: @@ -37,10 +37,10 @@ the status of your running jobs with ``flux jobs``: $ flux jobs JOBID USER NAME ST NTASKS NNODES TIME INFO - ƒSUEFPDH fluxuser my_other_s R 1 1 1.842s + ƒSUEFPDH fluxuser my_other_s R 1 1 1.842s ƒM5k8m7m fluxuser my_compute R 4 2 3.255s -And that's it! If you have any questions, please +And that's it! If you have any questions, please `let us know `_. ------------------------------------- From f12b66779dc37a4cc5c1b00f7beb3ef83af89235 Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Mon, 13 Feb 2023 07:33:10 -0800 Subject: [PATCH 5/7] tutorials: correct code blocks colorization Problem: The code blocks in flux-mini-submit were syntax highlighted to 'sh', but they should just be 'console' output. Update all code blocks to be 'console' colorized. --- tutorials/commands/flux-mini-submit.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tutorials/commands/flux-mini-submit.rst b/tutorials/commands/flux-mini-submit.rst index 2e89ce8e..550b0372 100644 --- a/tutorials/commands/flux-mini-submit.rst +++ b/tutorials/commands/flux-mini-submit.rst @@ -14,7 +14,7 @@ From within a Flux instance, you can submit your job on the command line with optional arguments, resource options, per task options, and per resource options: -.. code-block:: sh +.. code-block:: console $ flux mini submit --nodes=2 --ntasks=4 --cores-per-task=2 ./my_compute_script.py 120 ƒM5k8m7m @@ -33,7 +33,7 @@ details, please see :core:man1:`flux-mini`. A :ref:`jobID` (e.g., ``ƒSUEFPDH``) is returned for every job submitted. You can view the status of your running jobs with ``flux jobs``: -.. code-block:: sh +.. code-block:: console $ flux jobs JOBID USER NAME ST NTASKS NNODES TIME INFO @@ -47,14 +47,14 @@ And that's it! If you have any questions, please More Examples of Submitting Flux Jobs ------------------------------------- -.. code-block:: sh +.. code-block:: console $ flux mini submit --nodes=2 --queue=foo --name=my_special_job ./my_job.py This submits a job to the `foo` queue across two nodes, and sets a custom name to the job. -.. code-block:: sh +.. code-block:: console $ flux mini submit --dry-run ./my_cool_job.py @@ -62,7 +62,7 @@ If you don't want your job to actually run, but you are interested in looking at the :ref:`jobspec` for your job, include the ``--dry-run`` option when you submit your job. -.. code-block:: sh +.. code-block:: console $ flux mini submit --output=job-{{id}}.out ./my_super_cool_job.py ƒ3D78hc3q From a2608904e8e478c67df2b2349a77c910295e2e2d Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Mon, 13 Feb 2023 07:42:01 -0800 Subject: [PATCH 6/7] tutorials: use jobid not jobID Problem: The convention is to use lowercase "jobid" to reference jobids. Change jobID to jobid in flux-mini-submit tutorial. --- tutorials/commands/flux-mini-submit.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tutorials/commands/flux-mini-submit.rst b/tutorials/commands/flux-mini-submit.rst index 550b0372..6cfa31e3 100644 --- a/tutorials/commands/flux-mini-submit.rst +++ b/tutorials/commands/flux-mini-submit.rst @@ -30,7 +30,7 @@ task on just one node with 2 cores allocated for the task. There are many different options to customize your job submission. For further details, please see :core:man1:`flux-mini`. -A :ref:`jobID` (e.g., ``ƒSUEFPDH``) is returned for every job submitted. You can view +A :ref:`jobid` (e.g., ``ƒSUEFPDH``) is returned for every job submitted. You can view the status of your running jobs with ``flux jobs``: .. code-block:: console From c361e3cd0ab0ff97fb7c6090204591a7bdc4b326 Mon Sep 17 00:00:00 2001 From: Al Chu11 Date: Mon, 13 Feb 2023 07:32:38 -0800 Subject: [PATCH 7/7] tutorials: add flux mini run tutorial Problem: The flux mini submit tutorial was also supposed to have a flux mini run example, but was forgotten. Add flux mini run tutorial to flux mini submit. --- tutorials/commands/flux-mini-submit.rst | 30 +++++++++++++++++++++++++ tutorials/commands/index.rst | 2 +- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/tutorials/commands/flux-mini-submit.rst b/tutorials/commands/flux-mini-submit.rst index 6cfa31e3..8a7fca07 100644 --- a/tutorials/commands/flux-mini-submit.rst +++ b/tutorials/commands/flux-mini-submit.rst @@ -1,4 +1,5 @@ .. _flux-mini-submit: +.. _flux-mini-run: ========================== How to Submit Jobs in Flux @@ -40,6 +41,35 @@ the status of your running jobs with ``flux jobs``: ƒSUEFPDH fluxuser my_other_s R 1 1 1.842s ƒM5k8m7m fluxuser my_compute R 4 2 3.255s +----------------------- +Interactively Run a Job +----------------------- + +If you wish to run a job interactively, e.g. see standard output as it runs, you can +use the ``flux mini run`` command. It is identical to ``flux mini submit`` except it +will handle stdio and it will block until the job has finished. For example: + +.. code-block:: console + + $ flux mini run bash -c "echo start; sleep 5; echo done" + start + done + +In the above example, we run a small bash script that will output "start", sleep for 5 seconds, +and then echo "done". Unlike ``flux mini submit``, you'll notice it does not output a jobid. +If we check for the status of this job with ``flux jobs`` after it has run, you will not find the +job listed because it is no longer running. Instead run ``flux jobs -a`` which will list all jobs, +including completed jobs. + +.. code-block:: console + + $ flux jobs -a + JOBID USER NAME ST NTASKS NNODES TIME INFO + f2HnvmZy achu bash CD 1 1 5.119s catalyst159 + + +You will see that job is in the "CD" state or "Completed" state. + And that's it! If you have any questions, please `let us know `_. diff --git a/tutorials/commands/index.rst b/tutorials/commands/index.rst index 57bed74b..14f769cf 100644 --- a/tutorials/commands/index.rst +++ b/tutorials/commands/index.rst @@ -6,7 +6,7 @@ Command Tutorials Welcome to the Command Tutorials! These tutorials should help you to map specific Flux commands with your use case, and then see detailed usage. - - ``flux mini submit`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance" + - ``flux mini submit/flux mini run`` (:ref:`flux-mini-submit`): "Submit a job in a Flux instance" - ``flux proxy`` (:ref:`ssh-across-clusters`): "Send commands to a Flux instance across clusters using ssh" This section is currently 🚧️ under construction 🚧️, so please come back later to see more command tutorials!