Run Jobs in a Batch#

To run multiple jobs in a batch, you first need to create the batch, and then create the jobs that will be added to the batch. Once all of the jobs have been added, you can start the batch.

To run batches with the CONVERGE Horizon web interface or the command-line interface (CLI), follow the instructions below. Starting in CONVERGE Studio 6, you can also run and manage batches from CONVERGE Studio.

Create the Batch#

Follow these steps to create a new batch:

  1. Go to Batches and click Create Batch in the upper right.

  2. Enter a name for the batch.

  3. Click Create Batch.

When the batch is created, the Batch Details page will open. This page is mostly empty, because you haven’t added any jobs yet. Note that some details, such as Mode and Stop on Failure, initially display default values. You will have an opportunity to change those settings when you start the batch.

To create a batch with the CLI instead of in the web interface, use the batch:create command.

Add Jobs to the Batch#

Once you have created a batch, you can create new jobs to add to the batch. You cannot add jobs that are already created.

Follow the usual process for creating jobs. Make sure you select your batch in the Details > Optional Batch field—this is how the job is added to the batch. You can add up to 100 jobs per batch. Each job will be assigned an order in the batch based on when it was created (1 for the first job created, 2 for the second job created, and so forth). If you plan to run the jobs consecutively, create the jobs in the order you want them to run.

Each job you add to the batch will be created with an initial status of Batched. CONVERGE Horizon will not request hardware for these jobs until you start the batch. Each job will run on its own job cluster.

To add jobs to the batch with the CLI instead of in the web interface, use the job:create command with the --batch_id option.

Start the Batch#

When you are done adding jobs to a batch, follow the steps below to start the batch. You will not be able to add more jobs to the batch after it has started.

  1. Go to Batches and click on the name of the batch to open the Batch Details page.

  2. Review the list of jobs in the batch to confirm that jobs have been added correctly. If you plan to run the jobs consecutively, make sure the order of the jobs is correct.

  3. Click Start Batch in the lower right.

    • By default, the jobs will run concurrently, meaning they will be added to the queue at the same time (the actual start times may differ depending on how long it takes to acquire hardware for each job). Check Run jobs consecutively if you want to run jobs in the order in which they were created. In Consecutive mode, each job is added to the queue only after the previous job has stopped.

    • By default, the failure of one job does not affect other jobs in the batch. Check Stop batch on first failure if you want to stop all jobs in the batch if one job fails.

To start batches with the CLI instead of in the web interface, use the batch:start command.

Manage Your Batches#

You can check the status of your batches on the Batches page. By default, only your own batches are listed here. Adjust the filters at the top right to see batches with specific owners or statuses.

Batch status is distinct from job status. To see the status of each job, click on the name of the batch to open the Batch Details page. The jobs in the batch, along with their statuses and their order in the batch, are listed here. If you are running the jobs consecutively, some will remain at the Batched status until it is their turn to run.

The Cost column on the Batches page shows the total cost of all jobs in each batch. While a batch is running, this total is updated every 15 minutes with the latest accrued costs of all running jobs. There is no billing transaction for the batch itself; billing is done separately for every job in the batch.

Stop a Batch#

If you want to stop all jobs in a batch, follow the steps below. This action will affect all jobs that have not already stopped or failed. Jobs that are currently running will be stopped with a STOP file.

  1. Go to the Batches page.

  2. Click on the name of the batch to open the Batch Details page.

  3. Click Stop Batch in the lower right. You will be given the option to discard output files for all jobs that are stopped.

To stop batches with the CLI instead of in the web interface, use the batch:stop command.