Users

These pages are for people interested in installing and using Jupyter Scheduler.

For configuration options, please refer to our operator’s guide.

Installation

Jupyter Scheduler has a lab (client) extension and a server extension. Both are required to be able to schedule and run notebooks. If you install Jupyter Scheduler via the JupyterLab extension manager, you might only install the client extension and not the server extension.

Recommended: Install Jupyter Scheduler from the PyPI registry via pip:

pip install jupyter_scheduler

This automatically enables the lab and server extensions. You can verify this by running

jupyter server extension list
jupyter labextension list

and checking that both the jupyter_scheduler server extension and the @jupyterlab/scheduler prebuilt lab extension are enabled.

Attention

Starting with v2.7.0, Jupyter Scheduler requires SQLAlchemy 2.x instead of SQLAlchemy 1.x.

Attention

JupyterLab 3 reached its end of maintenance date on May 15, 2024. As a result, we will not backport new features to the v1 branch supporting JupyterLab 3 after this date. Fixes for critical issues will still be backported until December 31, 2024. If you are still using JupyterLab 3, we strongly encourage you to upgrade to JupyterLab 4 as soon as possible. For more information, see JupyterLab 3 end of maintenance on the Jupyter Blog.

Use

Jupyter Scheduler runs Jupyter notebooks in the background, either once or on a schedule. You can create jobs (single run of an individual notebook) and job definitions (scheduled recurring notebook jobs). When the scheduler runs your notebook, it makes a copy of the input file. The scheduler uses unique names for the input and output files so that rerunning the same notebook produces new files every time.

Creating a job or job definition

Choose a notebook

To create a job or job definition from a file browser, right-click on a notebook in the file browser and choose “Create Notebook Job” from the context menu:

To create a job or job definition from an open notebook, click on a “Create a notebook job” button in the top toolbar of the open notebook:

Submit the Create Job form

Give your notebook job or job definition a name, choose an environment to run it in, select its output formats, and provide parameters that are set as local variables when your notebook gets executed. This parameterized execution is similar to the one used in Papermill.

If you check “Run job with input folder”, the scheduled job will have access to all files within the same folder as the input file. The scheduler will copy all files from the input file to a staging directory at runtime, and will make these files and any side effect files created during the job run available for download after the job has finished. Use caution with this option if your input file’s directory has many large files in it.

To create a job that runs once, select “Run now” in the “Schedule” section, and click “Create”.

To create a job definition that runs repeatedly on a schedule, select “Run on a schedule” in the “Schedule” section. You can use shortcuts to, for example, run your notebook every hour or every day.

You can also specify a custom schedule in crontab format by selecting “Custom schedule” in the “Interval” drop-down menu.

Using list view

Once you’ve created a job or job schedule, you can use the “Notebook Jobs” icon in the launcher to view lists of jobs and job definitions.

Each item’s name links to a detail view.

You can sort the list by clicking one of the headers (for example, click the “Job name” header to sort the list alphabetically based on “Job Name”).

You can also delete each job or stop a running job by clicking the buttons in the “Actions” column. The job definitions list also has a button to pause and resume a job definition. Paused job definitions do not start jobs.

When one of your jobs has completed, you can download its input and output files by clicking the “Download job files” button in the “Output files” column.

After clicking the download button, links to files appear in the “Input file” and “Output files” columns.

Details view

You can click the job or job definition name in the list view to see details.

Job detail

The job detail page displays information about one job. It includes action buttons to reload the detail view, stop the job (if it’s in progress), download output files (if it’s finished), and delete a job.

Job definition detail

The job definition detail page displays information about one job definition. It includes action buttons to reload, pause, resume, edit, and delete the job definition. It also includes a list of jobs created by this job definition.

You can run a new job based on the input file snapshot by clicking the “Run Job” button on top of the detail view.