Manage jobs in the Microsoft Sentinel data lake

A job is a one-time or scheduled task that runs against data in the Microsoft Sentinel data lake. The Jobs page, under Microsoft Sentinel > Data lake exploration in the Microsoft Defender portal, lists every job in your tenant so you can monitor and manage them from one place.

The data lake supports three job types:

• KQL jobs run a Kusto Query Language (KQL) query against the data lake tier and promote the results to the analytics tier. You create them from the KQL queries editor or the Jobs page. For more information, see KQL jobs.
• Notebook jobs run a notebook on a schedule against data in the lake. You create them from the notebook authoring experience. For more information, see Notebook jobs.
• Graph jobs run a custom graph built in the Microsoft Sentinel Visual Studio Code extension, either on demand or on a schedule. You create them when you publish or refresh a custom graph. For more information, see Custom graphs overview and Graph jobs in this article.

The Jobs management page provides the following functions:

• View all jobs in the Microsoft Sentinel data lake, including KQL, notebook, and graph jobs.
• View a summary of KQL, notebook, and graph jobs.
• View details of all jobs and apply filter to narrow down the list.
• View recent job health issues.
• Create a new job to run a KQL query. For more information on creating jobs, see Create jobs in the Microsoft Sentinel data lake using KQL. Notebook jobs and graph jobs can't be created from this page.
• Edit job details for KQL jobs.
• Disable a job, preventing it from running until you enable it again.
• Enable a job, allowing it to run again after being disabled.
• View job history, including the run times, and statuses of the job.
• Delete a job, removing it from the list of jobs. This action is permanent and can't be undone.

KQL jobs

A KQL job runs a Kusto Query Language (KQL) query against the data lake tier and writes the results to the analytics tier, either once or on a schedule. You can create KQL jobs in the KQL queries editor or from the Jobs page. For more information about creating KQL jobs, see Create jobs in the Microsoft Sentinel data lake using KQL.

The Jobs page supports the full KQL job lifecycle: you can create, edit, enable or disable, view run history for, and delete a KQL job from this page. To find KQL jobs, open the Jobs page and look for rows where the Job type column shows KQL. For step-by-step procedures, see Manage jobs.

Notebook jobs

A notebook job is a scheduled run of a notebook against data in the Microsoft Sentinel data lake. Notebook jobs are created from the VS Code extension when you schedule a notebook, not from the Jobs page. For more information about creating, editing, and scheduling notebook jobs, see Notebook jobs.

To find notebook jobs, open the Jobs page and look for rows where the Job type column shows Notebook. Selecting a notebook job opens a details panel that shows:

• Job name and description.
• Job type: Notebook.
• Status (enabled or disabled) and most recent run status.
• Schedule configuration.
• Last run and next run times.

Graph jobs

A graph job is a scheduled notebook-based custom graph job in your Microsoft Sentinel data lake tenant. Graph jobs are created when you publish a custom graph from Sentinel VS Code extension or when graphs are subsequently refreshed per your schedule configurations. For more information about creating custom graphs, see Create custom graphs and Custom graphs overview.

To find graph jobs, open the Jobs page and look for rows where the Job type column shows Graph. Selecting a graph job opens a details panel that shows:

• Job name and description.
• Job type: Graph.
• Status (enabled or disabled) and most recent run status.
• Schedule configuration.
• Pool size.
• Last run and next run times.

Permissions

Microsoft Entra ID roles provide broad access across all workspaces in the data lake. To read tables across all workspaces, write to the analytics tier, and schedule jobs using KQL queries, you must have one of the supported Microsoft Entra ID roles. For more information on roles and permissions, see Microsoft Sentinel data lake roles and permissions.

Manage jobs

The Jobs page shows a list of jobs, including the job name, status, job type, last and next run dates, and the recent job health. You can filter the jobs by status, last run date, and creation date. The Jobs status column indicates whether the job is enabled or disabled. The Job type column indicates whether the job is a KQL job, a Notebook job, or a Graph job.

The Recent health issues column shows whether the job encountered any issues in its recent runs as per the filters. Select the link to view the job's health details.

To create a job from the jobs page, select Create a new KQL job. For more information on creating jobs, see Create jobs in the Microsoft Sentinel data lake using KQL.

Job details

To see a job's details, select the job from the table. The job details panel opens, showing the job's details. You can enable and disable a job, view its history, edit, or delete it. Select the Destination table link to open the table in the KQL query editor in Advanced hunting.
The query can be copied by selecting Copy query.

View a job's run history

To view the history of a job, select View history in the job details panel. The job history panel opens, showing a list of job run times and statuses. The row count reflects the number of rows sent to the destination table in the analytics tier.

Manage KQL jobs

The jobs page supports the full lifecycle of KQL jobs, including creating, editing, enabling or disabling, and deleting a KQL job.

Edit a KQL job

To edit a job, select Edit in the job details panel. The job details panel opens, allowing you to edit the following fields:

• Job description.
• KQL query. The query can be updated but must return the same output schema as the original query. For example, you can change the time range in the query, but you can't change the columns returned by the query.
• Job schedule. You can change the job to run once or on a schedule, or change the schedule.

Select Next to continue to the next screen.

After you edit the job, select Submit to save the changes. The job is updated and runs according to the new schedule or query.

Enable or disable a KQL job

To enable or disable a job, select Enable or Disable in the job details panel. When a job is disabled, it won't run until you enable it again. The status of the job changes to reflect whether it's enabled or disabled.

Delete a KQL job

To delete a job, select Delete in the job details panel. A confirmation dialog appears, asking you to confirm the deletion. If you confirm, the job is permanently deleted and can't be recovered. You can't delete a running job.

Considerations and limitations

For information on considerations and limitations when managing KQL jobs in the Microsoft Sentinel data lake, see KQL jobs.

Next steps

• Overview of the Microsoft Sentinel data lake
• Create jobs in the Microsoft Sentinel data lake using KQL
• Notebook jobs
• Custom graphs overview
• Data lake exploration - KQL queries
• Microsoft Sentinel data lake roles and permissions