All Things Microsoft AI

User Groups Mar 3: Ohio North Database Training Mar 11: Azure Cleveland Mar 19: GLUG .NET Mar 25: Cleveland C# User Group   Conferences Feb – Jun: Agent Camp Mar 16: Memphis Agent Camp  

Yesterday I delivered a presentation on "Data Cleansing using Databricks".  Listed below are the questions that came up as well as the answers related to Databricks.   Does state transfer from one cell to another in the Databricks notebook? Yes, state does transfer from one cell to another, as long as you're running in the same notebook session and on the same cluster.   Can Databricks Jobs (aka "pipelines") be accessed through a notebook? Yes, you can fully interact with Jobs from a notebook using the Databricks REST API or the Databricks CLI. Some of these interactions from a notebook include: Trigger a job run Check job run status Cancel a run Retrieve job metadata   However, you can't directly "open" the Jobs UI from a notebook, and you can't modify job definitions without using the API   Listed below is an example for accessing a Job from a notebook import requests import json   token = dbutils.secre...