Jupyter Notebook in Cognite Data Fusion

Jupyter Notebook is integrated with Cognite Data Fusion (CDF) and you can create interactive notebook documents that can contain live code, equations, visualizations, and media.

Beta

The features described in this section are currently in beta testing and are subject to change.

You can store the notebooks permanently or temporarily:

• Permanently — all files are stored in Cognite Data Fusion and you can share them with other users. You can't change the default files and folders.

• Temporarily — all files are stored temporarily in your browser. Save your notebooks externally to avoid losing them when clearing the browser cache.

All notebooks operate in a secure and private sandbox environment and have no direct access to your computer.

To learn more, refer to the Getting started guide for JupyterLab.

Create a notebook and store it permanently

To create a notebook and store it permanently:

1. Navigate to Data management > Build solutions > Jupyter Notebook.

2. In the Stored in Cognite Data Fusion section, select the Notebooks folder, and then select the data set folder you want to work with.

   Each sub-folder in the Notebooks folder is mapped to a data set that you need read/write access to. Only data sets where you have the proper access level will be listed. If the Notebooks folder is empty, request the necessary capabilities and create a data set to add your folder.

3. Select + and select the notebook Python file (.ipynb).

4. Save the changes before leaving Jupyter Notebook.

Each Python file (.ipynb) has an authentication helper to run the code with the current user's credentials, ensuring that they can work with data relevant to their role and projects.

 from cognite.client import CogniteClient
 # Authenticate under current user credentials
 client = CogniteClient()

If several users are working on the same file, Jupyter Notebook doesn't resolve conflicts and you risk losing changes.

attention

Only files with the .ipynb extension are saved in Cognite Data Fusion. Other file types are deleted when you leave Jupyter Notebook.

Create a notebook and store it temporarily

To create a notebook and store it temporarily:

1. Navigate to Data management > Build solutions > Jupyter Notebook.

2. On the starting page, select + and select the file to create.

3. Save the changes before leaving Jupyter Notebook.

Each Python file (.ipynb) has an authentication helper to run the code with the current user's credentials, ensuring that they can work with data relevant to their role and projects.

 from cognite.client import CogniteClient
 # Authenticate under current user credentials
 client = CogniteClient()

If several users are working on the same file, Jupyter Notebook doesn't solve conflicts and you risk losing changes.

Cognite Copilot

Cognite Copilot is integrated into Jupyter Notebook. It uses the Cognite Python SDK to help streamline your data analysis and engineering tasks.

You can use Copilot to:

• Generate code snippets — input your coding intentions in natural language to generate Python code snippets based on your description.
• Change existing code — change and improve the existing code with suggestions from Copilot.

Using Copilot doesn't guarantee that the code is error-free or that the answers are always accurate. To make sure the information you receive is accurate:

• Verify the source of the answers to make sure it's correct.
• Check the code just like you do with code written by engineers.
• Use clear instructions and code commands, for example, "Find P90 values for Dataframe rows."
• Split complex tasks, for example, have one small task per cell.
• Use the Cognite Data Fusion terminology.

To learn how Copilot complies with the existing privacy, security and compliance commitments, refer to the Trust and security article.

Use Copilot

1. In the notebooks with the .ipynb extension, select Copilot in the first empty cell or a cell with existing code > Generate code.

2. Write a short instruction prompt and select Generate.

   Copilot has access to code in other cells and can understand what variables and functions you have in scope. You can also use it to update the code already in the cell.

   

Limitations

The Jupyter Notebook environment in Cognite Data Fusion has these limitations:

• Python package compatibility — only packages written in Python are supported. The Can't find a pure Python 3 wheel for ‘\<package\>’ error indicates that a package relies on native libraries incompatible with Pyodide. For more details and workarounds, refer to the Pyodide documentation.

• Network restrictions — the browser environment limits network access due to cross-origin resource sharing (CORS) policies. This can affect the access to external data sources and services.

• Hardware constraints — your device's hardware resources can limit browser-based environments. This includes access to graphics processing units (GPUs) and constraints on available memory and processing power.

• Execution limitations — the platform doesn't support long-running processes, and users can't interrupt the execution of notebook cells once they've started. This can affect the management and control of running code.

• Concurrency and timing — multithreading isn't supported. Also, certain functions, such as sleep(), may not perform as expected under Pyodide.