Files in data modeling

Cognite's core data model uses CogniteFile to represent files. The actual file content is handled by the File service. This article describes how to use files with the core data model.

CogniteFile in the core data model

CogniteFile uses the assets property to associate a file with one or more CogniteAssets, and other properties to hold metadata of the file.

The access control for CogniteFile is handled by the datamodelinstances access-control list (ACL). This is the only ACL required to access files in the core data model.

Working with files using the Python SDK

Create, retrieve and delete files

To create a file, use the Cognite Python SDK CogniteFileApply data class:

    from pathlib import Path
    
    from cognite.client.data_classes.data_modeling import NodeId
    from cognite.client.data_classes.data_modeling.cdm.v1 import CogniteFileApply, CogniteFile
    
    # Create a file node in the specified space:
    client.data_modeling.instances.apply(
        CogniteFileApply(name="FileName", space="Space", external_id="File001")
    )

    # Retrieve the created file node:
    file_info_retrieved = client.data_modeling.instances.retrieve_nodes(
        nodes=NodeId("Space", "File001"), node_cls=CogniteFile
    )
    print(file_info_retrieved)

    # When you have created the file node, to upload the content:
    client.files.upload_content("file.txt", instance_id=NodeId("Space", "File001"))

    # To download the content:
    client.files.download_to_path("downloaded_file.txt", instance_id=NodeId("Space", "File001"))

    # To delete a file, delete the node:
    client.data_modeling.instances.delete(nodes=NodeId("Space", "File001"))

Associate files to CogniteAsset

To associate files to CogniteAsset, provide the instance identifiers of the asset nodes:

    from cognite.client.data_classes.data_modeling import DirectRelationReference, NodeId
    from cognite.client.data_classes.data_modeling.cdm.v1 import CogniteFileApply, CogniteFile

    # Create or modify a file node with the asset references:
    client.data_modeling.instances.apply(
        CogniteFileApply(
            space="Space",
            name = "FileName",
            external_id="File001",
            assets=[
                DirectRelationReference("Space", "Asset001"),
                DirectRelationReference("Space", "Asset002"),
            ],
        )
    )

    # Retrieve the file node to see the asset references:
    file_info_retrieved = client.data_modeling.instances.retrieve_nodes(
        nodes=NodeId("Space", "File001"), node_cls=CogniteFile
    )
    print(file_info_retrieved)

Creating and uploading files using the API

You can ingest files represented with CogniteFile directly through the API. Create a file node in the Data Modeling Service, request an upload link, and upload the file contents. You'll need to replace the placeholders in brackets {} used for the endpoint URLs and payload examples with appropriate values for your data model and files.

Step 1. Create an instance in the Data Modeling Service (DMS)

API documentation: Create or update nodes/edges

Endpoint URL: https://{cdf_cluster}.cognitedata.com/api/v1/projects/{cdf_project_name}/models/instances

Payload example:

{
    "items": [
        {
            "instanceType": "node",
            "space": "{instance_space}",
            "externalId": "{external_id}",
            "sources": [
                {
                    "source": {
                        "type": "view",
                        "space": "cdf_cdm",
                        "externalId": "CogniteFile",
                        "version": "v1"
                    },
                    "properties": {
                        "name": "{file_name}",
                        "mimeType": "{mime_type}"
                    }
                }
            ]
        }
    ]
}

It's important that you include a source object mapping to the CogniteFile view in the list of sources in the payload. Without this, you'll receive an error when requesting an upload link in the next step, because the instance won't be able to be identified by the File service. If you've created a custom view extending CogniteFile that you want to target instead, however, replace the values for space, externalId and version in the source object with the corresponding values for your custom view:

{
    "source": {
        "type": "view",
        "space": "{custom_space}",
        "externalId": "{CustomFileView}",
        "version": "{view_version}"
    }
}

ヒント

You aren't required to provide the mimeType property in the properties object, but it's useful for visibility in Search and for endpoints like Engineering diagram detection to be able to process the file.

Step 2. Request an upload file link

API documentation: Get upload file link

Endpoint URL: https://{cdf_cluster}.cognitedata.com/api/v1/projects/{cdf_project_name}/files/uploadlink

Payload example:

{
    "items": [
        {
            "instanceId": {
                "space": "{instance_space}",
                "externalId": "{external_id}"
            }
        }
    ]
}

情報

If you are uploading a file larger than 5 GiB you'll need to request a Multipart file upload link instead.

Step 3. Upload the file contents

Send a HTTP PUT request to the uploadUrl (or uploadUrls if you're using a multipart file upload link) in the response result from the previous query, with the relevant 'Content-Type' and 'Content-Length' headers according to the content of the file.

Learn more:

• Get upload file link
• Get multipart file upload link