Upload 360° image

You can upload 360° images to Cognite Data Fusion (CDF) and display them in various applications with the Reveal 3D viewer. 360° images are captured as part of a point cloud scan, such as an E57 format file, but you can also capture and upload them manually or using robots.

When displayed in Reveal 3D viewer, the locations of the 360° images are displayed as round markers in the 3D view. Select a marker to enter the 360° image, and use your mouse to rotate the camera in any direction. You can select other markers to move to nearby 360° images.

You need to upload the image files to Cognite Data Fusion (CDF). Each station has six cube map images with a specific set of metadata. These cube map images together form a 360° image, representing six directions (left, front, right, back, top, and bottom). You can access the cube map images separately through file exports from the lidar scanning software or in E57 point cloud files.

tips

You can also use a set of 360° panoramic images (equirectangular images). Before you upload the images to CDF, convert them to cube map images. For example, use a tool, py360convert.

Get started

You can upload 360° images to CDF in two ways:

By uploading E57 files with 360° images, if you have a point cloud file set in the E57 file format that includes both 360-degree images and point cloud data. A zip file of E57 files is also supported.
By uploading other formatted 360° images, if you have 360-degree images and related custom metadata in separate, other (not E57) formatted files.

After uploading in the user interface or using a custom script, you can view 360° images in Search by adding the 360° image collection to a scene. To view 360° images in custom Reveal applications, see Reveal documentation.

varning

Uploading the 360° image metadata (pose information) to the Events API is no longer possible. This functionality has been deprecated and will stop working in the future.

Access capabilities

See access capabilities to add the necessary capabilities to upload and view 360° images.

Limitations

3D API processing now supports extracting 360° images from E57 files for a 3D model with a data set when starting from a script.

varning

The CDF user interface currently doesn't support triggering extraction 360° images when either the source fileId or modelId has a data set.

The extracted 360° image files will be stored in the same data set as the 3D model. If the 3D model doesn't have a data set, the 3D API will also check and use any data set associated with the source file specified for the revision.

If you want the extracted 360° images to be available in a different data set, change the data set association for the files after processing is complete using file update requests.

If a data set is used in a modelId or source fileId, specify a dmsInstancesSpaceId value in the indexingOptions for the revision's request body. See Trigger 360° image extraction from E57 files using Python for more information. This specifies the space where the data model instances for 360° images are stored.

E57 files with 360° images

Follow the steps to upload the E57 files to CDF, trigger point cloud filtering and indexing, and opt-in to 360° image extraction.

Upload your E57 files to CDF. If these files are large:
- Upload the files in the ZIP format using the file extractor.
- Or, upload them as individual E57 files, referenced through a JSON file using a custom Python script.
Copy the main/primary file ID (Individual E57, ZIP, or JSON).
Navigate to Data management > Configure > 3D > New > Model.
Enter a model name, leave the data set field empty, and then select Upload new model.
Upload the file, if needed, or select Choose existing uploaded file to process to create a 3D model and enter the file ID (from step 2). Select Upload and process to start processing the point cloud.
Wait a few minutes for the point cloud processing to complete. Refresh the page, if necessary.
Once the model's status changes to Done, select the model > ... > Filtering and indexing.
Select Use the file ID from the previous revision.
Select Extract 360° images and clear Generate tiles option.
Optional. Expand the Filtering section and manage more settings.
Select OK to start the point cloud processing.

You can view the processed point cloud model once the processing is complete.

Trigger 360° image extraction from E57 files using Python

Use Python to trigger the 360° image extraction from E57 files. Set extractPhotoSphereImages to true to turn on the setting. Adjust the file_id, model_id, and dmsInstancesSpaceId.

The example shows the Python script that triggers the extraction of 360° images.

from cognite.client import CogniteClient
client = CogniteClient()

# Specify previously created model ID
model_id = 12346789

# Specify the E57 fileId to be processed for the revision
file_id = 987654321

# Specify the spaceId where the Image360 DMS instances will be stored
space_id = "threed_data"

session = client.iam.sessions.create()

payload = {
  "items": [
    {
      "fileId": file_id,
      "nonce": session.nonce,
      "pipelineConfiguration":
      {
        "indexingOptions" :
        {
          "extractPhotosphereImages": "true",
          "dmsInstancesSpaceId": space_id
        }
      }
    }
  ]
}

response = client.post(f"/api/v1/projects/{client.config.project}/3d/models/{model_id}/revisions", payload)

Custom script for other formatted 360° images

Follow these steps to upload the cube map images to CDF for 360° formats other than E57 and specify the location and orientation of the 360° images in the data modeling service (DMS) instances.

Upload the cubemap images to the Files API, for example, using a Python script. Each image file needs an externalID and a specific set of metadata.

A unique externalID should be set for each file, and it should be the same as referenced from the corresponding Image360 instances image-360-cubemap-front and similar property fields.

Name	Value	Example
`site_id`	A common identifier across all 360° images in the set. You will use this identifier when including the images in a scene.	“site-S01-area-a03”
`site_name`	A human-readable site name.	“Norway Site S01 Area 03”
`station_id`	A unique identifier for the scanning position.	“s01-a03-0012”
`station_name`	A human-readable station name.	“Area 03 - 0012”
`image_type`	Always “cubemap”.	“cubemap”
`image_resolution`	The resolution of the cubemap image. The recommended resolution of cubemap images is 2048 x 2048 pixels.	2048
`face`	The direction of the image (one of left, front, right, back, top, and bottom).	“left”

For each 360° image collection, use the Data modeling API to create a set of instances to specify the location and orientation of the images. For more information about ingesting and removing instances, see Ingesting instances into a graph or Create or update nodges/edges.

Three data model views are made explicitly for 360° images:

Image360Collection
Station360
Image360

If your project admin has allowed this, the models will be available with the cdf_360_image_schema system data model in all CDF projects.

You can store the instances in the same space, when ingesting instances for a collection through these three data model views. The DMS instances should have the following metadata:

Image360Collection

{
  "items": [
  {
      "instanceType": "node",
      "space": collection-space-id,
      "externalId": collection-external-id,
      "sources": [
      {
         "source": {
            "type": "view",
            "space": "cdf_360_image_schema",
            "externalId": "Image360Collection",
            "version": "v1"
         },
         "properties": {
            "label": collection-label
         }
      }
      ]
  }
 ],

"autoCreateDirectRelations": true,
"autoCreateStartNodes": false,
"autoCreateEndNodes": false,
"skipOnVersionConflict": false,
"replace": false
}

Value	Description
`collection_label`	The name of the collection, for example, “site-factory1-area-a01”.
`collection-space-id`	The space ID to store the `Image360Collection` instance.
`collection-external-id`	The external ID of the `Image360Collection` instance.

Station360

{
   "items": [
      {
         "instanceType": "node",
         "space": station-space-id,
         "externalId": station-external-id,
         "sources": [
         {
            "source": {
               "type": "view",
               "space": "cdf_360_image_schema",
               "externalId": "Station360",
               "version": "v1"
            },
            "properties": {
               "label": station-label
            }
         }
         ]
      }
   ],
   "autoCreateDirectRelations": true,
   "autoCreateStartNodes": false,
   "autoCreateEndNodes": false,
   "skipOnVersionConflict": false,
   "replace": false
}

Value	Description
`station-space-id`	The space ID to store the `Station360` instance.
`station-external-id`	The external ID of the `Station360` instance.
`station-label`	The name of the station, for example, “station-0012”.

Image360

{
   "items": [
      {
         "instanceType": "node",
         "space": image-360-space-id,
         "externalId": image-360-external-id,
         "sources": [
         {
            "source": {
               "type": "view",
               "space": "cdf_360_image_schema",
               "externalId": "Image360",
               "version": "v1"
            },
            "properties": {
               "image": {
                  "label": image-360-label,
                  "translationX": image-360-translation-x,
                  "translationY": image-360-translation-y,
                  "translationZ": image-360-translation-z,
                  "eulerRotationX": image-360-euler-rotation-x,
                  "eulerRotationY": image-360-euler-rotation-y,
                  "eulerRotationZ": image-360-euler-rotation-z,
                  "scaleX": image-360-scale-x,
                  "scaleY": image-360-scale-y,
                  "scaleZ": image-360-scale-z,
                  "cubeMapFront": image-360-cubemap-front,
                  "cubeMapBack": image-360-cubemap-back,
                  "cubeMapLeft": image-360-cubemap-left,
                  "cubeMapRight": image-360-cubemap-right,
                  "cubeMapTop": image-360-cubemap-top,
                  "cubeMapBottom": image-360-cubemap-bottom,
                  "collection": image-360-collection,
                  "station": image-360-station,
                  "timeTaken": image-360-time-taken
               }
            }
         }
         ]
      }
   ],
   "autoCreateDirectRelations": true,
   "autoCreateStartNodes": false,
   "autoCreateEndNodes": false,
   "skipOnVersionConflict": false,
   "replace": false
   }

Value	Description
`image-360-space-id`	The space ID to store the `Image360` instance. Normally, the same as the space ID for the `Image360Collection` and `Station360` instances.
`image-360-external-id`	The external ID of the `Image360` image set instance. Note that there's only one `Image360` instance for each set of 6 images.
`image-360-label`	The name of the image set, for example, “station-0012-images”.
`image-360-translation-x`, `image-360-translation-y`, `image-360-translation-z`	The translation coordinates of the image set.
`image-360-euler-rotation-x`, `image-360-euler-rotation-y`, `image-360-euler-rotation-z`	The rotation angles for the front image in the image set. The rotation angles are in radians.
`image-360-scale-x`, `image-360-scale-y`, `image-360-scale-z`	The scale factors of the image set. Normally, 1.0.
`image-360-cubemap-front`, `image-360-cubemap-back`, `image-360-cubemap-left`, `image-360-cubemap-right`, `image-360-cubemap-top`, `image-360-cubemap-bottom`	The external IDs of the cube map images stored in the Files API.
`image-360-collection`	The external ID of the `Image360Collection` instance.
`image-360-station`	The name of the `Station360` instance.
`image-360-time-taken`	The time when the images were taken.

Add 360° images to the scene

To see the 360° images in CDF programs and applications using the Reveal 3D viewer, add 360° images to the scene.

For the E57 file (set) with both point cloud and 360° images, add 3D data when you configure or edit the scene. See Unified 3D contextualization for more information.

obs!

The 360° images extracted from the E57 file set will be put in a 360° image collection named site-<modelId>-<revisionId>. For example, site-111-222, where 111 is the model ID and 222 is the revision ID.

Get started​

Access capabilities​

Limitations​

E57 files with 360° images​

Trigger 360° image extraction from E57 files using Python​

Custom script for other formatted 360° images​

Image360Collection​

Station360​

Image360​

Add 360° images to the scene​