Skip to main content

Filtering

Cognite Data Fusion (CDF) supports boolean logic using AND, OR, and NOT to build complex queries. You can also combine resource item attribute filters, such as equals, prefix, and range.

The default set of CDF filters lets you filter all CDF resources in these contexts:

  • CDF resources, for example, assets and events, by basic fields, metadata keys, and values.

  • Full-text search results. Use filters to narrow down the returned search results. For example, you can use a filter to return only "PDFs from last week" from your full-text search query.

  • Aggregate results. Use filters to reduce the number of returned aggregate buckets. For example, you can use a filter to return "metadata key names and their count for all keys with the prefix character '_'.

Serialization/deserialization​

You can express filters as JSON. The root must always be an object.

Properties​

CDF uses the same grammar to express a reference everywhere the API uses a resource attribute: a JSON array of strings to express the attribute/property. This is true for leaf filters, aggregate, sort, etc.

There are no naming constraints for the allowed property names. For leaf filters, CDF uses "property" to refer to the field(s) the filter operates on.

Example: nesting and sorting​

In the example object below, you can refer to the nested values as ["metadata", "../type"]. Or, you can use another nested property ["metadata", "type"] and then address the un-nested property as ["type"].

{
  "id": 1234,
  "metadata": {
    "../type": "critical",
    "type": "high"
  },
  "type": "alert"
}

To sort the /list result on metadata.type and type properties, send the following request:

{
  "sort": [
    {
      "property": ["metadata", "type"],
      "order": "asc",
      "nulls": "last"
    },
    {
      "property": ["type"],
      "order": "desc"
    }
  ]
}
note

You can sort by several properties in the same query.

Example: filtering​

This example defines a filter using JSON:

{
  "or": [
    {
      "not": {
        "and": [
          {
            "equals": {
              "property": ["type"],
              "value": "alert"
            }
          },
          {
            "in": {
              "property": ["subtype"],
              "values": ["critical", "warning"]
            }
          },
          {
            "range": {
              "property": ["rating"],
              "gte": 5,
              "lt": 10
            }
          }
        ]
      }
    },
    {
      "and": [
        {
          "equals": {
            "property": ["metadata", "priority"],
            "value": "highest"
          }
        },
        {
          "equals": {
            "property": ["subtype"],
            "value": "critical"
          }
        }
      ]
    }
  ]
}

A simplified, less verbose version could look like this:

(
    ("metadata.priority"=highest AND subtype=critical)
    OR
    (
        NOT (type=alert AND subtype IN (critical, warning) AND 5 <= rating < 10)
    )
)

Even if the JSON object is more verbose, the object is easy to express and handle programmatically. JSON is also unambiguous and doesn't impose rules on naming attributes/properties. You also can't mistake a property name for a keyword in JSON.

Below is another example of how using JSON to express a simple intent: "filter items where the attribute source is equal to the value git."

{
  "equals": {
    "property": ["source"],
    "value": "git"
  }
}

Filter language specification​

FilterSyntax
filter<bool_filter> | <leaf_filter>
bool_filter<and> | <or> | <not>
and{"and": [<filter>, …]}
or{"or": [<filter>, …]}
not{"not": <filter>}
property<list_of_strings>
leaf_filter<equals> | <in> | <range> | <prefix> | <exists> | <containsAny> | <containsAll>
equals{"equals": {"property": <property>, "value": <string_value>}}
in{"in": {"property": <property>, "values": <list_value>}}
range{"range": {"property": <property>, "(gte|gt|lte|lt)": <value1> [, "(gte|gt|lte|lt)": <value\2>]}}
prefix{"prefix": {"property": <property>, "value": <string_value>}}
exists{"exists": {"property": <property>}}
containsAny{"containsAny": {"property": <property>, "values": <list_value>}}
containsAll{"containsAll": {"property": <property>, "values": <list_value>}}
info

Different CDF resources may support a different subset of the existing leaf filters. Refer to the documentation for the specific CDF resource for details.

Boolean filters​

AND​

To express boolean AND logic, use the and filter expressed in JSON:

{"and": [ filter1, filter2, …, filterN]}

OR​

To express boolean OR logic, use the or filter (expressed in JSON):

{"or": [ filter1, filter2, …, filterN]}

NOT​

To express the negated logic of a boolean value, use the not filter (expressed in JSON):

{"not": filter}

Leaf filters​

note

The supported leaf filters may differ depending on the CDF resource.

Equals​

Filter items where the property (named value) equals the specified value. This filter works for properties with values that aren't arrays/lists.

Filter items where the property (name) equals the given name and has a specific value. This filter works for properties with values that aren't arrays/lists.

{
  "equals": {
    "property": ["rootId"],
    "value": 12432432423
  }
}

In​

Filter items where the property value equals one of the values specified in the list. This filter expects the value stored by the property to be a single value (not a list).

{
  "in": {
    "property": ["datasetId"],
    "values": [1, 2, 3]
  }
}

Range​

Filter items where the property value is in the range specified by the filter boundaries: gt|gte,|lt|lte.

{
  "range": {
    "property": ["count"],
    "gte": 1,
    "lte": 2
  }
}

You must specify gt, lt, lte, and/or gte, and at least one boundary value to compare the property with.

Range behavior depends on the type of the property you are comparing with. The filter language does not require that typing must exist. If all the values are stored as strings, the range filter will behave as if comparing string values.

This filter works for properties that are not typed as arrays/lists.

note

Some CDF resources don't support open-range filters. To avoid errors, specify both upper and lower boundaries. See the API documentation for details.

Prefix​

Filter items where the property value starts with the specified string. This filter works for properties that aren't typed as array/list.

{
  "prefix": {
    "property": ["metadata", "vendor"],
    "value": "si"
  }
}

Exists​

Filter items where the property value is something other than null (empty). This filter works for properties of any type.

{
  "exists": {
    "property": ["externalId"]
  }
}

ContainsAny​

Filter items where the property values equal one or more of the specified values from the filter. This filter works for properties that are of type array/list.

{
  "containsAny": {
    "property": ["labels"],
    "values": ["pump", "tube"]
  }
}

ContainsAll​

Filter items where the values from the property match all the values specified. This filter works for properties that are of type array/list.

{
  "containsAll": {
    "property": ["assetIds"],
    "values": [23234324, 2423423]
  }
}
Last updated: