> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cognite.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Set up the SAP extractor

> Learn how to set up and configure the SAP extractor to connect to SAP endpoints and extract data to CDF.

## Before you start

* Assign [access capabilities](/cdf/access/guides/capabilities#sap-extractor) for the extractor to *write* data to the respective CDF destination resources.

* Set up the SAP endpoints you want to extract data from.

* Check the [server requirements](/cdf/integration/guides/extraction/sap/sap_server_requirements) for the extractor.

* Create a configuration file according to the [configuration settings](/cdf/integration/guides/extraction/configuration/sap). The file must be in YAML format.

## Connect to SAP

The extractor supports two different protocols for connecting to SAP: OData and SOAP.

<Tabs>
  <Tab title="OData">
    The extractor connects to OData V2 and OData V4 endpoints in the [SAP NetWeaver Gateway](https://help.sap.com/saphelp_em92/helpdata/en/ec/aeea50ca692309e10000000a445394/content.htm?no_cache=true).
    SAP OData is available in multiple SAP ERP versions, such as:

    | SAP version           | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
    | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
    | SAP ERP 6.0           | We recommend the [SAP Gateway Service builder](https://help.sap.com/docs/SAP_NETWEAVER_AS_ABAP_751_IP/68bf513362174d54b58cddec28794093/cddd22512c312314e10000000a44176d.html), which automates and generates OData entities from the SAP standard data schemas. There are multiple ways of mapping SAP entities to OData entities. To ensure that schema mapping is done correctly on the entity level, use the **Import DDIC Structure** function for structures or database table that you expose through the OData service. [See this guide](https://blogs.sap.com/2021/05/06/a-step-by-step-process-to-create-odata-services-in-sap-sap-hana-system) for creating an OData service in SAP ERP 6.0. |
    | SAP S/4HANA OnPremise | We recommend using the standard OData endpoints delivered in the SAP S/4HANA installation. You'll find the endpoints at the [SAP Business Accelerator Hub](https://api.sap.com). To extract data using SAP OData predefined schemas, activate the standard endpoints on the [/IWFND/MAINT\_SERVICES](https://help.sap.com/doc/saphelp_nw75/7.5.5/en-US/bb/2bfe50645c741ae10000000a423f68/content.htm?no_cache=true) transaction. For instance, the standard endpoint for [SAP plant maintenance (PM) work orders](https://api.sap.com/api/OP_API_MAINTENANCEORDER_0002/overview).                                                                                                                      |
    | SAP S/4HANA Cloud     | The OData endpoints are available inside the cloud communication scenarios predefined by SAP. You'll find the standard endpoints at the [SAP Business Accelerator Hub](https://api.sap.com). For instance, the standard endpoint for [SAP plant maintenance (PM) work orders](https://api.sap.com/api/CE_API_MAINTENANCEORDER_0002/overview).                                                                                                                                                                                                                                                                                                                                                          |

    See [the SAP documentation](https://help.sap.com/doc/saphelp_ssb/1.0/en-US/71/d07b52a3566f54e10000000a44176d/content.htm?no_cache=true) for more information about SAP OData endpoints.

    <Note>
      We recommend using OData to connect to and extract data from SAP.
    </Note>
  </Tab>

  <Tab title="SOAP">
    The extractor connects to  [SAP ABAP Web Services](https://help.sap.com/doc/saphelp_nw74/7.4.16/en-us/48/52347a08e672d0e10000000a42189c/content.htm?no_cache=true) exposed through the SOAP protocol.

    Every published SAP Web service will have a corresponding XML-based description, accessible in the Web Service Description Language (a WSDL document). The extractor connects to the Web service by reading the definitions in the WSDL URL and triggering the defined operations.

    For more information about SAP ABAP Web Services, see [SAP's official documentation](https://help.sap.com/docs/SAP_NETWEAVER_750/684cffda9cbc4187ad7dad790b03b983/cf9fb513ced74bbf82cac2231a358086.html).
  </Tab>
</Tabs>

## Secure SAP connections with self-signed certificates

The extractor uses HTTPS to securely connect to SAP OData services. If your SAP system uses an internal or self-signed certificate, update the extractor’s certificate bundle to include it so connections between SAP, the VM environment, and the CDF service remain trusted.

### Export the existing certificate bundle

1. Locate the public CA bundle in the `certifi` library:

   ```python theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
   import certifi

   # Get the path to the CA bundle
   ca_bundle_path = certifi.where()
   print(f"CA bundle located at: {ca_bundle_path}")
   ```
2. Copy the file from the printed path and save it on your desktop as `cacert.pem`.
3. Add the SAP self-signed certificate
   1. Obtain the SAP self-signed certificate in PEM format.
   2. Open the certificate in a text editor.
   3. Open `cacert.pem` from step 1.
   4. Prepend the SAP certificate contents to the top of the file.
   5. Save the updated file.

Now, `cacert.pem` contains both public and SAP self-signed certificates.

### Configure the extractor

1. Set the environment variable `REQUESTS_CA_BUNDLE` to the path of the updated `cacert.pem`:

<Tabs>
  <Tab title="Windows (PowerShell)">
    ```powershell theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
    $env:REQUESTS_CA_BUNDLE="C:\Users\<username>\Desktop\cacert.pem"
    ```
  </Tab>

  <Tab title="Linux/MacOS (Bash)">
    ```bash theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
    export REQUESTS_CA_BUNDLE=/home/<username>/Desktop/cacert.pem
    ```
  </Tab>
</Tabs>

2. Run the extractor in the same shell where the environment variable is defined.

<Note>
  The certificate must be manually added to the bundle on the VM to ensure both SAP and CDF connections work correctly.
</Note>

## Run as a Windows executable file

<Steps>
  <Step title="Download the SAP extractor package">
    Navigate to <span class="ui-element">Data fusion</span> > <span class="ui-element">Integrate</span> > <span class="ui-element">Extractors</span> and find the SAP extractor's package for **Windows executable**. Download and decompress the zip file.
  </Step>

  <Step title="Run the executable file">
    Open a command line window and run the executable file with the configuration file as an argument.

    In this example, the configuration file is named *config.yml* and saved in the same folder as the executable file:

    ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
    > .\sap_extractor_standalone<VERSION>-win32.exe .\config.yml
    ```

    <Info>
      You stop the extractor by pressing **Ctrl+C** on your keyboard. The log file is stored in the [configured path](/cdf/integration/guides/extraction/configuration/sap#logger.file).
    </Info>
  </Step>
</Steps>

## Run as a Windows service

<Steps>
  <Step title="Download the Windows service package">
    Navigate to <span class="ui-element">Data fusion</span> > <span class="ui-element">Integrate</span> > **Extractors** and find the SAP extractor's installation package for **Windows service**. Download and decompress the zip file to the same directory as a configuration file.

    <Warning>
      You must name the configuration file `config.yaml`.
    </Warning>
  </Step>

  <Step title="Install the service">
    As an administrator, open up a command line window in the folder you placed the executable file and the configuration file. Run the following command:

    ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
    > .\sap_extractor_service<VERSION>-win32.exe install
    ```
  </Step>

  <Step title="Configure the service">
    Open **Services** in Windows and find the **Cognite SAP extractor** service. Right-click the service and select **Properties**. Configure the service according to your requirements.
  </Step>
</Steps>

## Run as a Linux executable

<Steps>
  <Step title="Download the Linux executable package">
    Navigate to <span class="ui-element">Data fusion</span> > <span class="ui-element">Integrate</span> > **Extractors** and find the SAP extractor's package for **Linux executable**. Download and decompress the zip file.
  </Step>

  <Step title="Run the executable file">
    Open a command line window and run the executable file with the configuration file as an argument.

    In this example, the configuration file is named *config.yml* and saved in the same folder as the executable file:

    ```yaml theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
    > ./sap_extractor-<VERSION>-linux path/to/the/folder/config.yaml
    ```
  </Step>
</Steps>

## Pagination

When you extract data from SAP OData, use different pagination methods: no pagination, client-side pagination, and server-side pagination.

Use the [pagination-type](/cdf/integration/guides/extraction/configuration/sap#endpoints) configuration setting to specify the pagination type to use when running full-load queries.

<Info>
  Pagination is available when you extract data from SAP OData endpoints.
</Info>

<Tabs>
  <Tab title="No pagination">
    No pagination means that the extractor fetches all data available from the SAP OData endpoint without using chunking logic on the server or client. Therefore, the SAP extractor may time out and return an error while waiting for the response from SAP.
  </Tab>

  <Tab title="Client-side pagination">
    OData client-side pagination uses query parameters from the client to define a record offset. This limits the volume of data retrieved from the server.

    A client-side pagination request is built with the following URI parameters:

    * **\$top**: specifies the number of records to return in a single batch. The default and maximum value is 1,000 records.
    * **\$skip**: specifies the number of records to bypass (skip) from the total data set before returning the desired subset.

    For example, a query with `$skip=2000&$top=500` returns the fifth page of data, assuming there's data available and the page size is 500 records.
  </Tab>

  <Tab title="Server-side pagination">
    The SAP server controls server-side pagination. The server generates a cursor to control the next batch of requests. This cursor represents a pointer to the start of the next page in the full data set and is returned to the calling SAP extractor.
    The server uses a `$skiptoken` value to resume pagination from the position identified by the cursor.
  </Tab>
</Tabs>

<Tip>
  The pagination type you use in your extractor configuration depends on the SAP OData endpoint implementation your extractor connects to.
</Tip>

## Load data incrementally (OData only)

If the OData entities have an **incremental field**, you can set up the extractor to only process new or updated entities since the last extraction.

For example, you can use the S/4HANA standard OData entity `Maintenance Order` and enter the field `LastChangeDateTime` in the [`incremental-field`](/cdf/integration/guides/extraction/configuration/sap#endpoints) configuration parameter for incremental delta queries to the CDF staging area.

The incremental field in SAP must be an `Edm.DateTimeOffset` field—for example, `LastChangeDateTime` on the `Maintenance Order` entity.

<Warning>
  Your SAP OData endpoint must return records ordered by the incremental field in **ascending** order. The extractor updates the state store with the timestamp from the last processed row. If records arrive in descending order (latest records first), the last row contains the oldest timestamp and the state store never advances. On each run, the extractor then re-fetches data from that old date and places extra load on the SAP server.
</Warning>

The extractor depends on client-side pagination for incremental load queries. When you configure the SAP extractor for a custom SAP OData endpoint, make sure your SAP implementation supports client-side pagination and the `$orderby` operation. See [Client-side pagination implementation](https://help.sap.com/doc/saphelp_ssb/1.0/en-US/71/d07b52a3566f54e10000000a44176d/content.htm?no_cache=true) for more information on the implementation provided by SAP.

## Schedule automatic runs

To schedule automatic runs on Windows, you can [run the extractors in Windows Task Scheduler](/cdf/integration/guides/extraction/run_extractor_win_task_scheduler).

To schedule automatic runs on Mac OS and Linux, use [cron expressions](https://en.wikipedia.org/wiki/Cron). To enter a new cron job, run `crontab -e` to edit the cron table file with the default system text editor.

Here's the format for a job in the cron table:

```bash showLineNumbers theme={"languages":{"custom":["/_languages/kuiper.json","../_languages/kuiper.json"]}}
<minute>  <hour>  <day of month (1-31)>  <month (1-12)>  <day of week (0-6 starting on Sunday)>  <command>
```

## Attachments

The extractor can extract attachments stored in SAP document frameworks, such as Generic Object Services (GOS), and ingest these into CDF Files.
The extractor connects to the SAP OData endpoint [API\_CV\_ATTACHMENT\_SRV](https://api.sap.com/api/OP_API_CV_ATTACHMENT_SRV_0001/overview) and fetches files that are linked to a standard SAP OData entity, such as maintenance orders.

See the [attachments](/cdf/integration/guides/extraction/configuration/sap#endpoints.attachments) configuration section.

<Note>
  You can extract attachments when you've connected to SAP S/4HANA servers.
</Note>