> For the complete documentation index, see [llms.txt](https://docs.oort.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.oort.io/integrations/splunk/aws-s3-splunk-cii-integration.md).
# AWS S3 Splunk Integration
## **Overview**
To establish an integration between Splunk and Cisco Identity Intelligence, please refer to the setup guide section in this document. This integration method utilizes the Splunk Add-on for AWS to ingest log data from your AWS S3 bucket. This method is frequently employed for batch processing of historical or high-volume log data for large organizations. It requires setting up AWS credentials and the accurate specification of the SQS queue region and URL.
## **Prerequisites**
Before you begin, please ensure you have the following:
1. Administrative access to your **Cisco Identity Intelligence**
2. Administrative access to your **Splunk Enterprise** or **Splunk Cloud**
3. The [**Cisco Security Cloud**](https://splunkbase.splunk.com/app/7404) **application** installed from Splunkbase
{% hint style="info" %}
The minimum required version of the Cisco Security Cloud application is 3.4.0 or higher
{% endhint %}
4. The Splunk Add-on for AWS **application** installed from Splunkbase
5. Appropriate permissions to manage AWS S3 buckets and Splunk data inputs
## **Setup Guide**
### Generate External ID
1. In CII, create/select an S3 notification target.
2. On Initial Setup, click Generate External ID.
3. Copy/save that value.
4. Use a fresh generated ID per S3 notification target
### Configure AWS S3
1. Create an S3 Bucket
a. Go to the **Amazon S3** console
b. Click **Create bucket**
c. Provide a **name** for the bucket, such as "splunk-cii-demo-set-up"for this example. Keep this name handy, it will be used throughout the setup process.
d. Click **Create bucket** to complete the process
2. Create an IAM Policy
a. Navigate to **IAM** > **Policies** > **Create Policy**
b. Select **JSON** in the policy editor and paste the following JSON:
{% hint style="info" %} **Note:** Replace `${BUCKET_NAME}` with a suitable name, such as `splunk-cii-demo-set-up`
{% endhint %}
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::${BUCKET_NAME}/*"
}
]
}
```
c. Click Next and provide the policy with a name (In this example, we will use S3WritePolicy)
d. Click Create policy
3. Create an IAM Role
a. Navigate to **IAM** > **Roles** > **Create Role**
b. Select **Custom trust policy** and paste the following JSON:
{% hint style="info" %} **Notes:**
* Replace `${CII_ACCOUNT_ID}` with the Identity Intelligence account ID for Splunk 988897525199
* Replace `${UNIQUE_EXTERNAL_ID}` with the External ID generated in Cisco Identity Intelligence for this S3 notification target
{% endhint %}
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${CII_ACCOUNT_ID}:root"
},
"Action": "sts:AssumeRole",
"Condition": {
"ForAnyValue:StringEquals": {
"sts:ExternalId": "${UNIQUE_EXTERNAL_ID}"
}
}
}
]
}
```
c. Click **Next**.
d. Attach the (S3WritePolicy) policy (created earlier)
e. Provide the role with the name CrossAccountS3WriteRole (This name is required and must not be changed)
f. Click **Create role**
4. Update the S3 Bucket Policy
a. Go to the **Amazon S3** console and select the **splunk-cii-demo-set-up** bucket.
b. Navigate to the **Permissions** tab > **Bucket Policy > Edit**
c. Paste the following JSON into the bucket policy editor:
**Notes:**
* Replace ${S3\_ACCOUNT\_ID} with your **AWS account ID**
* Replace ${ROLE\_NAME} with **CrossAccountS3WriteRole**
* Replace ${BUCKET\_NAME} with your bucket name, e.g. **splunk-cii-set-up**
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${S3_ACCOUNT_ID}:role/${ROLE_NAME}"
},
"Action": [
"s3:PutObject"
],
"Resource": "arn:aws:s3:::${BUCKET_NAME}/*"
}
]
}
```
d. Save changes
5. Create an SNS Topic
a. Go to the **Amazon SNS** console > **Create Topic**
b. Provide a name: SplunkSNS
c. In the **Access Policy**, select **Advanced** and paste the following JSON:\
\
Note: Replace `${BUCKET_NAME}` with your bucket name, e.g. splunk-cii-set-up
```json
{
"Version": "2008-10-17",
"Id": "PolicyForS3Access",
"Statement": [
{
"Sid": "AllowS3Publish",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "SNS:Publish",
"Resource": "*",
"Condition": {
"ArnLike": {
"aws:SourceArn": "arn:aws:s3:::${S3 BUCKET NAME}"
}
}
}
]
}
```
d. Click **Create topic**
6\. Configure S3 Event Notifications for the SNS Topic
* Go to the **Amazon S3** console and select your splunk-cii-set-up bucket.
* Navigate to the **Properties** tab > **Event Notifications**
* Click **Create event notification**
* Configure the event notification:
* **Event name**: Provide any name.
* **Prefix (optional)**: Use if you want to trigger events for specific folders.
* **Event types**: Select **Put** and **Post**.
* **Destination**: Select **SNS Topic** and choose SplunkSNS.
* Save changes
7\. Create Two SQS Queues
* Create Dead-Letter Queue (DLQ):
* Go to the **Amazon SQS** console > **Create Queue**
* Provide the name: SplunkDLQ
* Set **Visibility Timeout** to **300 seconds**
* Click **Create Queue**
8. **Create Main Queue:**
* Go to the **Amazon SQS** console > **Create Queue**
* Provide the name: SplunkMain
* Set **Visibility Timeout** to **300 seconds**
* Under **Dead-letter queue**, select **Enable** and choose SplunkDLQ
* Click **Create Queue**
**Important**: Configure the SQS visibility timeout to prevent multiple inputs from receiving and processing messages in a queue more than once. Set your SQS visibility timeout to 5 minutes or longer. If the visibility timeout for a message is reached before the message is fully processed by the SQS-based S3 input, the message reappears in the queue and is retrieved and processed again, resulting in duplicate data)
9\. Create an SNS Subscription for SQS
* Go to the **Amazon SNS** console > **Subscriptions** > **Create Subscription**.
* Configure the subscription:
* **Topic ARN**: Select SplunkSNS (created earlier)
* **Protocol**: Select **Amazon SQS**
* **Endpoint**: Select SplunkMain
* Click **Create Subscription**.
10\. Generate Access Key and Secret Key
* Go to **IAM** > **Users** and select your user.
* Navigate to the **Security Credentials** tab.
* Under **Access Keys**, click **Create Access Key**.
* Select **Command Line Interface (CLI)** and follow the instructions.
* Download the .csv file containing the **Access Key ID** and **Secret Access Key**.
### Configure Splunk
Note: Starting September 2025, CII no longer provides the audience value when creating an API client. However, the audience field is still required in the Splunk application. Until the application is updated, please use the default audience value [`https://api.oort.io`](https://api.oort.io/) for this field. We will update this documentation once the application no longer requires an audience value.
1. Log in to Splunk.
2. Under Apps, click **Cisco Security Cloud**.
3. On the Application Setup page, click **Configure Application**.
1. When prompted, enter the following values:
* **AWS Access Key ID**: From the .csv file.
* **AWS Secret Access Key**: From the .csv file.
* **SQS Queue URL**: Go to the **Amazon SQS** console, select SplunkMain, and copy the queue URL.
* **External ID**: Use the same CII-generated External ID that you added to the AWS IAM role trust policy.
* **S3 Bucket URL**: Enter a value in the format s3:// *bucket-name*.s3. *region-code*.amazonaws.com. You got this value in the previous section.
* **S3 Bucket Region: The** AWS region where your S3 bucket is hosted (e.g., us-east-1).
## **Troubleshooting**
**In Splunk:**
1. Navigate to **Apps** in Splunk and open the **Splunk Add-on for AWS**.
2. Go to the **Inputs** tab and verify if a new input named test\_splunk\_demo has been created.
3. Select the **Account** tab and find the entry for test\_splunk\_demo.
a. Ensure the account details are correct and match the credentials used during the integration setup.
**In AWS S3:**
1. **Check the S3 Bucket**:
a. Log in to your AWS console and go to the **S3** service.
b. Locate the bucket named splunk-cii-demo-set-up (or the bucket name used during the integration setup).
2. **Verify Data in the S3 Bucket**:
a. Check if data appears in the bucket after clicking **Test Connectivity** in Identity Intelligence.