---
title: "Securing S3"
date: "2019-04-03"
summary: Ozone supports S3 protocol, and uses AWS Signature Version 4 protocol which allows a seamless S3 experience.
weight: 5
menu:
   main:
      parent: Security
icon: cloud
---
<!---
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->

To access an S3 bucket, users need AWS access key ID and AWS secret. Both of
these are generated by going to AWS website. When you use Ozone's S3
protocol, you need the same AWS access key and secret.

Under Ozone, the clients can download the access key directly from Ozone.
The user needs to `kinit` first and once they have authenticated via kerberos
 they can download the S3 access key ID and AWS secret. Just like AWS S3,
 both of these are secrets that needs to be protected by the client since it
 gives full access to the S3 buckets.

## Obtain Secrets

S3 clients can get the secret access id and user secret from OzoneManager.

### Using the command line

For a regular user to get their own secret:
```bash
ozone s3 getsecret
```

An Ozone administrator can get a secret for a specific user by using the `-u` flag:
```bash
ozone s3 getsecret -u <username>
```

### Using the REST API

A user can get their own secret by making a `PUT` request to the `/secret` endpoint:
```bash
curl -X PUT --negotiate -u : https://localhost:9879/secret
```

An Ozone administrator can get a secret for a specific user by appending the username to the path:
```bash
curl -X PUT --negotiate -u : https://localhost:9879/secret/<username>
```

This command will talk to ozone, validate the user via Kerberos and generate
the AWS credentials. The values will be printed out on the screen. You can
set these values up in your _.aws_ file for automatic access while working
against Ozone S3 buckets.

<div class="alert alert-danger" role="alert">
 Please note: These S3 credentials are like your Kerberos passwords
 that give complete access to your buckets.
</div>


* Now you can proceed to setup these secrets in aws configs:

```bash
aws configure set default.s3.signature_version s3v4
aws configure set aws_access_key_id ${accessId}
aws configure set aws_secret_access_key ${secret}
aws configure set region us-west-1
```
Please refer to AWS S3 documentation on how to use S3 via command line or via
S3 API.

## Revoking Secrets via REST API

To invalidate/revoke the secret, use `ozone s3 revokesecret` command.
Alternatively, you can use the REST API endpoint to revoke the secret.
Ozone now provides a REST API endpoint that allows administrators to revoke S3 access secrets. This operation invalidates a secret, ensuring it can no longer be used for authentication.

### Endpoint Details

- **URL:** `http://localhost:9879/secret`
- **HTTP Method:** `DELETE`

### Authentication

The API leverages SPNEGO (Kerberos) authentication. The following curl options are used:
- `--negotiate` enables SPNEGO.
- `-u :` uses the current Kerberos ticket (an empty username is provided).

### Example 1: Revoke Secret for the Current User

This command revokes the secret for the currently authenticated user:

```bash
curl -X DELETE --negotiate -u : -v http://localhost:9879/secret
```

### Example 2: Revoke Secret by Username

This command revokes the secret for a specific user by appending the username as a query parameter. Replace `testuser` with the desired username:

```bash
curl -X DELETE --negotiate -u : -v "http://localhost:9879/secret?username=testuser"
```

### Response

- **Success:** Returns HTTP `200 OK` along with a confirmation message in JSON format.
- **Failure:** Returns an appropriate HTTP error status and message if there are issues (e.g., authentication failures).

### Testing and Verification

For a working example of these operations, refer to the [Secret Revoke Robot Test](https://raw.githubusercontent.com/apache/ozone/refs/heads/master/hadoop-ozone/dist/src/main/smoketest/s3/secretrevoke.robot). This test demonstrates both the default secret revocation and the revocation by username.

> **Note:** Ensure your Kerberos authentication is correctly configured, as secret revocation is a privileged operation.

## External S3 Secret Storage with HashiCorp Vault

By default, S3 secrets are stored in the Ozone Manager's RocksDB. For enhanced security, Ozone can be configured to use HashiCorp Vault as an external secret storage backend.

### Configuration

To enable Vault integration, you need to configure the following properties in `ozone-site.xml`:

| Property                                                 | Description                                                                                                                              |
| -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `ozone.secret.s3.store.provider`                         | The S3 secret storage provider to use. Set this to `org.apache.hadoop.ozone.s3.remote.vault.VaultS3SecretStorageProvider` to enable Vault. |
| `ozone.secret.s3.store.remote.vault.address`             | The address of the Vault server (e.g., `http://vault:8200`).                                                                             |
| `ozone.secret.s3.store.remote.vault.namespace`           | The Vault namespace to use.                                                                                                              |
| `ozone.secret.s3.store.remote.vault.enginever`           | The version of the Vault secrets engine (e.g., `2`).                                                                                     |
| `ozone.secret.s3.store.remote.vault.secretpath`          | The path where the secrets are stored in Vault.                                                                                          |
| `ozone.secret.s3.store.remote.vault.auth`                | The authentication method to use with Vault. Supported values are `TOKEN` and `APPROLE`.                                                 |
| `ozone.secret.s3.store.remote.vault.auth.token`          | The Vault authentication token. Required if `ozone.secret.s3.store.remote.vault.auth` is set to `TOKEN`.                                   |
| `ozone.secret.s3.store.remote.vault.auth.approle.id`     | The AppRole RoleID. Required if `ozone.secret.s3.store.remote.vault.auth` is set to `APPROLE`.                                            |
| `ozone.secret.s3.store.remote.vault.auth.approle.secret` | The AppRole SecretID. Required if `ozone.secret.s3.store.remote.vault.auth` is set to `APPROLE`.                                           |
| `ozone.secret.s3.store.remote.vault.auth.approle.path`   | The AppRole path. Required if `ozone.secret.s3.store.remote.vault.auth` is set to `APPROLE`.                                             |
| `ozone.secret.s3.store.remote.vault.trust.store.type` | The type of the trust store (e.g., `JKS`). |
| `ozone.secret.s3.store.remote.vault.trust.store.path` | The path to the trust store file.         |
| `ozone.secret.s3.store.remote.vault.trust.store.password` | The password for the trust store.         |
| `ozone.secret.s3.store.remote.vault.key.store.type`   | The type of the key store (e.g., `JKS`).   |
| `ozone.secret.s3.store.remote.vault.key.store.path`   | The path to the key store file.           |
| `ozone.secret.s3.store.remote.vault.key.store.password`   | The password for the key store.           |

### Example

Here is an example of how to configure Ozone to use Vault for S3 secret storage with token authentication:

```xml
<property>
  <name>ozone.secret.s3.store.provider</name>
  <value>org.apache.hadoop.ozone.s3.remote.vault.VaultS3SecretStorageProvider</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.address</name>
  <value>http://localhost:8200</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.enginever</name>
  <value>2</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.secretpath</name>
  <value>secret</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth</name>
  <value>TOKEN</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth.token</name>
  <value>your-vault-token</value>
</property>
```

### Example with SSL

Here is an example of how to configure Ozone to use Vault for S3 secret storage with SSL:

```xml
<property>
  <name>ozone.secret.s3.store.provider</name>
  <value>org.apache.hadoop.ozone.s3.remote.vault.VaultS3SecretStorageProvider</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.address</name>
  <value>https://localhost:8200</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.enginever</name>
  <value>2</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.secretpath</name>
  <value>secret</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth</name>
  <value>TOKEN</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.auth.token</name>
  <value>your-vault-token</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.trust.store.path</name>
  <value>/path/to/truststore.jks</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.trust.store.password</name>
  <value>truststore-password</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.key.store.path</name>
  <value>/path/to/keystore.jks</value>
</property>
<property>
  <name>ozone.secret.s3.store.remote.vault.key.store.password</name>
  <value>keystore-password</value>
</property>
```

### References

*   [HashiCorp Vault Documentation](https://developer.hashicorp.com/vault/docs)
