Object storage

Object storage is not available on projects that are part of an Upsun Cloud trial.

Object storage is an S3-compatible bucket attached to your application, provisioned and connected automatically once you set a size on the app’s resources. Object storage is integrated with the rest of the Upsun Cloud platform:

When you back up an environment, the bucket’s contents are included.
When you branch an environment, the bucket and its allocated size are cloned along with the environment’s other data.

When to use object storage

Use object storage when your app needs to store large or growing volumes of unstructured data (uploads, media, backups, generated artifacts) and you want to reach it through the S3 API. For small amounts of file data that a single instance reads and writes through the filesystem, a persistent mount is usually a better fit.

Set the object-storage size

CLI
Console

Set

Terminal

upsun resources:set --object-storage myapp:524288

The same resources:set --object-storage <APP_NAME>:<SIZE_IN_MB> command (size in MB) enables object storage if the app doesn’t have any yet, or resizes it if it does. Any value must be a multiple of the step size; this example uses the minimum allocation, 512 GB.

View

Terminal

upsun resources:get

When an app has object storage, the output includes an Object storage (MB) column.

Remove

Setting the storage size to 0, or removing the app, deletes the bucket and all of its contents on the next deployment. There’s no soft-delete or retention period. Back up any data you need before removing.

Set the bucket size to 0:

Terminal

upsun resources:set --object-storage myapp:0

Object storage sizing rules

Object storage is allocated in fixed steps, within a minimum and a maximum. The size you request must be a multiple of the step and fall within these bounds:

Rule	Value
Step	524288 MB (512 GB)
Minimum	524288 MB (512 GB)
Maximum	10485760 MB (10 TB)

Storage pricing

Object storage requires a minimum allocation of 512 GB and is billed on allocated capacity (the storage size that you specify, not the amount of data actually stored), prorated hourly. Backup storage is billed separately as its own line item, also based on allocated capacity. For pricing information, see the Upsun pricing page.

Relationship reference

Enabling object storage adds a relationship named object-storage to your app. This relationship name is reserved. If your configuration already declares a relationship called object-storage, the deployment fails and you must rename the relationship to proceed. The relationship provides the connection details for an S3-compatible endpoint. Upsun Cloud automatically generates service environment variables for it within your application container, in the $<RELATIONSHIP-NAME>_<SERVICE-PROPERTY> format. The prefix follows the relationship name, so a container that reaches the bucket through a relationship named bucket, as in Share a bucket across containers, gets BUCKET_ACCESS_KEY rather than OBJECT_STORAGE_ACCESS_KEY. Here is example information available through the service environment variables themselves, or through the PLATFORM_RELATIONSHIPS environment variable.

Service environment variables
PLATFORM_RELATIONSHIPS environment variable

You can obtain the complete list of available service environment variables in your app container by running upsun ssh env.Note that the information about the relationship can change when an app is redeployed or restarted. So your apps should only rely on the service environment variables directly rather than hard coding any values.

For some advanced use cases, you can use the PLATFORM_RELATIONSHIPS environment variable. The structure of the PLATFORM_RELATIONSHIPS environment variable can be obtained by running upsun relationships in your terminal:Here is an example of how to gather that information in a .environment file:

.environment

# Decode the built-in credentials object variable.
export RELATIONSHIPS_JSON="$(echo "$PLATFORM_RELATIONSHIPS" | base64 --decode)"

# Set environment variables for individual credentials.
export AWS_ACCESS_KEY_ID="$(echo "$RELATIONSHIPS_JSON" | jq -r '.["object-storage"][0].access_key')"
export AWS_SECRET_ACCESS_KEY="$(echo "$RELATIONSHIPS_JSON" | jq -r '.["object-storage"][0].secret_key')"
export AWS_DEFAULT_REGION="$(echo "$RELATIONSHIPS_JSON" | jq -r '.["object-storage"][0].region')"

# Some SDKs read AWS_REGION instead of AWS_DEFAULT_REGION.
export AWS_REGION="$AWS_DEFAULT_REGION"

# Read the bucket name rather than assuming it.
export S3_BUCKET="$(echo "$RELATIONSHIPS_JSON" | jq -r '.["object-storage"][0].buckets | keys[0]')"

The object-storage key has to match the relationship name. A container that reaches the bucket through a relationship named bucket uses .["bucket"][0] instead. A key that matches no relationship exports the literal string null rather than failing, so requests are signed with invalid credentials.

Connect from your app

The endpoint URL is http://<HOST>:<PORT>, built from the host and port values in the relationship. The endpoint is reachable over your project’s internal network only and isn’t exposed publicly. A single bucket is created per app, named after the app, and the relationship lists it in buckets, an object keyed by bucket name. So an app named myapp reaches its bucket at http://<HOST>:<PORT>/myapp/<KEY>. Take the name from the buckets keys rather than assuming it, so the same code keeps working in a container that reaches another app’s bucket through a shared relationship. When configuring your S3 client, note two requirements:

Use path-style addressing. The bucket name goes in the URL path (http://<HOST>:<PORT>/myapp/object.txt), not the hostname. Disable virtual-hosted-style addressing.
Use the credentials from the relationship. The endpoint authenticates every request, so configure your client with the access_key, secret_key, and region values from the relationship. Empty or arbitrary credentials are rejected.

Example (Python, boto3)

import base64
import json
import os

import boto3

relationships = json.loads(base64.b64decode(os.environ["PLATFORM_RELATIONSHIPS"]))
endpoint = relationships["object-storage"][0]

s3 = boto3.client(
    "s3",
    endpoint_url=f"http://{endpoint['host']}:{endpoint['port']}",
    aws_access_key_id=endpoint["access_key"],
    aws_secret_access_key=endpoint["secret_key"],
    region_name=endpoint["region"],
    config=boto3.session.Config(s3={"addressing_style": "path"}),
)

bucket = list(endpoint["buckets"].keys())[0]

s3.put_object(Bucket=bucket, Key="hello.txt", Body=b"Hello, World!")

Other containers can use an app’s bucket through a relationship:

A worker that doesn’t declare its own relationships block inherits the parent app’s relationships, including object-storage, with no extra configuration.
A worker with its own relationships block, another application, or a task can reach an application’s bucket by targeting that app’s object-storage endpoint:

.upsun/config.yaml

applications:
  myapp:
    # ...
    workers:
      queue:
        relationships:
          bucket: 'myapp:object-storage'

The relationship name on the left (bucket here) is yours to choose. Targeting an app that doesn’t have object storage configured fails the deployment.

Limitations

Each app gets a single bucket, named after the app. Multiple buckets per app aren’t supported.
A bucket belongs to one app. Other containers can share it through a relationship, but buckets can’t be reassigned between apps.
The endpoint is internal to the project and isn’t exposed publicly.
The following S3 features aren’t implemented: bucket versioning, object locking, server-side encryption (SSE), access control lists (ACLs), bucket policies, and lifecycle policies. Requests for them return a “not implemented” error.
Concurrent writes to the same object are last-write-wins with no object locking.

Inspect the bucket from an SSH session

You can explore your bucket directly from a running container using the AWS CLI.

Run S3 commands

The endpoint authenticates every request, so pass the credentials from the relationship. Inside the container they’re available as service environment variables, along with the host and port. Pointed at a custom endpoint with --endpoint-url, the AWS CLI uses path-style addressing on its own, so no extra configuration is needed.

Terminal

AWS_ACCESS_KEY_ID="$OBJECT_STORAGE_ACCESS_KEY" \
AWS_SECRET_ACCESS_KEY="$OBJECT_STORAGE_SECRET_KEY" \
AWS_DEFAULT_REGION="$OBJECT_STORAGE_REGION" \
aws s3 --endpoint-url "http://$OBJECT_STORAGE_HOST:$OBJECT_STORAGE_PORT" ls s3://<APP_NAME>/

Other standard S3 commands work the same way, with the same credentials and --endpoint-url flag:

Goal	Command
List objects	`ls s3://<APP_NAME>/`
Download a file	`cp s3://<APP_NAME>/path/to/file ./local-file`
Upload a file	`cp ./local-file s3://<APP_NAME>/path/to/file`
Sync a directory	`sync ./local-dir s3://<APP_NAME>/prefix/`
Delete an object	`rm s3://<APP_NAME>/path/to/file`

Get Started

Core Concepts

Configure Apps & Services

Build, Deploy & Run

Manage Projects

Manage Environments

Manage Resources

Observability & Metrics

Integrations

Troubleshooting

Resources

When to use object storage

Set the object-storage size

Object storage sizing rules

Storage pricing

Relationship reference

Connect from your app

Example (Python, boto3)

Limitations

Inspect the bucket from an SSH session

Run S3 commands

​When to use object storage

​Set the object-storage size

​Object storage sizing rules

​Storage pricing

​Relationship reference

​Connect from your app

​Example (Python, boto3)

​Share a bucket across containers

​Limitations

​Inspect the bucket from an SSH session

​Run S3 commands

When to use object storage

Set the object-storage size

Object storage sizing rules

Storage pricing

Relationship reference

Connect from your app

Example (Python, boto3)

Share a bucket across containers

Limitations

Inspect the bucket from an SSH session

Run S3 commands