Object storage

Object storage on Upsun adds an S3-compatible bucket to your application. You enable it by setting an object-storage size on the app’s resources. Upsun provisions the bucket and connects it to your app. Object storage is integrated with the rest of the platform. When you back up an environment, the bucket’s contents are included in the backup, and when you branch an environment, the bucket is cloned along with the environment’s other data. You manage it together with the rest of your project rather than as a separate service.

Object storage is currently a prerelease feature. To enable it on your project, open this prepopulated Support ticket and add your project ID before submitting.To get your project ID, run upsun projects and copy the corresponding ID.

When to use object storage

Use object storage when your app needs to store large or growing volumes of unstructured data (uploads, media, backups, generated artifacts) and you want to reach it through the S3 API. For small amounts of file data that a single instance reads and writes through the filesystem, a persistent mount is usually a better fit.

Enable object storage

Set an object-storage size on an app. The size is in MB, in the app:size format. Run upsun resources:set with the --object-storage option:

Terminal

upsun resources:set --object-storage <APP_NAME>:<SIZE_IN_MB>

For example, to give the myapp app 512 GB (524288 MB) of object storage:

Terminal

upsun resources:set --object-storage myapp:524288

To view the current allocation, run upsun resources:get. When an app has object storage, the output includes an Object storage (MB) column.

Sizing rules

Object storage is allocated in fixed steps, within a minimum and a maximum. The size you request must be a multiple of the step and fall within these bounds:

Rule	Value
Step	524288 MB (512 GB)
Minimum	524288 MB (512 GB)
Maximum	10485760 MB (10 TB)

Connect from your app

Enabling object storage adds a relationship named object-storage to your app. This relationship name is reserved. If your configuration already declares a relationship called object-storage, the deployment fails and you must rename the relationship to proceed. At runtime, the relationship is exposed through the PLATFORM_RELATIONSHIPS environment variable. The object-storage entry provides the connection details for an S3-compatible endpoint:

Field	Value
`scheme`	`http`
`host`	internal hostname of the service
`port`	port of the service

The endpoint URL is http://<HOST>:<PORT>. The endpoint is reachable over your project’s internal network only and isn’t exposed publicly. A single bucket is created per app, named after the app. So an app named myapp reaches its bucket at http://<HOST>:<PORT>/myapp/<KEY>. When configuring your S3 client, note two requirements:

Use path-style addressing. The bucket name goes in the URL path (http://<HOST>:<PORT>/myapp/object.txt), not the hostname. Disable virtual-hosted-style addressing.
No credentials are required. The endpoint doesn’t use access keys. Configure your client with empty credentials and any placeholder region.

Example (Python, boto3)

import base64
import json
import os

import boto3

relationships = json.loads(base64.b64decode(os.environ["PLATFORM_RELATIONSHIPS"]))
endpoint = relationships["object-storage"][0]

s3 = boto3.client(
    "s3",
    endpoint_url=f"http://{endpoint['host']}:{endpoint['port']}",
    aws_access_key_id="",
    aws_secret_access_key="",
    region_name="us-east-1",  # Placeholder; not used.
    config=boto3.session.Config(s3={"addressing_style": "path"}),
)

s3.put_object(Bucket="myapp", Key="hello.txt", Body=b"Hello, World!")

Other containers can use an app’s bucket through a relationship:

A worker that doesn’t declare its own relationships block inherits the parent app’s relationships, including object-storage, with no extra configuration.
A worker with its own relationships block, another application, or a task can reach an application’s bucket by targeting that app’s object-storage endpoint:

.upsun/config.yaml

applications:
  myapp:
    # ...
    workers:
      queue:
        relationships:
          bucket: 'myapp:object-storage'

The relationship name on the left (bucket here) is yours to choose. Targeting an app that doesn’t have object storage configured fails the deployment.

Resize and remove

To resize, pass a new value that is a valid multiple of the step:

Terminal

upsun resources:set --object-storage myapp:1048576

Set the size to 0 to remove object storage from the app:

Terminal

upsun resources:set --object-storage myapp:0

Setting the size to 0, or removing the app, deletes the bucket and all of its contents on the next deployment. There’s no soft-delete or retention period. Back up any data you need before removing.

When you branch an environment and clone its data, the object-storage size is inherited along with the data, so the new environment keeps its bucket.

Limitations

Each app gets a single bucket, named after the app. Multiple buckets per app aren’t supported.
A bucket belongs to one app. Other containers can share it through a relationship, but buckets can’t be reassigned between apps.
The endpoint is internal to the project and isn’t exposed publicly.
The following S3 features aren’t implemented: bucket versioning, object locking, server-side encryption (SSE), access control lists (ACLs), bucket policies, and lifecycle policies. Requests for them return a “not implemented” error.
Concurrent writes to the same object are last-write-wins with no object locking.

Inspect the bucket from an SSH session

You can explore your bucket directly from a running container using the AWS CLI.

Run S3 commands

The endpoint requires no real credentials — set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION to any arbitrary value. Use http://<HOST>:<PORT> as the endpoint URL, as described in Connect from your app.

Terminal

AWS_ACCESS_KEY_ID=fake \
AWS_SECRET_ACCESS_KEY=fake \
AWS_DEFAULT_REGION=fake \
aws s3 --endpoint-url http://<HOST>:<PORT> ls s3://<APP_NAME>/

Other standard S3 commands work the same way:

Goal	Command
List objects	`ls s3://<APP_NAME>/`
Download a file	`cp s3://<APP_NAME>/path/to/file ./local-file`
Upload a file	`cp ./local-file s3://<APP_NAME>/path/to/file`
Sync a directory	`sync ./local-dir s3://<APP_NAME>/prefix/`
Delete an object	`rm s3://<APP_NAME>/path/to/file`

Get Started

Core Concepts

Configure Apps & Services

Build, Deploy & Run

Manage Projects

Manage Environments

Manage Resources

Observability & Metrics

Integrations

Troubleshooting

Resources

When to use object storage

Enable object storage

Sizing rules

Connect from your app

Example (Python, boto3)

Resize and remove

Limitations

Inspect the bucket from an SSH session

Run S3 commands

​When to use object storage

​Enable object storage

​Sizing rules

​Connect from your app

​Example (Python, boto3)

​Share a bucket across containers

​Resize and remove

​Limitations

​Inspect the bucket from an SSH session

​Run S3 commands

When to use object storage

Enable object storage

Sizing rules

Connect from your app

Example (Python, boto3)

Share a bucket across containers

Resize and remove

Limitations

Inspect the bucket from an SSH session

Run S3 commands