> ## Documentation Index
> Fetch the complete documentation index at: https://developer.upsun.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Why we moved from LVM to Ceph for container storage

> Upsun migrated from LVM to Ceph for container storage to separate compute from storage, enable instant volume migration, and make VMs disposable again.

export const PostMeta = ({data = {}}) => {
  const {author, date, image} = data;
  const authors = Array.isArray(author) ? author : author ? [author] : [];
  const resolveAuthor = slug => {
    const entry = AUTHOR_MAP[slug] || ({});
    const name = entry.name || slug;
    const github = entry.github || null;
    const linkedin = entry.linkedin || null;
    const url = github ? `https://github.com/${github}` : linkedin || null;
    const avatarUrl = github ? `https://github.com/${github}.png?size=64` : null;
    return {
      name,
      url,
      avatarUrl
    };
  };
  const formattedDate = date ? new Date(date).toLocaleDateString('en-US', {
    year: 'numeric',
    month: 'long',
    day: 'numeric'
  }) : null;
  if (!image && authors.length === 0 && !formattedDate) return null;
  const AUTHOR_MAP = {
    "aaron-collier": {
      "name": "Aaron Collier"
    },
    "aaron-dudenhofer": {
      "name": "Aaron Dudenhofer"
    },
    "aaron-porter": {
      "name": "Aaron Porter"
    },
    "adriaan-odendaal": {
      "name": "Adriaan Odendaal"
    },
    "ajmal": {
      "name": "Ajmal Siddiqui"
    },
    "akalipetis": {
      "name": "Antonis Kalipetis"
    },
    "alexander-varwijk": {
      "name": "Alexander Varwijk"
    },
    "alicia-bevilacqua": {
      "name": "Alicia Bevilacqua"
    },
    "amelie-deguerry": {
      "name": "Amelie Deguerry"
    },
    "anacidre": {
      "name": "Ana Cidre",
      "linkedin": "https://www.linkedin.com/in/ana-cidre"
    },
    "andoni": {
      "name": "Andoni Auzmendi"
    },
    "andrei-taranu": {
      "name": "Andrei (Alex) Taranu",
      "linkedin": "https://www.linkedin.com/in/andrei-alex-taranu/"
    },
    "andrew-baxter": {
      "name": "Andrew Baxter"
    },
    "andrew-melck": {
      "name": "Andrew Melck"
    },
    "antoine-crochet-damais": {
      "name": "Antoine Crochet Damais"
    },
    "augustin-delaporte": {
      "name": "Augustin Delaporte",
      "linkedin": "https://www.linkedin.com/in/augustindelaporte/"
    },
    "branislav-bujisic": {
      "name": "Branislav Bujisic"
    },
    "carl-smith": {
      "name": "Carl Smith"
    },
    "caroline-leroy": {
      "name": "Caroline Leroy"
    },
    "cati-mayer": {
      "name": "Cati Mayer"
    },
    "catplat": {
      "name": "C Trinkwon"
    },
    "ceelolulu": {
      "name": "Celeste van der Watt"
    },
    "chadwcarlson": {
      "name": "Chad Carlson",
      "github": "chadwcarlson",
      "linkedin": "https://www.linkedin.com/in/chadwcarlson"
    },
    "chris-ward": {
      "name": "Chris Ward"
    },
    "chris-yates": {
      "name": "Chris Yates"
    },
    "christian-sieber": {
      "name": "Christian Sieber"
    },
    "christopher-lockheardt": {
      "name": "Christopher Lockheardt"
    },
    "christopher-skene": {
      "name": "Christopher Skene"
    },
    "chuck-morgan": {
      "name": "Chuck Morgan"
    },
    "corey-dockendorf": {
      "name": "Corey Dockendorf"
    },
    "crell": {
      "name": "Crell"
    },
    "damz": {
      "name": "Damz"
    },
    "dan-morrison": {
      "name": "Dan Morrison"
    },
    "davidbonachera": {
      "name": "David Bonachera",
      "github": "davidbonachera",
      "linkedin": "https://www.linkedin.com/in/davidbonachera"
    },
    "dereliahmet1": {
      "name": "Ahmet Faruk Dereli"
    },
    "devicezero": {
      "name": "Jonas Kröger",
      "github": "devicezero",
      "linkedin": "https://www.linkedin.com/in/jonaskroeger/"
    },
    "doug-goldberg": {
      "name": "Doug Goldberg"
    },
    "duncan-naves": {
      "name": "Duncan Naves",
      "github": "duncannaves",
      "linkedin": "https://www.linkedin.com/in/duncan-naves-a94423aa"
    },
    "erika-bustamante": {
      "name": "Erika Bustamante"
    },
    "fabpot": {
      "name": "Fabien Potencier"
    },
    "flovntp": {
      "name": "Florent Huck",
      "github": "flovntp",
      "linkedin": "https://www.linkedin.com/in/florenthuck"
    },
    "fred-plais": {
      "name": "Fred Plais"
    },
    "gauthier-garnier": {
      "name": "Gauthier Garnier"
    },
    "gilzow": {
      "name": "Paul Gilzow"
    },
    "gmoigneu": {
      "name": "Guillaume Moigneu",
      "github": "gmoigneu",
      "linkedin": "https://www.linkedin.com/in/guillaumemoigneu/"
    },
    "gregqualls": {
      "name": "Greg Qualls"
    },
    "guguss": {
      "name": "Augustin Delaporte"
    },
    "haylee-millar": {
      "name": "Haylee Millar"
    },
    "ivana-kotur": {
      "name": "Ivana Kotur"
    },
    "jackrabbithanna": {
      "name": "Mark Hanna"
    },
    "jared-wright": {
      "name": "Jared Wright",
      "github": "jww-sh",
      "linkedin": "https://www.linkedin.com/in/jaredwaynewright"
    },
    "jessica-orozco": {
      "name": "Jessica Orozco"
    },
    "joey-stanford": {
      "name": "Joey Stanford"
    },
    "john-grubb": {
      "name": "John Grubb"
    },
    "jonas-kruger": {
      "name": "Jonas Kruger"
    },
    "kathryn-frazer": {
      "name": "Kathryn Frazer"
    },
    "kemiojo": {
      "name": "Kemi Elizabeth Ojogbede"
    },
    "kieronsambrook-smith": {
      "name": "Kieronsambrook Smith"
    },
    "laurent-arnoud": {
      "name": "Laurent Arnoud",
      "linkedin": "https://www.linkedin.com/in/laurent-arnoud-861b44121/"
    },
    "letoya-boyne": {
      "name": "Letoya Boyne"
    },
    "lolautruche": {
      "name": "Jérôme Vieilledent"
    },
    "lyly-lepinay": {
      "name": "Lyly Lepinay"
    },
    "manauwar-alam": {
      "name": "Manauwar Alam"
    },
    "marc-antoine-porri": {
      "name": "Marc Antoine Porri"
    },
    "maria-antinkaapo": {
      "name": "Maria Antinkaapo"
    },
    "maria-de-anton": {
      "name": "Maria De Anton"
    },
    "mark-dorison": {
      "name": "Mark Dorison"
    },
    "markus-hausammann": {
      "name": "Markus Hausammann"
    },
    "mary-thomas": {
      "name": "Mary Thomas"
    },
    "mathias-bolt-lesniak": {
      "name": "Mathias Bolt Lesniak"
    },
    "mathieu-strauch": {
      "name": "Mathieu Strauch"
    },
    "matthias-van-woensel": {
      "name": "Matthias Van Woensel",
      "linkedin": "https://www.linkedin.com/in/matthias-van-woensel-267a069"
    },
    "michael-sharp": {
      "name": "Michael Sharp"
    },
    "mupsi": {
      "name": "Marine Gandy"
    },
    "natalie-harper": {
      "name": "Natalie Harper"
    },
    "ngommenginger": {
      "name": "Nicolas Gommenginger",
      "linkedin": "https://www.linkedin.com/in/nicolas-gommenginger"
    },
    "nicholas-bennison": {
      "name": "Nicholas Bennison"
    },
    "nicholas-vahalik": {
      "name": "Nicholas Vahalik"
    },
    "nick-hardiman": {
      "name": "Nick Hardiman"
    },
    "nickanderegg": {
      "name": "Nickanderegg"
    },
    "nicolas-grekas": {
      "name": "Nicolas Grekas",
      "github": "nicolas-grekas",
      "linkedin": "https://www.linkedin.com/in/nicolasgrekas/"
    },
    "niti-malwade": {
      "name": "Niti Malwade"
    },
    "opensocialteam": {
      "name": "Opensocialteam"
    },
    "ori-pekelman": {
      "name": "Ori Pekelman"
    },
    "otavio-santana": {
      "name": "Otavio Santana"
    },
    "palwandi": {
      "name": "Pawan Alwandi",
      "github": "pawpy",
      "linkedin": "https://www.linkedin.com/in/pawanalwandi"
    },
    "patrick-boest": {
      "name": "Patrick Boest"
    },
    "patrick-dawkins": {
      "name": "Patrick Dawkins",
      "github": "pjcdawkins",
      "linkedin": "https://www.linkedin.com/in/patrickdawkins"
    },
    "patrick-klima": {
      "name": "Patrick Klima"
    },
    "pjcdawkins": {
      "name": "Pjcdawkins"
    },
    "prineet-kaurbhurji": {
      "name": "Prineet Kaurbhurji"
    },
    "quentin-sinig": {
      "name": "Quentin Sinig"
    },
    "ralt": {
      "name": "Florian Margaine",
      "github": "ralt",
      "linkedin": "https://www.linkedin.com/in/florian-margaine-43971136"
    },
    "ramanathanramakrishnamurthy": {
      "name": "Ramanathanramakrishnamurthy"
    },
    "remi-lejeune": {
      "name": "Rémi Lejeune"
    },
    "ribel": {
      "name": "Taras Kruts"
    },
    "robert-douglass": {
      "name": "Robert Douglass"
    },
    "rudy-weber": {
      "name": "Rudy Weber"
    },
    "ryan-hicks": {
      "name": "Ryan Hicks"
    },
    "sabri-helal": {
      "name": "Sabri Helal"
    },
    "savannah-bergeron": {
      "name": "Savannah Bergeron"
    },
    "shannon-vettes": {
      "name": "Shannon Vettes"
    },
    "shawn-ogasawara": {
      "name": "Shawn Ogasawara",
      "linkedin": "https://www.linkedin.com/in/shawn-ogasawara-83a9a0/"
    },
    "shawna-spoor": {
      "name": "Shawna Spoor"
    },
    "shedrack-akintayo": {
      "name": "Shedrack Akintayo"
    },
    "simon-ruggier": {
      "name": "Simon Ruggier"
    },
    "sophie-van-der-kindere": {
      "name": "Sophie Van Der Kindere"
    },
    "stefanos-thampis": {
      "name": "Stefanos Thampis"
    },
    "stephen-weinberg": {
      "name": "Stephen Weinberg"
    },
    "sukhman-virk": {
      "name": "Sukhman Virk"
    },
    "sumaira-nazir": {
      "name": "Sumaira Nazir"
    },
    "sumer": {
      "name": "Sümer Cip"
    },
    "syed-raza": {
      "name": "Syed Raza"
    },
    "tamara-bacchia": {
      "name": "Tamara Bacchia"
    },
    "tara-arnold": {
      "name": "Tara Arnold"
    },
    "theosakamg": {
      "name": "Mickael Gaillard",
      "github": "theosakamg"
    },
    "thomasdiluccio": {
      "name": "Thomas di Luccio"
    },
    "tim-anderson": {
      "name": "Tim Anderson"
    },
    "tom-helmer-hansen": {
      "name": "Tom Helmer Hansen"
    },
    "tylermills": {
      "name": "Tyler Mills"
    },
    "upsun": {
      "name": "Upsun"
    },
    "veronika-tolkachova": {
      "name": "Veronika Tolkachova",
      "linkedin": "https://www.linkedin.com/in/veronika-tolkachova-169167a2"
    },
    "vince-parker": {
      "name": "Vince Parker"
    },
    "vinnie-russo": {
      "name": "Vincenzo Russo"
    },
    "vrobert78": {
      "name": "Vincent Robert",
      "github": "vrobert78",
      "linkedin": "https://www.linkedin.com/in/vincent-robert-498a883"
    },
    "yuriy-babenko": {
      "name": "Yuriy Babenko"
    },
    "yuriy-gerasimov": {
      "name": "Yuriy Gerasimov"
    }
  };
  return <div className="post-meta">
      {(authors.length > 0 || formattedDate) && <div className="post-meta-info">
          {authors.length > 0 && <div className="post-meta-authors">
              {authors.map(slug => {
    const {name, url, avatarUrl} = resolveAuthor(slug);
    const inner = <>
                    {avatarUrl && <img src={avatarUrl} alt={name} className="post-meta-avatar" />}
                    <span className="post-meta-author-name">{name}</span>
                  </>;
    return url ? <a key={slug} href={url} target="_blank" rel="noopener noreferrer" className="post-meta-author">
                    {inner}
                  </a> : <span key={slug} className="post-meta-author">{inner}</span>;
  })}
            </div>}
          {authors.length > 0 && formattedDate && <span className="post-meta-separator" aria-hidden="true">·</span>}
          {formattedDate && <span className="post-meta-date">{formattedDate}</span>}
        </div>}
      {image && <img src={image} alt="" className="post-meta-image" aria-hidden="true" />}
    </div>;
};

<PostMeta data={{ author: ["ralt"], date: "2026-04-02T06:00:00.000Z", image: "/images/posts/how-it-works/why-upsun-moved-from-lvm-to-ceph/why-upsun-moved-from-lvm-to-ceph.png" }} />

Data cloning is one of Upsun's core features. When you create a [preview environment](https://developer.upsun.com/docs/glossary#preview-environment), we clone your production disk, including your database, your files, everything. This has to be fast (seconds, not minutes), and it has to be correct. The entire capability depends on the storage layer underneath.

We started with LVM. We eventually moved to Ceph. The reasons tell you a lot about what it takes to run containers at scale.

## The feature that drives everything: data cloning

Preview environments are only useful if they have real data. A staging environment with an empty database doesn't tell you much about how your application behaves in production.

Copy-on-write (CoW) cloning makes this practical. Instead of copying hundreds of gigabytes of data, you create a logical snapshot that shares the underlying blocks with the original. New writes go to new blocks. Reads fall through to the shared data. The clone is instant and costs almost no additional storage until changes accumulate.

LVM supports this through LV snapshots. When you create a preview environment, we'd snapshot the production logical volume and hand the snapshot to the new container. Fast, efficient, and it worked well for years.

## LVM: what worked and what didn't

Our LVM setup ran on AWS at the time (we're multi-cloud now, but we'll use EBS terminology here since that's what we were working with).

LVM did the job initially. LV snapshots gave us CoW cloning, and we could provision volumes quickly on a single VM. But as we scaled, several problems became hard to ignore.

The most fundamental issue was that volumes were tied to a specific VM. Each logical volume lived on a specific host's EBS storage. Moving a container to a different VM meant stopping it, detaching the EBS volume, reattaching it to the new host, and restarting. This took minutes, not seconds, and created downtime.

This also meant all clones had to live on the same VM as production. LV snapshots exist within the same volume group, so your preview environments had to run on the same physical host as your production environment. No isolation between production and development workloads.

VM death was catastrophic, and in the cloud, VMs die. Not constantly, but regularly enough that you have to design for it. This isn't dedicated hardware sitting in a rack for five years. If a VM went down, every container on it went down with it. Recovery was slow and sequential: bring the VM back, check the file systems, restart containers one by one. You couldn't spin them up elsewhere because the data was physically attached to that host.

Backup granularity was another pain point. You couldn't surgically restore one customer's volume from an EBS snapshot. Restoring a single volume meant restoring the entire disk.

Adding storage capacity was also painful. You had to expand the EBS volume, then resize the physical volume, the volume group, and the logical volume. Multiple steps, each with its own failure modes, and you had to do it per VM.

All of this made VMs into pets. Each one had unique, irreplaceable data on its local storage. You couldn't terminate one and replace it with a fresh instance. Every host was special, and that's the opposite of what you want in a cloud-native infrastructure.

## Why Ceph

[Ceph](https://ceph.io/) (specifically Ceph RBD, not CephFS) checked most of our boxes. It supports CoW cloning natively, replicates data across nodes, and on top of that gives you network-attached block devices that can be mapped to any VM in under a second.

Storage and compute become separate concerns. Ceph runs on dedicated storage nodes, and compute VMs don't store any persistent data. You can optimize each fleet independently: storage-optimized instances for Ceph OSDs, compute-optimized instances for running containers. With EBS, you're technically not paying for storage compute directly, but it's baked into the price and you don't control it. With Ceph, you manage it yourself, which is more work but also means you can tune it to your actual workload.

Because volumes live on the Ceph cluster and not on any particular VM, you can map and unmap them from any host instantly. Container migration becomes trivial. Need to move a workload? Unmap the RBD image, map it on the new VM, start the container. The data doesn't move because it was never on the VM in the first place.

CoW cloning still works. Ceph RBD supports snapshots and cloning natively. There's a depth limit (we flatten at 16 parent levels) for clone chains to keep performance predictable. The cloning itself is still instant.

Since volumes aren't tied to hosts, your preview environment can run on a completely different VM than production. Better isolation, better resource allocation. Production and development workloads no longer compete for the same host resources.

Ceph also replicates data across multiple storage nodes. Losing one storage VM doesn't lose data. The cluster heals itself by re-replicating from surviving copies. And if a compute VM dies, you spin up the containers on another VM and map their volumes. No data recovery, no file system checks. The volumes were never on the failed host.

Scaling storage capacity is also straightforward. Need more space? Add new storage nodes to the cluster. Ceph rebalances automatically. No per-VM EBS resizing, no pv/vg/lv dance.

The net result is that VMs become cattle again. Compute VMs are stateless and disposable. You can terminate any of them and replace them with fresh instances. Auto-scaling becomes straightforward.

## Trade-offs

Ceph isn't free of downsides.

The blast radius shifts. With LVM, a VM failure affected only the containers on that VM. With Ceph, a storage cluster issue can potentially affect every container using that cluster. The failure mode is different: less frequent, but wider.

Operational complexity also increases. Running a Ceph cluster requires specific expertise. Monitoring, capacity planning, OSD management, placement group tuning. It's a meaningful operational investment.

The risk-to-reward ratio is worth it. The ability to treat compute VMs as disposable, to migrate containers quickly, and to scale storage independently of compute changes the operational model fundamentally.

For details on how we handle backups on top of Ceph, see how Ceph snapshots enable incremental full backups.

## The storage layer shapes everything

The choice of storage architecture isn't a background infrastructure decision. It directly determines what features you can offer and how reliably you can offer them. Preview environments with instant data cloning, fast container migration, self-healing after hardware failures: all of these trace back to the storage layer.

LVM worked until it didn't. Ceph gave us the flexibility to treat compute and storage as independent concerns, and more importantly, it let us treat our infrastructure as cattle, not pets. VMs come and go, containers move between hosts, and nothing breaks. That's the foundation everything else is built on.