Storage

SeaweedFS35.9k★

SeaweedFS is self-hosted storage for a lot of files: one Go binary serving an S3-compatible object store, a mountable filesystem and an Iceberg lakehouse from the same data. Its edge over Ceph and RustFS is small files: each one is a single disk read, so a hundred million thumbnails stay fast. The core is free under Apache 2.0. weed server -s3 gets a test box running in minutes. Production is real ops: masters, volume servers with real disks, a filer backed by a store like Postgres, and a replication or erasure coding plan picked on purpose. Enterprise is a license key on the same cluster: $2/TB per month or $20/TB per year, free for dev and test under 25TB. It adds undelete, point-in-time recovery, self-healing and custom erasure coding. Solo and small teams: skip it, a hosted bucket is less work. Mid-size teams with a painful S3 bill: free self-hosted. Large clusters: Enterprise is cheap insurance. The catch is security. Since June the project has published 25 security advisories, nine critical, including an S3 gateway that let any network peer inject admin identities. All patched, but keep cluster ports off the internet and stay current.

Ceph17.2k★

Ceph is a distributed storage platform that gives you object, block, and file storage from one cluster. Drop in commodity hardware, run a few daemons, and you get S3-compatible object storage, virtual disk volumes, and a POSIX file system with replication and self-healing built in. Cloud providers and CERN run on it in production. Dual-licensed under LGPL. Self-hosting is real work. You commit to Monitor, OSD, and MDS daemons across at least three nodes, capacity headroom, network throughput, and the operational depth to debug placement-group issues when they happen. Rook (the Kubernetes operator) makes it easier than it used to be, but Ceph rewards storage expertise and punishes teams without it. Solo and small teams: don't. Use S3, Garage or RustFS, or a managed NAS. Mid-sized teams with predictable storage and ops capacity: Ceph at scale beats the cloud bill and your data stays yours. Large teams: this is the OSS standard for self-hosted exabyte-class storage. The catch: if your storage layer goes down, your whole platform goes down. Don't run Ceph because it's free. Run it because you've decided to be the storage team.

minio-go3.0k★

Most applications end up putting files in object storage, and S3 is the API everyone speaks. Minio-go is the Go SDK for that. It works with AWS S3, MinIO, Google Cloud Storage, and any S3-compatible service. Picture the Go equivalent of boto3 for Python, but S3-focused. This is a client library, not MinIO itself. You import it into your Go application and use it to interact with object storage. Apache 2.0, maintained by the MinIO team. The SDK is free. What you pay for is the storage backend you connect it to, whether that's AWS S3 ($0.023/GB/mo), MinIO on your own servers (free), or MinIO's commercial SUBNET support ($10/TB/mo minimum). The catch: this is specifically for Go. If you're in Python, Node, Java, or anything else, MinIO has separate SDKs for those. And it's an S3-compatible SDK. If you need broader cloud storage abstractions (Azure Blob, etc.), you might want a cloud-agnostic library instead.

gcsfuse2.3k★

gcsfuse is Google's official FUSE adapter that mounts a Cloud Storage bucket as a local directory. Read and write files like any other path, with the bytes living in GCS. Apache 2.0 and free, generally available with Google support. The headline use case is machine-learning training pipelines that want a filesystem interface to bucket data without staging copies. Setup is the standard FUSE story: install the binary, run 'gcsfuse <bucket> <mount-point>', you're done. It has file caching to speed up repeated reads and streaming writes for big uploads without buffering everything. Works on Linux and macOS. For solo developers and small teams on Google Cloud, this is a clean way to point a process at a bucket without rewriting it to use the GCS API. For ML teams, the caching can materially accelerate dataset access during training. The software is free; the actual GCS data operations bill at Google's standard rates. The catch: this is not a POSIX filesystem. Some operations are slow or behave differently than a local disk: small-file workloads, frequent overwrites, and metadata-heavy patterns hurt. For real database I/O, do not mount and run, you'll regret it. For sequential reads, large files, and ML training, it's a fit.

storage1.4k★

This is Supabase's file storage service built on top of S3. Picture a managed file system with built-in auth rules, image transformations, and resumable uploads. You set policies in SQL (same as your database RLS), and files respect the same auth you already configured. Apache 2.0. The storage API supports presigned URLs, on-the-fly image resizing, and resumable uploads for large files using the TUS protocol. Supabase's free tier gives you 1GB of storage and 2GB bandwidth. Pro at $25/mo bumps that to 100GB storage and 250GB bandwidth, with overage at $0.021/GB stored and $0.09/GB transferred. Image transformations cost extra at $5 per 100 origin images. Self-hosting is possible. The storage server is open source and runs against any S3-compatible backend (MinIO, R2, actual AWS S3). Self-hosted means unlimited storage, no bandwidth fees, but you manage the infra. The catch: this is tightly coupled to the Supabase ecosystem. If you're not already on Supabase, there's no reason to use this over direct S3, Cloudflare R2, or similar. And the 1GB free tier is small; one app with user-uploaded images blows through that fast.