Gabe Rocks!

🖥️ Tech

🤔 Rambling

🐷 Health

Self-hosting: Speed, decentralization and efficiency

This post is an attempt to navel-gaze on the finer points of self-hosting your own sites & services. I’m going to put down some of the important but easily over-looked considerations that can really make a big difference. The goal is to help plan to make the most of the hardware you have access to without breaking the bank. There won’t be too much in-depth technical details, but more of an overview of things to think about to guide decision-making. If you’re looking for specific instructions, Reclaiming Territory in Cyberspace is a good place to start. I’m also writing this as a reference to myself for any future server-side projects I intend to build.

I dream of a future where ordinary people express themselves from social web sites & systems that are wholly their own. This requires a bit more than mere static sites. If speed was the only consideration, we could all simply put our data on a rented cloud servers leveraging high-performance SSDs and memory. This has two major problems; cost, and lack of independence. For significant amounts of data this would cost quite a bit, effectively filtering out anyone interested in sharing non-trivial amounts of multimedia content. Realistically, if we want a vibrant open web, people are going to want fast sites & services without too much overhead.

The good news is that local storage and compute is relatively cheap. Self-hosting in 2025 means that your largest constraints are bandwidth and latency. With some clever techniques, you can intelligently leverage limited cloud resources to make the best of your personal systems. By optimizing your setup, you can make the most out of even relatively trivial computing resources. Today the bar is quite low to be able to broadcast so much to the world.

Speed is relative. Some systems and services are inherently faster than others. Static sites are king when it comes to speed on the web, but with proper care other systems can be relatively close. In addition to the privacy risks, serving sites and media from your home is generally not advisable. Your home internet connection won’t be able to handle any non-trivial amount of burst traffic, and will likely leave your visitors waiting to load any media. This gets much worse in situations where residential users only have access to so much bandwidth during a given month.

Gotta go fast, while doing more with less

With enough of the big picture out of the way, let’s consider practical steps. Regardless of what systems and services we choose to run, we have to contend with physics and real-world costs. Hardware can only work so hard and data can only move so fast. The goal is to make the best use of available resources by contending with their limitations. In some situations it may make sense to be extremely selective, but in others it may make sense to take advantage of everything that is available.

Consider the following table, outlining the trade-offs of retrieving data from various methods:

Type Retrieval Scheme Cost Speed Throughput Independence
Network Enterprise ISP Extreme 💰 High Extreme 🌊 ?
Network Residential ISP Moderate Slow 🐢 Low 💧 Low
Network Mobile ISP High Slow 🐢 Low None 🔇
Network Sneakernet Moderate Varied 🤷 Varied 🤷 Perfect 💎
Network Postage Cheap 🫰 Glacial 🥶 Varied 🤷 High
File access Memory (Cloud) Extreme 💰 Extreme ⚡ Extreme 🌊 None 🔇
File access Memory (Local) Low High High Perfect 💎
File access File System (Cloud) Moderate High High None 🔇
File access File System (Local) Slow High High Perfect 💎
File access Object Storage (Cloud) Low Extreme ⚡ High None 🔇
File 🤡 Generative AI (Cloud) Absurd 💸 Fast High None 🔇
File 🤡 Generative AI (Local) High Slow 🐢 Low 💧 High

The point of this chart is to point out that there are inherent trade-offs when it comes to leveraging particular resources. You can specialize by focusing on a few, or handle the complexity of trying to ‘have your cake and eat it too’. By understanding this, you can understand that there are opportunities to leverage (but not necessarily depend on) particular tools to help enhance your systems.

Storage localization

The main point of self-hosting is to keep your data under your control. This is best done with your data on machines you control, rather than cloud providers. This has another significant advantage, storage devices themselves are often relatively cheap. You can store your data on high-end SSDs, and make what you want accessible to either your network or even the public. This can be a LOT cheaper than buying large amounts of cloud storage at the price of some initial cost. That said, if you intend to serve a large volume of data, or to a large audience, trade-offs may have to be considered.

Compute optimization

Just like cloud storage, cloud compute can become expensive quite quickly for demanding tasks. By self-hosting you can pay up-front for your compute needs which often saves you money in the long run. An example of this would be using powerful machines at home as remote runners for a PeerTube instance running on an inexpensive VPS. This would allow you to speed up processing videos without sacrificing on availability to the public Internet. Also similarly to storage, there are serious privacy & security benefits of doing the majority of compute tasks on your own machines.

Mirrors and caching

For small or simple sites and services, redundant mirrors can allow your setup to exist across multiple networks (such as darknets) but a ’thin-cache’ is a simple way of achieving the same goal. By having the external facing server (such as a cheap VPS or mini-pc running a darknet daemon) operate as a server-side cache, you can still reduce the latency penalty of self-hosting. This allows your media to be highly available despite having a relatively modest setup.

Eliminating waste

Since you’re likely dealing with constrained resources, or at least a ‘shoe-string’ budget, it makes sense to optimize things however you can. Even if you’re not aiming to be on the 512KB club, it is absolutely worth your time to consider how you can save your guests time, and yourself resources.

Content: Text is king

It may be worth applying the logic of “this meeting could have been an e-mail” to your own content. If a visual medium isn’t entirely necessary, text may suffice. At minimum, treating text content as a first-class format is critical. Text is universally accessible, and plays nice with text-to-speech systems as well as translation. This also applies to file formats, plain-text data such as html tables and svg images can often be much lighter than images for simple information. With some creativity, a great deal of information can be presented with hardly any bandwidth at all.

Compression: Minimize all the things

Hosting media files can be the biggest challenge for small self-hosting setups. If you’re not familiar with how to compress audio & video files you may be unnecessarily using up loads of bandwidth and storage. For a great deal of content it’s not entirely likely that you need full 4K resolution to get your ideas across. The smaller your media files are the easier it is to mirror them and the faster they load for slow connections. FFmpeg is my go-to for compressing audio & video. Remember that you can always keep your full-quality copies for local use!

Client-side: Make it their problem

With some careful consideration, it can be effective to optimize your setup to have much of the work done on the client-side. At minimum you’re going to want to make intelligent use of client-side caching to reduce load on your services. Another valuable pattern is to lazy load media files on your pages. These simple but effective optimizations can make a big difference on constrained networks.

Network optimization

Since the optimal strategy is to divide tasks between at least two machines, it makes sense to make use of tools like SSH tunnels and VPNs to provide a secure ‘backbone’ for them to communicate. It can be useful to use a cheap VPS to act as the public cache/mirror for your sites & services rather than just purchasing a tunnel to the Internet directly. Ultimately it will depend on your bandwidth and latency as well as the nature and volume of media you’re sharing. The cool thing is that with a deliberately distributed setup, you can make your systems available not only for the ‘clearnet’ but darknets or even nearby networks as well. With enough flexibility the options are truly endless.

Hope this helps!

I am always excited to see more people get into self-hosting and I’m hoping that people find this helpful. In an ideal world, the web would be small and filled with ordinary people expressing themselves easily and seamlessly. I believe that achieving that dream requires for there to be more resources and tools for people to not just self-host but self-host well and effectively. If nothing else I hope that this minor overview can be a small contribution towards that.

I’ve really just scratched the surface of this topic, but there is certainly more to cover.
Any feedback you may have would be greatly appreciated if I’ve missed some major blind-spots.


Reply to this post

Performance Self-Hosting Network Storage
Prev B @ Next