Setup a lightweight home server usable as desktop workstation or headless, with carefully selected apps to make your life easier and give you the benefits of the "private cloud"!
This guide uses a declarative methodology, not only to describe and run containerized applications (via docker-compose), but also to install and configure the server and all necessary tools via bash scripts. See What is a Container? to get a quick understanding why Docker is now the default way to deploy, run and manage web applications and how it differs from virtual machines.
See Justification on the What & the Why and definitely don't start buying stuff before reading Hardware Recommendations. Most information available online for pc building and NAS devices do not consider long term stability and low power consumption with fault tolerant components: they focus on downloading stuff and just storing them. My server uses just 4 WATT, less than a phone charger, comparable to a Raspberry Pi, much less than a Synology (a popular ready-to-use NAS system) yet way more powerful and futureproof.
Have a look at the overview of all applications and services that you will have up and running smoothly with this guide.
Note: I had zero experience when I started and learned everything by googling, spending time on fora, reddit and in documentations and by hours and days of trial&error. I made lots of mistakes. Now, in case of disaster I will use the scripts in this repository myself to get up and running again. I am documenting this because I haven't found a single source online that provides all necessary information to get up and running. Also, lot's of things have been carefully chosen after testing alternatives. You can save lots of time with this guide! :)
- The OS used is Ubuntu Budgie, because it is one of the most light-weight and extremely user-friendly of all Linux options. As this script is for beginners, it will help to have an intuitive OS to set everything up. Ofcourse, you can run the server headless (without UI, even without a monitor) .
- Please follow the OS Installation Guide. Step 3 (BtrFS filesystem) is required for this guide!
- In addition, consider running my post-install script Ubuntu Budgie Post Install Script. It's meant for home desktops and laptops but it also takes care of some OS essentials and generally recommended (by experts) btrfs subvolumes. At least use the parts of the script that make sense, especially setting up subvolumes.
- Make sure you have a good text editor installed such as Pluma (
sudo apt install pluma), this is done by the post-install script. - I had zero Linux experience when I started, so you don't need it, as long as you are ready to Google everything, especially some basic Linux commands.
- Your router port forwarding:
- The minimum set of services should be exposed via portforwarding to your server IP: TCP ports 80 and 443 for remote HTTPS access, UDP port 51820 for Wireguard-VPN access via PiVPN, TCP and UDP port 22000 for syncing devices via Syncthing.
- other containers, applications or services including SSH will only be accessible via VPN.
- Acquiring your own domain (mydomain.com) for easy and secure (TLS) HTTPS access. This is a requirement for this guide, get it via GoDaddy.com or Porkbun.com. Each exposed service will be accessible via subdomain.mydomain.com and the connection will be encrypted. The exposed services have proper authentication methods (2FA, FileRun and Bitwarden).
- Download this repository to your Downloads folder: Click the green "Code" button top left > Download as Zip.
- Open a Terminal (CTRL+ALT+T) or hit the Budgie start button and start typing "Terminal" or "Tilix.
NOTES:
- Opening a script or textfile in Terminal (instead of a normal UI text editor like Pluma) can sometimes prevent you from messing up the file:
nano /path/to/file.shnote in some cases you need elevated (root) privileges, to do that, prefix a command withsudo. - My system user account is called
asterix, I use variables instead of personal names, but that is not always possible. Make sure you replace "asterix" with your systems username (and read Folder Structure! Because "asterix" is also very important in my folder structure).
Prepare the filesystem. Install fs tools, understand their goal, tailor to your needs.
Move files to your server data pool and create your folder structure. Note my folder structure is simple.
Continue to Docker & server setup and use the bash script to automatically or manually install essential tools, apply basic configuration + required stuff for specific docker services. Get up and running in minutes via Docker Compose: this is the unique part of this guide, a complete and carefully built working Docker-Compose.yml file with variables.
Nightly maintenance of your server such as cleanup, backup and disks protection tasks.
Setup NFS a zero-overhead solution used in datacenters, the fastest way to share files/folders with other devices (laptops/PCs) via your local home network.
VPN client configs for yourself and others you trust to access non-exposed services, to manage your server remotely and to use your own adblocker remotely.
Almost everything will run isolated in Docker containers. The setup is easy with the provided docker-compose.yml file, which is a declarative way to pull the images from the internet, create containers and configure everything with a single command! See the subguide for Docker Compose on how to get up and running, this is the unique part of this guide, a complete and carefully built working Docker-Compose.yml file with variables. and the correct, well-maintained docker images have been selected, sometimes you spend hours finding the right one as there are often multiple available.
You can easily find other applications via https://hub.docker.com/ Below a description of each application that are in Docker-Compose.yml. Choose the ones you need. The only exceptions -apps that run natively on the OS for specific reasons- are Netdata, PiVPN and AdGuard Home. These apps have very easy installation instructions.
Netdata - via Native Install
Monitoring of system resources, temperature, storage, memory as well as per-docker container resource info.
There are other more bloated alternatives (Prometheus+Grafana) that is overkill in a homeserver situation. Netdata requires lm-sensors.
Runs natively just because it is such a deeply integrated to get sensor access etc. If you run it in Docker, you might have to fix that access yourself.
Portainer - via Docker
An complete overview of your containers and related elements in a nice visual UI, allowing you to easily check the status, inspect issues, stop, restart, update or remove containers that you launched via Docker Compose. Strangely, the tool cannot inform you of updates.
Organizr - via Docker
A a customisable homepage to have quick access to all your services/applications.
UniFi Controller - via Docker
Mobile App: Unifi Network
Ubiquiti UniFi wireless access points are the best. Recommended for good WiFi in your home. If you don't use their access points you do not need this. If you do have their APs, this is only needed to setup once.
Caddy - via docker caddy proxy
reverse-proxy for HTTPS access to the services that you want to expose online. Takes care of certification renewal etc.
Caddy already extremely simplifies the whole https process to allow browsers and apps A+ secure connection to your server. Docker Caddy Proxy goes one step further and allows you to set it up per container with just 2 lines! Alternatives like Traefik are needlessly complicated.
By default only the password manager (Bitwarden), file+Office cloud (FileRun, OnlyOffice), Firefox Sync server and Syncthing are accessible via web.
All other apps are only available via VPN or within your local network. You can easily expose other apps such as Jellyfin by adding a few labels to its container.
PiVPN - via Native Install
Mobile Apps: WireGuard + Automate
Using the Wireguard VPN protocol, easy and secure access to your non-exposed applications (including SSH & SFTP) on your server. Allows you to always use your own DNS (AdGuard Home + Unbound), giving you the same ad-free, secure internet access while outside of your home network, while still allowing direct regular internet access (bypasses the tunnel, only DNS + server IP access goes via the tunnel). Optionally, when in a less secure public environment, let all traffic on your mobile go via the tunnel.
AdGuardHome - via Docker with Unbound - via Docker
Unbound is a recursive DNS resolver. By using Unbound, no 3rd party will know the full URLs of the sites you are visiting (your ISP, local and international DNS providers).
AdGuardHome is a DNS based malware & ad filter, blocking ad requests but also blocking known malware, coinmining and phishing sites!
After AGH filters the requests, the remaining DNS requests are forwarded to Unbound, which chops it up in pieces and contacts the end-point DNS providers to get the necessary IP for you to visit the site.
This way, not 1 company in the world has your complete DNS requests. Compare this to the hyped encrypted DNS (DoH): your request is decrypted at the provider, the provider and all end-point DNS providers see your un-encrypted request.
By blocking on DNS request level, you easily block 5-15% of internet traffic requests, significantly reducing the data needed to load websites, run apps and play games.
All devices connected to your router such as home speakers, smart devices, mediaplayer etc are automatically protected.
This setup can also be used used remotely via split tunnel VPN (see PiVPN). This means you have 1 adfiltering and DNS resolver for all devices, anywhere in the world.
Bitwarden - via Docker
Mobile App: Bitwarden
Easily the best, user friendly password manager out there. Open source and therefore fully audited to be secure. The mobile apps are extremely easy to use.
Additionally allows you to securely share passwords and personal files or documents (IDs, salary slips, insurance) with others via Bitwarden Send.
By usingbitwarden_rs, written in the modern language RUST, it uses exponentially less resources than the conventional Bitwarden-server.
FileRun instead of NextCloud - via Docker
Mobile Apps: CX File Explorer and FolderSync (for phone backup).
FileRun is a very fast, lightweight and feature-rich selfhosted alternative to Dropbox/GoogleDrive/OneDrive. Nextcloud, being much slower and overloaded with additional apps, can't compete on speed and user-friendliness. Also, with FileRun each user has a dedicated folder on your server and unlike Nextcloud, FileRun does not need to periodically scan your filesystem for changes.
FileRun support WebDAV, ElasticSeach for in-file search, extremely fast scrolling through large photo albums, encryption, guest users, shortened sharing links etc.
Limits compared to Nextcloud: It is not open-source and the free version allows 10 users only. I use it for myself and direct family/friends only. It has no calendar/contacts/calls etc features like Nextcloud.
The Nextcloud mobile app works with FileRun but CX File Explorer (4.8 stars) is so much better and easier to use. It is a swift and friendly Android file manager that allows you to add your FileRun instance via WebDAV. Compared to the Nextcloud app, it allows you to easily switch between your local storage and your cloud, copying files betweeen them.\
FolderSync is THE app for Android when you run your own filecloud, allowing you to sync the data of your apps (photos, chat apps, backup of your 2FA app (Aegis), home screen settings etc.) to your server, instead of to Google Drive. It also allows local sync: moving all app-specific backup files (like whatsapp\databases) to a single backup dir first before syncing it to your server.
OnlyOffice DocumentServer - via Docker
Your own selfhosted Google Docs/Office365 alternative! This works well with both FileRun and NextCloud.
Syncthing - via Docker
Syncthing is the fastest and most lightweight solution for 2-way syncing, allowing you to sync user files on your laptop or other users PC/laptops/NAS to your server.
FileRun (like Nextcloud) can also do syncing via WebDAV, but is not very suitable for realtime 2-way with your laptops/desktops, for GBs or even Terabytes of data.
Syncthing can also be used on mobile devices, but for backup purposes, FileRun (via webDAV) is much more suitable (2-way sync is not a backup). If you sync the same folders between multiple devices, Syncthing uses p2p principle to keep them all in sync, reducing the load on each device/optimally using the bandwith of each device.
Firefox Sync - via Docker
By running your own Firefox Sync server, all your history, bookmarks, cookies, logins of Firefox on all your devices (phones, tablets, laptops) can be synced with your own server instead of Mozilla.
Compare this to Google Chrome syncing to your Google Account or Safari syncing to iCloud. It also means you have a backup of your browser profile. This tool has been provided by Mozilla. This is the only browser that allows you to use your own server to sync your browser account!
Paperless - via Docker
Scan files and auto-organise for your administration archive with a webUI to see and manage them. Background of Paperless. No more paper archives!
Jellyfin - via Docker
Mobile & TV Apps: Jellyfin clients (for series/movies), Gelli (amazing Music Player)
A mediaserver to serve clients (Web, Android, iOS, iPadOS, Tizen, LG WebOS, Windows) your tvshows, movies and music in a slick and easy to use interface just like the famous streaming giants do.
Jellyfin is user-friendly and has easy features that you might miss from the streaming giants such as watched status management etc.
The mediaserver can transcode media on the fly to your clients, adjusting for available bandwith. It can use hardware encoding capabilities of your server.
By using the Gelli app, Jellyfin competes with music servers such as SubSonic/AirSonic. Gelli is more slick and in active development.
Allows you to listen to your old AudioCDs! A HiRes Audio alternative to Spotify/Apple Music etc.
Sonarr (tvshows), Radarr (movies) Bazarr (subtitles), Jackett (torrentproxy) - via Docker
A visual, user-friendly tool allowing you to search & add your favourite TV shows (Sonarr) or Movies (Radarr) and subtitles (Bazarr), see a schedule of when the next episodes will air and completely take care of obtaining the requires files (by searching magnets/torrents via Jackett, a proxy for all torrentsites) and organising them, all in order to get a full-blown Nextflix experience served by JellyFin.| For years I have messed with FlexGet, but it can't beat Sonarr.
Transmission + PIA Wireguard VPN_ - via Docker
Mobile App: Transmission Remote
Sonarr, Radarr, Jackett (automatically) add stuff to Transmission which is a p2p client. It should run behind the chosen VPN provider.Many alternatives. Transmission is lightweight and originally has a bit better integration with the tools mentioned + allows for port change via the VPN provider.
Via thedocker-wireguard-piaimage created bythrnz, your downloads are obscured while still allowing you to reach high speeds via the open port in the VPN tunnel, and you can even automatically change the port in Transmission when PIA assigns a new open port, which happens every 90 days.