How I'm slowly leveraging my Guix server into a self-hosting feature-creep octopus
Published on
Hello again, it's been a bit of a dry month or so. Nothing super eventful has come out over the last month, I would say. Some of my audio projects are backed up by me, needlessly exploring non-PureData options for programming. I came across a project called seamstress and I spent some time rigging a small testbed with my Launchpad in it using Lua. I don't think I like Lua, so I'd like to consider using Fennel instead. But I'm not there yet, nor do I know if it's "good" enough for my needs.
Anyways, I'm here to talk about something other than audio programming, and it's back on the topic of self-hosting services. Let's begin.
I got relatively annoyed at myself for hosting a website I feel like I sometimes have long periods of inactivity with. I promise I would write... Then I don't. I draft a number of pieces between here and my published posts that never see the day of light. I sometimes get political, start something, read it, and get upset that it is seemingly low-quality and out of touch with my core writing mentality. I consider my blog a place where I can collect my thoughts and occasionally dig back years after to see what I got up to and what I lost somewhere along the way. I did it recently and re-discovered some old but critical info. That was helpful. Maybe it can be for others.
After looking at my views on my Cloudflare dashboard, I realize that my views are almost largely inflated by Chinese traffic, which makes me think my website is largely part of some weird, obscure bot-farm that takes my website and uses it for whatever they do. Be it some AI training farm, or scraping for vulnerabilities on a live hot-server for cryptojacking purposes. I don't feel like all my traffic is organic, and I'm inclined to make some dramatic changes to my website sooner or later. This in turn got my wheels spinning towards what I can do better about my website.
I am looking to actively get off Zola at some point since I have a number of issues, but I've run into so many bottlenecks with developing my own solutions, or even with finding alternatives. I ran a number of experiments on my Guix server to see if I can find one working static site generator that wasn't JavaScript-based, and I found effectively zero workable candidates to replace Zola from the Guix channels.
I tried my hardest to go Guix-only, but it seems like that has it's limitations at times. It might be that Guix isn't a simple solution to my problem, but in turn, I overlooked something I already use in practice: Docker.
Docker is a love-it or hate-it kind of product on the internet. People praise Docker, but hate the company behind Docker, which is fair. People would rather use NixOS, which is a pain in the ass to have to deal with and the documentation is poor, and also has controversy, to build their systems and have it statically defined. This works, until you have a crazy setup that requires a lot of finnicking around with in order to get it working.
I think Docker is far easier to work with in my opinion, and having battle-tested it for a bit, I think it's the simplest approach one can have towards projects like these. It's nice to not have to think about the package installation process if a bunch of Docker containers can take that level of difficulty away. You don't have to worry about version pinning, because you can do that with Docker. You will occasionally have to update Docker containers, but you already do that with any other system out there currently.
Where am I going with this? Ah yes.
I said somewhere previously I don't think Nextcloud is the perfect all-in-one solution. If you wanted to go a truly free-software route, you do not get that with most Nextcloud applications on mobile devices, due to some apps having a poor push notification story. Some work, some don't, and others are seemingly straight-up abandoned. Nextcloud is a large piece of software, and the beaten-path extensions for Nextcloud like bookmarks, chat or tasks are third-parties to the main centerpiece.
That being said, I actually continued to use Nextcloud for over a month since then. Not as a primary space or anything, but as a nice solution for cloud storage issues I've been having recently. I have over ten years of photos laying around, and Nextcloud was the best solution I could find to manage that. Videos... Not so much, because of how my Cloudflare tunnel limits data, and because Nextcloud has a memory limitation to requests anyway. Whatever.
I imported all my photos from all my devices into Nextcloud via the web and app interfaces, and because of Nextcloud, it generated a timeline of all my photos and made it dead simple for me to manage them and delete photos I simply no longer needed for whatever reason (bad photo, blurry, etc). I could have done this with Google Photos, but it's a victory I achieved because of a free piece of software.
My bookmark story is also a nightmare because the bookmark app for Nextcloud does not function and let you post bookmarks, but reading bookmarks is still functional. If you enable bookmark backups it should also be able to dump bookmarks into a folder on your Nextcloud files pane, which makes it accessible too via the app. I've been using the bookmarks application to help me sort out my Yu-Gi-Oh! decks, which has been helpful for sharing with my friends.
My Nextcloud is currently hosted by using a Docker Compose stack, which is basically mashing several different containers together to make sense. I could have defined my entire system via NixOS, but I didn't have to. My Guix OS is the base server layer, while Docker serves as an additional service layer on top of the base operating system. I don't need to rely on Guix to be my all-in-one knife, nor do I have to constantly tinker/reload the system configuration in Guix over and over to get productive results.
You could argue there's performance hits by using Docker to do the application layer, but my hardware is already old as hell, so there's not much hope there. The CPU itself is a fourth generation i5 from God knows how many years ago. I'd like to move away from it, but only if I see some success with this. Right now I don't have full-disk encryption for the entire Nextcloud stack because that would be an incredibly expensive thing and would butcher my iops. Perhaps if I can get my hands on CPUs from the last ~3 generations or so I'll make the transition, but we're not there yet.
One thing I started to play around with in my head is the idea of using my Guix server to host all my applications. Right now, this website is served by Cloudflare, and my files are all lodged into a Git repository hooked up to Cloudflare. I don't know if I want to be permanently attached to Cloudflare like this, other than the very handy tunneling functionality I have (which I could easily move to Tailscale one day... If that need arises).
I have a few web projects I'd like to connect, but I don't know if I care for hosting flat files on these Git page services anymore. I feel like I could do more without that level of statically-defined, so here's my attempt.
I currently have a leaderboard online that I crafted myself using our Steam user data to tell us how many games we have completed to one hundred percent completion (according to in-game achievements). This was normally published by me manually re-compiling data from Steam and doing fresh pulls on net data. It's thankfully a free API, but I'd rather not look a free service as "totally free", I don't want to abuse it. This is a fun thing for my friends and I, we all have Steam Decks and we use it as a minor contest and research tool to help us figure out what to play next.
Previously, this was held in a Git repository, but right now, it isn't. This is served from my Guix operating system in my house. It uses the same Cloudflare tunnel technology from my Nextcloud instance, but this time it connects to a Caddy server to serve static files instead. Caddy is a swiss army knife of proxying traffic, and it works well (until I don't understand it).
For this I use a Caddy Docker container to serve from a Caddyfile I define, and mount the static files I would like to serve to somewhere inside the Caddy's container volume that it understands. I connect it to something like /var/www/html
in Caddy's container, and that's all it takes to simply serve some static files. It was really easy. So easy that this was my Caddyfile.
# Caddyfile
:9999 {
root * /var/www/html
encode gzip
file_server
}
The Docker setup at present isn't difficult, but it's not a lively application. My static files need to be re-built from time to time, and it also needs to be re-built when there are underlying code changes to my messy Racket codebase. So there's three aspects I need to figure out:
Some of this is cron
-able, but I think I need to stub a small little application somewhere to occasionally run the git pull
command on the system. This isn't a complicated program, probably under about a hundred lines or so, but this idea can help me move my blog into the future somewhat.
Right now my blog depends on me manually installing Zola and re-building my site with singular commits each time. The commits are then read by my Git provider and Cloudflare picks up on the changes. This is a very fast process, I would say, and only takes a few minutes to be reflected in the web in real time, but having static files means there's limitations to what my website can do. As it stands, I don't have any means of handling something like WebMentions. I think it's an incredibly interesting topic, but not an easy one to solve for, as not all static site software can support this means. Which is why I would love to have my own static site publishing software, but at this point, I am still not there yet.
I am considering notion of using Docker to publish my website, just like with the leaderboard, and use something like Caddy to serve files and Ruby's Jekyll to publish the website. I can leverage Caddy's seemingly pretty darn good file server and Jekyll's longstanding history as a static site generator, and maybe one day I can grow to have a WebMention system in place somehow.
I'm thinking somewhere in the next week I will start experimenting with a new Docker setup for the website so I can start moving forward. I apologize for the lack of updates, but this is the next thing I will be focusing my time on.
I've seen a lot of people bring up the Ghost publishing software as a next big thing. Soon, Ghost will be federating with the ActivityPub protocol, making it possible to follow Ghost-based publications from a Mastodon account. This is a crazy feat, and it's exciting to think about what can be done with the ActivityPub protocol now.
This got me thinking: is Ghost for me?
I spun up a Docker container for Ghost and, while I was able to boot up a Ghost instance, I don't feel like it made me any better a writer than before. My theme was not my own, and it increased appearance burden on my shoulders because I know I'd like to have more say in the final product than just picking a theme. I think it's a good base for people with no web skills who simply want to blog, and that's totally fine. However, I think my current path is my best bet. I don't want to sell my website as a subscription, and I find it's tooling a little bit... Lacking, to say the least.
Next time you hear from me, I will be writing more about PureData and maybe even the website a little bit. Until next time, thanks for reading!