Of the people who write code in their spare time, many are building an app for a specific purpose. And while I've done that as well (e.g. with Obskurnee), most of my tinkering time is spent on my 'personal stack'. Rather than having a specific purpose of doing X or Y, its purpose is to provide whatever tools I need, all in one place.

To make it sound less generic: I use it for keeping track of my book collection(s), for playtime analytics, for my universal search, there are some chat-based controls for various parts, ...

And as much as I enjoy making these features, I only do it because I want to actually use them. Preferably yesterday. The less time I have to spend on them, the better. So I found a set of components/apps that work very well for me.

These are the basic components from most to least important.

NocoDB for viewing and editing data
A custom dotnet webapp (which I call botfire5) backed by a Postgres DB - this is where I add any code that I need executed
Metabase for exploring data and making fancy dashboards
n8n for various workflows

First off: NocoDB makes it very easy to start collecting data. It's easy to access, create or modify tables; it's also very flexible when it comes to automated data handling. One of the greatest benefits of it is that I can either manipulate the data by directly connecting to the underlying Postgres DB or via NocoDB's API, whatever best fits the current use case. Another is the fact that this automatically gets me a rudimentary UI - no endless coding of CRUD pages required.

Once there's a way to store and manipulate data, more complex processing comes next. For that, I use a custom web app (botfire5) with a Vue.js front end. As an example, this app hosts the 'universal search' UI (built with Meilisearch and Algolia UI components), the Matrix bots and various background services. Recently, I decided I wanted Audiobookshelf contents to be searchable in the search; so I made this app load data from the ABS API nightly, format it, and post it to Meilisearch.

Those two parts are indispensable. But the next two are also used very often.

Metabase is a "BI" tool - that means you point it at a database you already have, and you can dig around in the data, make visualisations, charts, dashboards.

Last but not least, n8n for automations. It has connectors for a whole bunch of services, and a great visual designer/debugging tool. It's the last on this list because I could just handle all of these things in code, but it would be a pain and I don't wanna.

Synergies

These apps work very well together. Let me illustrate with simplified example use case - I decided to track my videogame purchases.

First, I added a few tables to NocoDB to keep track of data.

Then I created a "Form" view in Noco, so I can have a simple page where I just enter the name of the game I just bought, the store, and the price. (Date is 'today' by default.)

Next, I used Noco's webhooks to trigger a workflow in n8n that tries matching the purchase to my pre-existing game database. If it's found, it's linked. If it isn't, a new one is created.

Then I realized I have many games with next to no metadata - it took me a long time until I started to jot down the genres at least. IGDB does provide an excellent API, but there's no practical way to connect to it directly from n8n. So I wrote a controller for botfire5 (my homebrew app) that accepts a game's name, tries looking it up in IGDB, and returns a JSON with all the data I'm interested in. I deliberately decoupled it from the rest of the logic, because this way I get a nicely reusable component for later.

After that, it was trivial to add a webhook to Noco that calls n8n; n8n checks if the metadata is already present and if not, it calls the botfire5 API, which fetches it. n8n then stores the data in Noco.

So I had a form where I can just write "Surviving Mars", select "GOG" from a dropdown and write the price; the purchase is logged, the game added to the DB if it was missing, and metadata automatically fetched from IGDB.

Then I realized I'm always curious and end up going to NocoDB right after to check out the loaded metadata. So I added a few extra branches to the n8n flow and now a description of the game is added to the personal newsletter I get every other day.

I also get more detailed stats sent to me in a Matrix message. That's also where info about any potential issues is sent, such as when there are suspected duplicities, or the game was found in IGDB but the name match is suspicious, et cetera.

Then I went to NocoDB and made nice charts that track and visualize spending, genres, things like that.

Then the last thing that bugged me was how annoying it is to search my game DB. Noco does have search, but it's limited and only searches a single column. So I went to Postgres and wrote a view that returns all data about all my games, already formatted the way Meilisearch likes it. Next comes a new n8n workflow, which is scheduled to trigger every few hours, that just loads the data directly from the Postgres view and sends it to Meilisearch.

That got me an excellent search capability across all relevant fields, with highlights and everything!

But the search result card was generic. It could be much nicer if it were formatted specifically for games - with things like expected playtime, cover art, and so on. So I went back to botfire5 and added a specialized result card.

Ok, so was all of that a lot of work? Definitely. Described like this, it may look like a nice, linear process, but in reality, it happened over multiple years. Working this way suits me - I build the minimal thing that works, use it, see what I do and do not like, then iterate. All the work was split into free evenings, adding a feature here and there, and I ended up with a system that I use often and with joy.

Here comes the kicker: most of the (sub-)features were relatively small. It often feels like playing with lego (or playing Factorio) - If I only have a couple of hours to work after the kids go to bed, I can add a new brick /component and get a small improvement immediately. This setup just works for me, at least for now.

Drawbacks

The drawbacks are insignificant to me at the moment, but they do exist. Ignoring the usual ones that come with running and administering a whole bunch of apps with different lifecycles, etc, the biggest one probably is: it's not very transferable.

If I showed you the things it does, and you said "cool, can I have it?" - well, not really.

Unfortunately, there isn't a straightforward way to deploy NocoDB schema changes from one environment to another, for example. I have to deal with this when moving changes between my dev and prod environments; I don't think there's any reasonable way of just handing them off to someone else for import.

The same goes for Metabase query and dashboard definitions.

The best one here is probably n8n, which I think does let you copy-paste and import/export flow components.

And the custom app is of course a) only tailored for me and b) entangled with the rest of the system, most notably n8n.

And if that made you go - "bummer, but at least it's open source, so I can replicate this" - yes! Go right ahead! I have often shared various bits and bobs from botfire5 on this blog in case someone wants to steal them. However, you may have a tough time if you're a total beginner. If, for example, one knew how to code, but knew nothing about self-hosting in general and all these apps in particular, it's a lot of work to get into it. None of the technologies are particularly daunting, at least not at this scale, but they do have their peculiarities; and there are a lot of them. For me, adding a new part to the stack that touches all the components is a matter of a few evenings, because I'm used to them. But if I had to start by learning Postgres, Noco's API, n8n troubleshooting, Meilisearch configuration, Vue state management, plus a myriad other things (every API such as IGDB or Audiobookshelf is different, has a different auth process, ...) - it's a whole lot.

Good news, though! All of that is again learnable in small increments. That's how I did it. You can certainly do it as well. It is a lot of fun. Just don't expect it to be quick.