2 min read personal stack

MeiliSearch >>> ElasticSearch

In this post, I'm talking about using these search engines for small-scale, personal project deployments. If you're interested in large ones, no need to read further.

I wish more open-source projects used MeiliSearch. Elastic seems to be the de-facto standard everyone reaches for automatically. But it's such a resource hog.

I run plenty of apps on my server(s) - over 2 dozen. Most of them are only used by me, and none have more than 10 users. I have 2 servers, and each of them has 8GB of RAM, with no prospect of adding more.[1]

So when I try a new app[2] and I notice it uses ElasticSearch, I'm wary. It always ends up being the same story - I start the (docker) stack and look at its memory usage. The app uses, say, 400-800MB RAM. The database is often Postgres, which takes up about 50-80MB.[3] And then bam, 1 000 MB RAM for ElasticSearch.

What is ES doing? Well, beats me! Apparently, it needs that much RAM just to handle... absolutely no data at all. Because the app has just started and is empty.

I just can't afford to run that, on a server with 8GB RAM and 20 other things already running on it.

For comparison - when I start an empty MeiliSearch app, it usually sits at 100MB or less. My Universal Personal Search app, which currently indexes some 40 000 documents, uses about 500MB while I'm using it.

"What are you talking about‽" I hear you say. "40k documents is nothing!"
Well, yes! That's my point exactly. At the moment, this is my most data-intensive app so far. Most of my apps have an order (or two) of magnitude less data.
And the open source apps I'm talking about here also have many self-hosted deployments that are often not much bigger than mine.

Again, to be clear: this is just an "old man yells at cloud" rant. (Because ES doesn't fit into my cloud, ha ha, get it) I'm not saying your use-case doesn't absolutely need ES. I'm also not talking about your hyper-optimized enterprise stack.

I just wish more open-source projects defaulted to a search engine I can actually run.

  1. One of them is an old second-hand laptop with hardwired RAM, so it's literally impossible to have more. ↩︎

  2. most recently TubeArchivist ↩︎

  3. Which is insane. Postgres is so good. ↩︎

zblesk

zblesk