rqlite creator here, happy to answer any questions.
As for reliability - it's a fault-tolerant, highly available system. Reliability is the reason it exists. :-) If you're asking about quality and test coverage, you might like to check out these resources:
>First of all, does anyone believe that highly scrutinized and bureaucratic functions are general high quality services?
This is the only part of your response that doesn't quite sit right with me. There could be many "highly scrutinized and bureaucratic functions" out there that are working very well, you just don't notice because they work so well. There could be a selection-effect here.
Quality is a big deal for me[1]. But I think you're defining "quality" too narrowly in this context. "Quality" could also mean "allows everyone, at scale, reliably, to do what they need to do." The US Tax Filing system (and its associated software) meets that goal.
Oftentimes, broadly accessible services are lower quality than more personalized ones, to the consumer.
As an example in US government bureaucracy, government software teams digitizing forms at one point weren’t allowed to utilize features like autofill or automatically filling fields based on previous answers because it would relatively disadvantage users using paper forms.
Government capabilities do need to serve everyone, and from the perspective of the whole society that is beneficial, but they are often are of low quality to the individual consuming them for this very reason.
Let’s exclude taxes, because obviously many people would hate them under any circumstances. Does the government do a good job providing the other services people interact with regularly? Do people love their visits ti the DMV? Are they satisfied with their interactions with the police? Heck, in my town, just renewing a dog license is a pain.
> The US Tax Filing system (and its associated software) meets that goal.
I disagree with the argument that the US Tax Filing system meets the goal of:
> "allows everyone, at scale, reliably, to do what they need to do."
It may do so for easy / common cases of W2 salaried employees but step a little outside of the norm (foreign sourced income, tax treaties etc.) and software gives up and shows you a PDF of relevant forms and requires you to become an expert in tax code and to keep your own multi year running calculation of carryovers and things to proceed. I'm glossing over all of the detail about how complex this really is but wouldn't expect the average, even very intelligent person to succeed in filing a correct return without a professional's help.
my anecdata is that I have always filed manually by myself, but every time had a small adjustment made by the IRS... indeed filing a correct return the 1st time seems close to impossible.
Will it allow you to reach the same scale in terms of data set size that Elasticsearch supports? Almost certainly no, but it might be enough depending on your use case.
Have you done any updated benchmarks on maximum supported writes/sec since the talk you gave comparing single-zone, single-region, etc.?
I recently had a potential use case for this, but it required somewhere around 600 writes/sec at a minimum, and it wasn’t clear what the ceiling was for rqlite without sacrificing durability guarantees.
rqlite creator here. I have performed a fair amount of performance testing, some of which I outlined in a talk to the CMU Database Group a few years ago. Details:
An important thing to note: this testing was done 4+ years ago, on moderately-powerful hardware for the time. With higher-end, more modern hardware you may get even better results.
Thanks for the reply. Yeah, that’s the talk I was referring to.
I suppose I could try my own benchmark out tbf. I’m curious to see what it can do on today’s hardware. I would think it’s mostly network-bound for Raft consensus, though the 10x ping time increase you demonstrated without an appreciable drop in writes suggests it’s more complex than that.
I did introduce Queued Writes[1] since that talk, allowing you to trade off performance versus immediate durability. It may interest you -- network is much less of a factor then, and you should get a 10-100x increase in throughput.
Nit: dqlite is a library, it is not a network-exposed database like rqlite is. Sure, it requires connecting to other nodes over the network, but local access is via in-process. In contrast one connects with rqlite over the network - HTTP specifically.
rqlite[1] author here. Just to be clear, rqlite is not SQLite but rewritten in Go. rqlite uses the vanilla C code, and calls it from Go[2]. I consider that an important advantage over other approaches -- rqlite gets all the benefits of rock-solid[3] SQLite. As result there are no questions about the quality of the database engine.
You are right that when someone (a human) submits a PR it didn't cost me anything (short of my time to review it). But those folks are not a team, not someone I could rely on or direct. Open-source projects -- successful ones -- often turn into a company, and then hire a dev team. We all know this.
I have no plans to commercialize rqlite, and I certainly couldn't afford a team of human developers. But I've got Copilot (and Gemini when I use it) now. So, in a sense, I now do have a team. And it's allowed me to fix bugs and add small features I wouldn't have bothered to in the past. It's definitely faster (20 mins to fire up my computer, write the code, push the PR vs. 5 mins to create the GitHub issue, assign to Copilot, review, and merge).
Case in point: I'm currently adding change-data-capture to rqlite. Development is going faster, but it's also more erratic because I'm reviewing more, and coding less. It reminds me of when I've been a TL of a software team.
There is a new continuum. "Team" is just a convenient word to emphasize that "Tools" are moving significantly in the "Teams" direction.
The post emphasizes the degree this is true/not.
Different people are going to emphasize changing attributes of new situations using different pre-existing words/concepts. That's sensible use of language.
No, it's clickbait and that's why this submission got flagged, sorry.
A team is comprised of people. Being able to prompt an LLM to create a pull request based on specifications is very useful, but it's not a team member, the same way that VSCode isn't a team member even though autocomplete is a massive productivity increase, the same way that pypi isn't a team member even though a central third party dependency repository makes development significantly faster than not having one.
If this article were "I get a massive productivity boost from $41.73/month in developer tools" it'd be honest. As it is, it's dishonest clickbait.
Ok, that's cool that you can develop faster now, but as the other comment: it is a tool, not the cost of a team. It still for me a very strange comparison.
As for reliability - it's a fault-tolerant, highly available system. Reliability is the reason it exists. :-) If you're asking about quality and test coverage, you might like to check out these resources:
- https://rqlite.io/docs/design/
- https://rqlite.io/docs/design/#blog-posts
- https://philipotoole.com/how-is-rqlite-tested/