I read about Erlang on /r/programming, which was having a new fad for a new shiny language, as was the custom back then.

And I desperately followed all the /r/programming fads because I was worried that I'd end up irrelevant if I wasn't skilled up in the latest Haskell web framework.

But one of those fads was Erlang, and it intrigued me so much, that I ended up printing Mr Armstrong's thesis, stuck it in a manila folder, and read it, slowly, cover to cover while waiting for, or riding on, my bus to my contact centre job, and I've still got it in my bookcase.

His thinking on resiliency in the face of inevitable failures, and on safe concurrency, has shaped my thinking, and proved invaluable repeatedly and is very relevant today. It's like their phone switches were distributed microservices running in a container orchestrator, before it was cool.

I am now inspired to do the same, thank you!

A (worker) thread dying isn't an issue in Rust, Go, C# and etc. Sure, each goes about error handling in a slightly different way either opt-in or opt-out or enforced (when unwrapping Error<T, E>) but other than that the advantages of Erlang/Elixir have faded over time because the industry has caught up.

p.s.: C# has not one but two re-entrancy syntax options - 'async/await' and 'IEnumerable<T>/yield return'. You can use latter to conveniently implement state machines.

It really hasn't. People fixate on the idea of just running some processes, and just catching some errors.

And yet, non of the languages that "solved this" can give you Erlang's supervision trees that are built on Erlang and Erlang VM's basic functionality. Well, Akka tried, and re-implemented half of Erlang in the process :)

But other advantages did fade: multi-machine configurations are solved by kubernetes. And it no longer matters that you can orchestrate multiple processes doing something when even CI/CD now looks like "download a hundred docker containers for even the smallest off tasks and execute those".

> p.s.: C# has not one but two re-entrancy syntax options - 'async/await' and 'IEnumerable<T>/yield return'.

What I meant by re-entrancy in the VM is this:

Every process in Erlang gets a certain number of reductions. Where a reduction is a function call, or a message passed. Every time a function is called or a message is passed, the reduction count for that process is reduced by one. Once it reaches zero, the process is paused, and a different process running on the same scheduler is executed. Once that reaches zero, the next one is executed etc.

Once all processes are executed the reduction counter is reset, the process is woken up and resumed.

On top of that, if a process waits for a message, it can be paused indefinitely long, and resumed only when the message it waits for arrives.

So, all functions that this process executes have to be re-entrant. And it doesn't mean just the functions written in Erlang itself. It means all functions, including the ones in the VM: I/O (disk, network), error handling, date handling, regexps... You name it.

Erlang (and by extension Elixir) has the advantage that the programmer can think at a higher level about their system. You don't have to write or configure a scheduler. You don't have to invent supervision trees. You can be sure that the concurrently-running parts of your system cannot possibly affect each other's memory footprint (though Rust gives a robust answer to this problem as well).

It doesn't make a perfect fit for every problem, but there is still a decent-sized space of problems -- I'd say "highly concurrent, but not highly parallel" -- where Erlang gives the programmer a headstart.

And in those cases there is simply no need for e.g. supervision trees because there is nothing to restart. You still need stuff like retrying, but this is supported by all major concurrency libraries / effect systems. In fact, Erlang has fallen behind here in terms of what the language offers (not what the BEAM offers, the BEAM is still top notch imho)

State is mostly moved to either the database or message queues or similar, which is pretty good.

Namely: The message queues are part of the language. They're built right in, for ease of use. Your caching layer is built into the language, for ease of use and faster performance.

It's true that handling state is hard in most programming languages. But that's neither the only nor the most important reason why people try to keep applications stateless.

I don't use Erlang but my understanding is that while it is not exactly fully pre-emptive, there are safeguards in place to ensure process fairness without developer foresight.

That's why synchronously blocking a thread is not a complete loss of throughput. It used to be worse but starting from .NET 6, threadpool was rewritten in C# and can actively detect blocked threads and inject more to deal with the issue.

Additionally, another commenter above mistakenly called Rust "bare metal" which it is not because for async it is usually paired with tokio or async-std which (by default, configurable) spawn 1:1 worker threads per CPU physical threads and actively manage those too.

p.s.: the goal of cooperative multi-tasking is precisely to alleviate the issues that come with pre-emptive one. I think Java's project Loom approach is a mistake and made sense 10 years ago but not today, with every modern language adopting async/await semantics.

...but the problems stated are real. I'm excited to hear that this might be fixed in .net 6 but it'll be a while before that rolls out to most deployments.

It has never been a big issue in the first place because by now everyone knows not to 'Thread.Sleep(500)' or 'File.ReadAllBytes' in methods that can be executed by threadpool and use 'await Task.Delay(500)' or 'await File.ReadAllBytesAsync' instead. And even then you would run into threadpool starvation only under load and when quickly exhausting newly spawned threads. It is a relatively niche problem, not the main cornerstone of runtime design some make it out to be.

Also, .NET 6 is old news and has been released on Nov 8, 2021. It is the deployment target for many enterprise projects nowadays.

Knowing that file IO is too heavy and has *Async counterpart methods is somewhat obvious to a veteran, but other long running methods are not so obvious. In this case you would need to profile your use case to understand that certain calculations/methods might be best farmed off to a different threadpool.

Unity still uses Mono and has a very low max thread pool size, for example. The thread pool is easily starved in the latest version of that engine and I'm sure it's more common than you think.

Relatively niche, perhaps, but a critical problem when stumbled upon none the less. Again, I like async/await but there are certainly foot guns left to remove.

Again, and I cannot stress this enough, we're discussing somewhat niche feature. You have to take into account that even the standard library still has a lot of semi-blocking code, simply due to the nature of certain system calls or networking code. From runtime standpoint, blocking or computationally heavy logic - there is no difference, it will scale the amount of threads to account for fairness automatically. It's that blocking just has extra cost due to being "better" at holding threads (you don't have to think about it). .NET 6 is just comparatively better at dealing with such scenarios but your app would work fine in PROD 9 times out of 10 with invalid code before or after that. It's a difference between running 'Task.Run(() => /* use up thread for no reason for seconds / minutes */))' in a 100s iterations loop going from terrible to very bad.

It's pointless to "fight against words". Just trust the runtime to do its thing right. That's why its baseline cost is somewhat higher than that of Golang or Rust/Tokio - you pay more upfront to get foolproof solution that has really good multi-threaded scaling.

If you don't want to believe the above, just look at average C# solutions on Github. There are no "special magic to learn", that's just how people write code new to the language or otherwise.

p.s.: This situation reminds me one of my colleagues who would always come up with an excuse for his point regardless of context. It's counter-productive and self-defeating.

If your Rust thread panics while it holds a Mutex, you've got a bit of a mess. Especially if it was halfway through updating shared mutable state. Probably similar in Go or C#, but I haven't used Go and only did cargo cult programming in C#, I didn't read any sources or see warnings about crashing in threads or async/await.

Go channels are nice but they don’t come close to Erlang message passing. In go you can’t just ignore if the channel is bounded or unbounded, open or closed. Writing to a closed channel with blow you up. It takes some time to learn it. Messages in Erl are easy fifo serial execution.

If you're willing to make your "lightweight processes" OS threads you could kind of make it work. E.g. Rust gives you both panic hooks (to notify everyone else that you died) and catch_unwind to contain the effect of a panic (which generally stops at a thread boundary anyways). But of course that only scales to a couple hundred or thousand threads, so you probably have to sacrifice a lot of granularity.

And any library that links to C/C++ code has the potential to bring the whole process down (unless you make your "lightweight processes" just "OS processes", but that just makes the scaling problems worse)

It’s explained in many documents about lightweight processes, of course for elixir/erlang/beam but also for Go and Crystal and even going back to Solaris Internals and modern Project Loom for upcoming JVM situations

So firing off and monitoring processes becomes second nature easily. Rarely so in other languages

Meanwhile, your OS thread will just keep eating CPU, but everything else is fine.

You're right about the number of threads though. It takes a lot more memory to run an OS thread than a BEAM process, and I'm not sure OS schedulers will manage that many threads even if the memory is there (but I could be wrong... getting things to work nicely at 50k threads may be sufficient for millions)

Doing this at such a granular level is also one of the reasons that you can run a database that thousands of processes are accessing, within the same runtime, without a performance impact to the overall system.

You can recover() panic() just fine in Go.

The nuance is that most languages let you build reliable programs _if your code is correct_ - if you're using defers, context handlers, finalizer, cleaning up state in shared data structure, etc.

Erlang's goal is to be reliable "in the presence of software errors", that is, even if your code is buggy. If a request handler process dies inside an Erlang web app, whatever file or socket it opened, memory it allocated, shared resource it acquired (e.g. a DB connection) will be reclaimed. This is true without having to write any error handling in your code.

The way it's done is that the VM handles the basic stuff like memory and files, and it provides signals that libraries (like a DB connection pool for instance) can use to know if a process fails and clean up as needed. In other words the process that fails is not responsible for its own clean up.

At some point some code must of course be correct for this to work. Like, if the DB connection pool library doesn't monitor processes that borrow connections, it could leak the connection when such a process dies. But the point is that this part (the "error kernel") can be a small, well-tested part of the overall system; whereas in a classic program, the entire codebase has to handle errors correctly to guarantee overall stability.

It's very hard to build Erlang-like versions of supervision trees using those tools.

Also by the very nature of Go your application state is as likely as not to be fucked, so even if you did handroll monitors and links, and thus could build supervision trees, they wouldn’t be of much use.

You can catch_unwind() panic!() in Rust too :-).

"Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang." - Virding's Law ;)

So first you have to implement all those things. And then use them.

I mean, you can write erlang-way in almost any language, just in this case you need to adopt all the libraries to follow the same principles.

Some languages implement similar error handling strategy by just creating a separate process per request (hello, php). We know how to clean after a worker dies (just let the worker die). Just in that case supervisor strategy is very simple.

Pure magic, that can't be recreated in any regular language :)