> You can teach someone to use a fixed set of macros. But again, in that case why use a language with macro features?
Because sometimes writing a simple macro makes it easier to write (and read) other code.
> If you're never going to use a non-built-in macro with the complexity of the built-in macros, why does the language need this complex macro system rather than a set of built-in keywords and a simpler extension system?
There's a lot to unpack here. A good lisp developer will very rarely need to write a macro that approaches the complexity of the loop macro. Firstly, that's not never. Being able to write a hairy macro is more like an air-bag than power windows. You're glad to have it when you need it, but you mostly forget its there the rest of the time. Secondly the same macro system that lets you write something like loop also lets you write simpler macros that make code both easier to read and harder to write incorrectly.
Lisp does not have a "complex" macro system. It has one of the simplest extension systems of any language I've used that has an extension system.
There are also plenty of people who think loop is too complicated and too inflexible and so avoid it.
Finally everything you said about macros and loop can be applied to other languages and the standard library. If nobody's going to write anything as complex as printf(), why does C need variadic functions? If nobody's going to write anything as complex as itertools, why does python need higher-order functions and generators?
I don't actually see the extensibility of lisp as its "killer feature" but I definitely see it as a net positive, while a lot of people seem to see it as a net negative. I blame books like "Let Over Lambda" which are more to be seen as examples of what you can do rather than what you should do.
There's lots of things to hate about C, but the fact that you can do:
enum { TRUE = 0, FALSE=1 };
is not one of them. There's plenty to hate about lisp, but the fact that it's possible to write bad macros is not one of them.
> Finally everything you said about macros and loop can be applied to other languages and the standard library. If nobody's going to write anything as complex as printf(), why does C need variadic functions? If nobody's going to write anything as complex as itertools, why does python need higher-order functions and generators?
IMO the essence of good language design is balancing expressiveness with enough constraint to make code understandable. I agree that printf is a mistake (string interpolation is enough of a special case that it's better as a language-level builtin). As for itertools I'd say that I do write code that's just as complex as the itertools builtins, but also that the itertools builtins don't hurt code readability so much because they're all (more or less) legitimate functions that can be reasoned about compositionally. Code that contains a call to a complex itertools builtin is not particularly hard to understand, because even the most complex itertools builtin returns a plain(ish) value that obeys simple rules. Not so for code that contains a call to a complex macro.
> Lisp does not have a "complex" macro system. It has one of the simplest extension systems of any language I've used that has an extension system.
"complex macro" system rather than complex "macro system", but yeah, I expressed myself badly. The problem isn't complexity per se (and as you say, the actual system is very simple) but that it's an unconstrained one where a macro can do almost anything.
> There's plenty to hate about lisp, but the fact that it's possible to write bad macros is not one of them.
I disagree. In my experience the main reason people give for moving away from lisp on any given project is "maintainability", and if you dig a little deeper that usually boils down to "we wrote too many bad macros". And a lot of the "limited tooling" complaints boil down to the effectiveness of IDEs etc. being limited by the presence of macros.
A program can do almost anything; that's the result of internal Turing completeness, plus unfettered external platform access. Macros do not cause this. A program can do anything using nothing but functions.
Macros are supposed to be used when they organize some particular anything in a way that makes it easier to understand and maintain (or even more performant) compared to the best available macro-free approach.
Everything can be abused: control structures, data structures, functions, variables, ...
I think most programmers would agree that running a code-generation program at build time is something that should be approached with a little more caution (not saying that you blanket shouldn't do it, but you should check the reputation of the code generator, what the output looks like, how well your debugger/profiler can handle it...) than using a plain-code library. But a lot of people seem to somehow fail to realise that using a macro is the same thing.
Everything can be abused, which is why it's best to write everything in the most restrictive environment that still allows expressing what you need to. See Dhall or Noether for what that looks like for everyday code. At the language design level, there may be cases where users need to apply an arbitrary Turing-complete AST->AST function, but it shouldn't be the first thing they reach for; if there are features (such as do notation) or patterns (such as Python decorators) that allow expressing most macro use cases via a more constrained (and therefore less maintainability-impacting) feature, that's a big win.
Because sometimes writing a simple macro makes it easier to write (and read) other code.
> If you're never going to use a non-built-in macro with the complexity of the built-in macros, why does the language need this complex macro system rather than a set of built-in keywords and a simpler extension system?
There's a lot to unpack here. A good lisp developer will very rarely need to write a macro that approaches the complexity of the loop macro. Firstly, that's not never. Being able to write a hairy macro is more like an air-bag than power windows. You're glad to have it when you need it, but you mostly forget its there the rest of the time. Secondly the same macro system that lets you write something like loop also lets you write simpler macros that make code both easier to read and harder to write incorrectly.
Lisp does not have a "complex" macro system. It has one of the simplest extension systems of any language I've used that has an extension system.
There are also plenty of people who think loop is too complicated and too inflexible and so avoid it.
Finally everything you said about macros and loop can be applied to other languages and the standard library. If nobody's going to write anything as complex as printf(), why does C need variadic functions? If nobody's going to write anything as complex as itertools, why does python need higher-order functions and generators?
I don't actually see the extensibility of lisp as its "killer feature" but I definitely see it as a net positive, while a lot of people seem to see it as a net negative. I blame books like "Let Over Lambda" which are more to be seen as examples of what you can do rather than what you should do.
There's lots of things to hate about C, but the fact that you can do:
is not one of them. There's plenty to hate about lisp, but the fact that it's possible to write bad macros is not one of them.