You're not doing interactive development on production. Production is the case where you just build a system image and deploy it without compile times. This is done all of the time, for example with Pumas.
This is a binary view of development-verses-production. I appreciate languages that treat this as a continuum and let me make my own trade-offs.
Common Lisp, Smalltalk, and Erlang all treat the process as running indefinitely. This gives those languages a certain something special.
I was hoping to find the same thing in Julia. It's not there. It seems like a significant missed opportunity to me. These dismissive comments make it sound like you guys might not know what you are missing.
We legally cannot do things like say live code change the production system underlying the analysis of ongoing clinical trials without going through the whole software development lifecycle required by the FDA (I.e. the Pumas example again), so even if it did exist all a lot of the production systems we're running legally couldn't use it. But also, you can do it with Revise+RuntimeGeneratedFunctions, obviously that solves the mentioned world age issue so I don't see what the fuss is all about? I don't see how offering a solution is dismissive?
but you can adhere to legal requirements and go through all the code and test reviews and still have live update capability. i don't see why they are exclusive. moreover, there are scenarios where taking the system off-line for an update would cause a critical failure
Okay, so then use RuntimeGeneratedFunctions+Revise. Done. Julia can do it, but having a setup so that people can do instant review of changes on the live production system is still a little bizarre and I wouldn't do it even though we could.
Yeah. This forum doesn't seem like the best place to have the discussion, alas. Maybe the corner of a room at JuliaCon with a few Lisp refugees one day :).
Coming from a Lisp background Julia fits like a glove in the beginning but some aspects, like the inability to redefine structs, I just crash into like a brick wall. It makes it hard to imagine writing (say) a Blender/Paraview type of application in Julia.
I suppose that Julia is heavily influenced by Scheme and Dylan but not especially by Common Lisp (or Smalltalk) where the idea of being unable to redefine types at runtime is jarring.
I wonder if Julia advocates would do better to just acknowledge that this point of view exists, even if experience with Julia might change it over time, rather than dismissing it (yes, very much so) as empty "fuss." But there aren't that many Lisp/Smalltalk hackers in the world so you can probably afford to alienate them if you want to...
It's acknowledged. The full redefinition of any function in any context was solved by RuntimeGeneratedFunctions and we use it extensively throughout SciML, ModelingToolkit, Pumas, etc. so it's fair to say struct redefinitions are the only thing left. That said, "struct redefinitions" are done all of the time in many contexts: if you know you may need to do this, you can just use a named tuple which acts just like an anonymous type and obeys dispatch on a named form that you can then hijack at runtime. With smart engineering then you can be redefining everything on the fly, and we do this through MTK quite regularly. That said, it would be nice to do the last feat of full redefinition of non-anonymous structs, and the latest proposal for how to do struct redefinition is https://github.com/JuliaLang/julia/issues/40399. So we both built and provided solutions for how to do it, showed how to do it in libraries, tutorials, videos, etc. How is that not acknowledging the point of view and doing something about it?
You might want to try and acknowledge the other point of view where I note that, hey, we did make this work but it was a smaller deal then we thought because we legally cannot employ it in many production contexts. We're still going to work out a few details for fun and ease of debugging, but given that we have extensively looked into and thought deeply about the whole issue, we have noticed that the last little bits are less useful than we had thought (at least in the contexts and applications I have been discussing, like clinical trial analysis). That doesn't mean we won't finish the last pieces, but given how much of it you can already do and how many teaching materials show how to do work around the issues in any real-world context, and how little of a real-world application the last few bits have, it shouldn't be surprising that the last pieces haven't been a huge priority. So instead of looking narrowly at one factor, I encourage you to take a more global view.
I'll concede that there seems to be a lot of work done on this in Julia that I'm not aware of. Thank you for the references, I'll check them out and then gladly concede the rest the next time this topic comes up.
In certain common lisp deployment you might do that.
There are legends from Common Lisp users where this is very much done.
> When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone.
> Such fast turnaround on bug fixes put us into an impossibly tempting position. If we could catch and fix a bug while the user was still
on the phone, it was very tempting for us to give the user the impression that they were imagining it. And so we sometimes (to their delight) had the customer support people tell the user to
just try logging in again and see if they still had the problem. And of course when the user logged back in they'd get the newly released version of the software with the bug fixed, and everything
would work fine. I realize this was a bit sneaky of us, but it was also a lot of fun.
I know that this is not possible in Julia like that. But just because it's not possible in Julia doesn't mean what they did was somehow wrong when it worked well for them.
Call me a cynic but I would hope almost no devs had the power to live update the production codebase. That sounds crazy to me. To me, that kind of thing should only be done with (approved/verified) versions which are cut and tested before deployment, and deployment should act like an unchangeable immutable compiled binary until the next binary is ready. Not only is this legally required in many scientific cases, but also production should have a lot more care.
This kind of thing sounds fine for a dev server though, in which case Revise+RuntimeGeneratedFunctions will get it done in the cases I've seen.
I understand where you’re coming from, it’s the production robustness paradigm of someone who has worked with such systems and knows what kind of controls are required to release code. eg live editing like that would almost certainly go against SOC2 controls.
However, I’m also tempted very much by a system that allows for fixing things so quickly, because it provides for a delightful customer experience. Most customers don’t report most issues, only when something is persistently wrong or is something they can’t get around. So being able to quickly fix it this way seems genuinely amazing. If I were to take this idea further… maybe some sort of customer isolated canary deploy/ feature flag might be the way to express this customer experience in todays world.
The problem is always that you might break other things with your fix. Maybe a battery of testing before canary deploys can raise confidence about the validity of the fix.
Also you can very much not redefine functions on the fly due to world age (assuming the caller doesn't pay for invoke-latest).