"Sadly, something essentially killed lisp innovation around the time Common Lisp got standardized"
I think it was timing, more than anything else.
The movement to create Common Lisp came right on the cusp of several important shifts in computing. The first was from specialized hardware to standardized platforms. Lisp bet heavily on the former as embodied in Lisp machines at the very time that ISA and x86 machines proliferated.
The second shift was from command lines to GUI's and the Common Lisp standard lacks facilities for windows and other GUI elements. There is no equivalent of Swing in Common Lisp.
At the time it was standardized, Common Lisp just missed the explosion of the commercial internet and the distribution advantages it afforded Java just a few years later.
Finally, the way in which the Lisp community tended (and still tends) to shun Microsoft Windows, probably played an important role in limiting its general adoption. After the demise of Lisp machines, the remaining segment of the Lisp community tended toward FOSS partisanship - somewhat unsurprising since RMS emerged from the Lisp community and it was closed source Lisp code which first led him to action.
In short, Lisp was both unlucky and placed bad bets relative to the current state of affairs. The ultimate effect of a bias toward FOSS remains undetermined.
It was not just "timing" that killed Lisp (if one can even say that Lisp is dead -- I use it in my day-to-day work, and I am not alone). While the world's Lisp hackers were working on lofty AI dreams, the world's Unix hackers were spreading C to every college, university, and business out there. Had an open, portable, low-end OS like Unix been written in Lisp, I suspect the programming world would be a much different place today.
The article suggested that Lisp innovation was dead, not Lisp itself. Written before Clojure was a viable option (per the author's comment in this discussion), that's a pretty easy position to defend.
Whether moving Lisp to the portable JVM and adding concurrency and other features it offers counts as innovation, is a matter for reasonable discussion.
It seems that you were not really there during the 80s.
First 'Lisp' can't bet. It's a programming language. As a community it is extremely diverse.
Let's get the history of Common Lisp right. Common Lisp was created to unite the successors of Maclisp (and related languages) and to make it feasible to create and deploy applications written in it. During the development of Common Lisp from 1982 on it was immediately implemented on all kinds of machines: especially on Unix, Windows and Macs. LUCID CL was a commercial system for Unix, LispWorks started on Unix, moved to Windows and the Macs, Allegro CL was on Unix and moved to Windows and later to the Mac. Golden CL was on Windows. Exper CL on the Mac. Macintosh Common Lisp on the Mac. KCL/AKCL/... on C. CLISP in C. Plus tons more.
When I worked in an AI Lab in the early 90s, developers were using: Macintosh Common Lisp on Macs and Allegro CL on SPARC/Solaris (used in 'planning and configuration'). We had a LispWorks license for SPARC/Solaris for an image processing project. Golden CL on Compaq/Windows. Lucid CL was used on SPARC/Solaris for a commercial project. Lisp Machines were not used anymore.
It just happened that at the end of the 80s / beginning of the 90s the development of applications which were typically developed in Lisp moved to C++. Reason: better deployment options on machines of that time and 'momentum'. Some then even moved to Java later. Few moved back when Common Lisp came back during the early 20xx years.
Common Lisp lacks 'standard lacks facilities' - just like most languages. Java has Swing, but it's not really successful. Then came SWT. ...
It's not that the Common Lisp community did not try to develop a GUI library standard. During the standardization process there were efforts in that direction. For example 'Common Windows'. But the funding died out and when new platforms appeared (Cocoa, GTK, ...) Common Lisp was out of fashion. The result is that the best GUI toolkits for Common Lisp are commercial: CAPI from LispWorks and Allegro's GUI Toolkit. CAPI runs natively on top of Windows, Cocoa and GTK+.
Common Lisp implementors support the development of Windows applications usually through the commercial offerings: Allegro CL and LispWorks. With some minor alternatives. It's just that these tools are expensive - as expensive as comparable options for other languages.
"During the development of Common Lisp from 1982 on it was immediately implemented on all kinds of machines: especially on Unix, Windows and Macs."
We are both sweeping across history in the same manner, since obviously there were no Macs or Windows machines in 1982, and thus no immediate development for such machines.
Or rather, we are both creating a sketch. Mine, I admit is quick and contains convenient anthromophization.
Commercialization of Lisp was largely focused on Lisp machines. There was no TurboLisp for the upcoming generation of programmers. It never made its way into the mainstream before the internet revolution, and was, as you point out increasingly fragmented (and under financed) by the mid 1990's.
I said from 1982 on. In 1982 Lisp ran on a variety of architectures. Common Lisp systems were offered in 1984 maybe.
Commercialization of Common Lisp was huge on Unix from day one. Lucid was a commercial vendor. You might want to read the description of RIchard P. Gabriel of its business. Franz is a commercial vendor on Unix. Harlequin was one / now Lispworks. When Steve Jobs sold his first NeXTStation, a copy of Allegro CL was included. Every major Unix vendor in the 80s sold a Common Lisp implementation. SUN had the Symbolic Programming Environment (SPE). HP. IBM. Most of the major Lisp software of that day got ported to Unix (some also to Windows). Macsyma ran on Unix and Windows. KEE on Unix. Knowledge Craft. G2. etc etc.
I think it was timing, more than anything else.
The movement to create Common Lisp came right on the cusp of several important shifts in computing. The first was from specialized hardware to standardized platforms. Lisp bet heavily on the former as embodied in Lisp machines at the very time that ISA and x86 machines proliferated.
The second shift was from command lines to GUI's and the Common Lisp standard lacks facilities for windows and other GUI elements. There is no equivalent of Swing in Common Lisp.
At the time it was standardized, Common Lisp just missed the explosion of the commercial internet and the distribution advantages it afforded Java just a few years later.
Finally, the way in which the Lisp community tended (and still tends) to shun Microsoft Windows, probably played an important role in limiting its general adoption. After the demise of Lisp machines, the remaining segment of the Lisp community tended toward FOSS partisanship - somewhat unsurprising since RMS emerged from the Lisp community and it was closed source Lisp code which first led him to action.
In short, Lisp was both unlucky and placed bad bets relative to the current state of affairs. The ultimate effect of a bias toward FOSS remains undetermined.