Debugging Lisp: fix and resume a program from any point in the stack - how Common Lisp stands out.

To really discover what interactive, image-based CL development is about, my pointers: - ▶ this video (5min) my demo to show how to resume execution of a long running program, from the stack frame right before the bug, after having fixed and recompiled the function, without quitting the debugger hence without stopping the program, thus without restarting everything from zero. - link (20min)showing what saving and restoring an image does, differences between SBCL and LispWorks. - link developing a 3D graphics software. - link - debugging CL tools: link - and no: while we could deploy a ball of mud to production, we can also not and use traditional tools and techniques: build an executable from scratch, restart the app, reload state. But we can do everything in between too (inspect the running app, change it a bit, change it a whole lot (while staying in sync with local source code), use images to speed-up development etc).

false premises, false conclusions, and bad good ideas in-between. > I was surprised by the absence of a quickstart page to help me set up a simple server what about link ? Also I think that the READMEs of Mito (ORM) and Djula (templating) and easy-routes (routing) are very good. Heck, Djula's documentation is excellent. Double heck, even Caveman's README, which the author uses, has a good and sufficient getting-started page. Oh, I understand, the author uses Clack/Lack… where Hunchentoot is much better documented. there are of course more blog posts, some videos. I invite any blogger to consolidate community resources. More recent: link ps: > ;; If the template is not already compiled, it is compiled, and stored in TEMPLATE-REGISTRY . won't that prevent the template to compile changes? I suggest to follow Djula's doc. > I would add a way to invalidate the cache automatically if a template changes. yeah. So don't do that ;) (also +1 to Jach's remarks and explanations) > I created a macro that ensure the table gets created correctly hummm… I'd suggest to leave the migration step explicit. > some database helpers to perform a few CRUD actions on the db. I felt the need for CRUD helpers too when defining Mito classes. However those four ones are only similar functions with same parameters but another name :] (defun db-add (instance) "Inserts a model instance into the database." (insert-dao instance)) > I would say that there are better languages for developing modern web apps

> all the other languages have kept picking up nearly everything once-cutting-edge about Lisp not all of them have CL's set of features: excellent long-live REPL, image-based development, compiling to binaries, the condition system, the interactive debugger and excellent developper tools (resumable stack frames, never re-run a long operation from zero [1]), the whole language at compile-time, stable syntax, extensible syntax, stable ecosystem, fast implementations, numeric tower, first-class interop to Java (LispWorks, ABCL), commercial support, mobile platforms delivery (LispWorks)… New CL implementations get created (CLASP, CL on C++ with LLVM or NPT, in C) and CL compilers and libraries evolve too [2]. SBCL is now callable as a shared library, it got a SIMD module, rumors say a bear party after ELS 2024 escalated and that they are adding coroutines soon. ECL is targetting WASM, all other major implementations had new releases. New libraries unlock new possibilities (Coalton: Haskell-like typing on top of CL), companies still pick CL today (CL is all the rage in quantum computing?), the community creates more resources and more tools such as, last but not least, new editor plugins o/ link (VSCode, Pulsar, Jetbrains, Sublime, Jupyter… Lem of course etc) [1]: ▶ this video [2]: link

When you redefine a function in CL, every function that use this function is notified of the change and pickup the new version. And because it's functional and you're not supposed to modify the state outside your function, everything will keep churning. The issue with emacs is the sheer number of variables. And the fact they're global. That leads you to do stuff you shouldn't do. But the principles are essentially the same. And in CL, you can snapshot the current state of your program to a file. Then restart later, instead of reloading the environment from the code. A quick demo: ▶ this video

> the code that is currently executing can be manipulated live Typically when people say this about Lisp, they mean that they can redefine functions and objects live, having all the code adapt to use the new definitions, or when their code hits an error, they can not only redefine functions and objects live, but also examine the state of the call stack and change the values of variables live, and then let the code run again. This[0] is a clear explanation that I found. There's also a video[1] that shows the same thing. However, I don't know if those apply to Scheme. Also, I am pretty sure it is unrelated to Lisp being homoiconic, and that it's more about it being dynamically typed and interpretable. [0]: link [1]: ▶ this video

I suggest: do you find yourself often quitting the debugger and re-running the same test, the same task from the start? You can avoid that in CL. Just leave the debugger open, go to the buggy line ("v" on a stacktrace in Slime), fix the function, re-compile it on the fly (C-c C-c in Slime), come back to the debugger, resume the program from the stackframe you want, now it passes. If the first steps were taking 10 minutes, you just saved yourself some time. You don't need special needs for that, it's a fast workflow I use everyday. demo: ▶ this video debugging a complex stacktrace: Python VS CL: link Another important point: we debug everything while keeping the current state in memory (in the image). We don't have to re-create our test data. All the program state is kept live, until we restart the image.

not true, sorry! And if some languages took features, the combination of them is unique in CL. Image-based REPL interactivity, condition system, interactive debugger, restarts, compile-time computing, compile-time checks, incremental typing, speed, a different object system, stability, single-file binaries, connecting to a remote system… Here we restart a buggy program from a single point in the stack: ▶ this video so we don't re-run everything from zero.

*and not only paste, but write, in the source file, the new definition, re-compile it (C-c C-c), and resume execution from a stackframe before this call. (a clarification for folks who would think we must paste in the REPL) That's useful not only for breakpoints but for errors. We can fix an error without re-running everything from zero. ▶ this video

That's a short video I made to show this: ▶ this video It's a feature that is useful everyday, if only to speed up trivial things.

> lots of interoperability libraries That's true. For cases when you want to start with a good set of libraries (json, csv, databases, HTTP client, CLI args, language extensions…), I am putting up this collection together: link It can be used as a normal Quicklisp library, or as a core image (it then starts up instantly) or as a binary. It can run scripts nearly instantly too (so it isn't unlike Babashka). We are ironing out the details, not at v1.0 yet. > handling a runtime error by just fixing the broken code--in-place, without any restarts [from the blog] Also (second shameless plug) I should have illustrated this here: ▶ this video We run a long and intensive computation and, bad luck, we get an error in the last step. Instead of re-running everything again from zero, we get the interactive debugger, we go to the erroneous line, we compile the fixed function, we come back to the debugger, we choose a point on the stackframe to resume execution from (the last step), and we see our program pass. Hope this illustrates the feature well!

Best of both worlds: write unit tests with an interactive debugger o/ Tell the test runner to not catch errors and let them pass through up to the debugger. Have a test fail, get the debugger, go to the buggy line (without quitting the debugger), fix the bug, compile the function with a keystroke (yes, one function), come back to the debugger, restart not the full operation, but the function call, the debugger frame where the error happened, and see the test succeed. You ran the test exactly once and fixed everything in the process, very quickly. This is exactly what we do everyday with the debugger, with trivial or complex code, with unit tests or regular code. An example: ▶ this video not with unit tests, it's an example on how to restart a precise frame, how to avoid re-running everything.

Related to debugging, the follow up article: how to fix and resume a program from any point in the stack. Have the debugger pop-up, read the error message, go to the erroneous line ("v"), fix the code, compile the function (C-c C-c), come back to the debugger, press "r" on any point in the stack, see the program succeed. You did not have to re-run everything from zero. link ▶ this video and some more: link