[HN Gopher] Lisp and the Web: Creating Web Apps Through Lisp and...
___________________________________________________________________
Lisp and the Web: Creating Web Apps Through Lisp and GCE
Author : abhinav22
Score : 150 points
Date : 2021-04-16 06:44 UTC (15 hours ago)
(HTM) web link (medium.math.dev)
(TXT) w3m dump (medium.math.dev)
| fractallyte wrote:
| You could cut out a chunk of configuration by using Caddy web
| server (https://caddyserver.com/) instead of Nginx.
|
| Configuration is very simple (for example, reverse proxying is
| just three lines - for the entire config!), and SSL works out of
| the box!
| abhinav22 wrote:
| Thanks! I'll have a look. I find Certbot a bit fragile to use,
| but not sure if that's just the problem space (since you have
| to verify domain control before issuing SSL certificates).
| Perhaps this will be a more robust approach
| MawKKe wrote:
| https://github.com/ashok-khanna/common-lisp-by-example/blob/...
|
| Formatting queries like this is not a good idea at all. See:
| https://en.wikipedia.org/wiki/SQL_injection
| abhinav22 wrote:
| That's a good point. I will update the code to prevent any
| potential SQL injections so that others follow the right
| practice.
|
| Very nice catch! Thank you very much.
| fidesomnes wrote:
| is there a better method for lisp?
| hashkb wrote:
| Something like https://github.com/fukamachi/sxql
| abhinav22 wrote:
| The issue with my code is that somebody can put in arbitrary
| SQL queries within the data field (I think - not sure if it
| is 100% possible because their data would be enclosed within
| the programs SQL query, which may break their attempts)
|
| Best way around it is to put in checks to prevent the code
| from being anything nefarious. I'll have a think and figure
| out what's the best way of doing this, but I assume not
| constructing the sql strings from scratch but rather using
| postmodern's API (postmodern is Common Lisp's excellent
| interface to PostgreSQL) makes sense, or the alternate
| solution recommended in the other reply to your post.
| stevelosh wrote:
| Postmodern can do the right thing if you use prepared
| queries. See `defprepared`.
| abhinav22 wrote:
| Thanks! I will use that
| phoe-krk wrote:
| Not to be confused with the book _Lisp for the Web_ [2015] by
| Adam Tornhill, https://leanpub.com/lispweb
| tartoran wrote:
| This looks interesting. I started learning Scheme but felt that
| CL has some interesting pieces and this book seems quite
| inviting. I wish I could order a paperback for this book
| abhinav22 wrote:
| Author: I wanted to consolidate some of the steps I did to get
| the basic functionality for Common Lisp web applications (CRUD,
| User Authentication, Domains, etc.) as its a big pain to keep
| revisiting and troubleshooting which step I did wrong.
|
| So I saved these notes in guide form, hopefully its useful for
| others. The trick is to extend a lot of this, all the hard parts
| are here - the rest is just basic Lisp, HTML, JavaScript,
| building upon this platform.
|
| Highly recommend Lisp for Web Development, its *much, much
| better* than any of the other frameworks out there! So easy to
| use. Do learn HTML, CSS & JavaScript though. Esp. JavaScript as
| you would want to offload some of the computation to client-side
| to reduce your server costs.
|
| Big Thanks to u/linnilbobo on Reddit for his code on user
| authentication in Hunchentoot - that was one of the missing
| pieces for me.
| manjana wrote:
| > Highly recommend Lisp for Web Development, its _much, much
| better_ than any of the other frameworks out there!
|
| Could you elaborate a bit on this? I've never written a single
| line of Lisp so I get a bit curious.
| abhinav22 wrote:
| There are many views on this (so please don't take my word as
| gospel), but from my experience, Lisp gives the following
| benefits:
|
| - A fantastic and easy to use open source web server in
| Hunchentoot
|
| - Very easy integration into databases like PostgreSQL. Just
| looking at my guide shows how easy all this is (assuming you
| know lisp, but if you look at the number of lines of code -
| it's not that much)
|
| - Live debugging - I.e. I can recompile certain functions and
| don't need to restart my web server! That's a huge win in my
| opinion
|
| - It's hard to explain, but I find Lisp very transparent.
| With knowledge of JavaScript, HTML, CSS and Lisp, I am very
| comfortable building any web app. Whereas when I use various
| web frameworks like react, angular, etc, I have to spend so
| much learning it's idiosyncrasies (which to me is a waste of
| time since these frameworks keep changing), and then usually
| I hit a brick wall somewhere. Other frameworks in my opinion
| obfuscate too much - they solve for easy wins (which they
| do), but anything more complex and you are fighting against
| the language, spending hours on Stack Overflow for solutions
| to what should be simple problems. In my opinion
|
| I spent 1-2 weeks setting up a CRUD app with Angular and
| still don't know the best way to use it. With Lisp, I did the
| same in _one_ day. It was that experience that pushed to me
| to learning and using Lisp (I was a complete beginner at the
| time). For me that was enough proof
|
| - Stuff like NPM I don't like because there are so many
| dependencies (which break from time to time) and I'm worried
| somebody could be putting in bugs / trackers in these
| packages. I don't like having many dependencies
|
| - Ability to write DSLs (Domain Specific Language) where I
| can easily write macros to generate JavaScript and HTML for
| me from Lisp, increasing the cohesiveness of the whole
| program
|
| All that said, Lisp by itself is not enough.
| HTML/CSS/JavaScript are much more important - Lisp is the
| magic server side plug for all of this, and in my opinion,
| better than server side JavaScript because Lisp is a very
| well thought and well featured language (JavaScript is as
| well, but I think Lisp, at least to me, is slightly more
| enjoyable to write in).
|
| I find the issue with Lisp is that it's not used much, so
| outside of some very good packages, there are many parts
| which are easier to do in other languages / frameworks with
| much bigger ecosystems. For me, web development is the
| exception, because all you need is HTML/CSS/JS and a good
| server side language.
|
| Other languages are also very good, so this is just a
| personal opinion. If you like to invest your time in one core
| language, and not have to keep learning new web frameworks, I
| recommend Lisp.
|
| It's very good for solo programmers or small teams, but I do
| think it's not suitable in larger teams as I do find lisp
| code can be a bit harder to read and also most people don't
| use lisp.
|
| So in my uneducated view - if you have spare time, if you
| like working from first principles and have a scientific /
| mathematical mindset, and if you are working alone or in
| small teams, I highly recommend Lisp for web dev.
|
| If you are more business minded and want to scale your
| project quickly and hire more developers, then using an off
| the shelf package makes more sense.
|
| If you are time poor, stick to what you know - the
| differences aren't THAT great, and it's better to build
| something now than to spend most of your time learning a new
| language (unless you have time to devote to it - in which I
| highly recommend learning Lisp :) It really makes me smile
| every time I code in Lisp :)
| jfoutz wrote:
| This is hand-wavy and bordering on woo, also not specific to
| web development.
|
| In my experience, Lisp has this sort of brain amplification
| power. Whatever weird idea I have about how to think about a
| problem, I can kinda smush lisp into shape so it solves that
| problem. with language X, I have to think about how to I say
| what I'm thinking in X. It's kinda like spending some time
| working in C then jumping to python. It's far less hassle to
| get the idea out there. It's not exactly like that, but it
| has that flavor.
|
| Two sort of related points.
|
| Lisp is pretty transparent. If you need to think about
| pointers and stuff, you can do that and it'll mostly stay out
| of your way. But if you don't, you don't.
|
| Lisp will let you represent whatever crazy notion you have in
| your head. You may understand why the code works, but
| explaining to other people can be tough, because your
| internal abstractions are probably not what other people
| intuitively think about when looking at the same problem.
|
| Not saying any of the above is good or bad. It is different.
| With java or go or whatever, I tend to think a lot about how
| do I represent this in java or go or whatever. With lisp it's
| the closest I've ever gotten tot my idea out there, naked.
| That can be powerful, or scary, or both.
|
| You might take a look at "Paradigms of AI Programming: Case
| Studies in Common Lisp". That shows a lot of tricky problems
| tackled in a pretty breezy way. Want to build a computer
| algebra system? Not that tough to get something working. Work
| through Norvig's approach and you can get something pretty
| good working pretty quick.
| dreamcompiler wrote:
| One major Lisp thing I've been musing about lately...
|
| C programs don't understand other C programs. Of course the C
| compiler itself is an exception but in general you don't
| expect to be able to feed your compiled C .exe _another C
| source file_ and do anything useful with it. That 's crazy
| talk. Likewise with Java or C++ or Fortran or...
|
| But Lisp programs understand Lisp programs. The compiler is
| always available even at runtime. And not just the compiler
| but the lexical analyzer, the parser, and the evaluator too.
| Output is also easy: You can write a Lisp program that builds
| Lisp data structures dynamically in memory and then prints
| them to a text file as Lisp source code. Which you can later
| read back in and execute as a program.
|
| This kind of thing is so natural in Lisp that it opens up
| whole new ways to think about code. So why is this useful?
|
| One example: In many programs you need configuration files.
| So you write configuration files for C programs in XML or
| yaml or JSON and then use a library function to parse them.
| Why don't we write configuration files for C programs in C
| syntax? Well, because C syntax is rather inconvenient for
| this purpose and because there aren't many C libraries that
| parse C syntax. Besides, how do you handle includes, compiled
| libraries, macros, linking, etc? Again, crazy talk.
|
| Not so in Lisp. Lisp syntax is _very_ convenient for
| configuration files (because it 's s-expressions) and the
| parser is built-in. So you write configuration files in Lisp
| and parse them at runtime! If you change a setting at
| runtime, it takes not much more than one line of code to
| automatically print a new configuration file. Which you can
| then tweak with a text editor if you so desire.
|
| Another example: Macros. Macros use this same principle but
| there's no external file involved. Macros change Lisp's
| syntax so you can write more advanced programming constructs
| and translate them to basic Lisp at compile time. It's the
| same idea as Typescript-- Typescript is a Javascript program
| that converts a type-safe version of Javascript into basic
| Javascript. But Typescript was a one-off major project--like
| building a compiler. In Lisp the toolkit for source-to-source
| translation is built-in and Lisp programmers use it every
| day.
| bilegeek wrote:
| Thank you for this. You get into the intermediate steps, which
| just about every other tutorial I've seen on this glosses over.
| abhinav22 wrote:
| You are most welcome :) Yes, this hopefully is a complete guide
| from start to finish and without any gaps (which are not easy
| for us to solve without having a reasonable amount of
| background knowledge!)
|
| Thanks for the kind words
| kreetx wrote:
| Seems to be paywalled?
| abhinav22 wrote:
| Argh! Fixed now - sorry about that. Pressed the wrong button
| when publishing on Medium (ps I really like Medium, sometimes
| it gets a bit of a rap, but it's very easy to write and they
| are quite user friendly in my opinion at least :)
| notagoodidea wrote:
| To be picky, Medium may be writer-friendly but never was
| really reader-friendly. The page are heavy with banners
| blocking the view, no possibilities to know in advance which
| article is paywalled or not. I do not have a problem with
| authors publishing on Medium, just Medium itself.
|
| Besides that, nice and interesting write up!
| abhinav22 wrote:
| Thanks!
|
| I agree with you, it's not perfect for readers and until I
| caved and signed up as a member, I also was annoyed at
| hitting my monthly limit of articles.
|
| They have amazing SEO so inevitably many articles I need
| while researching a solution led me to Medium.com
|
| It's just a very hard market to serve (long form reading /
| journalism etc). That's why I'm not that bullish on
| substack because it's hard to make it good for readers and
| profitable for writers / the startup.
|
| Medium paid me $10, and I have about 10,000 reads in the
| last year. I didn't do it for the money, but that's an
| indication for those who do want to write for a side income
| - it's a hard business.
|
| For writing, its very good - to the extent, I use it as one
| of my main word processors lol. Then just copy it to
| something else for formatting. Very distraction free and
| the font is nice to make you want to write more.
|
| It would be great if a new competitor did what Medium did,
| but made it better for readers. IMO the business model
| should be like Wordpress but for writers, together with an
| overall platform with high quality SEO. Make the money
| selling the software and a bit with memberships on the main
| site.
|
| I feel for Medium and it's writers - there's so little
| money in it, that to some extent they have to resort to
| click bait titles, bad UI for readers, and high amount of
| not amazing quality content.
|
| Perhaps a donation model is best.
| exdsq wrote:
| Does anyone here use Common Lisp for their day job? I'd like to
| try it out some time (and am currently working through Paradigms
| of AI Programming by Norvig) but it feels like the language is so
| _bendable_ that it 'd be a minefield for a large team.
| mikelevins wrote:
| I use it in my day job and have done so on and off for a little
| over thirty years. I've used it in team settings, as well as
| solo. It's a large force multipler solo and on small teams, but
| it can also be productive for larger teams.
|
| You want at least one experienced Lisper who can mentor the
| less experienced and build guard rails where needed. The good
| news is that it's unusually easy to build guard rails with Lisp
| because of that same "bendable" quality you mentioned.
|
| Another thing that is unusually easy with Lisp is building
| code-analysis tools. The source code is made of standard Lisp
| data structures that you can operate on with standard Lisp
| functions, and you can do it all interaactively in the live
| environment. That makes it much easier to build linters and
| other analysis tools than it is in most other languages.
|
| You need someone on the team who is experienced enough to know
| how to do these things, and team-oriented enough to want to do
| it.
|
| You also need to be aware that you probably don't know what
| guard rails and analysis tools you need when you're starting
| out. You'll discover them along the way. So you need project
| management with enough foresight to invest in prevention up
| front in order to save on cures down the road.
| abhinav22 wrote:
| Wow - that's awesome! We would all be really grateful for a
| blog of your experiences, if you have time one day. Which
| field do you use it in? I have a stereotype that it's mostly
| used in scientific / quantitative fields
| mikelevins wrote:
| I've used it for a long time, so forgive me if this reply
| gets long.
|
| I'm not the only or primary author of all of these; only a
| few of them.
|
| - Panels [Apple, circa 1988]: an experimental graphical UI
| framework with a built-in constraint solver and extensible
| event-handling system
|
| - GATE [Apple, circa 1989]: a knowledge-based, distributed,
| automated testing system used to discover and report
| thousands of application-compatibility bugs in Mac system
| software before release
|
| - bauhaus [Apple, circa 1991-1994]: an experimental Lisp OS
| for Newton. The Newton OS (except the microkernel) was
| written in Dylan back when it was basically Scheme+CLOS;
| the Dylan compiler and development tools were written in
| Common Lisp.
|
| - SK8 [Apple, circa 1995-1997]: rapid-application-
| development system sometimes referred to as "HyperCard on
| Steroids".
|
| - Reactivity Application Firewall [Reactivity, circa
| 1998-2004; later Cisco]: Reactivity's product was a complex
| application-in-a-box that provided rule-based application-
| level security and integrity for networks. Cisco bought our
| company and folded it into their product line. I used
| Common Lisp for several code-analysis and documentation
| tools used in development of the product.
|
| - Clozure Common Lisp [Clozure Associates, circa
| 2006-present]: I wrote documentation and code for Clozure
| Common Lisp and some associated tools
|
| - Secure Outcomes LS1100 [Secure Outcomes, circa 2010]: I
| wrote the bulk of the control system and GUI for the Secure
| Outcomes fingerprint livescanner product, using Lispworks.
|
| - Delectus [my own product, circa 2010-present]: Delectus
| 1, a simple personal data manager, was written mostly in
| Scheme. Delectus 2, a more capable, flexible, and portable
| version, is still under development and is written entirely
| in Common Lisp.
|
| - Confidential project [Confidential client; 2020-present]:
| a knowledge-based, reasoning-based neurosymbolic machine-
| control system. As I mentioned, it's confidential, but I
| can say that it's designed to monitor and orchestrate a
| variety of different kinds of robots with human supervision
| in real time.
|
| I've omitted several free libraries and other open-source
| Lisp software that I've written over the years.
| abhinav22 wrote:
| That's so cool! Simply amazing. And so much through lisp
| and its dialects. Thanks for sharing.
|
| Do you find Lispworks superior to SBCL / Emacs?
|
| Any tips for other programmers on what to do / what not
| to do when it comes to production level lisp programs?
| mikelevins wrote:
| I don't find SBCL or Lispworks superior overall. Each one
| is better than the other in specific, narrow ways. For
| example, you can't beat SBCL for availability of
| compatible open-source libraries. On the other hand, you
| can't beat Lispworks for a portable GUI framework.
|
| I use both.
|
| As for tips, there are probably a lot, but I have to take
| a break from HN right now to attend a work meeting. :-)
| abhinav22 wrote:
| Thanks for all your insights today :-) Have a nice rest
| of the day!
| kagevf wrote:
| Can LispWorks (easily) use quicklisp libraries?
| mikelevins wrote:
| Yes. Piece of cake.
|
| SBCL is better in the sense that almost everyone who
| writes an open-source library for Quicklisp distribution
| is going to test it with SBCL, so most of the time
| Quicklisp libraries just work with SBCL.
|
| Testing with other Lisps, including Lispworks, is not
| quite so automatic, so there's a little greater
| likelihood that something might fail to work.
|
| That being said, most libraries I've tried with Lispworks
| have worked without any trouble.
| blacktriangle wrote:
| Out of curiosity, how does hiring go at these teams? I'm
| really leaning towards CL for my next product but the thought
| of having to hunt down CL devs to hire is giving me pause.
| This is a boodstrapped product, so not able to afford a bunch
| of devs and need to scale up fast, just need to be able to
| find 2 or 3 CL devs.
| mikelevins wrote:
| I don't think I can generalize usefully about hiring. I can
| generally find Lispers to help me, but that's at least
| partly because I've been one for a long time and as a
| consequence I have a handful of contacts I can turn to who
| will either sign up or help me find someone.
|
| (If you want me to see if I can help, send me mail at
| mikel@evins.net and we can talk it over.)
|
| In most of the cases on my list of past projects, the
| people who organized the projects were Lispers themselves
| or had contacts in the Lisp community, and recruiting from
| that community was part of the early project setup.
|
| The bad news is that the community of Lispers is small
| compared to those of more popular languages. The good news
| is that Lispers are generally eager to work in Lisp, which
| can be helpful in recruiting. For example, I prioritize
| Lisp opportunities over others, and I'll work for less
| money if I like the people and I get to work in Lisp.
|
| I have been part of hiring and mentoring some programmers
| that were not experienced with Lisp. In all but one case,
| they were interested in learning it, and made quick
| progress.
|
| I've worked on a few teams of mixed Lispers and non-
| Lispers, ranging in size from about three to about sixty.
| I've found that you want one or more experienced Lispers
| who are willing and able to take on a mentorship role. With
| mentors, good programmers can pick up Lisp and its style
| and idioms pretty quickly (assuming they want to). Without
| mentors it can be rough going.
| fidesomnes wrote:
| I used to work at an AI firm that used it. Large Lisp codebases
| are a minefield with near impossible security and debugging
| issues. I still love it as a programming language tho.
| ska wrote:
| > Large Lisp codebases are a minefield with near impossible
| security and debugging issues.
|
| Sounds about like most large codebases. Languages can't save
| you from this, it seems, only discipline can.
| medo-bear wrote:
| There are a number of large companies that use CL quite
| effectively for safety critial purposes. Can you please give
| more details about your experience?
| exdsq wrote:
| Out of interest can you give some examples?
| medo-bear wrote:
| siscog in portugal is one example but there are others,
| historical and current. it is also used in quantum
| computing and financial markets. gramarly and google
| flights are perhaps the most famous current users of the
| software.
|
| edit: i adjusted my answer because i thought i was
| answering to parent
| abhinav22 wrote:
| I would love to hear more on the security challenges and
| debugging issues, if you have time. It's not used as much in
| industry, so any insights are really very valuable.
|
| Also so cool that you worked in AI! That's my dream field :)
| blacktriangle wrote:
| I saw a fun debugging issue the other day on r/lisp that
| almost made me want to walk away from lisp for good.
|
| You have a function (defun my-val () "a string")
|
| Referentially transparent, easy to reason about right?
| Wrong. Lisp strings are mutable, and everything is a
| reference. So if you do something like this
|
| (replace (my-val) "b" :start1 0 :end1 1)
|
| (my-val) => "b string"
|
| I appreciate that CL doesn't force everything to be
| immutable, but it seems like at least strings would benefit
| from being treated as immutable values.
| abhinav22 wrote:
| Interesting example. Yes, it's not a perfect language by
| any means :-) Some things are better in Lisp, some things
| are better in other languages as well for sure!
| kagevf wrote:
| I remember that ... it was pretty surprising!
|
| I thought I could do (string "a string") to prevent it,
| but that didn't work.
|
| (concatenate 'string "a string") worked, though ...
| dj_mc_merlin wrote:
| I skimmed over the article, but I think it misses my favourite
| part: running a Swank server for live debugging! Something along
| the lines of:
|
| (bt:make-thread (lambda () (swank:create-server :port port :dont-
| close t)) then connect with M-x slime-connect
|
| I've debugged many tricky to reproduce issues like this. Also
| it's just damn cool.
| abhinav22 wrote:
| Would you be able to expand on this? I'm very basic when it
| comes to debugging in Lisp :)
|
| So you are running a second server to test issues away from
| production? I thought this is possible in other languages /
| frameworks, so I must be missing something
|
| I read the cook book but still didn't get the main benefits of
| it. However the quote was very cool:
|
| That's how Ron Garret debugged the Deep Space 1 spacecraft from
| the earth in 1999:
|
| We were able to debug and fix a race condition that had not
| shown up during ground testing. (Debugging a program running on
| a $100M piece of hardware that is 100 million miles away is an
| interesting experience. Having a read-eval-print loop running
| on the spacecraft proved invaluable in finding and fixing the
| problem.
| dj_mc_merlin wrote:
| You have your normal application, accessible on some port
| serving HTTP requests (or whatever). Then you start a thread
| (with `bt`, bordeaux-threads, a very common multi-threading
| library) and run a Swank server listening on some other port
| on it. Swank is the backend used by SLIME, it allows
| manipulation of the Lisp environment. A remote server is
| still the same Lisp environment as when you run it on your
| machine, so you can use Emacs to connect to it (M-x slime-
| connect) and debug live.
|
| Swank doesn't allow remote connections though, otherwise
| anyone could connect and mess with your server. You must
| connect through the loopback interface. Easiest way is to set
| up an ssh tunnel, like: ssh abc@example.com -L
| 4200:localhost:4200
|
| Whether you can call the Swank server a second server or not
| is a somewhat philosophical question I guess. There's one
| application serving different kinds of requests on different
| ports.
| abhinav22 wrote:
| Does that mean the swank thread still gets all the inputs
| of the live process and you can debug in isolation there
| and then push changes to the main thread? Sorry if I'm
| stupid (no need to reply), I need to read into this more.
| jhgb wrote:
| You don't need to "push changes to the main thread", you
| can mutate the state of the whole process during
| debugging, and in Lisp that includes compiling and
| loading new code that gets used immediately by whatever
| application you're trying to debug.
| mikelevins wrote:
| It's probably worth mentioning that you usually don't
| want to make changes in a live production environment,
| and even when you do, you probably want to capture those
| changes in a file as well, so that you have a record of
| what you did. It's possible to make arbitrary changes to
| a live, running system in Lisp with no other record of
| what changes you made, but it's not usually a good idea
| to do that.
| jhgb wrote:
| > So you are running a second server to test issues away from
| production? I thought this is possible in other languages /
| frameworks, so I must be missing something
|
| The Swank connection runs on a different _thread_ , not on a
| different _server_.
| abhinav22 wrote:
| Ah that might be able to help bridge the gap for me. I
| still don't quite follow, but I guess having multiple
| threads allows for easier testing of the interface with
| other components, perhaps? Sorry I am out of my depth on
| this one. Will Google more tonight
| kagevf wrote:
| This repo has an example of running hunchentoot with a swank
| server: https://github.com/vseloved/flight-
| recorder/blob/master/hunc...
___________________________________________________________________
(page generated 2021-04-16 22:02 UTC)