[HN Gopher] Kina Knowledge, using Common Lisp extensively in the...
___________________________________________________________________
Kina Knowledge, using Common Lisp extensively in their document
processing stack
Author : p_l
Score : 82 points
Date : 2021-10-22 20:01 UTC (2 hours ago)
(HTM) web link (lisp-journey.gitlab.io)
(TXT) w3m dump (lisp-journey.gitlab.io)
| pphysch wrote:
| > Lisp allows us to scale dramatically and manage a large code
| base.
|
| Wow, really? How big is your company?
|
| > Right now, in our core company we have three people, two here
| in Virginia and one in Mexico City.
| sitzkrieg wrote:
| oh no no, scale only means hiring tons of programmers after
| winning funding rounds in these parts
| pphysch wrote:
| Apparently, scale means LOC of Lisps (or LOL, if you will).
| p_l wrote:
| Sounds like scaling developer power all right.
|
| Reminds me of quotes from Google SRE materials about scaling
| "SRE per managed server count".
| thorweur234 wrote:
| You know Google's flight scheduling software runs on Borg right
| ?
|
| After they acquired ITA, they have few openings in between, and
| are yet, apparently, one of the biggest consumers of quotas.
| the_optimist wrote:
| What's a quota?
| dataangel wrote:
| I think he means they one of the biggest consumers of
| cluster resources. Each group within google probably has a
| quota for how many resources they are allowed to use. It's
| unclear here if their consumption is so huge because they
| are successful or inefficient though...
| the_optimist wrote:
| Seem's like you're trying for sarcasm with the rhetorical
| question, but it's unclear. Scale what? Team size?
| [deleted]
| Abhinav2000 wrote:
| This part is really interesting:
|
| > It is fast - our spatial classifier takes only milliseconds to
| come to a conclusion about a page (there is additional time prior
| to this step due to the OpenCV processing - but not too much) and
| identify it and doesn't require expensive hardware. Most of our
| instances run on ARM-64, which at least at AWS, is 30% or so
| cheaper than x86-64. The s-expression structures align to
| document structures nicely and allow a nice representation that
| doesn't lose fidelity to the original layouts and hierarchies.
|
| The article also mentioned they have 3 programmers and 100k lines
| of code, that sounds impressive.
| neilv wrote:
| > _The article also mentioned they have 3 programmers_
|
| One of the problems we had with promoting commercial use of
| Scheme and then Racket was that -- although some companies were
| using it to great success -- there weren't any job postings for
| it.
|
| It was the norm for a single programmer to be doing the work
| that would normally be a team (sometimes multiple teams).
|
| And the knowledge of that success wouldn't be well-known.
| (Because they liked to focus on the work, or because the larger
| team of business etc. people they were in was also small, or,
| in at least one case, the business person thought "we use Lisp"
| would kill business deals even though the code wasn't customer-
| visible.)
|
| So there would be no success stories, no job postings
| mentioning it as something people should learn, etc.
|
| Which, I suppose was good for open communities self-limiting
| themselves to people who were genuine enthusiasts not motivated
| by money, and with no need to posture as influencers or do SEO,
| but... not so great for bringing in large developer base,
| getting lots of startups using it, etc.
| pphysch wrote:
| > It was the norm for a single programmer to be doing the
| work that would normally be a team (sometimes multiple
| teams).
|
| Be careful with what you conclude from this observation. Is
| it that A) Lispers are generally 10x Programmers, or that B)
| Lisps are generally poor languages for team environments?
| math-dev wrote:
| I can't imagine the choice of language being strongly
| correlated with programmer skill (it would be elitist to
| think so, and everybody feels best in the language they
| spend the most time in).
|
| I think C) Lisp is well suited to certain applications,
| from what I can see, web development or niche areas where
| exploratory programming is required. The downside of Lisp
| is lack of good GUI (CAPI is the best they can offer, but
| this is not as good as other language implementations for
| various reasons) and not having the backing of Apple,
| Microsoft, Google or Facebook, and thus lacking in APIs.
|
| But given its dynamic development (its a real joy), its
| very well suited to exploratory programming.
| csande17 wrote:
| > It was the norm for a single programmer to be doing the
| work that would normally be a team (sometimes multiple
| teams).
|
| One thing that surprised me when I entered the professional
| world was just how much this is seen as a _downside_. At
| almost every level, companies will choose technologies and
| techniques that allow them to hire more headcount, even if
| that results in a worse end product.
|
| Many startups value the appearance of having a large
| actively-hiring engineering team, and many BigCo middle
| managers want lots of direct reports. They don't usually care
| how much work gets done per employee, and sometimes they
| don't even care about the total amount of work that gets done
| across the organization. It's all about getting warm bodies
| into seats, either to impress investors or to gain status
| within the organization.
| hcarvalhoalves wrote:
| Hiring thousands of developers is a success metric now.
| Jtsummers wrote:
| Management is still one of the fastest ways to a high
| paying job. Managers get reward (implicitly or
| explicitly) for being in charge of more people, either
| with better pay in the same position or by appearing to
| be more competitive when going for promotions. This
| creates a perverse incentive where someone can make
| decisions which promote a higher headcount at the cost of
| actual capability or efficiency.
|
| It's often not even deliberate. No one sits down and
| says, "This language or tool kit requires a higher
| headcount, so I'll select it for my project". But they
| aren't motivated to find a more efficient approach that
| results in them being put in charge of a smaller team
| (because this would cost them and not reward them).
| shaunxcode wrote:
| Really interested to play with dlisp if/when it is open sourced!
| gjvc wrote:
| Yes! This sounds really promising.
| dang wrote:
| Have they said whether it's a complete CL implementation or
| just a subset? I have the impression that it gets
| asymptotically more difficult as you approach the finish line.
| p_l wrote:
| Looks like it's CL-inspired so kinda like subset.
| aidenn0 wrote:
| Assuming you aren't just talking about how any project
| becomes harder to fix bugs in once you've fixed the easy
| bugs, it only gets more difficult as your approach the finish
| line if you don't plan on making a CL implementation from the
| start.
|
| It's really easy to make something that kind-of looks like CL
| by use of clever smoke and mirrors, and many "Lisp in X"
| projects do so. See e.g. parenscript's LOOP implementation.
|
| Other notes:
|
| - Implementing CLOS is a massive undertaking, but there are
| two reasonably good portable implementations of it (Closette
| and PCL).
|
| - The numeric tower is sufficiently loosely specified that
| you can typically get away with using any existing bignum
| library.
|
| - LOOP and FORMAT are annoying to implement just because they
| are big DSLs. I'm also not aware of any portable
| implementations of them (though SICL might have one).
|
| - Most of the rest of the effort will be spent on
| implementing things that interact with the system.
| Streams/files/pathnames are all annoying
| p_l wrote:
| LOOP can be cheated a bit with, as it originated in
| separate semi-portable package AFAIK, just like for CLOS
| you can use PCL for heavy lifting. Specifically there was
| MIT LOOP package that wasn't exactly ANSI compliant but got
| you significant portion of the way there.
|
| Also, if you're willing, it's possible to just lift a bunch
| of code from SBCL/CMUCL due to permissive license.
|
| MIT LOOP https://www.cs.cmu.edu/afs/cs/project/ai-
| repository/ai/lang/...
|
| few other versions including symbolics update to MIT which
| is supposedly closer to ANSI:
| https://www.cs.cmu.edu/afs/cs/project/ai-
| repository/ai/lang/...
___________________________________________________________________
(page generated 2021-10-22 23:00 UTC)