[HN Gopher] From First Principles: Why Scala?
___________________________________________________________________
From First Principles: Why Scala?
Author : lihaoyi
Score : 227 points
Date : 2021-02-11 13:47 UTC (9 hours ago)
(HTM) web link (www.lihaoyi.com)
(TXT) w3m dump (www.lihaoyi.com)
| dbsmith83 wrote:
| Scala is nice for doing things with Spark compared to say, Java,
| but I really think that Kotlin can do just as well. The only
| advantage Scala has with Spark is the syntactic sugar. Once there
| is an API for Kotlin (beyond the preview version), I will be
| dropping Scala like a bad habit. Kotlin is also concise, has
| better IntelliJ support, less "implicit magic", and better Java
| interop.
|
| https://github.com/JetBrains/kotlin-spark-api
| bjarneh wrote:
| > has better IntelliJ support
|
| That's gonna be hard to beat for most languages I guess, the
| best IntelliJ support will most likely always go to Kotlin :-)
| brabel wrote:
| That's not actually true, IntelliJ support for Java has
| always been much better, and still is today! Just try auto-
| completion in Java VS Kotlin in your IDE... it's much faster
| in Java files, something I've been complaining about as I've
| been using more Kotlin than Java, recently. Check also how
| many refactorings are available in your Java files compared
| to the Kotlin ones.
|
| Jetbrains knows its main source of income is still, by far,
| Java, so they don't drop that ball.
| zmmmmm wrote:
| If only Kotlin would introduce a non-ridiculous way to declare
| map and list literals it would be a great data science
| language. As it is, it falls over at the first hurdle where you
| want to specify some data inline. One reason I've really stuck
| with Groovy for JVM-based data sciency stuff.
| ibains wrote:
| Prophecy.io - low code product for data engineering on Spark - is
| Scala based - we love it! We do compiler work - and packrat
| parser combinators are great.
|
| Basic things are supported well. We use Play. We have GraphQL in
| web services - use Sangria. Slick FRM for database access is very
| clean. We use Akka for live web socket connections to Spark for
| interactive execution.
|
| We extend Spark code from outside when needed - adding functions
| to classes in imported libraries.
|
| It's a learning curve for new engineers, but we hire the best and
| they like Scala after using it for a bit. SBT is confusing as
| hell, build times were painful but getting better. Debugging is
| sometimes hard (implicits)
|
| One thing I've learnt us that if you're handed a gun, you don't
| have to start shooting at your feet just because you can. We used
| C++ at NVIDIA sensibly a decade back - using OOP but still
| avoiding horrible parts of C++. We use Scala sensibly now.
|
| Can't imagine another language that would work for us. I've used
| C, C++, F# before (building compilers and databases)- don't know
| Kotlin. Only other language we use is Golang for our kubernetes
| operator (excluding UI)
| polote wrote:
| Damn, you submitted the link 5 times in 5 days, at what number
| did you plan to stop?
| https://news.ycombinator.com/submitted?id=lihaoyi
| sdfhbdf wrote:
| I guess 8.
|
| They did that with a YouTube link sometime before [1]. Stopped
| after 8 failed submissions.
|
| [1]: https://news.ycombinator.com/item?id=23810499
| [deleted]
| chrisjarvis wrote:
| Great article, you have convinced me to give Scala another try :D
| The first time around made me feel very, very stupid.
| adenozine wrote:
| If I've said it once I've said it a thousand times: the day Scala
| kills sbt officially will be the day thousands of devs consider a
| return to it. Until then, I couldn't dare.
| denimnerd42 wrote:
| we use it with Maven. no issues
| holtalanm wrote:
| curious. what exactly is wrong with sbt? I used it once of
| twice when I was learning scala and I liked it a LOT more than
| gradle/maven. To the point where I actually opted to use it
| with my Kotlin projects later.
| arcsin wrote:
| From the same author as OP:
| https://www.lihaoyi.com/post/SowhatswrongwithSBT.html
| Fiahil wrote:
| Being better than maven is not a high bar, you should try
| cargo/rust, to have a meaningful comparison of a good build
| system.
|
| The only problem is, once you've tried out Rust, it will be
| difficult for you to come back to Scala.
| ratherbefuddled wrote:
| You need a phd in sbt to do things like "copy the jars", or
| "have a second set of dependencies".
|
| The proponents of the approach that sbt takes have failed to
| realise that build tools have zero value whatsoever, every
| second spent on them is a second wasted.
|
| What you want is a good mix simplicity (yes copy paste is
| fine) and some basic examples to follow, and the ability to
| break out into script if you are doing something unusual.
| the_af wrote:
| To me, sbt is confusing as hell.
|
| I no longer program in Scala, but I used to for my previous
| job and I regularly came across build files I couldn't
| understand -- and when I asked the person who wrote them,
| _they_ couldn 't explain their build properly either.
|
| And this happened in enough cases that I wouldn't consider it
| an exception or a problem with a particular developer. SBT
| used to have two entirely separate syntaxes, and it was a
| mess translating from one to the other. I don't know if this
| is still the case (I know one of the two syntaxes got
| deprecated, can't remember which one now).
| vikramkr wrote:
| It looks like its a build tool? Are you obligated to use that
| with scala or something?
| the_af wrote:
| You're not obligated, but much like Maven is _the_ build tool
| for Java, sbt is _the_ build tool for Scala.
|
| You use alternatives at your own peril. This means you won't
| be able to understand other people's builds, for example.
|
| I know some Scala projects use Maven instead. Whether that's
| a good idea is debatable!
| lihaoyi wrote:
| > You use alternatives at your own peril. This means you
| won't be able to understand other people's builds, for
| example.
|
| The tradeoff is with SBT, you won't be able to understand
| your own builds!
|
| (This is kind of a joke, but not really...)
| the_af wrote:
| Sadly, you're completely right. It's not a joke: it's
| happened to me. I still have nightmares about sbt.
| runT1ME wrote:
| SBT in the last few years is nothing like SBT of five years
| ago. I agree it was a convoluted mess at one point, and now I
| much prefer it to Gradle.
| sk5t wrote:
| Scala compiles well enough under Gradle--got it in a roughly
| dozen-module multi-language project right now, and it's no
| trouble. I suppose Gradle might (or might not?) come up short
| if you want automatic deprecation/replacement updates applied
| to source.
| lihaoyi wrote:
| I use Scala with Mill and Bazel for my OSS and professional
| work respectively. I haven't used SBT in 3 years. The post-SBT
| world is wonderful!
| MrPowers wrote:
| Perhaps the better way to phrase this is "the day Lightbend
| deprecates SBT and tells the community to use Mill", but I
| agree with the sentiment.
|
| Scala needs a new "official stack", hopefully Mill +
| utest/Munit + scalafmt (with opinionated settings).
|
| Scala 3 + SBT + Scalatest + whatever formatting isn't going to
| grab folk's attention.
| adenozine wrote:
| Exactly! You are far more eloquent than I am, from phone on
| porcelain throne.
| user1241320 wrote:
| > Scala needs a new "official stack"
|
| Maybe so... but so do Python or Javascript. It seems to me
| some other languages have only one stack (rust: cargo,
| rustfmt...; the .net world as well). Does it make them
| better? (honest question). Then again, what's Java official
| stack?
| user1241320 wrote:
| I'd pick sbt over mvn no doubt. Haven't played with either fury
| or mill, so can't comment on those (even gradle... I never
| really used it). But coming from years of Java/Maven I think
| sbt is way better. Things start complicating a bit if you have
| lots of plugins or macros probably.
| bsg75 wrote:
| For new Scala devs, how much of a challenge is it to avoid sbt?
|
| Especially for those coming from languages like Python and Go,
| where the former has no build system (ignoring setup.py), or an
| almost universal one (go build, sometimes make).
|
| All of the projects I have touched are documented using sbt,
| and transitioning the build process for existing code projects
| while a new language learner is a double cognitive load.
| yw3410 wrote:
| It's easy; there's mill, maven and fury. Mill is quite good -
| but I've had to dive into the source code a few times to do
| things.
|
| If you're dealing with dependencies only, then it's a
| straightforward change. OTOH I've never had an issue with SBT
| since it's cleanup a couple of years back so YMMV.
| thinkharderdev wrote:
| I don't know. I have no real love for SBT but I have yet to
| find a build tool that I actually like and that covers all the
| use cases and has the ecosystem that I want. Maybe it is just a
| really hard problem and extremely resistant to creating a
| simple interface. Unless of course you consciously decide to
| punt on the hard cases (which if fine sometimes).
| AzzieElbab wrote:
| Scala is awesome. It is meant to be a scalable language where you
| go from level 1 to say 5 as you learn. Problems arise when you
| put a level 5 guy into a level 1 team and letting him loose. No
| common style guide, no team training, just write those type
| signatures approaching tweets max length and let the rest wonder
| wth is going on
| agbell wrote:
| Great Analysis! I agree that languages are becoming more like
| Scala.
|
| I'm not sure about the JIT part though. Aren't many of the recent
| language success stories now about AOT compiled languages like
| Go, Rust and Swift?
| dboreham wrote:
| Scala is AOT. The JIT part is just to speed up execution of the
| machine code its compiler targets, it not being the native ISA
| typically.
| darksaints wrote:
| There are very few optimizations that a JIT can improve over a
| static compilation of a strongly-typed language. Strong typing
| brings immense capability for static analysis and subsequent
| optimizations. The most successful JIT optimizations out there
| typically are ones that make dynamic languages about as fast as
| statically typed languages by making assumptions about the
| runtime types of objects, while trapping/reoptimizing when
| those assumptions don't hold true. It also does pretty well
| finding hot loops and selectively optimizing those, but PGO
| gives you that too without any performance hit in your final
| executable.
|
| A strongly typed language like Scala has the potential for
| massive optimizations under the whole program optimization
| umbrella...and no JIT would ever dare to try those types of
| optimizations, because they are extremely memory and compute
| intensive. They'd have to pause execution for minutes at a time
| just to figure out what to do.
|
| I think it is unfortunate that Scala was designed for the JVM
| first. It brought a lot of syntactic baggage for interop
| purposes, and it saddled it with weird things like reflection
| and dynamic class loading, making it hard to statically
| compile. The ideas behind scala's type system are extremely
| powerful and could have fundamentally changed programming for
| the better, but all we got was a JVM me-too that was quickly
| usurped by Kotlin. I don't think it is dead or a lost
| cause...but the legacy that the JVM has on Scala is one that
| slowed it down, not sped it up (like what was claimed would
| happen with JVM languages).
| agbell wrote:
| I agree that a AOT scala would be great. But don't JITs have
| the ability to react to real world usage and do hotspot
| optimization and stuff. I assume they are very complicated to
| build, and if you do thing ahead of time you should, but
| aren't the things that can only be done in a JIT important?
| mark_l_watson wrote:
| I am working on an AI book using Swift. Yesterday in a meeting,
| my manager was asking why Swift. I compared it to Scala, but
| built on LLVM, and a rich CoreML and other library
| infrastructure. REPL + closures, very fast build times.
|
| I haven't had much opportunity to use Scala, but I did take
| Martin's Functional Programming with Scala course years ago,
| and that was great.
| yw3410 wrote:
| Complete aside, but how is Swift these days? I used it
| literally a week after release and promptly stopped because
| the compiler kept segfaulting - I imagine it's more stable
| now?
| agbell wrote:
| Scala is a great language. I haven't done that coursera
| course, but many people swear by it.
|
| To me, Swift and Kotlin look like they have taken inspiration
| from Scala. I think Kotlin explicitly did. That idea transfer
| from one area to another is how we get better langauges.
| zepto wrote:
| Having programmed professionally in both Scala, and Swift,
| and enjoyed both, I can see basically nothing that Swift
| takes from Scala.
|
| The common ancestor is OCaml, and the ML languages in
| general.
|
| To be fair, that may change as Swift adopts Actor based
| concurrency.
| asimpletune wrote:
| I wish swift were like Scala. It attempts to look and feel
| like it, but there's one huge aspect of Scala that is missing
| in swift that basically makes swift a world apart, which is
| expressions vs statements.
|
| In Scala everything is an expression, meaning everything
| evaluates to some result even if that result is void.
|
| In swift everything is a statement. There's a big difference
| because statements don't return things, only functions do.
|
| It sounds minor, especially since swift has pattern matching
| and first class supper for functions etc, but the truth is
| these are all improvements over a more oop (in the Java
| sense) way of programming. They don't however make the
| languages very similar because at a more basic so level
| they're very different.
|
| I would even go as far as to say that rust is more like Scala
| than swift is, although it's been a while since I coded in
| rust.
| ebruchez wrote:
| > In swift everything is a statement. There's a big
| difference because statements don't return things, only
| functions do.
|
| That sounds absolutely awful.
| asimpletune wrote:
| It's such a small, non-flashy language feature, that
| honestly matters much more than almost everything else.
| ebruchez wrote:
| > Aren't many of the recent language success stories now about
| AOT compiled languages like Go, Rust and Swift?
|
| Scala also has Scala Native which, well, compiles to native.
| It's not yet as polished as Scala JVM and Scala.js, but the
| plan is to make it so.
|
| https://scala-native.readthedocs.io/en/v0.4.0/
| [deleted]
| tanin wrote:
| I'm a big fan of Scala, though I think the sweet spot for Scala
| is for a small team with pre-existing Scala's expertise. For a
| one-person project, I definitely use Scala. I used to use Ruby a
| lot.
|
| You can move fast with safety with the caveat that the code
| quality is harder to manage/control.
|
| Contrasting this with golang where they really take the guard
| rails to the other extreme (e.g. can't compile if a variable is
| unused, not even compiling just to test it out locally).
|
| Another weak-point of Scala is that the learning curve is steep,
| especially when it involves SBT, the defacto build system for
| Scala.
| davidfstr wrote:
| The author argues that dynamic languages like (JavaScript,
| Python) have Poor IDE and Tooling Support and are Unmanageable in
| large programs.
|
| I'll counter that if you add a type checker to one of these
| languages, such as TypeScript (for JavaScript) or mypy (for
| Python), then all of those problems go away.
|
| I'm personally particularly excited about the possibilities
| unlocked in the Python community by type checking, which I wrote
| about recently here:
| https://dafoster.net/articles/2021/01/26/python%27s-type-che...
|
| Additionally, RE "Poor performance" the author is presumably
| talking about CPU-bound performance. If you happen to be writing
| network-based or I/O bound programs, which you always are in
| JS/TypeScript and sometimes are in Python (for web applications),
| then the effective performance is more affected by the
| network/database/disk than the programming language.
| xvilka wrote:
| Problem with Scala that most projects use sbt but it's not even
| packaged in many Linux distributions. With the Haskell doing
| strong and Rust, both compile into the native out of the box
| there are no reasons to use Scala unless you need particular Java
| libraries.
| occupy_paul_st wrote:
| I definitely agree that many newer languages are strongly
| inspired by Scala, but I don't see a strong argument for why to
| choose Scala over one of a newer hybrid language. My own
| intuition is that these newer languages have had the benefit of
| being able to learn from Scala's mistakes. A lot of them have
| intentionally sacrificed some of the Scala's flexibility to be
| easier to learn and use.
|
| I found Scala to be an extremely enlightening language: it taught
| me a lot about functional programming. But, the complex type
| system and the sacrifices needed to ensure Java compatibility
| entail many quirks that I find frustrating. I think it's actually
| a similar dynamic to what you mention: "retro-fitting these
| features onto an existing language never fits quite as nicely as
| a language that was designed with them from scratch."
|
| BTW, I am a huge fan of your writing: the Principle of Least
| Power is a sacred programming text to me .
| pegasus wrote:
| Scala has learned from Scala's mistakes too. Scala 3 cleans up
| a lot of the mess that had accumulated over the years and is on
| the cusp of being rolled out this month.
| imtringued wrote:
| >the sacrifices needed to ensure Java compatibility entail many
| quirks that I find frustrating.
|
| That reminds me of that dead JVM language called Ceylon. It
| didn't compromise anything to ensure Java compatibility. The
| combination of OOP+FP was executed with zero friction.
| Meanwhile Scala did it so poorly it created a stupid myth that
| OOP and FP shouldn't mix.
|
| Of course everyone knows that nobody used Ceylon, not even
| Redhat who sponsored the language. The compiler was also dog
| slow. Typechecking complicated type unions/intersections can
| get really slow with bigger projects. The metamodel also
| created a lot of JS bloat if you were brave enough to use it in
| the browser. Lots of practical complaints that ultimately make
| you stop using it. But oh man the idea and design. It was very
| well thought out and enjoyable to program in.
| stickfigure wrote:
| Ceylon was (is?) really beautiful. At least type
| unions/intersections found their way into Typescript.
|
| IMO Gavin's biggest mistake was picking Eclipse for the IDE
| platform at a time when every professional was moving to
| IntelliJ.
| davnn wrote:
| > BTW, I am a huge fan of your writing: the Principle of Least
| Power is a sacred programming text to me .
|
| That principle is actually directly from the CTM book, which
| seems to have been a great inspiration for Martin Odersky in
| the language design.
| yawaramin wrote:
| > I don't see a strong argument for why to choose Scala over
| one of a newer hybrid language.
|
| Scala has been in production use and getting bugfixes for
| longer than some of the newer languages have existed. I feel
| like a re-read of
| https://www.joelonsoftware.com/2000/04/06/things-you-should-...
| is warranted.
| Cthulhu_ wrote:
| I feel like Scala started a trend in language design, where
| every language adds features from other languages. Not because
| there's a need, but because some people are used to language X.
|
| Example: adding OOP in PHP and Javascript. In both examples,
| especially the first attempts, were half-baked. Why bolt on
| half a language feature? Both languages would be much better
| served with dependency management and / or modules, which for
| both languages came out of the community at first.
|
| Counter-example would be Go, that resists change - especially
| if it's requested with a "this feature is in language X! I NEED
| it!". From the FAQ:
| https://golang.org/doc/faq#Why_doesnt_Go_have_feature_X
| (followed by some features other languages have. Generics /
| type arguments is one that will probably be added in an
| upcoming version of Go, but they only considered it once they
| fully understood the problem and need)
| mrkeen wrote:
| > Counter-example would be Go, that resists change
|
| It will happen, just slower. Just as half-baked.
| lmm wrote:
| > I definitely agree that many newer languages are strongly
| inspired by Scala, but I don't see a strong argument for why to
| choose Scala over one of a newer hybrid language. My own
| intuition is that these newer languages have had the benefit of
| being able to learn from Scala's mistakes. A lot of them have
| intentionally sacrificed some of the Scala's flexibility to be
| easier to learn and use.
|
| I don't believe any of those new alternatives has made better
| choices; some of them haven't had time to accumulate as many
| warts as Scala yet, but all of the ones I've seen have design
| decisions that make them inevitable. Higher-kinded types still
| work much better than every other way of achieving the same
| things that's been found - and if you want both higher-kinded
| types and decent tooling/library support, Scala is still pretty
| much the only option. (Haskell exists, but even if you consider
| its tooling good enough, implicit pervasive laziness has huge
| costs).
| playing_colours wrote:
| What newer hybrid languages besides Kotlin can cover the
| benefits you can get with Scala: a solid ecosystem (both for
| Scala and Java) for developing web applications, microservices,
| data processing, and good performance.
|
| Julia, Rust, and Elixir are all great, but popularity,
| ecosystem are not there yet.
| dunefox wrote:
| For Julia there are PyCall and RCall, so the ecosystem is
| less of an issue than with other languages. Also, its
| ecosystem has been growing quickly over the last 2 years or
| so.
|
| https://github.com/JuliaPy/PyCall.jl
|
| https://github.com/JuliaInterop/RCall.jl
| jolux wrote:
| F#
| lihaoyi wrote:
| I actually came to Scala via F#. You can even find exact
| dates of those transitions in my stackoverflow history.
|
| Scala and F# are sinilar enough to basically be the same
| language, though I found F# had a number of warts and
| idiosyncracies that made me move on to Scala. Scala didn't
| have all of these (though of course it had warts of its
| own!) and is what stuck with me for the long term
| smabie wrote:
| F# has a much more primitive type system than Scala and
| isn't particularly impressive imo. And it doesn't have
| functors ala OCaml.
| occupy_paul_st wrote:
| I actually haven't seen a language that can do everything
| that Scala can do, so in a lot of senses I agree with you!
|
| The alternative is to use multiple different languages to
| fill different niches. In theory this sounds suboptimal but I
| think it might actually be easier to learn multiple simpler
| languages than it is to learn Scala.
|
| BTW according to PYPL: Swift, Kotlin, TypeScript, and Rust
| are now more popular than Scala.
|
| https://pypl.github.io/PYPL.html
| dlandis wrote:
| > While most programming languages allow you to take an
| expression (1213, "moo") and infer the type (Int, String), Scala
| allows you to go the other direction: take the type (Int, String)
| and infer a value (1213, "moo")! This is tremendously useful in a
| whole range of different scenarios
|
| That's an interesting way to put it (and pretty neat), but I
| guess I'm not totally convinced it is that useful in practice. I
| saw the linked example about the parser using implicit objects
| but I think that could be rewritten in a number of different ways
| without relying on that feature.
| yw3410 wrote:
| DI in Java world is another example where people require this
| type of feature (unsafely) via reflection.
| valenterry wrote:
| It is for example very usefull to generate random test data
| (property based testing) in a very elegant way, without
| manually wiring up a lot of stuff.
| prionassembly wrote:
| I want to switch away from Python, at least for personal
| projects. I want to broaden my range, experiment with weird
| paradigms (Raku) or get code that's a little faster (eg Rust).
|
| The problem is that I use Python as a "glue language" and heavily
| rely on excellent third party libraries. I don't want to learn
| how to write a minimum spanning theorem algorithm; I'm used to
| have it next to several other convenient graph utilities in
| networkx. There's so much stuff for Python -- where else will you
| find something like giotto-tda?
|
| I'm not gloating about Python, I feel stuck.
| esarbe wrote:
| Scala is just an amazing language. I'm very happy that the worst
| of the 'Scala as a worse Haskell' days is over and that the Scala
| community is finding it's own programming style.
|
| I remember years ago when I had a heated discussion about local
| mutable state. An that time it felt like heresy to even suggest
| it. I just recently had another discussion with the same
| programmer and he was like: "Yeah, that works".
|
| Good times.
| dkarl wrote:
| Unfortunately, "Scala as a worse Haskell" is alive and well.
| The /r/Scala subreddit, for example, is dominated by folks who
| tell every Scala-curious beginner who drops by that the
| accepted right way to write Scala is the pure FP ecosystem and
| everything else died a long time ago.
|
| There are beautiful, simple, elegant ways to write Scala, but
| I'm afraid that the community isn't converging on a single
| concrete style, and a consensus around an abstract set of ideas
| about style isn't useful. Everybody agrees on words like
| "simple" and "elegant" and "readable," but I've seen some real
| abominations in Scala that people are tragically proud of. I'm
| afraid that Scala has attracted too many of the wrong kind of
| programmer, the kind that identifies their professional value
| with their ability to implement complex solutions that are
| intractable to their peers. The problem is not that you can't
| write simple code in Scala, but that people choose not to.
|
| I don't want to blame the pure FP style of Scala itself,
| because I've seen the same problems in the "better Java" style
| as well, and because I'm still thinking that I might be able to
| write good code in this style and might even come to prefer it.
| But oh man, the Scala programmers I've personally worked with
| who embrace pure FP have really screwed up priorities.
| uDontKnowMe wrote:
| Yep this is a huge cultural problem with the Scala community.
| On paper and in marketing material, Scala is flexible enough
| to empower you to slip into mutable/OOP style or more FP
| style. But in practice the pure FP enthusiasts are the ones
| who totally dominate the ecosystem (aside from Haoyi Li). You
| simply cannot just ignore the Haskell Larpers/Wannabe's and
| get on with it, because they infect all the community
| watering holes like Reddit, Gitter and Discourse. Every time
| one talks about using non-pure-FP features of the language,
| they have to go through the same tired patronizing lectures
| about how "yes I know this is not 'safe'" and "yes I know
| what I'm doing" and "I'm very sorry that I am not as smart as
| you smart FP people" before you can actually talk about what
| you want to talk about.
| randmeerkat wrote:
| I still find that Scala devs think they're writing Haskell.
| From what I've seen in the community, no one cares about
| shipping code, they care about whether or not it's "pure".
| thinkharderdev wrote:
| To give them their credit. The argument from the purists is
| that purity is itself about shipping code. As a long-time
| scala developer who has started using the pure-FP ecosystem
| (cats,cats-effect mainly) I am starting to see their point.
| It takes some time to grok but once you do then it is
| amazingly productive to work with.
| randmeerkat wrote:
| Maybe you're on an exceptional team, but I've never seen a
| Scala team that could deliver faster than a Python / Ruby /
| Java team.
|
| Scala teams also have a hard time hiring and if they hire a
| dev that's new to Scala they then have to deal with a very
| long ramp up time.
| thinkharderdev wrote:
| Maybe. I certainly don't have data on it but am also not
| sure what sort of data would actually prove that point.
| Teams that choose Scala may just be solving different
| sorts of problems. Or maybe they take longer and deliver
| a higher quality product. All I know if that I am
| personally way more productive in Scala than any other
| language I've used (and I have used most of them at one
| point or another).
| randmeerkat wrote:
| > Or maybe they take longer and deliver a higher quality
| product.
|
| Without going into how do you define a higher quality
| product, does quality really matter that much..? Most
| code is going to be re-written every few years anyway.
| Software engineers aren't building architectural wonders
| that will last hundreds or maybe thousands of years. So
| is the complexity of a language like Scala really worth
| it, just to deliver a product that will be obsolete in a
| few years anyway?
| esarbe wrote:
| Well, I've been working in financial and data services
| for many, many years now and I can tell you as much: many
| of the code bases I've been dealing with (and am dealing
| with) written in less strictly typed languages (much
| Python, lots and lots a lot Java) are riddled with hidden
| runtime errors that you'll just never see in Scala.
|
| If there's a bug, it's usually some modelling problem or
| some conceptual issues, not the implementation of the
| code. I've much more confidence in the code I've written
| in Scala than the code I've written in Python or Java and
| I never ever had to get up and pick up the phone in the
| middle of the night because some production issues with
| code written Scala.
| asimpletune wrote:
| Local mutable state as in a var in a function or how did you
| mean?
| chaorace wrote:
| Basically that, yes. The idea is, as long as the mutable
| state starts and ends within the confines of an individual
| function, you can get away with treating that function as if
| it were "truly" FP.
|
| The main argument against this is that it's a concession that
| makes the codebase less coherent. The more hardline argument
| being that the encapsulated mutable state is still more prone
| to bugs and should be almost always be avoided, except at
| great cost.
| asimpletune wrote:
| Yeah, this is totally fine.
|
| It's rare that you need to use vars in Scala, but still if
| it's scoped to the function then you know you can mentally
| discount it once the function terminates.
| lfmunoz4 wrote:
| What's the benefit of Scala over Kotlin? To me it seems Kotlin
| has all the advantages of Scala plus none of the downsides. Also
| with Android going to Kotlin the community is bigger and Kotlin
| Co-routines is probably the best multi-threading library ever
| created.
| aynyc wrote:
| Is Scala used outside of Spark and Akka? Spark is my company
| chosen analytic engine, Scala was/is the main language for
| majority of the production workload. In the last two years or so,
| more devs are writing PySpark and SparkSQL jobs than Scala.
| dominotw wrote:
| def mirrors my experience too. Vast majority of spark jobs were
| easily ported to sql/dbt and the remaining ones are in pyspark.
| I used to use a lot of scala spark in backend data processing
| in 2016 but now its almost down to zero.
|
| scala is real big impediment to making data processing
| accessible to general public in your company. order of
| preference now at my company is,
|
| 1. sql 2. pyspark 3. java spark 4. scala spark
|
| eg: shopify found that 70% of their pyspark could be converted
| to just sql https://shopify.engineering/build-production-grade-
| workflow-...
| MrPowers wrote:
| I've rolled out Scala based Spark interfaces to non-
| programmers in Databricks notebooks, so it's definitely
| possible, but only if you stick with the basic language
| features.
|
| Here's a more detailed PySpark vs Scala comparison in case
| folks are interested: https://mungingdata.com/apache-
| spark/python-pyspark-scala-wh...
|
| I think Scala Spark (using 10% of the language features) is a
| better technical decision (because it provides huge benefits
| like fat JARs, shading, better text editor support, etc), but
| the worse overall choice for most organizations because
| people are generally terrified of Scala.
|
| They'd rather do nothing than write Scala code. I can
| empathize with their position.
| aynyc wrote:
| > scala is real big impediment to making data processing
| accessible to general public in your company
|
| Ding Ding Ding! Presto/Athena now is becoming huge in BI
| ecosystem. We don't really use Spark for ad-hoc BI anymore,
| we use it for data science and large repetitive workload.
| [deleted]
| jatcwang wrote:
| I've spent the last 5 years working professionally in Scala and
| I've never touched spark and I know plenty others like me. (I'm
| mainly a backend developer)
|
| I think it's quite location-dependent. From my own
| surroundings, you can easily find backend Scala jobs in the
| bigger hubs in Europe, and seems like there are plenty in the
| US as well.
| UglyGenius wrote:
| There's a blooming ecosystem of pure functional programming
| libraries. Or even two ecosystems: Typelevel and ZIO.
|
| My company uses Scala a lot with with these ecosystems (my team
| is on Typelevel, another one is on ZIO) and in fact Spark and
| Akka is what we're trying to avoid as much as possible. Once I
| got used to the composable nature of FP and static types - both
| Spark and Akka just feel clumsy and unintuitive, like
| maintainers of Spark chose Scala because it was a "Java with
| fancy syntax" back then, ignored most of benefits of static
| types, HKT, immutability, type classes etc. If there's future
| behind Scala - it definitely doesn't lie in Apache land. Akka
| is slightly different and probably has higher potential, but
| just an overkill for most of use cases it's advertised for.
| the_only_law wrote:
| I want to like Scala and Akka, but they just seem to get used
| very little in the "real world". I recall some HN comments
| complaining about how a lot of the Scala they were writing for
| Spark was more or less a series of procedural API calls rather
| than using the language to its potential.
| sdfhbdf wrote:
| It's used in my workplace for ElasticSearch interactions. It's
| for search, product recommendations, discovery, sorting in an
| ecommerce.
| atbpaca wrote:
| I think that the Scala experience will get even better with Scala
| 3, tons of well thought refactorings for metaprogramming and
| typeclasses, among others. Also, looking forward to (optional)
| indentation based syntax.
| neysofu wrote:
| It's funny, because my experience is almost the opposite. My
| Scala skills are a bit rusty at this point and I promised
| myself to get back in the game once Scala 3 was released. Most
| of it I like. The new syntax, however, is really putting me
| off. My #1 problem with Scala has always been its huge upfront
| complexity and the quality of libraries, which are often over-
| engineered and not nearly enough UX-centric (cough implicits
| everywhere). Scala 3 was supposed to be a breath of fresh air,
| but I don't want to invest more time in a language that instead
| of getting simpler further complicates something as basic as
| the syntax. This change tells me that I strongly disagree with
| the core leadership when it comes to the future of Scala and
| what kind of language it should be.
| user1241320 wrote:
| Opposite for me. After years of Scala (which I like) I
| recently tried to port a few things to Scala 3 and it really
| feels like some of the wtf-implicit part is gone. We'll still
| need to make a complete idea about this version 3 though.
| bjarneh wrote:
| Scala is one of those great languages with that one Achilles
| heel. The speed of that compiler: $ time scalac
| Hello.scala real 0m2,907s user 0m7,901s
| sys 0m0,308s $ cat Hello.scala object
| Hello{ def main(args: Array[String]){
| println("hello world") } }
|
| Implicit type conversion can be debated I guess, but otherwise a
| very beautiful language.
| _a1_ wrote:
| Using scalac is not a standard use case.
|
| Standard use case is to use sbt (well, or mill, since we're in
| lihaoyi's thread :D). Sbt starts up slow, but with bloop, or
| sbtn (native client of sbt), the compilation is really fast
| (much faster than C++ for example): $ time
| sbtc compile
| [info] entering *experimental* thin client - BEEP WHIRR
| [info] terminate the server with `shutdown` > compile
| [info] compiling 1 Scala source to
| /home/.../target/scala-2.13/classes ... [info] compile
| completed [success] Total time: 0 s, completed Feb 11,
| 2021 4:21:51 PM sbtc compile 0,08s user 0,02s system
| 21% cpu 0,492 total
| thinkharderdev wrote:
| Yeah, set 1.4 was a huge improvement. I wrote a lot of scala
| several years ago and just got back into it this year and the
| dev experience is much nicer now with the SBT sever model
| lihaoyi wrote:
| Yeah cold compiles are slow, but most modern tools keep a hot
| compiler to re-use and once warm a hot compile of hello world
| is more like ~200ms. There's a bit of complexity managing these
| daemons, but that's something that the build tool normaly
| handles for you transparently
| bjarneh wrote:
| Yes it gets better, and the 'fsc' (Fast Scala Compiler) does
| improve on the situation somewhat; but I've always found it
| annoyingly slow. It should be mentioned that I haven't looked
| very hard at that language in many years for that very
| reason; so things could have improved somewhat. And from what
| I understand, the next version of Scala (Dotty) is going
| address this issue (build speed).
| lihaoyi wrote:
| The compiler has sped up 3x in the past few years. That's a
| very big speedup, and the current experience is likely
| nothing like what you remember
| rubyn00bie wrote:
| I loved Scala until a minor version updated somewhere around
| 2.9-2.11 destroyed all my hope. Eight years later I still refused
| to use it to build anything. Yet, with that said, IMHO, it can be
| fucking great _second_ language.
|
| If you haven't used a statically typed language before, i.e.
| you're from Ruby, JS, Python, et. al then it's one bangin' ass
| brain rodeo. I'd highly recommend spending a couple months trying
| to learn it. Once you're done, you will almost certainly spend
| near zero effort learning any other typesystem save the MLs
| (which would, in some ways, probably be a better rodeo).
|
| One of my favorite "ah-ha! that is totally how it fucking works"
| moments with Scala was going to see how `Function` worked only to
| find this bad boi: https://www.scala-
| lang.org/api/current/scala/Function22.html
|
| A lot of time people think there's more magic going on than there
| is and just seeing that's how a function works with 22 arguments
| was pretty illuminating for me coming from Ruby.
| sireat wrote:
| I actually like Scala.
|
| I teach it to complete beginners to programming and generally
| it goes great - there is a huge demand for junior Scala
| developers in my neck of woods.
|
| However when I found that 22 argument monster a while ago it
| was a bit of a wtf moment to me.
|
| Why 22? Why not 21 or 23 ? Is it one of those no one is going
| to need more than 22 arguments?
| fmakunbound wrote:
| > only to find this bad boi: https://www.scala-
| lang.org/api/current/scala/Function22.html
|
| Dang. I just checked in the Common Lisp I use and it's a bit
| higher. What's the reason for having to have at least 22
| concrete function signatures like that? CL-
| USER> call-arguments-limit 4611686018427387903 (62
| bits, #x3FFFFFFFFFFFFFFF)
| rbonvall wrote:
| Originally there was no way of abstracting over an arbitrary
| number of type parameters in the language, so a different
| trait was required for each arity. The choice of 22 was
| arbitrary, I think.
|
| This is no longer true in Scala 3, so there will be no such
| limit: http://dotty.epfl.ch/docs/reference/dropped-
| features/limit22...
| MrPowers wrote:
| I am a Scala programmer & think it's a great language. Here are
| some arguments for why not Scala:
|
| * Li's libs (os-lib, upickle, utest) have clean public
| interfaces, but most Scala ecosystem libs are hard to use, see
| the JSON alternatives for examples:
| https://www.lihaoyi.com/post/uJsonfastflexibleandintuitiveJS...
|
| * The Mill build tool looks a lot better than SBT, but seems like
| everyone is still using SBT
|
| * Scala minor version are binary incompatible, so maintaining
| Scala projects is a big pain. Upgrading Spark from Scala 2.11 to
| Scala 2.12 was a massive undertaking for example.
|
| * Scala has tons of language features and lets people do crazy
| things in the code. Hard to win technical arguments with Scala
| geniuses that like using complicated language features.
|
| * Scalatest is stil used by most projects and is annoying to use,
| as described here: https://github.com/lihaoyi/utest#why-utest.
| The overuse of DSLs in Scala is really annoying. Too many DSLs is
| another example of something I consider to be an antipattern, but
| there is no Scala community consensus on the responsible use of
| DSLs.
|
| I'm optimistic about Scala. There are some folks that love the
| language and are continuously improving the ecosystem. Scala 3
| will have to sell a better story about ditching legacy tooling
| and giving users a better default stack if it wants to compete
| with modern Go/Rust/Python.
| dominotw wrote:
| > Hard to win technical arguments with Scala geniuses that like
| using complicated language features.
|
| I was on a team building good old crud apps using monads,
| monoids, categories, combinators, effects cats, seamless and
| bunch of other nonsense that i've now purged from my brain.
|
| You could've easily mistaken our team for a programming
| language research group at a university.
|
| This is literally the number one reason i would never choose
| scala. There is no upper bound to amount of stupidity one can
| indulge in. Our code was so convoluted that even intellij had
| trouble understanding what the hell was going on and would spit
| out compilation errors when there were none.
|
| I now work with clojure team and i feel a sense of relief and
| weight taken off of my head from all the useless stuff i had to
| learn and use.
| runT1ME wrote:
| >I was on a team building good old crud apps using monads,
| monoids, categories
|
| Here's the thing, if you look at a basic Spring crud app, you
| are also using Monads, Monoids, Categories, Traverse, etc.
| but you aren't expressing it in the type system. Seriously go
| look at modern Spring's flux stuff, it's all there minus the
| type classes.
|
| I've seen teams that tried to over engineer Spring, teams
| that tried to over engineer Node applications, and yes there
| are teams that over engineer Functional style Scala.
|
| All the teams I worked with (and managed) at Verizon leaned
| pretty heavily into FP style Scala without much over
| engineering and the experience was extremely pleasant. The
| only production issues in my three years there I remember
| were performance related, finding out how to get more
| throughput or lower latency out of FP Scala. I literally
| can't remember any 'bugs' that made it to production.
| dominotw wrote:
| > Seriously go look at modern Spring's flux stuff, it's all
| there minus the type classes.
|
| I don't use spring or plan to use it. Not quite sure why it
| needs to be looked up. What are you trying to say?
| runT1ME wrote:
| I'm postulating that any modern CRUD app framework has
| Monads, Monoids, and other categorically inspired
| structure, even if they don't call it that or have a way
| to abstract over it.
| yw3410 wrote:
| Verizon's Scala projects are very cool - I wish that they
| got more limelight and documentation/support.
| wtetzner wrote:
| Yeah, I'd say that Spring is the Java equivalent of the
| same problem (over-engineering), just using reflection and
| runtime bytecode generation instead.
|
| I agree that using Monads, Monoids, etc. isn't necessarily
| indicative of over engineering in itself. If used well they
| can make the code clearer/simpler.
| MrPowers wrote:
| In the Spark world, you can use a tiny subset of the Scala
| features and enjoy huge productivity gains over the other
| language APIs (Java & Python). Those productivity gains are
| wiped out as more crazy language features get used.
|
| I don't think the super complex language features should be
| removed. Li's libs do some crazy stuff under the hood, but
| provide a clean, Python-like public interface. Most devs
| aren't that good and complex underlying implementations leak
| and yield complex public interfaces.
|
| Lots of folks would love Scala codebases that only use 10% of
| the available language features and none of the complex
| frameworks. But, like you mentioned, it's a hard language to
| use responsibly.
| brabel wrote:
| > Lots of folks would love Scala codebases that only use
| 10% of the available language features and none of the
| complex frameworks.
|
| They already do it: it's called Kotlin.
| [deleted]
| noisy_boy wrote:
| I think the trick to using Scala is to read the high
| level/key features of the language (case classes/pattern
| matching/using Option/Some/None, type aliases etc) and
| basically stick to them. A good yardstick for what are
| those features is the Scala Book[0]. This is what I read
| before I started to actually code in Scala.
|
| That coupled with side-effect free functional style (as
| much possible without overstretching) uses the power of
| Scala and keeps the code concise and readable. That is how
| I use it and find it very pleasurable to write in Scala
| compared to Java.
|
| [0]: https://docs.scala-lang.org/overviews/scala-
| book/introductio...
| bcrosby95 wrote:
| > Lots of folks would love Scala codebases that only use
| 10% of the available language features and none of the
| complex frameworks. But, like you mentioned, it's a hard
| language to use responsibly.
|
| Does this have the same problem in large languages such as
| C++ though? Where everyone thinks there's an optimal subset
| of the language, but no one agrees on what that optimal
| subset is?
| MrPowers wrote:
| Yep, there is even a talk on the Scala Infinity War which
| argues for one subset version of Scala that it needs in
| order to survive: https://www.youtube.com/watch?v=v8IQ-X2
| HkGE&ab_channel=Scala...
|
| For me, Scala is a multi-paradigm language with tons of
| features. The community just needs different types of
| style guides for different apps. vars are horrifying for
| the functional crowd, but cool with me for example.
| [deleted]
| brainzap wrote:
| Clojure still sounds like research group to me.
| [deleted]
| [deleted]
| Kototama wrote:
| Tell that to Amazon, Apple, Netflix, Cognitect and all
| enterprises that produce reliable and performant software
| with it everyday.
| randmeerkat wrote:
| I don't know of anyone at those companies using clojure
| and I haven't seen any tech blogposts by those companies
| about using clojure. If clojure exists at those companies
| I imagine it's a small very niche team.
| yawaramin wrote:
| And Walmart:
| https://news.ycombinator.com/item?id=19728502
| joshlemer wrote:
| One very public example of a Clojure shop through and
| through is NuBank out of Brazil, who employs some 700
| "Clojure developers" according to them. In fact they are
| so committed to Clojure they bought Cognitect last year.
| Kototama wrote:
| Don't bother, I have the feeling randmeerkat will argue
| that if the team is so big, the language must be not
| expressive!
| jjav wrote:
| > You could've easily mistaken our team for a programming
| language research group at a university.
|
| I had the misfortune of working with basically this same
| team, on scala (of course).
|
| I argued for maintainable simple technology but scala was too
| hip to pass, for many.
|
| Still remember one meeting trying to decipher a bug and while
| reviewing the code in question the lead scala fan said "It is
| not reasonable to expect to understand what code does by
| looking at it. I'll need to go research this for a few days."
|
| I hope to never see scala (or its ilk) ever again. Give me
| the most stable language and environment, where every
| question is a FAQ and every odd behavior is documented in
| every book and I never need to fall into the rabbit hole of
| language traps. Then I can just work on building the product,
| which is the whole point.
| meowzero wrote:
| I love Scala. But arguing with intellectuals who _HAVE_ to
| use these feature just because they can is a struggle I hate
| dealing with.
| dionian wrote:
| Just because you don't like it doesn't mean you should be a
| good fit for their team.
| phendrenad2 wrote:
| > I was on a team building good old crud apps using monads,
| monoids, categories, combinators, effects cats, seamless and
| bunch of other nonsense that i've now purged from my brain.
|
| Ah I had the same experience.
|
| That this is allowed to happen just shows that putting naive
| non-technical managers in charge of coders is a disaster.
| Cthulhu_ wrote:
| This is my biggest gripe with Scala as well. I'm sure it's a
| great language, but I've often seen it used to add complexity
| where none is necessary. It's abused to give developers a
| sense of accomplishment and intellectual superiority.
|
| In fact, in one project my former employer was involved in,
| one main reason they picked Scala was to, on the one hand,
| weed out the chaff from their existing team of .net
| developers (in a "shape up or ship out" kind of fashion), and
| on the other to weed out the 95% of mediocre Java developers.
| dboreham wrote:
| > sense of accomplishment and intellectual superiority
|
| This, I feel, is underneath a very large proportion of
| everything in our industry.
| dmux wrote:
| >...I've often seen it used to add complexity where none is
| necessary.
|
| This reminded me of a great blurb about Niklaus Wirth's
| approach to languages:
|
| >Wirth's philosophy of programming languages is that a
| complex language is not required to solve a complex
| problem. On the contrary, he believes that l languages
| containing complex features whose purposes are to solve
| complex problems actually hinder the problem solving
| effort. The idea is that it is difficult enough for a
| programmer to solve a complex problem without having to
| also cope with the complexity of the language. Having a
| simple programming language as a tool reduces the total
| complexity of the problem to be solved. [0]
|
| [0] Pg.2 http://www.cslab.pepperdine.edu/warford/ComputingF
| undamental...
| augustk wrote:
| Niklaus Wirth designed and implemented the languages
| Pascal, Modula-2 and Oberon of which the last one is the
| smallest.
|
| https://miasap.se/obnc/oberon-report.html
| abhgh wrote:
| What makes this quote really interesting is that Wirth
| was Martin Odersky's (guy who designed Scala) PhD
| advisor!
| dominotw wrote:
| > weed out the chaff from their existing team of .net
| developers
|
| This is scala equivalent of HN/pg lisp folklore.
| dkarl wrote:
| I'm on exactly that kind of team now. They've been working on
| a very simple problem for several years now and have an
| enormous, sophisticated, but still buggy and unstable
| solution.
|
| They're really smart people, and they had to be extremely
| capable programmers to get this far, but the embarrassing
| question is, how would a team of mediocre developers have
| tackled this problem? They would have picked a mediocre
| language and probably written a naive, straightforward
| solution, and after a few months of squashing bugs and
| developing production workarounds, it would have been a
| workable, tolerable system. They would have spent the last
| year working on something else.
|
| The naive solution would have more inherent unreliability,
| but our vast and intricate solution, designed to scale to the
| moon and be more self-healing than the T2000, is never going
| to approach the same reliability as a naive solution, because
| it's too complex (and the code is too hard for mortals to
| read and reason about) for us ever to iron out the bugs.
|
| They make a huge deal out of how much safer and easier to
| reason about our code is thanks to the FP discipline, but why
| do we have just as many concurrency-related production issues
| as a typical team using blocking operations and threadpools?
| Why do we have _more_ incorrectness caused by swallowed
| errors than I came to expect in projects that relied on
| exceptions?
|
| I'm still keen on mastering this style of Scala, because I
| think the benefits can be had, but it annoys me that some of
| my teammates are happy to use those stated benefits to
| justify their programming adventures without seeming to care
| if they actually materialize.
| tikhonj wrote:
| > _how would a team of mediocre developers have tackled
| this problem?_
|
| In my experience, mediocre developers would stick to a
| simple language like Python and only use simple Python
| features... and write a completely impenetrable, 100%
| coupled and wholly unreadable mess that would make you
| _wish_ for over-engineered Scala. Don 't even get me
| started on what happens with "simple" distributed computing
| tools like Hive...
|
| I see people complain about languages like Scala leading to
| over-engineering and I just don't get it: have you _not_
| worked with sub-par Python or JavaScript code? Sure, poorly
| designed complex solutions have problems, but so do under-
| abstracted codebases! Bad code is bad code, and I don 't
| think it's meaningful to say that "over-engineering" is
| categorically worse than "under-engineering".
|
| I actually worked with somebody who wrote pathologically
| over-complicated Haskell. Had some friction between us over
| that. His Python code? Somehow even _worse_.
|
| No serious language I've seen at either end of the
| abstraction-friendly spectrum (not Haskell/Scala nor
| Go/Java/Python/etc) puts a meaningful floor on code
| quality. An inexpressive language can have a low ceiling
| for how _good_ code is, but restrict or simplify the
| language however you like and people will still happily
| write utterly unworkable code.
|
| I've found that languages--especially less common languages
| thanks to the "nobody gets fired for IBM" effect--get used
| as scapegoats for not talking about cultural issues. If
| people on the team are writing clearly poor, over-
| complicated and bug-riddled code, it means there is some
| sort of technical leadership shortcoming, and it would not
| be any better if they were using Java instead. (I mean,
| have you _seen_ Enterprise(tm) Java(tm) codebases?) And, at
| the same time, it 's clear that a Scala codebase with
| tasteful technical direction and leadership--conveyed
| through team culture, code reviews, shared expectations...
| etc--can be absolutely great to work in.
|
| At this point, I'm leery of blaming languages for
| programming problems without a clear mechanism. "The
| language lets people do bad things" isn't a real mechanism
| because _all_ languages do bad things. "The language
| attracts people who like complexity" seems specious too.
| (Again, let's not blame languages for hiring and cultural
| problems!) "The languages defaults incentivize poor code"
| is a better argument, although it can be a bit fuzzy. And
| arguments like "language A doesn't have capability B that
| we need" or "language C allows classes of bugs other
| languages don't which empirically cause problems" are the
| most compelling of all.
| yw3410 wrote:
| I am continually perplexed at the swallowing of exceptions
| in certain FP communities.
|
| Result types, Either et cetera make it extremely easy to
| swallow errors by flatMapping thoughtlessly losing the
| context of where they were - you typically /want/ the call
| stack when you hit into an exceptional flow.
| Kototama wrote:
| It makes the code impure (that is with side-effects). One
| can also argue that if you can encode the potential
| errors in the type they are not exceptional.
|
| It's not that bad though, because you get much less of
| these kind of errors than in a typical Java program (for
| example).
| yw3410 wrote:
| Yes, it is a side-effect - and does make the code impure;
| but it's an /exceptional/ flow like OOM, infinite
| looping, stack overflows which also break equational
| reasoning.
|
| My argument is that as a programmer you /choose/ the base
| lemmas which you're comfortable with - with an exception
| you're saying for a large swathe of your code, it will
| assume a lemma. When is /does/ break you get to point the
| finger at it with a stack trace saying here is where the
| lemma was broken. As an aside there are nicer ways to do
| this ala contracts, but exceptions aren't a /bad/ way.
|
| The equivalent of try.toOption is try catch everything
| and discarding. The equivalent of flatMapping without
| adding surrounding context is equivalent to try catch
| rethrowing. Both of these are /much/ too common in FP
| codebases.
| joshlemer wrote:
| This is spot on! Way too often in Scala codebases, which
| tend to wrap IO calls in Future or similar is that you'll
| have chains of side effecting futures like
| readFromDb() .flatMap(x => doSomethingElse(x))
| .flatMap(x => doSomethingElseAgain(x))
| .flatMap(x => onceAgain(x))
|
| And this all gets passed up the call stack to one generic
| thing that basically does nothing in the case of error,
| maybe logging it and that's it. It's tempting to write
| code like this because it's so easy and looks so clean
| and readable, but in reality it is not how you build
| reliable software because what one must do in response to
| a failure from the first call is not the same as what one
| must do for a failure of the second call, or third or
| fourth, but this flatMap().flatMap().flatMap() style
| seduces the programmer into thinking they've handled
| errors when really they've just lumped the entire
| workflow as one big chain that can fail at any point and
| in practice usually just ignore all error cases.
|
| One strategy I find does slightly mitigate this is to use
| future.transformWith[S](f: Try[T] => Future[S]). This at
| least presents to you the opportunity to think about
| error handling a bit more consciously, but I still find
| that the chaining operator approach nudges you away from
| thinking about error handling because it becomes quickly
| difficult to read.
|
| It's probably to do with how limited the control
| structures are when dealing in Futures. You don't have
| while/if-else/etc, you must lift all your control into
| flatMaps and iterations are only doable via recursive
| calls with explicit accumulator state (yuck!) and
| nesting. So whereas a properly thoughtful treatment of
| error cases in synchronous code may take 20 or 30 lines
| of legible code, this gets tranformed in the async style
| to 20 or 30 levels of nested callbacks and recursive
| calls which becomes unreadable.
| dkarl wrote:
| Amen. I saw exactly the same problem with over-reliance
| on flatMap, either explicitly or in for expressions, in
| future-heavy Akka code that I do in cats and cats-effect
| code.
|
| People love the unifying abstractions underneath these
| types, and they love developing instincts about how to
| write code based on them, but from an application
| programming point of view, their instincts are often
| counterproductive. I don't think people take a
| mathematical enough viewpoint. The abstractions can't
| tell you what is important and unimportant. They can't
| tell you how your code should be shaped. They can't tell
| you which values should be transformed further and which
| should be short-circuited. They can't tell you which
| values deserve to be given a name for readability and
| which values should be anonymous.
|
| "All these values are monads, so I can combine them with
| a for expression" is a meaningless statement of a trivial
| mathematical fact, not a clue about how you should write
| your code.
|
| I think error handling code can be written in a
| straightforward style, but it isn't as pretty as people
| would like. I think the trap they fall into is holding
| onto elegance while they reach for correctness, instead
| of holding fast to correctness while they reach for
| elegance.
| mhitza wrote:
| Doesn't Scala have transformer libraries like ExceptT in
| Haskell. Aside, much of what this comment thread is
| about, is similar in Haskell-land as well.
|
| Anyway, a pattern I've started to use in Haskell with
| ExceptT is along the lines of handledIo
| = runExceptT $ do failable <- someIo
| can_handle <- anotherIo failable `catchError`
| handlerOfSubsetOfExceptionsOrRethrows return $
| updateState can_handle
|
| So the code still remains legible (minus the plethora of
| Haskell operators, data wrappers and unwrappers :)), I
| can trap individual exceptions along the way, which I can
| handle. And at the end unhandled exceptions are returned
| by the function as Left.
| dkarl wrote:
| As I am slowly acquiring the habits of thought that make
| it possible to read and write FP code at a reasonable
| speed, I'm horrified by how much idiomatic Scala FP code
| relies on projecting certain assumptions onto types and
| operations that seem, at first glance, to be neutral
| mathematical abstractions.
|
| For example, I find myself hating the Either type,
| because I feel like there is a socially established
| convention that one half of the Either is the type that
| matters, the value that you want, the value that is the
| point of the computation you're doing, and the other half
| is a garbage value that should never be directly handled.
| So I really feel like I should conform to the convention
| and reserve Either for cases where one of the possible
| types doesn't matter. But how often is it true that one
| side of the Either doesn't matter? People want me to
| encode success/failure in an Either type, but if I do
| that, are they going to treat failure with the care it
| deserves?
|
| I often handle Either (and Option) using pattern matching
| when I feel it's important to give both code paths equal
| importance and equal visibility in the code, but people
| change it because flatMap is supposedly more idiomatic,
| and they believe that eliminating pattern matching from
| their code is a sign of sophistication.
|
| I feel like this stems from a strong desire among FP
| folks for the happy path to be the only one visible in
| the code, and the non-happy path to work by invisible
| magic. Maybe there are some brilliant programmers who
| achieve this by careful programming, but there are people
| mimicking them who seem to rely more on faith than
| logical analysis. They just flatMap their way through
| everything and trust that this results in correct
| behavior for the "less important" cases.
|
| I'm sorry that this turned into a bit of a rant, but I'm
| entirely fed up with it, and it accounts for a lot of
| what I dislike about the code I work with on a daily
| basis.
| rovolo wrote:
| > I often handle Either (and Option) using pattern
| matching when I feel it's important to give both code
| paths equal importance and equal visibility in the code,
| but people change it because flatMap is supposedly more
| idiomatic, and they believe that eliminating pattern
| matching from their code is a sign of sophistication.
|
| Isn't this the same as letting exceptions bubble up in a
| non-FP language?
| yw3410 wrote:
| Yes - but you /also/ lose the stack trace and it's /far/
| too easy to do; hence my OP.
|
| It is /infuriating/ to be knee-deep trying to work out
| which of your 30 Eithers failed with a non-descript
| error.
| desbo wrote:
| You don't necessarily lose the stack trace. Typically the
| left side of an either is an Exception (or an error ADT
| that wraps one). When you want to handle the left case,
| you can log out the full trace as you would without
| Either.
|
| The Monad instance for Either means that chaining them
| together with flatMap has a short-circuiting effect and
| the first failure will stop the rest of the chain from
| being evaluated. I find this actually makes it easier to
| know where your errors are happening, and also allows you
| to centralise your error handling logic.
| kaba0 wrote:
| Hmm, when I first learnt Scala, I haven't had too
| advanced FP knowledge, so I am yet to have first-hand
| experience with this sort of exception-handling and I'm
| yet to decide how good it is.
|
| Compared to Haskell, it is probably better in some way
| because you have the proper stacktrace; but it "feels"
| impure a bit.. In a way Java's exceptions are already an
| Either type with the result type and the thrown Exception
| (with "auto-decomposition", unless checked exceptions)
| --- is the advantages like manual management of when
| mapping/flatmapping happens worth it in your opinion?
| Nonetheless thanks for the heads up, I might try out
| Scala again with the exception handling model you
| mentioned!
| dkarl wrote:
| It's supposed to work like that, but it's a lot easier to
| screw up. Smart people screw it up all the time, and it's
| hard to spot in code review, whereas average programmers
| have no problem avoiding swallowing exceptions once they
| realize it's important, and if they do mess up it stands
| out like a sore thumb in the code.
| Fellshard wrote:
| I've been working through 'Haskell From First
| Principles', and it turned on a lightbulb: the 'right'
| half of Either is the 'important' one because its type
| variable is free to change. instance
| Functor (Either a) where -- a is fixed here!
| fmap :: (b -> c) -> Either a b -> Either a c fmap
| _ (Left l) = Left l fmap f (Right r) = Right (f
| r)
|
| As a general rule, the last type parameter of a type
| carries special significance: the same applies to Tuples.
|
| You _can_ trivially construct a type where the two labels
| are swapped; it's just labels, Left and Right aren't
| intrinsically important, except insofar as they reflect
| the positions of the type arguments in written text.
| data Either' a b = Left b | Right a
| yw3410 wrote:
| This is also the reason we have the convention in Scala
| as well - the inference to partially apply the type works
| in a certain way. But I agree with the parent post, a
| more descriptive name would be better.
| clickok wrote:
| This was great, actually. I don't program in Scala, but
| it was very interesting to hear about the difference
| between types as abstractions vs types as they are used.
|
| For unfamiliar topics or when presented with uncommon
| insight, I believe rants, monologues, even diatribes are
| actually some of the best things to read.
| jinfiesto wrote:
| You shouldn't swallow errors like that in FP either. You
| can trap the error in an effect type and throw it at the
| top level of your app when your effect type gets run
| after all of the code that processes that error value has
| a chance to recover. See something like Zio for an
| example, though you can do similar things without Zio.
| randmeerkat wrote:
| If the team uses Zio, or maybe they've gotten lost in the
| religious war of Scalaz, vs Cats, vs Zio... One of the
| major issues I have with Scala is how fragmented the
| community is. Everyone has an opinion on how something
| should be done and everyone thinks that everyone else is
| wrong.
| spion wrote:
| And will you get a call trace when you do that?
| hderms wrote:
| we use error types that extend Exception specifically for
| the purpose of getting a call trace. A lot of people hate
| doing this but it's been extremely helpful on our project
| spion wrote:
| I think the solution is to have typed exception modelled
| as polymorphic variants. Something similar to Swift.
| dnanabkchsbxb wrote:
| Reading this thread is giving me some catharsis. I worked
| on a large project in a big tech company where the org's
| leadership was ideologically determined to build everything
| in Scala. We ended up with dozens of engineers and failed
| to deliver basic functionality. The leadership eventually
| left the company and now they are running a startup with a
| few of the Scala savants from the old team. They got lots
| of funding and they invited me to visit to get me to join.
| They showed me the awesome type system they had implemented
| and tried to demo it but the product was broken in five
| ways.
| jmfldn wrote:
| "I'm on exactly that kind of team now. They've been working
| on a very simple problem for several years now and have an
| enormous, sophisticated, but still buggy and unstable
| solution. They're really smart people, and they had to be
| extremely capable programmers to get this far, but the
| embarrassing question is, how would a team of mediocre
| developers have tackled this problem? They would have
| picked a mediocre language"
|
| I don't mean to be rude to your colleagues, but almost by
| implication in what you've said, they're not good
| programmers. Being a good programmer is nothing to do with
| leveraging a fancy FP language. It's about delivering
| working, maintainable, testable code. Whether you write
| buggy spaghetti code in Go or misuse Scala, its the same
| root problem.
|
| I've been a Scala dev for five years, worked on some
| massive projects and it's been a dream. As long as you
| agree on a style, don't go crazy with the language and use
| it with discernment, it's a joy to work with. It's a sharp
| tool though and it takes discernment to know how to wield
| it appropriately. I don't know if that's a downside, it's
| more a warning.
| jgwil2 wrote:
| I think people can be good programmers and poor software
| architects at the same time. Sounds like that's what has
| happened in this case: brilliant abstractions that are
| misused and result in a buggy product and less productive
| team.
| watwut wrote:
| Frankly, they are bad programmers. They are smart highly
| intelligent people who don't have either knowledge or
| aptitude or willingness to be good programmers.
|
| My pet peeve in this business are supposedly good
| programmers who get praised despite never having actual
| results. And it is not like they would be rare.
| facorreia wrote:
| True, and with people switching jobs every 12 to 24
| months (at least in the Bay Area) many of them never get
| to see the outcome of their choices, and don't learn from
| experience.
| jjav wrote:
| Unfortunately the leetcode whiteboard interviewing
| strongly encourages this type. Even almost requires it.
| If all energy is spent on that, it is time not spent
| learning how to build stable products which can be
| maintained long term.
|
| Because turns out that popping out algorithm exercise
| code after another into source control doesn't produce a
| good product. But that's what current interview fad
| measures, thus it's what it produces.
| jmfldn wrote:
| "I think people can be good programmers and poor software
| architects at the same time. Sounds like that's what has
| happened in this case: brilliant abstractions that are
| misused and result in a buggy product and less productive
| team."
|
| That's fair. I guess if by good programmer we mean,
| fluency with the language and algorithms / abstractions
| /types etc then perhaps they qualify. I really meant
| something more like "software engineer" ie. someone who
| can architect a simple, sane, working solution adhering
| to the usual best practices of good software
| construction. Maybe there's a missing piece between being
| a good coder vs scaling this knowledge up to the
| application level. The latter is indeed a much rarer
| skill and too often, people without it influence major
| decisions.
| jgwil2 wrote:
| I would guess that it's mainly a question of experience.
| When you have smart kids fresh out of school, they are
| going to be eager to use every tool in their toolbox.
| Eventually (hopefully) they will develop the wisdom to
| know when to deploy what.
| jmfldn wrote:
| Maybe the issue is that Scala requires a bit more wisdom
| that some languages. The problem is that the profession
| is filled with inexperienced people because of the rate
| at which new programmers are appearing. In that world you
| either want safe and, dare I say it, lowest common
| denominator languages, or you need more hierarchy where
| senior people take a stronger lead and set the rules. In
| other words, if you're going to use Rust, Scala, Haskell
| etc acknowledge the tradeoffs and potential footguns and
| insist that someone who knows what they're doing, leads
| the project in a very hands on way. Properly mentored
| juniors or mid level devs will have that moment of
| enlightenment where they see the point of these languages
| and don't just succumb to the blub paradox
| https://en.wiktionary.org/wiki/Blub_paradox.
| brabel wrote:
| I am fairly certain the jury is out on this question:
|
| It's always better to write simple, almost dumb code that
| anyone can understand than to use advanced abstractions
| from category theory or whatever.
|
| Why? Because every single study or anedocte I've ever heard
| is like yours: adding complex abstractions to code do NOT
| make it more reliable. But they do make it much harder to
| modify, understand, and fix.
|
| FP tought us that unconstrained mutable state is very bad,
| and that has helped us immensely in the mundane languages
| of the world (Java, C#, C++, Go). But I think it's time to
| learn the other lesson from FP: overly complex abstractions
| are also very bad and should NOT be used willy nilly as
| they simply have a very low or negative cost-benefit.
| dkarl wrote:
| I agree 100%, and I always bring it back to one simple
| observation that has been true of almost all the projects
| I've worked on: the limiting resource that programmers
| work with is their own time and brainpower. To my mind,
| much of the discipline of programming is centered around
| making frugal use of that scarce resource.
|
| I still think programmers should work with powerful
| tools, because "dumb" tools often force you to express
| things in convoluted ways, and you don't need a powerful
| language to make a huge mess. (Java proves that you only
| need single inheritance to produce virtually unlimited
| amounts of spaghettiness in real-world projects. The
| difference between Java and Scala is not that Scala
| enables teams to produce epic piles of FP crap, it's that
| we long ago stopped being shocked when teams produce epic
| piles of OO crap.)
|
| I think the programmers doing terrible things with Scala
| would be doing terrible things in any language, given the
| chance. What they need is hands-on technical management.
| When they have a wild idea, they need to be walked
| through an appropriate engineering decision-making
| process. They need somebody with the authority to tell
| them, "Ha ha, yeah, that design with etcd and Kafka and
| the robot space lasers would friggin' rock, how cool
| would that be, but on the other hand, we could just stick
| a REST API in front of a database and you'd be done in
| two weeks, so let's compare pros and cons."
| simias wrote:
| Finding the right level of abstraction is basically the
| one and only skill that matters for a software engineer
| IMO. I've got about 15 years of experience in programming
| and it's still something I'm actively working on.
|
| I disagree with the general take that "adding complex
| abstractions to code do NOT make it more reliable". Good
| use of abstraction makes code easier to write, easier to
| maintain and easier to extend. Good use of abstraction
| can make code more concise and more regular while at the
| same time allowing better diagnostics when you do
| something wrong.
|
| As a quick example: a generic JSON serialization library
| that can work with any type is probably a lot nicer to
| use that one that requires manual reimplementation for
| every class in your program. It'll be more complicated to
| write but it's probably well worth it in the end.
| Similarly a logging system that can abstract over several
| backends and log levels depending on the environment
| probably beats having a bunch of if/else every time you
| want to log a message.
|
| But one needs to remember the old saying: debugging code
| is harder than writing it, so if you write code that's as
| smart as you're capable of producing then by definition
| you're not smart enough to debug it.
| keybored wrote:
| "Complex abstraction" is a contradiction in terms since
| an abstraction is a relation A -> B where B needs to be
| simpler than A.
| jgwil2 wrote:
| From the client's point of view that may be true;
| abstraction should be the process of pushing complexity
| down. But in this case I think we're talking about code
| that has inadvertently introduced unnecessary complexity
| in an attempt to be hyper-generic.
| einpoklum wrote:
| > adding complex abstractions to code do NOT make it more
| reliable.
|
| Sometimes false. If the (more) complex abstractions are
| in well-tested libraries that haven't been written solely
| for your code.
|
| > But [complex abstractions] do make [your code] much
| harder to modify,
|
| Not necessarily. You can separate concerns; you can
| isolate changes; and you hopefully reduce the amount of
| code you've written yourself, significantly even.
|
| > [complex abstractions] do make [your code] much harder
| to understand
|
| This is as likely false as it is true.
|
| > [complex abstractions] do make [your code] much harder
| to fix
|
| Again, this very much depends. If you're using a well-
| tested, widely-used external library with those
| abstractions, it may well be easier to fix your own code.
| MikeDelta wrote:
| When I was young I though that good writers are the ones
| who write such complex sentences that nobody can
| understand. Later I learned that good writers are clear
| and understandable.
|
| I used to write complex code because I found it elegant.
| Now I write simple code because I enjoy breaking down a
| complex problem into its simplest form, which for me
| means I fully understand the problem.
| jgwil2 wrote:
| I think the writing metaphor is a good one; we are
| writing source code for other humans to read after all.
| aapppwe wrote:
| i think it depends if you need it, rust for example is
| quite complex, but it solves a class of issue that other
| language have trouble solving while maintaining speed
| jjav wrote:
| > the other lesson from FP: overly complex abstractions
| are also very bad
|
| Lisp, in its persistent failure to achieve world
| domination, has been teaching us this lesson for over 60
| years now!
|
| Infinitely powerful languages are fun and, yes, powerful.
| But as soon as the project has more than one or two
| people in it, they also result in runaway complexity that
| will hurt the product far more than any gains in
| expressibility.
| lihaoyi wrote:
| > Hard to win technical arguments with Scala geniuses that like
| using complicated language features.
|
| This is an interesting point. Perhaps part of the reason my
| current employer has been successful with Scala has been that
| we never were integrated into that part of the community.
|
| We hire non-Scala programmers and they write Scala without any
| training, and it generally turns out OK and ends up converging
| in a pretty boring style without any fanciness
| MrPowers wrote:
| Yep, the Spark codebase is a great example of what a big
| Scala codebase should look like. Nothing crazy, not too many
| traits, mainly just "regular functions".
|
| Imagine another codebase uses a feature like self-types
| extensively: https://docs.scala-lang.org/tour/self-types.html
|
| A small team has no good way to resolve the argument if self
| types should be used all over the place or avoided entirely.
| marcinzm wrote:
| > Yep, the Spark codebase is a great example of what a big
| Scala codebase should look like.
|
| A lot of the hardcore Scala community hates Spark which
| sort of summarizes the situation.
| hocuspocus wrote:
| This is a false dichotomy. You'll run into all sorts of
| pain and frustration as soon as Spark touches your
| codebase, no matter what kind of Scala you're writing.
| marcinzm wrote:
| I've almost every big data processing system out there
| and Spark causes no more pain than any other. Less than
| most. It's also a data processing system not a library so
| if you're integrating it into existing code (rather than
| writing code for it and running code on top of it) I'd
| argue you're doing it wrong.
| hocuspocus wrote:
| One big reason, maybe the biggest, to write Spark jobs in
| Scala and (not move to pyspark) is code reuse between
| different components. My team maintains a fairly sizeable
| library that is common to our Spark jobs and several web
| services. Spark _is_ a library dependency if you do
| anything remotely complex with it and it can easily creep
| up everywhere if you aren 't careful.
|
| Decoupling modules isn't always obvious in a codebase
| that's grown organically for 6-7 years now (long before I
| joined) and the cohabitation with Spark is inevitably
| going to cause some pains. A couple examples:
|
| - Play-json codecs aren't serializable (ironically
| enough, Circe, the "hardcore Scala" library is).
|
| - Any library that depends on Jackson is likely to cause
| binary compatibility issues due to the ancient versions
| shipped with Spark. Guava can be a problem too. Soon
| enough you'll need to shade a bunch of libraries in your
| Spark assembly.
|
| - We have a custom sparse matrix implementation that fits
| our domain well, it was completely broken by the new
| collections in Scala 2.13. It makes cross-publishing
| complicated if I don't want to be stuck to Scala 2.12
| because of Spark.
| valenterry wrote:
| Let's clarify some points for folks not so familiar with Scala.
|
| > * Scala minor version are binary incompatible, so maintaining
| Scala projects is a big pain. Upgrading Spark from Scala 2.11
| to Scala 2.12 was a massive undertaking for example.
|
| Scala just chose a strange naming scheme. Other languages would
| have just increased their major version instead. The scala
| minor version is increased every few years and not every month
| or so.
|
| > * Scala has tons of language features and lets people do
| crazy things in the code.
|
| Actually, that's not true. Or rather: compared to what
| language?
|
| Scala has surprisingly few language features, but the ones it
| has are very flexible and powerful. Take Kotlin for example. It
| has method extensions as a dedicated feature. Scala just has
| implicits which can be used for method extension.
|
| > * Scalatest is stil used by most projects and is annoying to
| use, as described here: https://github.com/lihaoyi/utest#why-
| utest. The overuse of DSLs in Scala is really annoying.
|
| I agree with the overuse of DSLs. Luckily that got much better,
| but older libraries like scalatest still suffer from that.
|
| > * Li's libs (os-lib, upickle, utest) have clean public
| interfaces, but most Scala ecosystem libs are hard to use, see
| the JSON alternatives for examples
|
| I think that just comes from using the library in a non-
| idiomatic way. In most applications, you will need to use the
| whole json anyways, and then you use (or can use) circe like
| that: { "id":
| "c730433b-082c-4984-9d66-855c243266f0", "name":
| "Foo", "counts": [1, 2, 3], "values": {
| "bar": true, "baz": 100.001, "qux":
| ["a", "b"] } } case class
| Data( id: String, name: String,
| count: List[Int], values: List[DataValue] )
| case class DataValue( bar: Boolean, baz:
| Float, qux: List[String] )
| import circe.generic.auto._ import io.circe.parser._
| val data = decode[Data]("...").get data.copy(name =
| data.name.reverse).asJson
|
| Yeah, that is more code, but as I said, in the vast majority
| for projects, you need all or most of the fields anyways, so
| all the structure definition is a one-time thing.
|
| The advantage is that the last line is plain Scala code. You
| don't even need to understand the json-library to do
| transformations and re-encode into json.
| yawaramin wrote:
| Did you mean: values: DataValue
|
| ?
|
| Otherwise the decoder is incorrect and, I'm not 100% sure
| but, is likely to throw a runtime error.
| _a1_ wrote:
| > Scala just has implicits which can be used for method
| extension.
|
| I don't think it's fair to say it like this. Scala's
| implicits can mean different things, depending on where
| they're used. Scala 3 even divides 'implicit' to multiple
| keywords.
|
| (kind of 'static' in C++ I guess, only more complicated)
| valenterry wrote:
| That's exactly what I said, no?
|
| Scala has one feature (implicits) but it can be used
| ("mean") for different things.
|
| Essentially, you can mark definitions as implicit and you
| can mark parameters as implicit. Yes, Scala 3 uses
| different keywords to make it easier to understand which is
| what, but both is still just the concept of things being
| implicit.
|
| Think about it: one without the other is completely
| useless. If you cannot define implicit parameters, then
| marking any value as implicit will not have any effect. The
| other way around too: you can mark your parameters as
| implicit as much as you want, if you can't define implicit
| values, you will always be forced to pass all parameters
| manually.
|
| Even implicit classes (excentions) are just syntactic sugar
| for regular methods that are marked implicit.
| jshmrsn wrote:
| I'm not a Scala programmer so I don't know who is more
| right here, but _ai_ was saying that calling three
| different features by one name does not mean there's
| really one feature. Which is different than saying one
| feature can be used in three different ways. The C++
| static example was used because in that case the same
| keyword was used for several literally different features
| to avoid adding additional reserved words.
| yw3410 wrote:
| It's literally one feature - each of the different "ways"
| gets rewritten.
|
| It's why Scala 2 and 3 are able to maintain pretty good
| interoperability.
| oldgradstudent wrote:
| > The overuse of DSLs in Scala is really annoying.
|
| Unless, of course, that specific DSL is the sole reason for
| choosing Scala, like with Chisel[1].
|
| [1] https://www.chisel-lang.org/
| bzzzt wrote:
| Overheard on a Java conference: "I love Scala, clients pay me
| big bucks to rewrite it into sane Java code".
| msluyter wrote:
| After a number of years with Python, I started a position
| working in Scala about 6 months ago. I really wanted to learn
| something new, and become acquainted with functional
| programming concepts.
|
| I agree with most of the above. A couple of additional
| thoughts:
|
| * sbt -- I still have a lot of coming up to speed to do here,
| but the manual is like 500 pages and it's somewhat
| overwhelming. There are tons of little oddities, like why can't
| I run `sbt --version` and instead have to do `sbt sbtVersion`?
|
| * The functional side is fascinating -- I'm still studying the
| cats library. I can almost describe a Monad! It's a pretty big
| mountain, though, and there are times where I have doubts
| whether the benefits will be worth it. Would love to hear some
| re-assurance! ;)
|
| * The ecosystem for microservices seems pretty closely tied to
| akka & lagom. These are quite complex in their own right and
| we've been having trouble with the latter in particular.
| Curious to learn about alternatives. ZIO?
|
| * Re: DSLs. Also not a huge fan of DSLs. One refreshing thing
| about python is that often configuration can just be in Python
| itself (as in Django, for example). See also:
|
| [1] https://erikbern.com/2018/08/30/i-dont-want-to-learn-your-
| ga...
|
| [2]
| https://github.com/cf020031308/cf020031308.github.io/blob/ma...
|
| [3]
| https://twitter.com/antonycourtney/status/589238574429515777
| randmeerkat wrote:
| > It's a pretty big mountain, though, and there are times
| where I have doubts whether the benefits will be worth it.
| Would love to hear some re-assurance! ;)
|
| It's not worth it, the entire language is a mountain of
| documentation and hard to understand concepts that just gets
| in the way of actually delivering product features for the
| business. It will help you to think differently about
| programming problems though.
| rapnie wrote:
| > Scala has tons of language features and lets people do crazy
| things in the code. Hard to win technical arguments with Scala
| geniuses that like using complicated language features.
|
| I also like(d) a lot about Scala, but this is what ultimately
| led me to other languages. There's too many language features
| cranked together. This makes a) for a steep learning curve and
| b) exposes you to expert's code that is so dense and 'smart'
| that its a PITA to decipher what it does.
|
| b) you'll encounter especially when incorporating libraries
| that are not fully stable, and you have to track down bugs in
| them (if only to know if your code is at fault, or the
| library's code).
|
| I had this with Akka when it was just released. One line of
| code and 2hrs of debugging to determine what it did, and if it
| was faulty or not.
| pdpi wrote:
| > Scala has tons of language features and lets people do crazy
| things in the code.
|
| In many ways, it's even worse than that. Scala has exactly two
| features (implicits and punctuation-free method calls) that
| allow you to build massively complicated libraries that pretend
| to be language features, and which work worse than equivalent
| features in languages that support them explicitly. A good
| example of this is typeclasses which are core features of
| haskell and rust, but are implemented by explicitly passing
| implicits (hah) in Scala.
| yw3410 wrote:
| Scala 3 has type-classes as a core feature.
| pdpi wrote:
| Right. We don't have a full production 3.0 release yet, so
| I tend to think of "Scala" as Scala 2 unless it's
| specifically qualified as Scala 3.
|
| Scala 3 seems to be a really good step in the right
| direction for me. There's a clear desire for many of the
| features that are currently achieved through burdensome
| hacks on top of implicits, and actually reifying several of
| them into core language features is a counterintuitive way
| to make things _simpler_ rather than more complicated.
| wirrbel wrote:
| I can agree with all your points (wrote a fair bit of Scala all
| though I wouldn't identify as a scala programmer).
|
| Scala is a weird case, I would normally be the perfect fanboy
| for it, I like functional programming, did my fair share of
| SML, Haskell, Clojure. Also I am not at all dogmatic and can
| even find joy in writing Java.
|
| However, Scala and I never got along well. It is - I think -
| too magic. I'd even prefer pure Java over it tbh.
|
| And then, the community is fairly toxic, which also made me not
| want to stick around.
| liquidify wrote:
| A lot of my buddies who use Scala seem to like it. I'm sure it
| has some to do with the fact that they can use Intellij, which I
| consider to be one of the best IDE's available.
| specialist wrote:
| I'm most interested in Scala's structural typing.
|
| Are there any studies comparing real world impact of structural
| vs nominal typing?
|
| Defects? Code size? Team size? Architectural choices, big and
| small, like use of Visitor design patterns?
|
| I've looked, but no joy.
|
| --
|
| FWIW, metaprogramming should be reserved for personal projects
| and small high trust teams. Definitely not for bog standard data
| processing and CRUD apps.
|
| I'm kinda curious how structural typing story plays out in this
| dichotomy.
| mwcampbell wrote:
| I dabbled with Scala several years ago, but I've been using
| Kotlin for a JVM-based project and am happy with it. My main
| reason for choosing Kotlin is smoother interop with the JVM
| world. For example:
|
| - Scala adds an Option type, whereas Kotlin adds nullability
| checking for existing object types.
|
| - Scala has its own convention for getters and setters, whereas
| Kotlin automatically turns JVM getters and setters into
| properties.
|
| - Scala defines its own set of collection classes, whereas Kotlin
| has zero-overhead wrappers over the JVM collection types.
|
| I haven't tried Kotlin/JS lately, but I expect that Kotlin's
| strong focus on interop with existing stuff would serve it well
| there too, especially when it comes to minimizing bundle size.
|
| Scala may be more powerful in some ways, but Kotlin gives me the
| conciseness I want, with great interop with the huge Java
| ecosystem.
| the_af wrote:
| > _Scala defines its own set of collection classes [...]_
|
| One (arguably) negative consequence of this is that Scala's
| collections don't play nice with Hibernate and similar ORMs.
| Due to the way they work, both Scala and Hibernate wanted to
| "take over" your collections, and obviously they cannot do so
| at the same time. Of course, it's arguable whether this is a
| flaw with Scala or with Hibernate, but for the programmers who
| just wants to get things done that's irrelevant.
|
| It used to be a big deal a few years ago, to the point teams in
| my company decided to either drop Hibernate or drop Scala.
|
| PS: if I remember correctly, early Scala adopters had reached
| the consensus that ORMs didn't play nice with Scala's FP style
| anyway, and so using Hibernate was a mistake. Possibly this
| attitude changed later.
| wtetzner wrote:
| > PS: if I remember correctly, early Scala adopters had
| reached the consensus that ORMs didn't play nice with Scala's
| FP style anyway, and so using Hibernate was a mistake.
| Possibly this attitude changed later.
|
| Having used Hibernate in a Java project, I would argue using
| Hibernate is _always_ a mistake. Way too much complexity
| /brokenness for basically no benefit.
| thelittlenag wrote:
| The two major SQL libraries in the Scala ecosystem these days
| are Doobie (https://tpolecat.github.io/doobie/) and Slick
| (https://scala-slick.org/).
|
| With Doobie you manually write your queries, and then map the
| results into the objects in your domain model. Nothing is
| generated for you. OTOH, nothing is hidden and you are free
| to write queries as optimized and specialized as you need.
| The real selling point of Doobie is a typesafe API for
| manipulating and combining queries, and fragments of queries,
| into larger wholes. This works very well when your
| application interfaces with a database it doesn't own.
|
| With Slick you get access to a DSL that lets you layout how
| your tables look. From there Slick offers an api that let's
| you treat SQL tables as-if they are basically mutable
| collections, with Slick handling all the SQL generation
| itself. You also get DDL, so that you can automate db
| creation and upgrades. This work very well when your
| application owns and controls the database it is connecting
| to.
|
| Both of these have diverged from that traditional ORM model.
| Slick bills itself as FRM, or Functional Relational Mapping.
| And Doobie is embedded queries on steroids.
| wtetzner wrote:
| Isn't JOOQ also usable with Scala?
| joshlemer wrote:
| Yes
| joshlemer wrote:
| Nobody should be recommending Slick to anyone for new
| projects. Slick will not be supported in the future.
| solicode wrote:
| I've heard that too but I think there are at least
| efforts to support Scala 3.
|
| Having said that I think Quill looks nicer
| (https://getquill.io/) if you want a DSL like that.
| yw3410 wrote:
| I seem to remember Stefan Ziegler asking around the
| channels about questions relating to how to do certain
| features to uplift Slick. Don't know if anything came out
| of it.
| thelittlenag wrote:
| Lightbend is planning to end support and development of
| Slick? This is the first I've heard of this. Can you
| point me towards what they've said?
| merb wrote:
| they never said that, but it's basically abadonware. look
| at the commit history:
| https://github.com/slick/slick/commits/master you can
| also look at the contributors page:
| https://github.com/slick/slick/graphs/contributors the
| guys who primarly contributed to it basically left
| lightbend, and in 2019 he shifted his priorities. it only
| had 28 commits since 2020 (including merge commits and
| changes to the build process) it's basically dead until
| somebody steps up.
| zmmmmm wrote:
| > 28 commits since 2020
|
| wow, the bar for "not dead" is really high these days :-)
| philipkglass wrote:
| I have had poor experiences with "fancy" SQL libraries in
| multiple languages. This includes Slick and Quill in Scala.
|
| I don't remember what the problems were with Slick that
| leave such a bad feeling when I hear its name -- that was 5
| years ago -- but I had problems with Quill just last year.
| I was trying to use it to generate an efficient "in" query
| against a two column composite primary key, and nothing
| seemed to work. Since it uses macro magic, one of my
| attempts triggered an internal compiler error instead of
| normal compiler feedback.
|
| I ended up dropping Quill for ScalikeJDBC:
|
| http://scalikejdbc.org/
|
| It seems to be less popular/active than other libraries,
| but it is dead simple to use, even for developers new to
| Scala. I write exactly the SQL I want just like I would in
| psql. There is little-to-no magic. I think that the only
| slightly magical feature I use is variable interpolation
| into SQL ("SQLInterpolation") that prevents injection
| attacks. It actually has capabilities to automatically map
| tables/columns into different structures and generate code
| for you, but my team doesn't use any of that. We just write
| SQL.
| ebruchez wrote:
| > Scala defines its own set of collection classes [...]
|
| And that is a huge benefit. Scala collections are really great.
|
| In _addition_ , Scala also has wrappers around Java collections
| for interop, or you can just use plain Java collections
| directly.
| zmmmmm wrote:
| > Scala collections are really great.
|
| It doesn't really matter how great they are, it kills JVM
| interop right there ... and its even worse if it doesn't and
| you get implicit conversions killing your performance.
|
| This is where there is a bit of bait and switch with the JVM
| story (applies to a lot of JVM languages) - "language has
| great feature X" and "use any library from the JVM ecosystem"
| - often turn out to be mutually exclusive in a practical
| sense. Groovy is the only JVM language I've found that seems
| to deliver a real JVM interop story, but that's explicitly
| because it is designed from the ground up for that.
| mwcampbell wrote:
| > Groovy is the only JVM language I've found that seems to
| deliver a real JVM interop story
|
| Have you tried Kotlin yet? If so, what interferes with its
| JVM interop story in your opinion? I'm happily using Kotlin
| with several Java libraries, and IIRC, no Kotlin-specific
| libraries except for the stdlib.
| strulovich wrote:
| This is a good list, and this is why Scala for Android was a no
| starter. The huge standard library and awkward conversions with
| Java made it an expensive cost on early phones (and probably
| still today).
|
| Kotlin also side steps the big standard library by using inline
| functions a lot (in addition to the mentioned advantage if
| extending existing Java classes).
|
| To really appreciate Kotlin's design, one should know some
| Scala.
| meowzero wrote:
| > - Scala adds an Option type, whereas Kotlin adds nullability
| checking for existing object types.
|
| I like null coalescing, but Scala uses a more generic way to
| handle nulls like Options. After working with both, I'm not
| sure which way I prefer. I do like not having to create new
| syntax for language features and could be handled with a monad.
| Async/await is another that can be handled with an IO, Future,
| ZIO, etc.
| einpoklum wrote:
| The article mostly lost me with two points:
|
| * It claims to be a compiled language "like C++ and Java" - not
| making a distinction between targeting multiple hardware
| platforms and targeting a cozy VM.
|
| * It ties itself strongly to the Java ecosystem.
|
| So, I guess I might consider it in the 'nicer Java' category -
| and it does seem nicer than Java, subjectively - but not outside
| of that ecosystem.
| [deleted]
| vemv wrote:
| > In contrast, cross-compilers for dynamic languages like Clojure
| tend to have a long-list of caveats and incompatibilites
|
| I don't think the author fully comprehended the linked resource
| (namely https://clojurescript.org/about/differences). Perhaps he
| just noted the its size.
|
| A good chunk of the document lists things in common, not
| differences. The actual differences have nothing to do with
| dynamic typing and are deliberate design decisions:
|
| - don't pretend the JVM and JS have identical runtimes when it
| comes to concurrency or numerics. Clojure is a language, not a
| platform abstraction.
|
| - Emit efficient, optimally minifiable javascript code, at the
| cost of offering something less traditionally lisp-y when it
| comes to eval, macros, and other forms of code loading. Those are
| still possible, but some discipline is imposed.
| lihaoyi wrote:
| > - don't pretend the JVM and JS have identical runtimes when
| it comes to concurrency or numerics. Clojure is a language, not
| a platform abstraction.
|
| > - Emit efficient, optimally minifiable javascript code, at
| the cost of offering something less traditionally lisp-y when
| it comes to eval, macros, and other forms of code loading.
| Those are still possible, but some discipline is imposed.
|
| This is exactly the kind of thing I'm talking about: Scala.js
| doesn't need to make these tradeoffs at all! Concurrency works
| identically (just no parallelism), numerics work identically,
| macros work identically. The list of pure-Scala things that
| behave differently in Scala.js is shockingly short
| (https://www.scala-js.org/doc/semantics.html)
|
| Scala.js sacrifices none of these and still emits efficient
| (sometimes faster than idiomatic JS!), optimally minifiable
| javascript code. The fact that Clojure needs to make these
| tradeoffs and Scala doesn't is largely due to Scala's static
| nature allowing more aggressive analysis and optimisation at
| compile time. For example, the reason Scala.js can provide
| exact semantics for numerics while maintaining high performance
| is precisely because it knows what exactly types things are,
| and can use that information to optimize the generated JS in a
| semantics-preserving way
| (http://lampwww.epfl.ch/~doeraene/presentations/jslongs-
| vmm20...)
| yw3410 wrote:
| ScalaJS is one of the projects which I don't understand why
| more people don't rave about. It's /extremely/ nice that I
| can have a single project which share code between them for
| validation; no need for swagger or OpenAPI.
| fulafel wrote:
| How is concurrency implemented in Scala.js? Eg switching
| tasks when you use Future for concurrent long running tasks
| that do IO or sleep?
| vemv wrote:
| The JVM offers real threads, and JS not. The JVM offers
| nanosecond-precision time measurement, and JS not. JS has a
| single type for representing numbers. And so on.
|
| How can those possibly be abstracted away and unified? If
| doing so, what would have the solution have to do with type
| inference at all?
|
| i.e., this is simply a difference in how one approaches
| platform interop (raw vs abstracted/unified). Clojure
| occasionally offers cross-platform abstractions (e.g.
| core.async) but it's not its main philosophy.
| wtetzner wrote:
| > The JVM offers real threads, and JS not.
|
| How does this make a difference to the Clojure interface to
| threads?
|
| After all, if you're running your JVM on hardware that
| doesn't support parallelism, you're not getting parallelism
| anyway.
|
| > JS has a single type for representing numbers.
|
| The comment you're replying to already covered that:
|
| > For example, the reason Scala.js can provide exact
| semantics for numerics while maintaining high performance
| is precisely because it knows what exactly types things
| are, and can use that information to optimize the generated
| JS in a semantics-preserving way
|
| > Clojure occasionally offers cross-platform abstractions
| (e.g. core.async) but it's not its main philosophy.
|
| I can understand providing different libraries for
| different platforms, but language semantics should remain
| (as much as is possible), e.g. numerics.
| wtetzner wrote:
| This is also what I love about js_of_ocaml. It preserves the
| language semantics.
___________________________________________________________________
(page generated 2021-02-11 23:01 UTC)