[HN Gopher] An Illustrated Introduction to Linear Algebra, Chapt...
___________________________________________________________________
An Illustrated Introduction to Linear Algebra, Chapter 2: The Dot
Product
Author : egonschiele
Score : 93 points
Date : 2025-11-03 13:28 UTC (9 hours ago)
(HTM) web link (www.ducktyped.org)
(TXT) w3m dump (www.ducktyped.org)
| seanhunter wrote:
| If you actually want to learn linear algebra, don't use this
| blogpost. It's real weaksauce compared to the wealth of free
| information and resources available online.
|
| Firstly, the _real_ illustrated guide to linear algebra is the
| youtube series "The Essence of linear algebra" by
| 3blue1brown[1]. It has fantastic visualisations for building
| intuition and in general is wildly superior to this, which seems
| fine but extremely superficial.
|
| If you're done with 3b1b and want to take things further, then
| the go-to is the excellent 18.06SC course by the late and
| legendary Gilbert Strang. It's amazing, it's free. [2]
|
| Still want more? OK now you're talking my language. If you are
| serious about linear algebra (Up to graduate level, after that
| you need something else) then you want the book "Linear Algebra
| Done Right" by Sheldon Axler. It's available for free from the
| author's website[3] and he has made a bunch of videos to
| supplement the book. There's also an RTD Math full lecture
| series[4] that follows the book and he explains each thing in a
| lot of detail (because Axler goes _fast_ , so it's beneficial to
| unpack the concepts a bit sometimes).
|
| [1]
| https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...
|
| [2] https://ocw.mit.edu/courses/18-06sc-linear-algebra-
| fall-2011...
|
| [3] https://linear.axler.net/ and
| https://www.youtube.com/watch?v=lkx2BJcnyxk&list=PLGAnmvB9m7...
|
| [4] https://www.youtube.com/watch?v=7eggsIan2Y4&list=PLd-
| yyEHYtI...
| sgdpk wrote:
| Second Axler's book! (probably as a second exposure after a
| first course, to really understand what's going on in Linear
| Algebra)
| seanhunter wrote:
| Yeah agree. I first picked it up before I self-studied the
| 18.06SC course and bounced off pretty hard, then I'm going
| through it again now and it's an absolute joy, but is really
| packed.
| incognito124 wrote:
| I also recommend Robert Beezer's "A First Course in Linear
| Algebra". Great for self-studying.
|
| http://linear.ups.edu/
| barrenko wrote:
| Thank you, that's a new one.
| selimthegrim wrote:
| Strang is still very much alive AFAIK
| tptacek wrote:
| Yep. He retired, is all.
| tptacek wrote:
| Every time this subject comes up, or really any math subject
| comes up, someone recommends 3Blue1Brown. I love 3Blue1Brown.
| Just like when The Shawshank Redemption plays on TBS for the
| 389248th time, I will stop what I'm doing and rewatch any
| 3Blue1Brown video as soon as it appears in my feed.
|
| But I'm not sure I've ever really learned anything from one of
| those videos. Appreciated something more? Absolutely. And
| maybe, sure, that's a kind of learning. But I cringe every time
| eigenvalues come up and people point to the 3B1B evector video.
|
| In fact, if your goal is to actually get any kind of facility
| with the concept, this "weaksauce" blog post probably has a
| better didactic strategy than 3B1B. It strips the concept down,
| provides specific, minimized worked examples, _and_ provides a
| useful framing for the concept (something basically at the core
| of 3B1B 's process).
|
| I learned linear algebra from Strang's 18.06. I later did a
| bunch of Axler helping my daughter through UIUC linear algebra.
| I like both. Strang is much closer to what the median HN person
| probably wants. In both cases though: don't do what I did at
| first, and just watch the videos and read the book. If you're
| not doing problems, you're probably not learning anything.
|
| This blog post comes closer to "actually doing problems" than
| 3B1B. Ergo: its sauce is stronger, not weaker.
|
| I came to the blog post expecting to roll my eyes. No
| discussion of inner product spaces? Not even a mention of
| conjugate symmetry? I was pleasantly surprised.
|
| It's not easy to come up with a simple, accessible framing for
| a topic like this, and, maybe, the dot product is particularly
| tricky to give an intuition for (I'll go out on a limb and say
| that neither Strang nor Axler do a particularly great job at it
| --- "it" being, explaining the "why" of the dot product to
| someone who doesn't really even know what a vector is). The
| post doesn't purport to teach all of linear algebra. It's just
| an exercise in trying to explain one small part of it.
|
| I'm not asking you to give the author a break, so much as
| suggesting that you're closing yourself off from appreciating
| different strategies for explaining complex topics, which is a
| valuable skill to have.
| creata wrote:
| But at the very least, surely a dot product explainer should
| talk about the two main ways of looking at a dot product!
| This article leaves out the "angle and norms"
| (||a||*||b||*cos(th)) interpretation entirely. It's like if
| someone gave you the formula for complex multiplication,
| without also showing you how it's all about rotations.
|
| And maybe I'm being a pedant, but the dot product should be
| between two things that are in the same space. In the
| Minnesota lottery example, the probabilities should be a row
| vector instead. It's the exact same calculation, so again,
| maybe a bit too pedantic.
| tptacek wrote:
| So does Strang! (I just checked, Linear Algebra &
| Applications 4E).
|
| Also: sir, this is a blog post. It's wild seeing people say
| "if you really want to understand this topic, pick up
| Axler". I mean, yeah, also if you were serious you could
| just enroll in your local community college's Linear
| Algebra course.
|
| My feeling is that a lot of the critique here is really
| signaling. For whatever reason, linear algebra is super
| high-status in this community, and people want to
| communicate that they've done something serious with it.
| (I'm sure I'm guilty of that too.)
| creata wrote:
| > It's wild seeing people say "if you really want to
| understand this topic, pick up Axler"
|
| I agree. I don't think "Linear Algebra Done Right" is a
| good fit for most people. It's way too dry, and I don't
| think his crusade against determinants helps the book. I
| don't know what a _good_ book suggestion would be,
| though. Maybe just nab the course notes /slides/exercises
| off some university's website?
|
| > this is a blog post
|
| Blog posts can be and often are amazing.
|
| > My feeling is that a lot of the critique here is really
| signaling.
|
| Weird accusation, so just to be clear, I haven't done
| anything serious with anything, ever.
| tptacek wrote:
| I wasn't referring to anybody in particular. But, like:
| what is the point of calling out a blog post for not
| presenting the angle interpretation of the dot product?
| How would that have fit with the goals of this post? You
| presented it as a defect, but that logic also suggests
| Strang's explanation is defective.
| creata wrote:
| > that logic also suggests Strang's explanation is
| defective
|
| I haven't read Strang's book, so I can't comment on that.
| But yeah, if it never mentions the formula
| ||a||*||b||*cos(th) or at least talks about how the dot
| product relates to parallel and orthogonal vectors, I
| would consider that a big hole in an introduction to the
| dot product.
|
| > How would that have fit with the goals of this post?
|
| Because the post is titled "an introduction to linear
| algebra... the dot product", and this is something that I
| believe should be in anything that considers itself an
| introduction to the dot product.
|
| You seem to disagree, and I'd like to ask: why? I think
| this a fundamental aspect of the dot product, again, just
| as fundamental as the relationship between complex
| multiplication and rotation. I think my view is common.
|
| > calling out a blog post
|
| I didn't intend to do anything as strong as "call out"
| the blog post. I just wanted to express surprise at
| someone so strongly praising ("its sauce is stronger
| [than 3B1B's video series]") an alright post.
| tptacek wrote:
| Well, he's one of the most famous and best-respected
| educators of linear algebra, and this is an unusually
| basic piece of linear algebra to be taking aim at him
| for, so one answer is: if you have to ask whether his
| approach is defective, you should first evaluate how
| strong your own understanding is. That's argument from
| authority, but then: Strang _is_ an authority. The
| typical push-pull on a message board is between Strang
| and Axler, and, if you want to find out if Axler is going
| to save your argument, flip to 6.A in LADR.
|
| The direct response to your question is: the centrality
| of the angular interpretation of the inner product is the
| kind of thing I feel like you'd say if your primary
| purpose for learning linear algebra is to program video
| games. Linear algebra isn't "about" geometry, and, in
| particular, Strang's teaching goal centers vector spaces
| and the relationship between spaces. You need inner
| products to apply linear transformations with matrices.
| You basically don't need the angular interpretation... at
| all? if your goal is to teach linear algebra as a tool
| for data manipulation and, as Strang did in his last
| course, as a vehicle for deep learning.
|
| (You need orthogonality, but you didn't say `a[?]b=0`,
| you said `|a||b|costh`).
|
| I'm not rating this blog post "higher" than 3B1B; the
| comparison doesn't even make sense. The blog post and the
| video series simply have different objectives.
| creata wrote:
| > That's argument from authority, but then: Strang is an
| authority.
|
| Yes, it's an argument from authority.
|
| > The typical push-pull on a message board is between
| Strang and Axler
|
| We're not playing Pokemon with linear algebra textbooks
| here...
|
| > the centrality of the angular interpretation
|
| It's not central. It's _one of_ the ways to think of it.
| I think 3B1B actually did a good job emphasizing this in
| his video series: there are many ways to look at vectors,
| and all of them are sometimes useful. They can be arrows
| in space, or sequences of numbers, or black boxes that
| obey the vector space axioms, or polynomials, and so on.
|
| > if your primary purpose for learning linear algebra is
| to program video games.
|
| What an odd guess. Seriously, I would be surprised if
| most math teachers didn't mention angles and norms,
| scalar projections, etc. An important part of math is
| being able to see things from multiple angles (no pun
| intended), and this is a useful angle to view the dot
| product from.
|
| > Linear algebra isn't "about" geometry
|
| Sort of. Linear algebra proper, the study of vector
| spaces and linear maps between them, is not about
| geometry, but geometry comes in almost precisely once you
| equip the vector space with an inner product, such as the
| dot product. A bare vector space has almost no geometric
| content, but the inner product gives you lengths, angles,
| isometries, orthogonality, and all that jazz.
|
| > Strang's teaching goal centers vector spaces and the
| relationship between spaces.
|
| That's a good goal.
|
| > You need inner products to apply linear transformations
| with matrices.
|
| I think you have a fundamental misunderstanding of linear
| algebra here. You _do not_ need an inner product to
| "apply linear transformations with matrices".
|
| > You basically don't need the angular interpretation...
| at all? if your goal is to teach linear algebra as a tool
| for data manipulation
|
| Off the top of my head, cosine similarity? It's not
| uncommonly used.
|
| But if you're teaching linear algebra for data
| manipulation, you rarely need the dot product, either.
| Most "dot products" in data manipulation, like the ones
| in this article, would be better expressed as row-vector-
| column-vector matrix products.
|
| > (You need orthogonality, but you didn't say `a[?]b=0`,
| you said `|a||b|costh`).
|
| I said "||a||*||b||*cos(th) or at least talks about how
| the dot product relates to parallel and orthogonal
| vectors". Anyway, it's hard for a person not to interpret
| orthogonality as a statement about angles, so I'm not
| sure what distinction you're trying to draw here.
|
| > I'm not rating this blog post "higher" than 3B1B
|
| I guess I misinterpreted "its sauce is stronger", then.
|
| Sorry for any mistakes; this took way too long to type.
| tptacek wrote:
| We're indeed not playing Pokemon with textbooks here
| since neither of the two textbooks we're discussing (2 of
| the 3 most famous for the subject) agree with you. I
| haven't read Lay; is that the one you read, and does it
| support this argument?
| creata wrote:
| I didn't read a linear algebra textbook; I studied pure
| math at university.
| ndriscoll wrote:
| You don't need inner products for linear transformations.
| You just need the idea of a basis and linearity. You
| define your transformation on a basis (which is all a
| matrix is: the list of where the map sends each basis
| element), and it is automatically defined everywhere else
| via linearity. The textbook my undergrad class used
| (Curtis) doesn't define inner products until after linear
| transformations and matrices, for example.
|
| The angular interpretation and geometry are basically the
| entire point of inner products (inner products are how
| you _define_ a large chunk of geometry). Angles and
| projections are the entire intuition behind talking about
| orthogonality, which is super important practically to
| basically every field.
| tptacek wrote:
| Re the more abstract approach to transformations, fair
| point, and I feel like that describes Axler well too. I'd
| soften my argument to just that being Strang's approach
| to bringing in the subject.
|
| I agree orthogonality is important. But Strang doesn't
| get to `a[?]b=0` by means of `costh`. You're halfway into
| the book before he's even defined the Euclidean norm. He
| derives orthogonality mostly algebraicaly; the only angle
| he talks about is p/2.
| seanhunter wrote:
| I take exception to the idea that I was somehow
| signalling, and frankly that's a pretty weird thing to
| say. It happens that I love maths and am studying it
| part-time alongside my work. I posted some links to
| things that I have found useful on my journey so far in
| the hope that they are useful to others.
| tptacek wrote:
| I don't think "taking exception" is helpful. You opened
| with "If you actually want to learn linear algebra, don't
| use this blogpost." Do you feel like that really needed
| to be said?
| commandlinefan wrote:
| When I was in school, my teachers and professors all insisted
| that the only way to learn a subject was to _practice_ it. If
| you want to learn math, work the problems. I was skeptical
| when I was young, but I have yet to find any other way to
| learn math (or anything else).
| photochemsyn wrote:
| The Axler text was discussed here (631 pts 295 comments):
|
| https://news.ycombinator.com/item?id=38060159
|
| For most people going into science and engineering as opposed
| to pure mathematics, Poole's "Linear Algebra: A Modern
| Introduction" is probably more suitable as it's heavy on
| applications, such as Markov chains, error-correcting codes,
| spatiel orientation in robotics, GPS calculations, etc.
|
| https://www.physicsforums.com/threads/linear-algebra-a-moder...
| beklein wrote:
| Comparing this blog post to a 500-page book or a multi-hour
| course and calling it "weaksauce" misses the point. This post
| is meant as an introduction to the dot product, and it does
| that really well. The formal definition (6.1) and explanation
| in Axler's book wouldn't make a good starting point for most
| people, it isn't even a good next step in my opinion. It's
| great that you're passionate about the topic, really, but
| helping more people discover math means meeting them where they
| are and appreciating content like this for what it's trying to
| do.
| nh23423fefe wrote:
| The post contains no geometry. Which is the only worthwhile
| content of dot products.
|
| Explaining the dot product by its implementation over R^n is
| pointless. Conflating 1-forms and vectors is pointless.
| tptacek wrote:
| The only worthwhile content of dot products is _geometry_?
| nh23423fefe wrote:
| of course. dot products are a symmetric form on vector
| spaces. they let you compute the spheres of radius r.
|
| given the sphere of radius r, for any pair of vectors v,w
| in the sphere -r^2 <= dot(v,w)=dot(w,v)
| <= r^2
|
| as w varies from v to -v the value moves from r^2 through
| 0 to -r^2
|
| this is how we define parallel perpendicular and
| antiparallel.
|
| the dot product is only meaningful in a geometric
| context. by definition it projects vectors down to
| scalars. fixing the scalar value finds the spheres, and
| for a sphere we can vary the vectors to compute cosines.
| bee_rider wrote:
| Find a copy of Yousef Saad's books (Iterative Methods, and the
| Eigenvalue Problems). Find a problem you want to solve, and
| implement one of the solutions he describes. If you don't
| understand something, that's what the first chapter is for.
| cultofmetatron wrote:
| mathacademy has a course on linear algebra. currently working my
| way back up from nothign to get to it. easily the best resource
| for learning math on the internet.
| tptacek wrote:
| I do love Math Academy (I signed up 9 months ago in the hopes
| of replacing my NYT Crossword habit with something more
| productive, and 9 months later I'm gearing up for multivariable
| calc, which is neat given that except for linear algebra, which
| I self-studied out of necessity for cryptography work, all my
| math education stopped in sophomore year of high school).
|
| It has a very different purpose than a post like this though!
| Also: there's probably more effort at exposition in this blog
| post than in all of Math Academy's coverage (that's not a dunk
| on Math Academy).
| cultofmetatron wrote:
| > here's probably more effort at exposition in this blog post
| than in all of Math Academy's coverage (that's not a dunk on
| Math Academy).
|
| haha definitely agree. a lot of these blog posts are great if
| you want to read about math. mathacademy is pretty much all
| the exposition chopped out and you spend 90% of your time
| doing math. I can see how some wouldn't like it but I think
| the problem solving aspect makes it was more useful for
| bruteforcing your way towards building intuition
| tptacek wrote:
| I like comparing it to Lingua Latina Familia Romana, a book
| that teaches Latin basically without any English; it opens
| in Latin and just keeps going that way and somehow you're
| able to follow along. Both are kind of trippy experiences.
| WrongOnInternet wrote:
| When I see the word "illustrated," I expect to see graphs or
| something that would help me visualize how linear algebra works.
| The only thing "illustrated" about this post is that he hand drew
| some table which could have been easily with some basic HTML+CSS.
| drdec wrote:
| I don't understand the down votes, I had the same reaction.
| Other posters have suggested some better resources, check those
| out
| vixen99 wrote:
| That's your preference. However "To illustrate is to make
| something more clear or visible. Children's books are
| illustrated with pictures. An example can illustrate an
| abstract idea. "illustrate" comes from the Latin illustrare 'to
| light up or enlighten.'"
|
| Quote from https://www.vocabulary.com/dictionary/illustrate
| sarchertech wrote:
| In the context of books or internet books illustrated almost
| exclusively means "with pictures".
| 0_____0 wrote:
| It's extremely obvious that the sense of "illustrated" meant
| here is "containing illustrations."
| tptacek wrote:
| What graphical illustration do you think this is missing? How
| would that make things better? Have you ever seen
| http://matrixmultiplication.xyz/? Great graphical illustration.
| Also: a really unhelpful way to understand matrix
| multiplication.
|
| This is part 2 of a series, all under the same name; the first
| part is _extensively_ illustrated (and I 'm not sure the part 1
| illustrations are all that helpful).
| griffzhowl wrote:
| Illustrating the dot product using the projection of one
| vector on another conveys the geometric idea. Then it's
| transparent why "orthogonal" means dot product = 0.
|
| The author seems to be taking a different tack though, and
| maybe doesn't want to be too tied to this particular
| geometric picture
| MaxBarraclough wrote:
| I agree that's a poor way to teach matrix multiplication.
| This topic arose last year. [0]
|
| Don't introduce any clumsy nonstandard transformations like
| 'rotating' a matrix, just highlight the relevant row and
| column. [1]
|
| [0] https://news.ycombinator.com/item?id=41402224
|
| [1] https://www.mathsisfun.com/algebra/images/matrix-
| multiply-a.... (from
| https://www.mathsisfun.com/algebra/matrix-multiplying.html )
| tptacek wrote:
| I kind of hate all these, because matrix multiplication is
| easiest to think of as simply repeated matrix-vector
| multiplication, which you need anyways (and earlier).
| egonschiele wrote:
| I think this is a reasonable expectation. I'm writing about the
| dot product and there is a geometric visualization for dot
| products, so it's fair to expect that. As I said in this
| comment (https://news.ycombinator.com/item?id=45802969) I
| focused on teaching the reader the bare minimum they would need
| to know to get to matrix multiplication.
|
| I use illustrations for a lot of other things though. I use
| them to pace my posts, to call out things that are especially
| important, to reinforce written ideas with images, and to give
| the reader's eye a place to rest.
|
| If you are looking for graphs, this video about the dot product
| from 3B1B may be more your speed:
| https://www.youtube.com/watch?v=LyGKycYT2v0
| photochemsyn wrote:
| Some hand-written (not AI-generated) prompts to consider:
|
| "An expert in university-level linear algebra, including solving
| systems of equations, matrices, determinants, eigenvalues and
| eigenvectors, symmetry calculations, etc. - is asked the
| following question by a student: "This is all great, professor,
| and linearity is also at the heart of calculus, eg the derivative
| as a linear transformation, but I would now like you to explain
| what distinguishes linear from non-linear algebra."
|
| "What kind of trouble can the student of physics and engineering
| and computation get into if they start assuming that their linear
| models are exact representations of reality?"
|
| "A student new to the machine learning field states confidently,
| 'machine learning is based on linear models' - but is that
| statement correct in general? Where do these models fail?"
|
| The point is that even though it takes a lot of time and effort
| to grasp the inner workings of linear models and the tools and
| techniques of linear algebra used to build such models,
| understanding their failure modes and limits is even more
| important. Many historical engineering disasters (and economic
| collapses, ahem) were due to over-extrapolation of and excessive
| faith in linear models.
| griffzhowl wrote:
| A great resource that isn't mentioned often is the linear algebra
| chapters in Birkhoff and Mac Lane's Survey of Modern Algebra.
| Chapters 7,8,9, and 10 (in the 4th and 5th editions anyway) are a
| self-contained book-within-a-book of about 200 pages on both the
| computational and theoretical aspects of vector spaces, matrices,
| linear transformations, and determinants.
|
| Many times I've been puzzled by a concept just to go there and
| find it made simple and obvious. It's a real golden nuggett...
| Plus if you then want to go further into groups, rings, fields,
| and Galois theory, that's also there.
| bsoles wrote:
| > Summary: A dot product is a weighted sum of two vectors.
|
| Nope. This is incorrect. The dot product is a weighted sum of a
| vector's elements, where the weights are the elements of the
| other vector. Weighted sum of two vectors would require a third
| entity to provide the weights.
| tptacek wrote:
| It's an interesting callout; if you go Google "weighted sum of
| two vectors", it's not too hard to find more authoritative
| sources (nothing as authoritative as Axler or Strang, of
| course) describing either a dot product or a linear combination
| in those terms.
| evanb wrote:
| A dot product _is_ a weighted sum of two vectors, but not in
| the way the author suggested. The author 's use is that one of
| the vectors is the weights and the other is 'the' vector, so
| the dot product is the weighted sum of ONE vector. It just so
| happens that because the author is not interested in the
| geometric interpretation of the dot product that they forgo the
| metric.
|
| On the other hand, it is common to need a _metric_ , which is
| actually the set of weights in the dot product. If `g` is the
| metric, dot(a, g, b) = np.einsum('x,xy,y->',
| np.conj(a), g, b)
|
| g doesn't have to be diagonal, but if you want the dot product
| to be symmetric in a and b it ought to be self-adjoint. Then
| you can find a basis where g is diagonal with real diagonal
| elements, which you can interpret as the weights.
| egonschiele wrote:
| Hey everyone, I'm the author. I'm seeing a lot of the same
| comments here, so I want to address them.
|
| I teach math by leading with examples. I try to show the
| intuition behind an idea, and why it is interesting. For this
| series, my reader is someone who knows algebra, and likes
| learning new things, especially when a teacher shows what is
| interesting about a topic.
|
| ## You didn't cover x about the dot product.
|
| I try to only teach as much as is necessary to get the student to
| the next point, which is matrix multiplication. I usually end up
| cutting a lot of material out of my chapters to keep them simple.
| In this case, I cut out a whole section on the properties of a
| dot product, as well as a discussion about inner and outer
| products, because those weren't necessary to get to matrix
| multiplication. I think this context was lost while posting to
| HN.
|
| ## 3B1B already has a series on this.
|
| I love 3B1B, but his style of teaching and mine are quite
| different. Even though we both teach visually, his videos are
| densely packed with information and his expectation is that you
| will watch the video a few times till you understand the topic.
| He also leads with math more than I do. My posts are written more
| like stories. My goal is they should be easy to get into, and by
| the time you have finished reading, you should understand more
| about the topic. I don't expect readers to read through multiple
| times. I personally learned linear algebra through Strang's
| videos and textbook, and those videos are awesome, but can be
| confusing. If you found the Strang or 3b1b videos confusing,
| hopefully my posts will make it easier for you to follow them. I
| think comment is spot on:
| https://news.ycombinator.com/item?id=45800657
|
| If these ideas resonate with you, I think you'll like this post,
| and if not, there are plenty of guides that go the more
| traditional route. You can also read the first post in the series
| and see if you like it: https://www.ducktyped.org/p/an-
| illustrated-introduction-to-l...
|
| For another example of my writing, see my series on AWS:
| https://www.ducktyped.org/p/a-mini-book-on-aws-networking-in...
| tptacek wrote:
| _I try to only teach as much as is necessary to get the student
| to the next point, which is matrix multiplication_
|
| Preemptively noting: this is also Strang's strategy.
___________________________________________________________________
(page generated 2025-11-03 23:01 UTC)