[HN Gopher] My Notes on Apple Math Notes
___________________________________________________________________
My Notes on Apple Math Notes
Author : mlajtos
Score : 149 points
Date : 2024-11-08 21:31 UTC (1 days ago)
(HTM) web link (mlajtos.mu)
(TXT) w3m dump (mlajtos.mu)
| alsetmusic wrote:
| Placing text representations of the symbols above them (as
| interpreted by the handwriting algo) seems so obvious that it's
| the sort of thing that I'll be frustrated if Apple doesn't adopt
| it. It generates confidence in the output.
|
| Some things aren't obvious until we see them. This is one.
| MBCook wrote:
| I don't know, it seems unnecessary to me.
|
| Why do I need a second copy on top of the one I just wrote?
|
| I don't think misinterpretation would happy nearly enough for
| it to be more useful than intrusive.
| wonger_ wrote:
| I like having confidence that the intermediate representation
| is correct before moving forward. Especially since OCR
| usually makes mistakes on handwriting. It's not a redundant
| copy, it's reassurance.
| mlajtos wrote:
| Yes, the handwriting recognition system needs to
| communicate results as soon as possible to instill trust in
| the user. I need to be able to spot and fix
| misinterpretations as quickly as possible, otherwise errors
| accumulate real fast.
| mlajtos wrote:
| I thought the same few years ago - putting recognized symbol
| above is simple and effective. But you can get the same effect
| (confidence in recognized input) with the Smart Script
| technique when done right.
|
| I briefly experimented with that in 2022 (nothing published
| though) and it worked pretty well. However, the implementation
| has to be good - you can't just beautify every symbol because
| it becomes distracting really fast. The less confidence system
| has about some symbol, the more it should fix it. High-
| confidence misclassifications will still be a theoretical
| problem, but I didn't experienced them that often.
| keyle wrote:
| Related, has anyone noticed with the new Sequoia, Apple Notes,
| which I use extensively, if you start searching or clicking as
| soon as the window open, you get a frozen application that won't
| recover - it must be killed? Is that being reported and if not
| where to?
|
| I don't really care for the suggestion that 2 + 2 = 4, I'd rather
| the app be responsive on open.
| neom wrote:
| My regular spotlight started doing that, if I do command space
| the box shows up and then spotlight hangs and then crashes, 2
| month old MBa with sequoia update.
| shmoogy wrote:
| I only get this when I paste and it's usually only edge or
| chromium apps (slack). I can't replicate it on demand to bug
| report, but it's very annoying when it happens.
| wpm wrote:
| Nuke your spotlight index. You can add your entire "Macintosh
| HD" to the Spotlight Privacy settings to exclude it from
| indexing, and then just remove it, and it'll force it to
| rebuild.
|
| You can also run `mdutil -E`.
| neom wrote:
| worked - tnx!!
| dagmx wrote:
| I haven't experienced it myself but you should file issues by
| going to the feedback page on the Apple site or going to
| applefeedback:// in a browser. The latter is better because it
| a shortcut to the embedded feedback app which collects
| diagnostic data.
| saagarjha wrote:
| If you take a sample of the application you might be able to
| figure out what it's doing.
| nullhole wrote:
| Suggestion for the website: ~double the height of the first video
| element; as it is, if the mouse is over the video, the play
| button / scrubber overlay will be displayed & obscure the video
| itself.
|
| (the quote at the bottom of the article is excellent and apt, too
| :))
| mlajtos wrote:
| Fixed, thank you for your suggestion!
|
| (I love that quote.)
| peppertree wrote:
| I wish there's a way to restrict math notes within a bounding
| box. I use notes with pencil to help my kids with their math
| homework and it started randomly trying to solve what I was
| writing. I like it but wish there's more control.
| epistasis wrote:
| AI demos have gone so wild that managers are apparently
| mistaking demos for products.
|
| I miss Steve Jobs, he would get the team back on track and
| drive the field forward more quickly by demanding reasonable
| UIs.
| MBCook wrote:
| What's wrong with this UI? It seems basically perfect. You
| write math equations and it gives you the answer.
| MBCook wrote:
| You can turn it off on a per note basis in the three dot menu.
|
| there is no way to default it to off, which I have seen some
| people request.
| happyopossum wrote:
| The 18.2 beta shows you the equation Apple thinks you wrote as
| soon as you draw the = .
| amelius wrote:
| Aren't we supposed to discuss technology here _before_ it hits
| the stores?
| mlajtos wrote:
| In the article I discuss tech that doesn't exist yet.
| wiredfool wrote:
| Not seeing it on the App Store, though it's one of the auto
| completes now. Is this really a thing, and if so, where is
| it?
| Jtsummers wrote:
| Notes, calculator, and freeform, though not exactly the
| same features in each. Freeform (which really ought to)
| doesn't suggest inserting graphs (on iOS, maybe on iPad?)
| for instance.
| wpm wrote:
| Maybe I'm a freak but almost all of the proposed "improvements"
| in this article I find distracting, annoying, and would turn me
| off of ever using the feature. I do NOT want my notes or content
| dancing around on the screen or having motion displayed because
| its evaluating equations or statements _while I 'm writing them_.
| It's also why I disable most autocomplete and warnings in code
| editors. I don't complaints about syntax errors for lines I
| _haven 't finished writing_.
|
| Like, why would I care about the value of `a` before I've
| finished writing the equation? Sometimes a tool just shutting the
| hell up is a good thing.
| paulcole wrote:
| Yeah it's almost like people design things without asking you
| specifically for feedback.
|
| Lots of people _like_ autocomplete and animations.
| treyfitty wrote:
| > I don't complaints about syntax error for lines I haven't
| finished writing
|
| How about "complaints about" syntax errors after you've
| finished writing?
|
| Jokes aside, to answer why knowing the value of 'a' might be
| useful before writing the full equation: ever work in excel and
| you're building up a calculation, and when finished, your
| answer isn't what you expected? Tracing that missing
| parentheses or brain-fart typo can be frustrating. I imagine
| that's the same reason why some people would care about the
| value of 'a' before they've finished.
| mlajtos wrote:
| Fair point. Interactivity is a spectrum that should match the
| user preference. Interactivity helps when you are learning the
| tool, because you can do & fix mistakes faster. But when you
| want to think deeply, it should stand away and be there only
| when you explicitly request it.
| fastasucan wrote:
| What makes me crazy about my developer setup in vscode right
| now is that it is complaining about unused imports (even auto
| removing them before I turned that off) when writing an import
| statement. However if I try to avoid that by referencing that
| module before import it is (naturally) complaining about
| unknown module referenced.
| pazimzadeh wrote:
| living graphs would be great.
| MBCook wrote:
| They are. Once you make a graph, you can do something like tap
| on one of the values in the equation and do a gesture to change
| the value and the graph will update live.
|
| I don't know if I'm explaining it well. It was shown in the
| original demo of math notes during WWDC.
| mlajtos wrote:
| It is true that in the WWDC24 keynote, Jenny Chen demoed a
| graph that could be changed by dragging a slider of one of
| the variables [0], but as you can see in demo in the article,
| graphs are dead - they do not respond to changing variables.
| Or maybe I am just doing something wrong...
|
| [0]: https://youtu.be/RXeOiIDNNek?t=2900
| Deadsunrise wrote:
| I just tested your example and is working fine on two
| different ipads on iPadOS 18.2 Beta 1 and 2.
| Jtsummers wrote:
| That works in 18.1. Tap the number you want to change in
| the equation and you get a slider. Change the number and
| the graph changes.
| odysseus wrote:
| In Apple Math Notes, why can't you type a simple list of numbers,
| either in a table or in columnar format, like:
|
| 239.12
|
| +242.33
|
| +673.34
|
| -------
|
| =
|
| And get an answer after the equals sign? (a la Soulver) Isn't
| this an obvious use case? Was Apple trying really hard not to
| copy Soulver's implementation?
|
| You can type them one after another on the same line and get an
| answer, but that's bad for readability. I want to add up numbers
| in a column like Soulver and a spreadsheet lets you do, and do
| that inside Notes.
| mlajtos wrote:
| You can do this in Math Notes - scribble numbers in a column,
| underline it and boom, there is your result. Doesn't work in
| typed text though.
| dhosek wrote:
| For quite a while, I've kept track of how many pages are left in
| the books I'm reading by having a note with entries along the
| lines of
|
| Vargas Llosa 727-516=211
|
| I'd forgotten about math notes / assumed it only applied to
| handwritten notes so the first time I updated one of these notes
| after updating my iOS it was a little shocking to retype the =
| and have the difference generated automatically instead of having
| to figure it out on my own. I've been holding off updating my
| Macs, but I can see this feature being really handy as a sort of
| lite spreadsheet replacement.
| mlajtos wrote:
| Yes, having an ability to do simple math across the whole
| ecosystem is extremely useful.
| rgreekguy wrote:
| I do not do many calculations, so my only issues where when I
| tried it out.
|
| If you open the calculator app and go to handwritten calculations
| (whatever they're called), you have all that cool stuff. But if
| you open Notes, you have to click "Insert result" every single
| time, for better or for worse.
|
| Also about UX improvements: lmao. There is still no way to add a
| new line with Scribble, or sanely write in a small text box (I
| guess, it's only in third-party applications I have had this
| issue).
| mlajtos wrote:
| Yes, math engine works differently when you are in Notes and
| Math Notes. There is a setting ("Insert results" vs. "Suggest
| results") that controls this behavior.
| spwa4 wrote:
| But the problem here is: you need to pay for software
| development. This requires a few years of effort by a capable
| machine learning engineer that's also a pretty good software
| developer. I would love to try this, mostly to become a capable
| machine learning engineer. But ... what's missing is the $, like
| in so many software demands.
|
| I looked up the paper on this, but there are no good models for
| it that generate proper syntax. So for one thing, whoever does
| this would need to grind out creating at minimum a few 10,000
| math exercises, to get the software to work.
|
| So this costs, say, ~400.000 USD, and 2 years of time, counting
| taxes, equipment, office, ... Perhaps "only" 100.000 if you're a
| professor and willing to use a PhD student (and deal with the
| issues that will give).
|
| Now I don't mean to disparage this proposal. But this seems to be
| how everything works except "improve our existing boring software
| 1%" ... And it's frustrating. At best, I can find machine
| learning jobs that boil down to "make LLMs solve tickets" or,
| worse "make LLMs write spam" (usually customize/personalize
| existing spam mails). Beats solving tickets myself, that it
| definitely does, but something like this ...
|
| I don't have trouble finding software development jobs. However,
| I cannot ever seem to do something like this. I would so love to
| make something like this, to really take a hard problem, work it
| and make something beautiful. But this is not a hobby effort.
| This requires spending a few years of effort making something
| beautiful.
|
| If this is done as a hobby project I can make a nice tech demo,
| no doubt, but it will be only what I make of it, and it'll be a
| limited cool tech demo, just like what Apple did (making a gradio
| that does a few examples correctly isn't very hard), what I'd use
| it for. Not because I don't want to make the next generation of
| advanced calculators, but because I'd never have the time to do
| that.
| mlajtos wrote:
| You are not wrong. In 2021, I did bootstrap everything on my
| own - dataset, segmentation algorithm, neural net, UI, math
| engine, etc. And in the end, it worked properly only for me.
| But that was my aim - own complete stack to be able to try out
| everything that might be interesting without being burdened by
| 3rd party solutions. It was a prototype that I could show to
| people - spread the idea. Doing product of that magnitude on my
| own would never fly.
|
| I don't think Apple did a "cool tech demo". In my view, they
| published first version of a very ambitious project. They
| worked on it for years with top talent. And they felt this is
| the minimum lovable product on which they will iterate.
|
| And if you would like to build product-ready solution similar
| to Math Notes, you can leverage existing stuff. MyScript.com
| have really good handwriting recognition system with bells and
| whistles. You can take a lot of shortcuts.
| spwa4 wrote:
| Well, I hope you'll open source it. But I will 100%
| understand you not doing this.
|
| And it's just a general observation. Being well paid to make
| a technically impressive GOOD product for people, I would so
| love to be in that situation. But barring a FANG job, I don't
| know what to do.
| tomashubelbauer wrote:
| Math Notes is pretty cool, I used it recently to figure out what
| spec AC I needed to buy to climate control my non-insulated barn
| workshop while I'm working there. However, I struggled to get it
| to convert units.
|
| An example with non-representative numbers: A =
| 10000 W B = A / 1 kW =
|
| This gives 10 000 000 W2. I was hoping for 10 kW.
| setopt wrote:
| Did you try "A / ( 1 kW )" just to see if that works? Sounds
| like it doesn't understand that 1 and kW are parts of the same
| "thing".
| tomashubelbauer wrote:
| Good call! Seems as though that just gives a unitless "10".
| Which is cool, I also struggled to make it give me unitless
| values I could use as factors without it starting to make up
| sci-fi units. But it could use some more of that Apple polish
| and just get this right.
| donatj wrote:
| I really want to like Apples handwriting recognition, it's half
| the reason I bought an Apple Pencil with my new iPad. It's just
| kind of difficult to use in practice.
|
| It's just not that good at recognizing my handwriting, and best I
| can tell has zero mechanism for teaching it to better understand.
|
| When enabling the "autocorrect" where it cleans up your writing,
| it so often replaces my handwriting with the entirely wrong word
| as to be infuriating. On top of that, it often does so long after
| I wrote it so I might not even notice that it happened until I
| try to read the note back later.
|
| As the author mentioned, having to switching to eraser and back
| is painful and annoying. I really wish the options for reflowing
| a document were a lot more intuitive. If I need to go back and
| add a sentence or two mid note I can try to draw a line to make
| space but it rarely works like one would expect.
|
| I think all in all the feature is neat but half baked, and they
| really need to do a bunch more user testing to get it into
| something that you would want to use rather than a neat tech
| demo.
| jerojero wrote:
| Live graphs, function definitions and support for more calculus
| would be something good for the future, as it is right now this
| is good for basic middle school math and quick calculations.
|
| Linear algebra, calculus notation, etc. these are necessary
| improvements. I think they might come in the future though, I
| know this will become on par with a graphic calculator.
| mlajtos wrote:
| Yes, the first version is already usable for everyday simple
| stuff. In the future, it will definitely gain more features.
| But you are wrong in one thing - it will become much-much more
| than graphing calculator.
| amai wrote:
| I think this might be relevant: https://mathpix.com/handwriting-
| recognition
___________________________________________________________________
(page generated 2024-11-09 23:01 UTC)