[HN Gopher] Building an email-to-calendar LLM
___________________________________________________________________
Building an email-to-calendar LLM
Author : bookmark99
Score : 57 points
Date : 2024-03-25 20:58 UTC (1 days ago)
(HTM) web link (ngacho.com)
(TXT) w3m dump (ngacho.com)
| bookmark99 wrote:
| My friend and I built a gmail add on to easily parse tasks from
| an email and add them to your calendar.
| rockwotj wrote:
| You can also do this with https://shortwave.com (and get
| iOS/Android/Web)
|
| https://twitter.com/Shortwave/status/1760723475923390598
| maliker wrote:
| I used to use calendar integrations for these kind of things.
| Then I realized I'd prefer to have the low priority stuff
| disappear until I have to deal with, so I switched to
| followupthen.com and have been happy with it. A nice effect is
| that it creates a paper trail so I know how many times I've put
| things off.
| denton-scratch wrote:
| Just use CALDAV; it's designed for making calendar entries
| automatically via email. I'm not hip with the fashion for putting
| an LLM into everything. I think it's lazy.
| darby_eight wrote:
| IDK about "lazy", but it's certainly an extremely expensive
| solution.
| refulgentis wrote:
| A 3B model runs on Android phones from 2 years ago at 6
| tkns/s.
| swsieber wrote:
| If I'm generating the data, sure, I'll use CALDAV.
|
| But for ingesting unstructured data I didn't generate, I'm
| going to reach for an LLM now (provided latency isn't an
| issue).
| IanCal wrote:
| That's a great solution to a different problem. Unless caldav
| has a process for extracting dates and actions from
| unstructured emails? But that doesn't seem related to caldav.
| politelemon wrote:
| I'm looking at the AppScript site.
|
| Is this like PowerAutomate, but for G Suite products?
|
| Also I don't know why, looking at the site makes me think it's a
| candidate that Google is going to kill off without warning:
| https://www.google.com/script/start/
| tmpz22 wrote:
| It stands out that it doesn't follow the Design System of many
| other Google products - wonder if that means it has its own
| fiefdom.
| simonw wrote:
| Apps Script is positively ancient at this point, first released
| in 2009: https://en.wikipedia.org/wiki/Google_Apps_Script
|
| It's one of the Google products I worry least about, mainly
| because there are 15+ years of existing Google Sheets documents
| that people have built using it at this point. I don't think
| even Google would lightly break THAT many of their existing
| (often paid) users.
| htrp wrote:
| > I don't think even Google would lightly break THAT many of
| their existing (often paid) users.
|
| Says everyone using GCP services that get deprecated.
| robertclaus wrote:
| "Setting up LLMs to output structured data is incredibly hard."
| resonated strongly with my experience working in similar one-off
| projects. I've almost always implemented some level of fuzzy-
| matching to validate and convert the LLM output back into my
| expected structured format.
|
| I've also noticed that the LLMs are much better at writing code
| than structured JSON (no real surprise given the popularity of
| code assistants). If it makes sense in the specific situation, I
| now have the LLM generate code and parse it into the right
| structure rather than requesting structured data directly:
|
| `generate_event("I need to do X", new Date("1-1-2025"))` seems to
| be more reliable to generate than `{ "description": "I need to do
| X", "when": "1-1-2025" }`
| refulgentis wrote:
| If you're doing it locally, it's likely got llama.cpp
| underneath it somewhere. Ask the dev to allow specifying a JSON
| schema via using its grammar feature.
| el_nahual wrote:
| As long as you can sanitize the LLM output somehow. You should
| never `eval` LLM code straight from the tap!
| jakecodes wrote:
| Hey! It's awesome to read other people's solutions to this.
|
| I've been working on solving this for the past 2 years or so and
| I went through much of the same struggles in the beginning until
| we came up with a solution which is fairly complex, to get LLM's
| to output data in a way we can use.
|
| The big problem is that 95% accuracy is not good enough for
| calendars. People lose confidence after 1 failed attempt. Trying
| to get LLM's to output JSON can have a 1 in 1000 invalid JSON
| problem which is unrecoverable. What I wound up doing is training
| models for the tasks with tremendous amounts of data. I did not
| use OpenAI's models as they were not right for the job. Would
| love feedback.
|
| convoke.ai
| ljlolel wrote:
| You can just force it to output into a specified BNF grammar
| this is quite easy
|
| https://www.imaurer.com/llama-cpp-grammars/
| darkest_hour wrote:
| https://www.ninjatech.ai/ aims to solve this
___________________________________________________________________
(page generated 2024-03-26 23:00 UTC)