[HN Gopher] Unlocking the power of time-series data with multimo...
       ___________________________________________________________________
        
       Unlocking the power of time-series data with multimodal models
        
       Author : alach11
       Score  : 59 points
       Date   : 2024-12-02 19:40 UTC (3 hours ago)
        
 (HTM) web link (research.google)
 (TXT) w3m dump (research.google)
        
       | alach11 wrote:
       | It kind of feels criminal to do time-series analysis with
       | multimodel models and not use any traditional numerical models to
       | provide a baseline result. It's an interesting result though.
        
         | AriedK wrote:
         | They mention using a IMU dataset that is collected using an
         | APDM Opal. https://www.apdm.com/wp-
         | content/uploads/2015/05/Opal-Publica... This publication
         | mentions a paper on p. 5839 (p 13 of the pdf) where a single
         | sensor on the waist (as used in the Google research) would lead
         | to an f1 score of 0.77 if I did my math correctly. In other
         | words, pretty close to a >1 shot plot analysis of gpt4o and
         | gemini pro1.5.
         | 
         | I would also be interested how the llm's would hold up to the
         | free-fall interrupt that's built in to some consumer grade
         | IMU's (BMA253 for instance), anyone here with experience in
         | this usecase?
        
       | emkee wrote:
       | This is really neat. I imagine this will be an entryway for LLMs
       | to creep into more classic data science / ML workloads.
        
       | bentcorner wrote:
       | I don't want to sound too dismissive of someone's hard work but I
       | was kind of hoping for something more sophisticated than showing
       | an LLM the image of a plot. Using the article's example, I would
       | be interested in understanding causes (or even just correlations)
       | of near falls - is it old people, or people who didn't take their
       | vitamins, or people who recently had an illness, etc.? What's the
       | best way of discovering these that isn't me slicing the data by X
       | and looking at the plot.
        
       | levocardia wrote:
       | To me, this basically says "LLMs aren't pre-trained on enough 1D
       | timeseries data" - there's a classic technique in time series
       | analysis where you just do a wavelet or FFT on the time series
       | and feed it into a convnet as an image, leveraging the massive
       | pre-training on, e.g. ImageNet. This "shouldn't" be the best way
       | to do it, since a giant network should learn a better internal
       | representation than something static like FFT or a wavelet
       | transform. But there's no 1D equivalent of ImageNet so it still
       | often works better than a 1D ConvNet trained from scratch.
       | 
       | Same applies here. An LLM trained on tons of time series should
       | be able to create its own internal representation that's much
       | more effective than looking at a static plot, since plots can't
       | represent patterns at all scales (indeed, a human plotting to
       | explore data will zoom in, zoom out, transform the timeseries,
       | etc.). But since LLMs don't have enough 1D timeseries
       | pretraining, the plot-as-image technique leverages the massive
       | amount of image pre-training.
        
       | richrichie wrote:
       | Kelly et al took similar approach to trading. The idea was that
       | human traders looked at charts on the screen and "intuitively"
       | made trading decisions.
       | 
       | https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3756587
        
       ___________________________________________________________________
       (page generated 2024-12-02 23:00 UTC)