[HN Gopher] Is there a balance to be struck between simple hiera...
       ___________________________________________________________________
        
       Is there a balance to be struck between simple hierarchical models
       and
        
       Author : luu
       Score  : 36 points
       Date   : 2025-05-07 15:15 UTC (4 days ago)
        
 (HTM) web link (statmodeling.stat.columbia.edu)
 (TXT) w3m dump (statmodeling.stat.columbia.edu)
        
       | Onawa wrote:
       | Full Title: Is there a balance to be struck between simple
       | hierarchical models and more complex hierarchical models that
       | augment the simple frameworks with more modeled interactions when
       | analyzing real data?
        
       | a-dub wrote:
       | "When working on your particular problem, start with simple
       | comparisons and then fit more and more complicated models until
       | you have what you want."
       | 
       | sounds algorithmic...
        
         | mnky9800n wrote:
         | Yes and you can even build symbolic engines that do this for
         | you. I think the real question we must ask ourselves as data
         | scientists or statisticians or whatever is whether we believe
         | these data models represent the space of data fully or by
         | happenstance. And if by happenstance is it because the data
         | doesn't capture the underlying processes that produced the data
         | or are they uncapturable in this way and function approximators
         | like neural networks or gradient booster machines are better.
         | And is that because those function approximators capture
         | interactions between the driving processes that otherwise go
         | unseen or is it because those processes have fractional
         | dimensions that control their impact that are not captured by
         | data models. This all is summed up well by Leo Breimans two
         | cultures paper in my opinion. I have gone back and forth on
         | which "culture" is the correct representation of how processes
         | produce data. If you buy that only function approximators truly
         | capture the complexity of whatever processes you are observing
         | then you have to wonder why physics works so well. That's
         | because, at least in my opinion, from the statistical point of
         | view physics has spent centuries developing equations that are
         | linear combinations of variables that are essentially data
         | models according to Leo. I hope this opinion generates
         | discussion because I don't know what the answer is or if it
         | matters that there is one.
        
           | a-dub wrote:
           | seems to me that one approach is fueled by data and the other
           | is fueled by understanding. in the former, the observations
           | form a view of behavior which is then modeled with high
           | fidelity. in the latter, active inquiry, adversarial data
           | collection and careful reasoning produce simpler models of
           | hypothsized underlying processes that often prove to have
           | nearly perfect generalization.
           | 
           | the interesting future is probably the one where the former
           | produces new building blocks for the latter. (ie, the
           | computer generates new simple and easy to understand
           | constructs from which it explains previously not understood
           | or well modeled phenomena.)
        
       | joe_the_user wrote:
       | Well, my impression is that the statistic paradigm itself limits
       | the complexity of a model through it's basic aims and measures.
       | Especially, a statistical model aims to be an unbiased predictor
       | of a variable whereas machine learning/"AI" just aims for
       | prediction and doesn't care about bias in the sense of
       | statistics.
        
         | klysm wrote:
         | I think they have totally different goals typically. For
         | example, let's say we are doing a sampling procedure. How do
         | you estimate the sampling error? I'm not aware of a machine
         | learning technique that will help, but you can use Bayesian and
         | MCMC techniques
        
         | usgroup wrote:
         | I think this is accurate but mostly because statistical
         | modelling aims for interpretable parameters. That very strongly
         | regularises complexity.
        
       ___________________________________________________________________
       (page generated 2025-05-11 23:01 UTC)