[HN Gopher] Why does a least squares fit appear to have a bias w...
       ___________________________________________________________________
        
       Why does a least squares fit appear to have a bias when applied to
       simple data?
        
       Author : azeemba
       Score  : 61 points
       Date   : 2026-01-04 20:25 UTC (2 hours ago)
        
 (HTM) web link (stats.stackexchange.com)
 (TXT) w3m dump (stats.stackexchange.com)
        
       | charlieyu1 wrote:
       | If you plot the regression line of y against x, and also x
       | against y, you would get two different lines.
       | 
       | I found it in the middle of teaching a stats class, and feel
       | embarrassed.
       | 
       | I guess normalising is one way to remove the bias.
        
       | dllu wrote:
       | You can think of it as: linear regression models only noise in y
       | and not x, whereas ellipse/eigenvector of the PCA models noise in
       | both x and y.
        
         | analog31 wrote:
         | That brings up an interesting issue, which is that many systems
         | do have more noise in y than in x. For instance, time series
         | data from an analog-to-digital converter, where time is based
         | on a crystal oscillator.
        
           | GardenLetter27 wrote:
           | This fact underlies a lot of causal inference.
        
       | sega_sai wrote:
       | The least squares and pca minimize different loss functions. One
       | is sum of squares of vertical(y) distances, another is is sum of
       | closest distances to the line. That introduces the differences.
        
         | ryang2718 wrote:
         | I find it helpful to view least as fitting the noise to a
         | Gaussian distribution.
        
           | LudwigNagasena wrote:
           | OLS estimator is the minimum-variance linear unbiased
           | estimator even without the assumption of Gaussian
           | distribution.
        
       | gpcz wrote:
       | You would probably get what you want with a Deming regression.
        
       ___________________________________________________________________
       (page generated 2026-01-04 23:00 UTC)