Post AXtHOsEx1ZVbeVGz3Y by DanielBrockman@mastodon.world
 (DIR) More posts by DanielBrockman@mastodon.world
 (DIR) Post #AXsboLLLlCGV4xjD9M by tiago@social.skewed.de
       2023-07-20T08:20:10Z
       
       0 likes, 1 repeats
       
       There's an ideology in ML that says that when there's no “ground truth,” results must be obtained via inference, and hence are inherently suspicious, sensitive to arbitrary assumptions, and subject to (demonstrably far-fetched) no-free-lunch theorems.This always struck me as deeply unscientific.There's no “ground truth” in Physics, Biology, or any other empirical science.Yet, there's substantial progress using general inference and induction frameworks, Occam's razor, etc.The wikipedia definition states: “Ground truth is information that is known to be real or true, provided by direct observation and measurement (i.e. empirical evidence) as opposed to information provided by inference.” This reads as if written by an untrained scientist.There's no such thing as “direct observation”: Every measurement is indirect and contains uncertainty.Empirical evidence is *always* inferential.No empirical information is “known to be true”. There are always error bars.The concept of “ground truth” is poison.
       
 (DIR) Post #AXssFtoNJawl47SxPs by PausalZ@fediscience.org
       2023-07-20T11:24:23Z
       
       0 likes, 0 repeats
       
       @tiago But suppose we want to assess a new statistical method. We can simulate data with a known mechanism and get our 'ground truth'. Isn't the ideology you mention more of a distinction between the mechanisms we know (simulation) and the mechanisms of the world (inferential)?
       
 (DIR) Post #AXstqOlLogCTRE1H4C by tiago@social.skewed.de
       2023-07-20T11:42:14Z
       
       0 likes, 0 repeats
       
       @PausalZ The parameters you impose when generating artificial data from a model are not empirical information.Indeed, the *only* situation where it makes sense to talk about “ground truth” is when we generate the data ourselves, and then we can use this known information to investigate a method's consistency.However, in an empirical context there's never a “ground truth.” All information available about a real system is at some level inferential.In ML, the term “ground truth” is most often used in empirical contexts, where some of the data has been labeled in some way. But the labels are just more data, not the “truth”.
       
 (DIR) Post #AXsxVKNlJ7TlaVC6hk by PausalZ@fediscience.org
       2023-07-20T12:23:12Z
       
       0 likes, 0 repeats
       
       @tiago Gotcha! Thanks, I'm mostly use to seeing 'ground truth' in the statistics literature, which has ground truth correspond to us generating the data ourselves
       
 (DIR) Post #AXtHOsEx1ZVbeVGz3Y by DanielBrockman@mastodon.world
       2023-07-20T16:06:10Z
       
       0 likes, 0 repeats
       
       @tiago Good morning, Tiago.I look out my kitchen window this morning & see a bumblebee on the lavender. I turn to my browser, go to Mastodon, & see a post from Tiago. I assert ground truths: Tiago's post exists. A bumblebee exists. I infer "Tiago's post" wasn't written by Donald Trump. I infer I'm not looking at a clever 16K UHD representation of a bumblebee. I pass the Cartesian test: I think, so I exist. Both inference & empirics give us truth.
       
 (DIR) Post #AXtIZWXS9OVTUAZMCu by tiago@social.skewed.de
       2023-07-20T16:19:17Z
       
       0 likes, 0 repeats
       
       @DanielBrockman A person with schizophrenia will make similar assertions about things that don't exist.Maybe the bumblebee you saw was actually a wasp, but you didn't notice.Etc, etc.Uncertainties can be very small, and in practice there's little difference between an error probability of 10^{-100} and zero.But epistemologically we cannot do empirical science by assuming we can perform measurements with zero error.
       
 (DIR) Post #AXtUDrAvwRxMZJTFK4 by DanielBrockman@mastodon.world
       2023-07-20T18:29:50Z
       
       0 likes, 0 repeats
       
       @tiago quite right. we never escape uncertainty.
       
 (DIR) Post #AXtr3KLxPudWp2Zj0a by EduardoAltmann@aus.social
       2023-07-20T22:45:34Z
       
       0 likes, 0 repeats
       
       @tiago Indeed! I think that the root of this "ideology" is the naive view that there is access to "truth" (or data) without an underlying "theory" (or  model).
       
 (DIR) Post #AXuR1QzLNBNbAiOWX2 by tiago@social.skewed.de
       2023-07-21T05:28:41Z
       
       0 likes, 0 repeats
       
       @EduardoAltmann Precisely! There's no “model-free” science, since science *is* modelling, all the way down to the instruments we use to collect data.