Post 9u91vFNqZ7LCsil37w by djoerd@idf.social
(DIR) More posts by djoerd@idf.social
(DIR) Post #9u6TZJJTMsNA10X58a by djoerd@idf.social
2020-04-16T14:09:42Z
0 likes, 0 repeats
Really inspiring keynote talk by Jamie Callan: What an exciting time to be an Information Retrieval researcher!
(DIR) Post #9u6TppIUDg1hqTwklc by djoerd@idf.social
2020-04-16T14:12:43Z
0 likes, 0 repeats
Jamie point out that we as an IR community are really benefiting from the leading work in NLP and ML. "We need to up our game"!
(DIR) Post #9u6U8J2qMCRlJpRRUO by djoerd@idf.social
2020-04-16T14:16:03Z
0 likes, 0 repeats
Question by Fabrizio Sebastiani: One big difference between IR and NLP is that the NLP community has all its publications open access! All our papers are behind paywalls. Is this a factor in the success of NLP?
(DIR) Post #9u6dZxopldbLBR9TNo by tfidf@fosstodon.org
2020-04-16T16:01:53Z
0 likes, 0 repeats
@djoerd ist there a recording of the keynote talk?
(DIR) Post #9u6mhoJJtz67b9EBVI by djoerd@idf.social
2020-04-16T17:44:09Z
0 likes, 0 repeats
@tfidf There was a live stream, which will be available here, I guess: https://www.youtube.com/channel/UCCL4B2Zm7YvAte2m4rFmx7A
(DIR) Post #9u6uLDujLHsByjT0Fs by arjen@idf.social
2020-04-16T19:09:41Z
0 likes, 0 repeats
@djoerd Jamie's answer was no. But "maybe" would've been mine.
(DIR) Post #9u91vFNqZ7LCsil37w by djoerd@idf.social
2020-04-17T19:44:05Z
0 likes, 0 repeats
@arjen Jamie's answer was "no, because we all put our publications on our personal home pages and we put our code online" (if I remember well, but there's always Youtube)
(DIR) Post #9u92KerkxY6T9Zrfua by djoerd@idf.social
2020-04-17T19:48:40Z
0 likes, 0 repeats
@arjen it's easy saying no if you're priviliged to work at CMU (or Radboud U. for that matter)
(DIR) Post #9u9DOcR12XIMGcRqsK by arjen@idf.social
2020-04-17T21:52:36Z
0 likes, 0 repeats
@djoerd I think he said no because he does not want to see it as an explanation why NLP/ML had more impact/development recent years. I also do not believe that the publication model explains that. NLP community is way larger than ours, IR is niche, and hard to study well (so less attractive if you need many top publications early career).
(DIR) Post #9uCHW6EJdvy5zdD5sm by djoerd@idf.social
2020-04-19T09:22:55Z
0 likes, 0 repeats
@arjen IR is not a niche, we're one of the oldest SIGs of ACM and companies doing IR are among the most valuable companies in the world!! When we were PhD students, and even when we organized SIGIR in 2007, the ACL conference was roughly the same size as SIGIR. NoW, ACL is almost 3 times bigger than SIGIR. We need to up our game!
(DIR) Post #9uCIIilreRfbQU5Rce by djoerd@idf.social
2020-04-19T09:31:44Z
0 likes, 0 repeats
@arjen Whether open access is the main explanation for our bad performance, compared to NLP and ML, we don't really know..., I can think of other explanations, but IR being a niche is not one of them, (sounds more like a self-fulfilling profecy, or an explanation after the fact)
(DIR) Post #9uCIyx96b1pYAh5yJk by arjen@idf.social
2020-04-19T09:39:21Z
0 likes, 0 repeats
@djoerd I beg to disagree. IR research properly done is - inherently - way more expensive to carry out, you cannot reuse corpora like you can in the other areas you mention. The reason is that in IR every request has a unique context, one specific user with their knowledge. While the properties of language are the same for all those users, **their** infoneeds are different. But it is fine to disagree :-) Open access would be great, we agree there!
(DIR) Post #9uD9X48YIz2HJfQM08 by djoerd@idf.social
2020-04-19T19:28:11Z
0 likes, 0 repeats
@arjen We agree, but for the sake of the argument: We *do* reuse corpora all the time. IR researchers practically invented the reusable test collection (maybe because it is a challenge), see Cleverdon's work in the 1960's, the Sparck-Jones & Van Rijsbergen "Report on the Need for and Provision for an ‘Ideal' Information Retrieval Test Collection" in 1975 and TREC that started in 1992.
(DIR) Post #9uEcDvKPvHeCK79sAK by arjen@idf.social
2020-04-20T12:24:22Z
0 likes, 0 repeats
@djoerd again true!But! :-)Progress on interesting new questions requires corpora that are far more expensive to construct than, say, holding out one attribute from an existing source, and then predicting that attribute again. I realize that I am not doing all NLP research justice, take e.g. multi-document summarization; but then again, in the NLP community, that is a topic also "avoided", whereas it would actually be incredibly more useful than yet another sentiment analysis approach, say.
(DIR) Post #9uEfpmgcnj4QO7I5AW by djoerd@idf.social
2020-04-20T13:04:50Z
0 likes, 0 repeats
@arjen Fun fact: The reason that I ended up in IR was that "we" have great test collections! (remember I did my PhD in an NLP group, and my supervisor suggested ACL and Digital Libraries over SIGIR)
(DIR) Post #9uEgCYXvXZ6BC254e8 by djoerd@idf.social
2020-04-20T13:08:58Z
0 likes, 0 repeats
@arjen (that and the fact that my supervisor gave her students the benefit of the doubt, even if they had unusual opinions) <3