[HN Gopher] Ask HN: Which recent research paper blew your mind?
       ___________________________________________________________________
        
       Ask HN: Which recent research paper blew your mind?
        
       .
        
       Author : froster
       Score  : 275 points
       Date   : 2023-07-24 12:34 UTC (10 hours ago)
        
       | d-- wrote:
       | "Enso: A Streaming Interface for NIC-Application Communication"
       | https://www.microsoft.com/en-us/research/uploads/prod/2023/0...
       | 
       | We've been using the same API to communicate with our NICs since
       | 1994. That API severely limits network throughput and latency. By
       | simply changing the API (no new NIC) you can get 6x higher
       | throughput in some apps and 43% lower latency.
       | 
       | Code runs on FPGA NIC only for now:
       | https://github.com/crossroadsfpga/enso
       | 
       | Won USENIX OSDI best paper award and best artifact award.
        
       | manvel_hn wrote:
       | Toolformer: Language Models Can Teach Themselves to Use Tools
       | https://arxiv.org/abs/2302.04761
       | 
       | Older one, but still very nice work.
        
         | jrmiii wrote:
         | I just posted the Voyager paper at exactly the same moment you
         | posted this - a lot of the same ideas.
         | https://voyager.minedojo.org/
        
       | javajosh wrote:
       | I think you meant "recently published paper" but others are
       | bringing up old stuff. I've always enjoyed Einstein's
       | (translated) papers. They are both precise and readable, and his
       | reputation for genius is well deserved!
       | 
       | This one is interesting, "On the Influence of Gravitation on the
       | Propagation of Light" you can read here:
       | https://einsteinpapers.press.princeton.edu/vol3-trans/393. This
       | was his initial stab at GR. It was a simple approach, later
       | abandoned, that considered light to slow down in a gravity well
       | rather than remaining constant and specifying that mass warps
       | spacetime. I suppose I like the idea that he pursued an idea that
       | didn't work, and was forced to do something far more complex.
       | It's kinda relatable.
        
         | DennisP wrote:
         | The book _Einstein 's Lost Key_ by German physicist Alexander
         | Unzicker is all about that idea and its history. Unzicker
         | argues that it's still a good idea, hasn't been falsified since
         | it makes much the same predictions as GR, and we should look
         | into it further.
        
       | jrmiii wrote:
       | I was impressed by the Voyager paper on a GPT-powered Minecraft
       | bot.
       | 
       | https://voyager.minedojo.org/
        
       | jeffbee wrote:
       | https://cseweb.ucsd.edu/~tullsen/halfandhalf.pdf
       | 
       | Half&Half: Demystifying Intel's Directional Branch Predictors for
       | Fast, Secure Partitioned Execution
        
         | 1letterunixname wrote:
         | Related:
         | https://www.cs.toronto.edu/syslab/pubs/demkea_context.pdf
        
       | cratermoon wrote:
       | On the Dangers of Stochastic Parrots: Can Language Models Be Too
       | Big? <https://dl.acm.org/doi/10.1145/3442188.3445922>
        
       | GregarianChild wrote:
       | Someone managed to GPU-accelerate program synthesis, a form of
       | symbolic ML. First time for ML that is not deep learning:
       | 
       | https://dl.acm.org/doi/10.1145/3591274
       | 
       | Deep learning took off precisely when the ImageNet paper dropped
       | around 2010. Before nobody believed that backprop can be GPU-
       | accelerated.
        
         | YeGoblynQueenne wrote:
         | That's cool and all, but the one thing that really made it
         | possible to train deep neural nets was the use of
         | backpropagation, and its polynomial time complexity.
         | 
         | By contrast, there are no known polynomial time algorithms for
         | program synthesis and the standard approach is to search some
         | large combinatorial space [1]. That's the case for all the
         | classical approaches: SMT, SAT, planning and scheduling, etc.
         | At the same time there's very powerful heuristics for all the
         | other classical problems that can solve many problem instances
         | efficiently.
         | 
         | ____________
         | 
         | [1] The one exception to this is Inductive Logic Programming,
         | i.e. the inductive synthesis of logic programs, for which we do
         | know a polynomial time algorithm (but that is my work so I'm
         | not pimping it here).
        
           | varranvar wrote:
           | I'm working on an inductive logic programming algorithm, and
           | from my understanding the search space for logic programs is
           | just as vast as other types of program synthesis. Do you have
           | any information you can share about the polynomial time
           | algorithm?
        
             | YeGoblynQueenne wrote:
             | Of course. It's published as an open-access paper here:
             | 
             | https://link.springer.com/article/10.1007/s10994-020-05945-
             | w
             | 
             | But I recommend the arxiv version where Springer couldn't
             | mess up my LaTex formatting:
             | 
             | https://arxiv.org/abs/2101.05050
             | 
             | (Not that my LaTex formatting is anything to write home
             | about!).
             | 
             | To clarify, the algorithm described in our paper doesn't
             | reduce the size of the search space for logic programs- it
             | sidesteps it.
             | 
             | Note I'm the corresponding author in the paper and my email
             | is in the Springer version (top of the page). I'm always
             | happy to answer questions about my work.
        
           | GregarianChild wrote:
           | > _backpropagation, and its polynomial time complexity_
           | 
           | How do you reconcile the NP-completeness result in [1] about
           | training neural networks with your claim?
           | 
           | [1] A. L. Blum, R. L. Rivest, Training a 3-Node Neural
           | Network is NP-Complete. https://proceedings.neurips.cc/paper/
           | 1988/file/3def184ad8f47...
        
         | d_burfoot wrote:
         | > ... 2010. Before nobody believed that backprop can be GPU-
         | accelerated.
         | 
         | When I was doing my master's in 2004-06, I talked to a guy
         | whose MSc thesis was about running NNs with GPUs. My thought
         | was: you're going to spend a TON of time fiddling with hacky
         | systems code like CUDA, to get basically a minor 2x or 4x
         | improvement in training time, for a type of ML algorithm that
         | wasn't even that useful: in that era the SVM was generally
         | considered to be superior to NNs.
         | 
         | So it wasn't that people thought it couldn't be done, it's that
         | nobody saw why this would be worthwhile. Nobody was going
         | around saying, "IF ONLY we could spend 20x more compute
         | training our NNs, then they would be amazingly powerful".
        
           | rch wrote:
           | I went to a talk on "general purpose GPU programming" at the
           | Colorado School of Mines around 2001 that covered exactly
           | that topic. It was very disappointing to have my interest in
           | FPGAs for this purpose be so entirely destroyed by a quirk of
           | graphics card design.
           | 
           | Hinton also addressed the contribution of hardware
           | performance advances to practical deep neural net
           | applications in his talks in the mid-2000s.
        
           | GregarianChild wrote:
           | Exactly.
           | 
           | It's easy to see in retrospect, but hard in prospect: the
           | original paper [1] on GPU acceleration of NNs reports a
           | measly 20x speedup. Assuming a bit of cherry-picking on the
           | author's side to make get the paper published, the 'real-
           | world speedup' will have been assumed by the readership to be
           | less. But this triggered a virtuos cycle of continuous
           | improvements at all levels that has been dubbed "winning the
           | hardware lottery" [2].
           | 
           | [1] K.-S. Oh, K. Jung, _GPU implementation of neural
           | networks._
           | 
           | [2] S. Hooker, _The Hardware Lottery._
           | https://arxiv.org/abs/2009.06489
        
         | claytonjy wrote:
         | > First time for ML that is not deep learning
         | 
         | What do you mean by this? Virtually all "classic" or "shallow"
         | ML can be GPU-accelerated, from linear regression to SVM to
         | GBM.
        
           | GregarianChild wrote:
           | Can you point me to papers with reproducible benchmarking
           | that achieves big speedups on those?
           | 
           | Modern GPUs are _GP_ -GPUs: where GP means _" general
           | purpose"_: you can run any code on GPGPUs. But if you want to
           | gain real speed-ups you will have to program in an awkward
           | style ("data parallel"). I am not aware of GPU acceleration
           | of the work-horses of symbolic AI, such as Prolog, or SMT
           | solving. There has been a lot of work on running SAT-solvers
           | on GPUs, but I don't think this has really succeeded so far.
        
             | claytonjy wrote:
             | I think we're conflating two things: shallow/classic ML is
             | not symbolic AI. I'm not sure "ML" even encompasses
             | anything "symbolic"; I see symbolic AI and ML as subfields
             | with little overlap.
             | 
             | I'm not saying symbolic AI has been GPU accelerated in the
             | past, but that non-deep ML has been.
        
               | saltcured wrote:
               | Back when I took AI courses in the early 90s, ML was
               | anything that was trained by data. It did not refer
               | exclusively to dense numerical or statistical methods. It
               | included decision trees, which I think of as being closer
               | to the symbolic camp...
        
               | GregarianChild wrote:
               | There is no agreement on the exact meaning of ML and AI.
               | They are often used interchangeably. And for good reason,
               | because it's all about getting computers to learn. We
               | should not squabble about semantics.
               | 
               | Can you point towards papers that report substantial GPU
               | acceleration on what you call "non-deep ML"?
        
               | chelmney wrote:
               | Let's not forget Gaussian processes:
               | https://gpytorch.ai/, https://www.kernel-
               | operations.io/keops/index.html Some of the work I've been
               | doing in this field only became feasible in a reasonable
               | amount of time due to the use of GPUs.
        
               | claytonjy wrote:
               | You're certainly right that AI/ML is often used
               | interchangeably, though I think the distinction is pretty
               | clear among practitioners: AI is the big circle, ML is a
               | subset of AI, deep learning is a subset of ML. Symbolic
               | AI is also a subset of AI, though I'm not familiar enough
               | with it to say how much it interects the others.
               | 
               | So ML _is_ AI, but just a small part of it.
               | 
               | As for papers, here's one showing GPU speedups for the 3
               | big GBM packages: https://arxiv.org/pdf/1809.04559.pdf.
               | Their setup shows a 3-7x speedup for XGBoost.
        
             | fancyfredbot wrote:
             | The parent comment is talking about SVM, which is not a
             | form of symbolic AI. SVM and other kernel based algorithms
             | contain naturally data parallel steps like summing
             | f(z-x[i]) for some kernel function f and all sample data
             | x[i]. These will work great on GPU once you have a few
             | thousand data points.
        
           | a-dub wrote:
           | computing a single matrix right divide is probably faster
           | done in place with the cpu using vector math instructions.
        
         | naillo wrote:
         | How is program synthesis the same as regular expression search?
         | Honest question
        
           | lmeyerov wrote:
           | This seems like an early paper and agreed with the
           | consternation.
           | 
           | The paper, in a modern context and based solely on the
           | abstract and having been in the community, is chipping at the
           | "uninteresting" part of the problem. Around that time,
           | program synthesis started switching to SMT (satisfiability
           | modulo theory) methods, meaning basically a super powerful &
           | general SAT solver for the broad search ("write a wild python
           | program") and then, for specialized subdomains, have a good
           | way to call out to optimized domain solvers ("write a tiny
           | bit of floating point math here"). The paper would solve what
           | the regex callout looks like.. which is specialized. We can
           | argue regex is one of the most minimal viable steps towards
           | moving to general programming on GPUs. Except as a person who
           | does SIMD & GPU computing, optimizing compute over finite
           | automata is not general nor representative and I don't expect
           | to change my thinking much about the broader computational
           | classes. To be fair to the authors... back then, synthesizing
           | regex & sql were hard in practice even for boring cases.
           | 
           | Separately, nowadays synthesis has shifted to neural
           | (copilot, gpt), and more interesting to me, neurosymbolic in
           | R&D land. We're doing a lot of (simple) neurosymbolic in
           | louie.ai, and I'm excited if/when we can get the SMT solver
           | side in. Making GPT call Z3 & Coq were some of the first
           | programs I tried with it :) Till then, there's a lot more
           | interesting low-hanging fruit from the AI/ML side vs solvers,
           | but feels like just a matter of time.
        
             | GregarianChild wrote:
             | The paper claims that there is no neural / deep learning
             | based solver that performs well on regular expression
             | inference.
             | 
             | Calls to Coq and Z3 will be _very_ slow and not competitive
             | with GPU compute.
        
           | GregarianChild wrote:
           | Good question. All supervised learning is a form of search
           | with three components:
           | 
           | - _Specification:_ what are you are looking for?
           | 
           | - _Search space:_ were are you looking?
           | 
           | - _Search mechanism:_ how are you going through the search
           | space?
           | 
           | Program synthesis is simply learning where the search space
           | is syntax. In deep learning, taking the ImageNet paper as an
           | example, the specification a bunch of photos with
           | annotations, the search space is multi-variate real functions
           | (encoded as matrix of floats) and the search mechanisms is
           | gradient descent (implemented as backprop) with a loss
           | function.
           | 
           | I think this paper uses regular expressions an example of how
           | to search fast over syntax. It claims not to be tied to
           | regular expressions.
        
             | abstractcontrol wrote:
             | https://deepai.org/publication/search-based-regular-
             | expressi...
             | 
             | Here is the full text.
             | 
             | Regexps aren't even Turing complete as far as I know, if
             | whatever they have in their paper works for arbitrary
             | programs it would be shocking. I'll give it a read.
             | 
             | *Edit*: The algorithm in the paper is a DP like algorithm
             | for building regexes. They use a matrix, and it has all the
             | potential strings to be checked on one axis, and all the
             | potential regex programs on the other axis, and in-between
             | values (the actual matrix values) are booleans saying
             | whether the string matches the program. The algorithm
             | builds the matrix iteratively.
             | 
             | I haven't understood how regex evaluation is done, probably
             | directly, but obviously this algorithm is only for checking
             | whether a particular regex program matches an output rather
             | than general purpose synthesis.
             | 
             | We'll have to wait for AI chips to really scale genetic
             | programming, GPUs won't cut it.
        
               | GregarianChild wrote:
               | > _genetic programming_
               | 
               | There is no magic in GP. It is just another form of
               | searching the space of programs, i.e. program synthesis.
               | The search mechanism is a local, stochastic search, known
               | to be especially inefficient (for example you may hit the
               | same program multiple times). What's good about GP is how
               | simple it is, so it's a good starting point.
        
       | dan-g wrote:
       | Generative Agents: Interactive Simulacra of Human Behavior[1].
       | Make sure to check out the recorded demo!
       | 
       | [1] https://arxiv.org/abs/2304.03442
        
       | maurits wrote:
       | LoRA: Low-Rank Adaptation of Large Language Models [1]
       | 
       | [1]: https://arxiv.org/abs/2106.09685
        
         | ComputerGuru wrote:
         | Off topic: LoRa is forever low-power long-range communication
         | for me (as in LoRaWAN).
        
           | awestroke wrote:
           | LoRA != LoRa
        
       | masfuerte wrote:
       | Grid-free Monte Carlo for PDEs with spatially varying
       | coefficients.
       | https://cs.dartmouth.edu/wjarosz/publications/sawhneyseyb22g...
        
       | aaron695 wrote:
       | RNA demethylation increases the yield and biomass of rice and
       | potato plants in field trials (2021)
       | 
       | https://www.nature.com/articles/s41587-021-00982-9
        
       | w-m wrote:
       | Integral Neural Networks (CVPR 2023 Award Candidate), a nifty way
       | of building resizable networks.
       | 
       | My understanding of this work: A forward pass for a (fully-
       | connected) layer of a neural network is just a dot product of the
       | layer input with the layer weights, followed by some activation
       | function. Both the input and the weights are vectors of the same,
       | fixed size.
       | 
       | Let's imagine that the discrete values that form these vectors
       | happen to be samples of two different continuous univariate
       | functions. Then we can view the dot product as an approximation
       | to the value of integrating the multiplication of the two
       | continuous functions.
       | 
       | Now instead of storing the weights of our network, we store some
       | values from which we can reconstruct a continuous function, and
       | then sample it where we want (in this case some trainable
       | interpolation nodes, which are convoluted with a cubic kernel).
       | This gives us the option to sample different-sized networks, but
       | they are all performing (an approximation to) the same operation.
       | After training with samples at different resolutions, you can
       | freely pick your network size at inference time.
       | 
       | You can also take pretrained networks, reorder the weights to
       | make the functions as smooth as possible, and then compress the
       | network, by downsampling. In their experiments, the networks lose
       | much less accuracy when being downsampled, compared to common
       | pruning approaches.
       | 
       | Paper:
       | https://openaccess.thecvf.com/content/CVPR2023/papers/Solods...
       | 
       | Code: https://github.com/TheStageAI/TorchIntegral
        
         | ninjaa wrote:
         | smart
        
         | dicroce wrote:
         | Damn. It's like jpeg for neural networks.
        
         | smaddox wrote:
         | Nice. I was wondering if something like this is possible a few
         | days ago. The next step would be somehow extending the
         | discrete->continuous concept to layers.
        
           | smaddox wrote:
           | Ahh, I guess that's been done, too: https://proceedings.neuri
           | ps.cc/paper_files/paper/2018/file/6...
           | 
           | Now we just need an iterative solver over both the structure
           | and the "weights", and we get both architecture search and
           | training at the same time
        
           | diracs_stache wrote:
           | After finally learning some complex integrals/residue theory
           | and seeing the connection to continuous and discrete signal
           | processing I was very happy that the "magic trick"
           | disappeared, your comment has me interested in pulling the
           | string farther. Thanks!
        
         | numbers_guy wrote:
         | Going just by your description this sounds like they are doing
         | operator learning. It's actually a very old idea. The proof
         | that started operator learning is from 1988 I believe.
         | Mathematicians have been playing around with the idea since
         | 2016 at least.
        
           | w-m wrote:
           | Indeed, this seems closely related, thanks for the pointer!
           | 
           | Unfortunately I'm not deep enough into the topic to
           | understand what their contribution to the theory part of it
           | is. (they have some Supplementary Material in [INN Supp]). In
           | the discussion of the Integral Neural Networks (INN) paper,
           | there's this paragraph about an operator learning
           | publication:
           | 
           | "In [24] the authors proposed deep neural networks with
           | layers defined as functional operators. Such networks are
           | designed for learning PDE solution operators, and its layers
           | are continuously parameterized by MLPs only along the kernel
           | dimensions. A re-discretization was investigated in terms of
           | training on smaller data resolution and testing on higher
           | input resolution. However, the proposed framework in [24]
           | does not include continuous connections between filters and
           | channels dimensions."
           | 
           | Also the weight permutation to perform the resampling on
           | pretrained networks in INNs seems to be novel? And I guess it
           | doesn't hurt that they're bringing new eyeballs to the topic,
           | by providing examples of common networks and a PyTorch
           | implementation.
           | 
           | [INN Supp]: https://openaccess.thecvf.com/content/CVPR2023/su
           | pplemental/...
           | 
           | [24]: Zongyi Li Nikola Kovachki. Neural operator: Graph
           | kernel network for partial differential equations. arXiv
           | preprint arXiv:2003.03485, 2020,
           | https://arxiv.org/abs/2003.03485
        
         | llaolleh wrote:
         | Supercool.
        
       | snarfed wrote:
       | Not recent but legendary: "Latency Lags Bandwidth" David
       | Patterson
       | http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.115...
       | 
       | Recent:
       | 
       | "How to Hack the Simulation?" Roman Yampolskiy
       | https://www.researchgate.net/publication/364811408_How_to_Ha...
       | 
       | "On the Computational Practicality of Private Information
       | Retrieval" Radu Sion, Bogdan Carbunar
       | https://zxr.io/research/sion2007pir.pdf
       | 
       | (via "Explained from scratch: private information retrieval using
       | homomorphic encryption," https://blintzbase.com/posts/pir-and-
       | fhe-from-scratch/ )
        
       | msravi wrote:
       | Error Bounds for Convolutional Codes and an Asymptotically
       | Optimum Decoding Algorithm [0]
       | 
       | Also known as the Viterbi algorithm. Every digital communication
       | device in existence today most likely has an implementation of
       | it.
       | 
       | Later proved optimal by Forney [1]
       | 
       | 0. https://www.essrl.wustl.edu/~jao/itrg/viterbi.pdf
       | 
       | 1.
       | https://www2.isye.gatech.edu/~yxie77/ece587/viterbi_algorith...
        
       | philipkglass wrote:
       | "Liquid solution centrifugation for safe, scalable, and efficient
       | isotope separation"
       | 
       | https://www.science.org/doi/10.1126/sciadv.adg8993
       | 
       | The authors show that a biological type laboratory
       | ultracentrifuge can efficiently function as a near-universal
       | isotope separator. Any element that can be dissolved as a salt in
       | water -- the entire periodic table, excepting the noble gases --
       | can be enriched according to its relative mass. This can reduce
       | the cost of refining certain isotopes like calcium-48 by orders
       | of magnitude compared to the previous best techniques.
       | 
       | Left unsaid, but implied by its universality: the new technique
       | is also a new approach to producing enriched fissile materials
       | for nuclear reactors and weapons. It requires less chemical
       | engineering sophistication than current processes which require
       | production and handling of gaseous uranium hexafluoride.
        
         | NoMoreNicksLeft wrote:
         | Surely something about the technique would make it impractical
         | for enriching uranium though... you're not going to be able to
         | produce kilograms of 20% U235 with this are you?
        
           | nukeman wrote:
           | Hypothetically, if the technique can enrich U, Pu, or other
           | fissile isotopes in decent quantities, you run into other
           | issues. Stirring an aqueous enriched uranium solution is a
           | great way to have a criticality accident (cf.
           | https://en.wikipedia.org/wiki/Criticality_accident). That's
           | not to say it's impossible to manage, just difficult. UF6,
           | the current compound of choice for enrichment, is in the gas
           | phase during the process.
        
             | Iwan-Zotow wrote:
             | yeah, especially if water ends up enriched in deuterium as
             | well
        
           | philipkglass wrote:
           | Not with off-the-shelf biological centrifuges, no. I don't
           | know enough about centrifuge design to judge how easy/hard it
           | is to build a 60,000 G centrifuge that can handle liters
           | instead of milliliters of liquid per cavity.
           | 
           | The other even more exotic possibility is to use something
           | like this to enrich ordinary reactor grade plutonium to
           | weapons grade plutonium-239. The amount of mass to process
           | with plutonium is orders of magnitude less because spent fuel
           | plutonium is already more than 50% Pu-239, versus 0.7% U-235
           | in natural uranium, and a bare sphere critical mass of Pu-239
           | is only 10 kg vs 52 kg for U-235:
           | 
           | https://en.wikipedia.org/wiki/Critical_mass#Critical_mass_of.
           | ..
           | 
           | The United States considered enriching waste fuel plutonium
           | to weapons grade in the 1980s, when it contemplated another
           | big nuclear weapons buildup against the USSR, but the laser
           | based separation technology to be used was much more
           | complicated than centrifuge separation. The project ended
           | shortly after the USSR dissolved. It was called the Special
           | Isotope Separation Project.
           | 
           | The 1988 environmental impact statement for the project gives
           | some background information:
           | 
           | https://www.energy.gov/sites/prod/files/2015/06/f24/EIS-0136.
           | ..
        
       | the_snooze wrote:
       | "Blue Is the New Black (Market): Privacy Leaks and Re-
       | Victimization from Police-Auctioned Cellphones"
       | 
       | https://krebsonsecurity.com/2023/05/re-victimization-from-po...
       | 
       | Researchers bought up a bunch of seized phones from police
       | auction sites and found about 25% of them were trivially
       | unlockable and still held sensitive data about suspects _and
       | victims_.
        
         | paxys wrote:
         | Is this really "mind blowing"? Knowing the competency of the
         | average police department, I'd consider it more par for the
         | course.
        
           | the_snooze wrote:
           | The result is obvious, but the question is demonstrably not.
           | Good researchers know how to ask interesting questions that
           | no one had bothered to ask before. Seeing clever work like
           | this makes me reflect and continually ask myself "What cool
           | angles am I missing?"
        
           | jjoonathan wrote:
           | Novelty is overrated. Finding proof and pushing for
           | accountability is so much more important.
        
             | klabb3 wrote:
             | Isn't novelty also arbitrary and subjective? I recall when
             | everyone seemed to think there was novelty in applying ML
             | to X, where X was something very specific. "Look, it works
             | here too!" I doubt that's considered novel now.
        
       | freedude wrote:
       | An analysis of studies pertaining to masks in Morbidity and
       | Mortality Weekly Report: Characteristics and quality of all
       | studies from 1978 to 2023
       | 
       | "0/77 were randomized studies."
       | 
       | https://www.medrxiv.org/content/10.1101/2023.07.07.23292338v...
       | 
       | Here is a pdf.
       | 
       | https://www.medrxiv.org/content/10.1101/2023.07.07.23292338v...
        
       | dooraven wrote:
       | Can Language Models Teach Weaker Agents? Teacher Explanations
       | Improve Students via Theory of Mind
       | 
       | https://arxiv.org/abs/2306.09299
       | 
       | TokenFlow: Consistent Diffusion Features for Consistent Video
       | Editing
       | 
       | https://huggingface.co/papers/2307.10373
       | 
       | Need to see code for second one.
        
       | danesparza wrote:
       | Not all of these are research papers. But all are fairly recent.
       | 
       | Gene linked to long COVID found in analysis of thousands of
       | patients https://www.nature.com/articles/d41586-023-02269-2
       | 
       | Surfactants safely take down mosquitoes without using
       | insecticides https://newatlas.com/science/surfactants-safely-
       | take-down-mo...
       | 
       | This is what our Milky Way galaxy looks like when viewed with
       | neutrinos https://arstechnica.com/science/2023/06/ghost-
       | particles-have...
        
         | tandr wrote:
         | >Surfactants safely take down mosquitoes without using
         | insecticides https://newatlas.com/science/surfactants-safely-
         | take-down-mo...
         | 
         | Curious - what would happen to a spider or an ant that ate a
         | couple of mosquitoes with that spice on top? Will they also
         | suffocate?
        
       | [deleted]
        
       | Uptrenda wrote:
       | I find the most interesting papers I read all come from the same
       | place: The National Library of Medicine https://www.nlm.nih.gov/
       | 
       | Here's some recent papers I liked:
       | 
       | - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8413749/ --
       | Lithium is used as a mood stabilizer for bipolar and in other
       | disorders. The form of Lithium used in psychiatry is Lithium
       | Carbonate. But other forms also exist. As a supplement: there is
       | Lithium Orotate which some people use to help them sleep, deal
       | with stress, and so on. This paper puts forwards the idea that
       | Lithium Orotate is preferable to Lithium Carbonate due to lower
       | quantities being needed for the same therapeutic results.
       | Resulting in less side-effects.
       | 
       | - https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1525098/ -- In
       | bipolar disorder its known that there are abnormalities in the
       | presence of brain derived neurotrophic growth factor (BDNF.)
       | What's interesting about this is treatments for bipolar help to
       | increase BDNF which may be something of interest to those who are
       | into nootropics.
       | 
       | The papers on this site are honestly some of the best written,
       | in-depth, and accessible works I've come across anywhere. There's
       | enough information here to live a better life if you're willing
       | to sift through papers. No joke.
        
         | billylo wrote:
         | I wonder if Google's Med-PaLM2 use those for training.
         | 
         | Best content + quality LLM + quality verifications => Useful
         | health advice.
        
       | carapace wrote:
       | "Cyclic Combinational Circuits", by Marc D. Riedel
       | 
       | > we present theoretical justification for the claim that the
       | optimal form of some [combinational] circuits requires cyclic
       | topologies. We exhibit families of cyclic circuits that are
       | optimal in the number of gates, and we prove lower bounds on the
       | size of equivalent acyclic circuits.
       | 
       | http://www.mriedel.ece.umn.edu/wiki/images/7/7a/Riedel_Cycli...
        
       | di4na wrote:
       | All the stuff coming out of the Koka and Effekt development.
       | 
       | In particular last week i read their FBiP2 paper
       | https://www.microsoft.com/en-us/research/uploads/prod/2023/0...
        
         | anfelor wrote:
         | Thanks, I am surprised and delighted to see our paper here! We
         | also wrote a follow-up recently, which you may enjoy:
         | https://www.microsoft.com/en-us/research/uploads/prod/2023/0...
        
           | di4na wrote:
           | Oh nice thanks!
           | 
           | I am slooooowwly working on my own language that is planned
           | to use work from both you and the Effekt team, so i may send
           | you all an email in the future when i start having questions.
        
             | anfelor wrote:
             | Sure, it would be great to chat! If you are interested in
             | contributing to Koka at some point, there might also be
             | opportunities there :)
        
         | afroisalreadyin wrote:
         | When I google for "koka effekt" I get pages in German about the
         | effects of cocain. They should maybe reconsider the "Koka" part
         | of the project name?
        
           | anfelor wrote:
           | These are two different languages, see https://effekt-
           | lang.org/ and https://koka-lang.github.io/koka/doc/index.html
           | -- although I believe that the FP^2 work mentioned by OP
           | currently only works in Koka (and perhaps soon in Lean;
           | https://leanprover.github.io/)
        
             | di4na wrote:
             | Indeed.
             | 
             | That said i am slowly writing my own language that will
             | probably use ideas from both :D
        
       | dargscisyhp wrote:
       | I thought the AlphaZero paper was pretty cool:
       | https://arxiv.org/abs/1712.01815
       | 
       | Not only did we get a whole new type of Chess engine, it was also
       | interesting to see how the engine thought of different openings
       | at various stages in its training. For instance, the Caro-Kann,
       | which is my weapon of choice, was favored quite heavily by it for
       | several hours and then seemingly rejected (perhaps it even
       | refuted it?!) near the end.
        
         | not-my-account wrote:
         | totally! MuZero is my favourite[1]
         | 
         | The super cool thing about MuZero is that it learns the
         | dynamics of the problem, i.e. you don't have to give it the
         | rules of the game, which makes the algorithm very general. For
         | example, DeepMind threw MuZero at video compression and found
         | that it can reduce video sizes by 6.28% (massive for something
         | like YouTube)[2][3].
         | 
         | Curious if anyone else knows examples of MuZero being deployed
         | outside of toy examples?
         | 
         | [1] https://arxiv.org/pdf/1911.08265.pdf [2]
         | https://arxiv.org/pdf/2202.06626.pdf [3]
         | https://www.deepmind.com/blog/muzeros-first-step-from-resear...
         | 
         | (edit s/Google/DeepMind)
        
       | ranprieur wrote:
       | Placebo Effect Grows in U.S., Thwarting Development of
       | Painkillers
       | 
       | https://www.scientificamerican.com/article/placebo-effect-gr...
       | 
       | The most interesting thing is that "placebo responses are rising
       | only in the United States."
        
         | 1letterunixname wrote:
         | I wonder if there is any correlation with either political or
         | socio-political volatility, e.g., bread and circuses,
         | conspiracy theories, cults, political dysfunction, and an
         | inability to double-blind test painkillers.
        
       | intended wrote:
       | Content Moderation / Trust and Safety person
       | 
       | Open AI's como paper, A Holistic Approach to Undesired Content
       | Detection in the Real World.
       | 
       | https://arxiv.org/pdf/2208.03274.pdf
       | 
       | Lots of interesting facts are strewn around the paper.
       | 
       | -----
       | 
       | The first paper that squarely talked about the language resource
       | gap in CS/ML. Before this came out, it was hard to explain just
       | how stark the gap between English and other languages was.
       | 
       | Lost in Translation: Large Language Models in Non-English Content
       | Analysis
       | 
       | https://cdt.org/insights/lost-in-translation-large-language-...
       | 
       | ----
       | 
       | This paper gets in for the title:
       | 
       | "I run the world's largest historical outreach project and it's
       | on a cesspool of a website." Moderating a public scholarship site
       | on Reddit: A case study of r/AskHistorians
       | 
       | https://drum.lib.umd.edu/bitstream/handle/1903/25576/CSCW_Pa...
       | 
       | ----
       | 
       | This was the first paper I ended up saving on online
       | misinformation. The early attempts to find solutions.
       | 
       | The Spreading of Misinformation online,
       | https://www.pnas.org/doi/10.1073/pnas.1517441113
       | 
       | What I liked here was the illustration of how messages cascade
       | differently based on the networks the message is traveling
       | through.
        
       | Jalad wrote:
       | "Bounding data races in space and time" was an interesting one I
       | saw recently! It's discussing the memory models of programming
       | languages, and how they can fail pretty horribly when data races
       | occur, and then talks about ways to avoid those. OCaml's
       | multicore support is based on this work, meaning it's memory
       | safety guarantees in when data races occur are pretty interesting
       | 
       | https://kcsrk.info/papers/pldi18-memory.pdf
       | 
       | https://youtube.com/watch?v=eXXzUzt_nAY
        
       | philipkglass wrote:
       | "Co-cultivation enhanced microbial protein production based on
       | autotrophic nitrogen-fixing hydrogen-oxidizing bacteria"
       | 
       | https://www.sciencedirect.com/science/article/abs/pii/S13858...
       | 
       | Certain bacteria can directly assimilate a mixture of hydrogen,
       | carbon dioxide, and nitrogen to produce protein. You could
       | consider it an alternative to bacterial nitrogen fixation in root
       | nodules with much higher productivity. Or you could consider it
       | an alternative to the Haber-Bosch process with much milder
       | reaction conditions -- ambient temperature and pressure. It's a
       | way to turn intermittent electricity into protein with simple,
       | robust equipment. I wouldn't be surprised if this or a related
       | development ultimately supplants much of the current demand for
       | synthetic nitrogen fertilizers.
        
         | hirundo wrote:
         | We hear that eating vegetables is more efficient in ecological
         | footprint than eating meat, since it cuts out the middle man.
         | Is it yet more efficient to cut out the plants and get dietary
         | protein from the bacteria that feed them?
        
           | philipkglass wrote:
           | Possibly, yes. This is the dream of so called "single cell
           | protein" production. One type of SCP has been sold for years
           | under the brand name Quorn (derived from a fungus rather than
           | bacteria).
           | 
           | Bacterial protein may trigger allergic reactions in people
           | and bacterial biomass is purine-rich which can also be a
           | problem for people prone to gout. It's possible that cell
           | engineering, directed evolutionary selection, or additional
           | post-growth processing can minimize these problems.
           | 
           | I personally think that the more likely path is using fast-
           | growing bacteria as feed for animal agriculture or
           | aquaculture. Solar panels are so efficient at sunlight
           | conversion compared to plants that you could farm salmon
           | protein starting from bacterial pellets grown on solar
           | derived hydrogen with per-hectare productivity comparable to
           | conventionally farming soy beans. But the solar farm can go
           | on saline, dry, contaminated, or otherwise agriculturally
           | useless land. And salmon has slightly greater nutritional
           | value than soy protein plus significantly greater market
           | value.
        
           | cratermoon wrote:
           | Plants use water, sunlight, and organic chemicals. In plants,
           | protein is made from soil nitrates. Bacteria don't feed the
           | plants, but they can fix nitrogen from the air into the soil.
           | If we really wanted to cut out the middleman, we'd turn the
           | nitrogen we breathe directly into proteins. For now, we have
           | to eat to obtain the essential amino acidd, Histidine;
           | Isoleucine; Leucine; Lysine; Methionine; Phenylalanine;
           | Threonine; Tryptophan; and Valine
        
       | gabitoju wrote:
       | "C-Store: A Column-oriented DBMS":
       | https://web.stanford.edu/class/cs345d-01/rl/cstore.pdf
       | 
       | By among others, the great Mike Stonebraker.
        
       | the-mitr wrote:
       | More is Different by P. W. Anderson (1972)'arguing that "at each
       | level of complexity entirely new properties appear" -- that is,
       | although, for example, chemistry is subject to the laws of
       | physics, we cannot infer the field of chemistry from our
       | knowledge of physics.' The paper https://cse-
       | robotics.engr.tamu.edu/dshell/cs689/papers/ander...
       | 
       | Also its impact
       | https://www.nature.com/articles/s42254-022-00483-x
        
         | codethief wrote:
         | > that is, although, for example, chemistry is subject to the
         | laws of physics, we cannot infer the field of chemistry from
         | our knowledge of physics
         | 
         | I mean we _could_ , with infinite computing power and enough
         | time to look into every interesting phenomenon (and to evaluate
         | the corresponding multi-particle Schrodinger equation
         | numerically) but there are simply too many such phenomena,
         | Schrodinger equations are tough to solve, and quantum mechanics
         | is also not a great level of abstraction for the reason you
         | mentioned.
        
       | edent wrote:
       | "Overview of SHARD: A System for Highly Available Replicated
       | Data" it's the first paper to introduce the concept of database
       | sharding. It was published in 1988 by the Computer Corporation of
       | America.
       | 
       | It is referenced hundreds of times in many classic papers.
       | 
       | But, here's the thing. _It doesn 't exist_.
       | 
       | Everyone cites Sarin, DeWitt & Rosenb[e|u]rg's paper but none
       | have ever seen it. I've emailed dozens of academics, libraries,
       | and archives - none of them have a copy.
       | 
       | So it blows my mind that something so influential is,
       | effectively, a myth.
        
         | cyrillite wrote:
         | I am infinitely disappointed to discover you are also the only
         | person who seems to care about this online. I found a website,
         | but it's you apparently.
         | 
         | Now I'm going to be bugged by this too! Great trivia also a
         | heck of a mystery
        
           | iib wrote:
           | It seems like he is not the only one (in case he did not use
           | a pseudonym here): https://en.wikipedia.org/wiki/Talk:Shard_(
           | database_architect...
        
         | mindcrime wrote:
         | Huh. OK, here's something that might be interesting. I found
         | another paper[1] that cites SHARD, but the citation is slightly
         | different. Instead of being a CCA memo, it shows it as a Xerox
         | memo:
         | 
         |  _Sunil Sarin, Mark DeWitt, and Ronni Rosenberg, "Overview of
         | SHARD: A System for Highly Available Replicated Data,"
         | Technical Report 162, Xerox Advanced Information Technology
         | (May 1988)._
         | 
         | EDIT:
         | 
         | OK, I think I get this now. I had read the Wikipedia blurb
         | about CCA being acquired by Rocket earlier, but only just now
         | did I keep reading further down to find this bit:
         | 
         |  _in 1984, CCA was purchased by Crowntek, a Toronto-based
         | company.[8] Crowntek sold Computer Corporation of America 's
         | Advanced Information Technology division to Xerox Corporation
         | in 1988.[9] The balance of CCA was acquired by Rocket Software,
         | a Boston-based developer of enterprise infrastructure
         | products,[2] in April 2010._
         | 
         | So it seems like the portion of CCA that would be of interest
         | here, is probably the bit that sent to Xerox. Maybe somebody at
         | Xerox could help turn up the missing document?
         | 
         | I doubt it will help, but I took a stab at pinging them on
         | <strike>Twitter</strike> X.
         | 
         | https://fogbeam.com/tweet_xerox_cca_paper.png
         | 
         | [1]: https://apps.dtic.mil/sti/pdfs/ADA209126.pdf
        
         | pjs_ wrote:
         | Going through the bibliography of other people's papers and
         | theses, looking for papers that you better cite "for good
         | luck", or because "you gotta cite that one" is a classic PhD
         | student behavior (I've done it) and it's not terribly
         | surprising that something like this can happen. In fact I'd
         | expect it to be much more widespread...
        
           | InitialLastName wrote:
           | One of the only notes I got from my MS thesis defense was one
           | of the professors being annoyed that I had cited a result
           | from someone else's paper that he had reported (effectively
           | the same but derived differently and less conclusively) in
           | one of his own papers. I added a note referring to his result
           | and a citation to his paper and everybody went home happy.
        
           | Delk wrote:
           | Feeling the need to cite a particular work out of convention
           | or for social reasons is understandable and very common
           | AFAIK, but I'd consider it a part of academic rigour to at
           | least take look at the work one is citing. Blindly citing
           | without ever even laying one's eyes on the work doesn't sound
           | quite right.
           | 
           | Of course if nobody can get their hands on a particular work,
           | as seems to be the case here, that makes things kind of hard.
           | But I'd expect most works you need to cite in a fast-moving
           | field such as CS to be available at least _somewhere_ , even
           | if it takes a bit of effort.
        
           | kfrzcode wrote:
           | Funny, in my high school literature class I clearly remember
           | being chastised for having sources in my works cited; but not
           | warranting their inclusion with an actual reference in the
           | work.
           | 
           | It's kind of wild that LLMs and other models/sequences will
           | be able to quickly suss out which papers have high levels of
           | referential integrity.
        
             | samstave wrote:
             | > _which papers have high levels of referential integrity_
             | 
             | The problem with this, as I see it IMO, is that there could
             | be references that are cited due to their influence on the
             | thought process/writing process of the work - thus citing
             | them gives contextual zeitgeist - and this is something
             | that AI would not be able to muster...
             | 
             | SO a LACK of referential integrity should show that it is
             | written by a human as opposed to an AI.
        
         | naasking wrote:
         | That's pretty interesting actually. Someone must have a copy
         | somewhere. Seems like a real failure of scholarship if it's
         | truly lost, and a serious argument against walled gardens-style
         | publishing.
        
         | yung_steezy wrote:
         | Is this an example of reference rot or did it never exist?
        
           | cinericius wrote:
           | If it did exist, there's some delicious irony in an original
           | paper on replicating data in a highly-available manner being
           | lost.
        
             | agumonkey wrote:
             | An endless runaway replica wave. Poetic.
        
         | chubot wrote:
         | Which papers cite it? Are they old papers, when perhaps it
         | still existed, or recent ones?
         | 
         | Very interesting either way!
        
         | littlestymaar wrote:
         | > It is referenced hundreds of times in many classic papers.
         | 
         | Wait, you mean people include papers they haven't even _opened_
         | in their references?!
        
           | HideousKojima wrote:
           | Happens far more than you think. It can be an innocent (sort
           | of) mistake, where authors see the citation in a previous
           | paper and simply copy it into their own.
        
             | WindyLakeReturn wrote:
             | This feels like something that would, at large scale, be
             | unhealthy for science as a whole. While existing papers
             | have already gone through their own quality checks, this
             | enables bad, misleading, or false statements to propagate
             | which can end up being a blow to the credibility of the
             | entire model. Shouldn't there be an ethical duty to due
             | one's due diligence?
        
               | square_usual wrote:
               | To be honest, this is probably one of the less unhealthy
               | common behaviours in academia :^)
        
             | littlestymaar wrote:
             | > where authors see the citation in a previous paper and
             | simply copy it into their own.
             | 
             | Without opening the paper to even read the abstract? To me
             | it doesn't sound like "innocent" at all, and borderline
             | malpractice...
        
               | irrational wrote:
               | Funny. I read your previous comment (with the ?! ending)
               | as sarcastic. Now I see you were serious. I would be
               | astonished if most authors have actually read even a
               | fraction of the sources they cite.
        
               | hansvm wrote:
               | There's a crowd who tilt the other way -- if I might
               | possibly have hinted at the idea before you then it's
               | borderline malpractice to not reference me. In many
               | fields it's common to directly reference what appear to
               | be the bigger transitive references then, even if they
               | didn't directly influence this work in particular. I'd
               | personally want to see a twidge more evidence before
               | bringing out the pitchforks.
        
               | HideousKojima wrote:
               | It's innocent in the sense that it's (usually) not
               | checked out of laziness/complacency as opposed to
               | malicious and intentional citation fraud.
        
         | antegamisou wrote:
         | > It is referenced hundreds of times in many classic papers.
         | 
         | According to Google Scholar, it's cited a measly 11 times .
         | 
         | https://scholar.google.com/scholar?cites=1491448744595502026...
        
           | mindcrime wrote:
           | While it may be fair to say the number isn't "hundreds" (or
           | maybe it is?) I will say that I'd take that Google Scholar
           | number with a grain of salt. Just poking around looking for
           | stuff in the spirit of this sub-thread, I've found 4 or 5
           | additional documents that cite the SHARD paper, and which
           | aren't on that Google Scholar list.
           | 
           | I've found that Google Scholar's coverage gets a little
           | sketchy on older stuff, and since we're talking way back in
           | the 1980's here, I don't think it's surprising that some
           | things are missing.
        
         | darkclouds wrote:
         | Still searching then?
         | 
         | https://shkspr.mobi/blog/2021/06/where-is-the-original-overv...
         | 
         | I can only find the Oracle reference to Sharding, which might
         | be the same thing or not.
         | https://docs.oracle.com/en/database/oracle/oracle-database/1...
         | 
         | Along with the wikipedia reference.
         | https://en.wikipedia.org/wiki/Shard_(database_architecture)
         | 
         | And a Science Direct reference.
         | https://www.sciencedirect.com/topics/computer-science/shardi...
         | 
         | Along with facebooks reference.
         | https://engineering.fb.com/2020/08/24/production-engineering...
         | 
         | And Wolverhamptons reference to Oracle Sharding. http://ora-
         | srv.wlv.ac.uk/oracle19c_doc/shard/sharding-overvi...
         | 
         | And Amazon's. https://aws.amazon.com/what-is/database-sharding/
         | 
         | So is the original paper a myth or was/is this demonstrating
         | the closed circuit nature of the dissemination of knowledge?
         | 
         | How many different ways do you cut up the data?
        
         | nocoiner wrote:
         | This is really interesting, thanks for posting it here!
         | 
         | On a semi-related topic, I love mysteries like these - mystery
         | songs, those Japanese kanji in Unicode that nobody knows what
         | they mean or where they came from, paper towns on maps.
         | 
         | If anyone else has anything else to read along similar lines,
         | please post it!
        
         | jansan wrote:
         | They should have kept the paper as highly available replicated
         | data.
        
         | bombcar wrote:
         | Do Sarin, DeWitt & Rosenb[e|u]rg exist? Are they still alive?
         | Tracking them down and going directly to the source would seem
         | to be the way to go. Perhaps even enlisting some "big names" in
         | the industry to ask around?
        
         | codethief wrote:
         | Have you contacted authors of papers citing the one you're
         | looking for, especially of papers that appeared shortly after /
         | in the 90s? Maybe one of them still has a paper copy lying
         | around somewhere.
         | 
         | I was in a similar situation before with some math paper from
         | the 50s that's nowhere to be found (neither online nor in
         | library indices) and you'd be surprised how many professors
         | still use paper copies.
        
         | johnmaguire wrote:
         | I found LinkedIn profiles for Sunil Sarin, Mark Dewitt, and
         | Ronni Rosenberg who all worked at CCA during this time period.
         | 
         | I've gone ahead and sent them each a message asking if they
         | might be able to make the paper available.
         | 
         | If you'd like to get in contact with them yourself and are
         | having trouble finding their LinkedIn, shoot me an email and
         | I'll be happy to provide you links.
        
           | johnmaguire wrote:
           | I received a response from Dr. Rosenberg:
           | 
           | > Yes, I was involved, 35 years ago! I believe it was an
           | internal CCA paper. I don't have a copy and I have no idea
           | how to get it. Sorry about that. It does seem to be the
           | earliest reference to "shard" in the DB context. (The other
           | early reference pointed to in Wikipedia is from much later,
           | 1997.)
           | 
           | > Fortunately, you need not go back 35 years to read about
           | sharding; it's easy to get current info. Cheers.
           | 
           | I've now sent a message to Andy Youniss, CEO of Rocket
           | Software to see if he can help.
        
             | mindcrime wrote:
             | _I 've now sent a message to Andy Youniss, CEO of Rocket
             | Software to see if he can help._
             | 
             | I suspect that if that memo lives on anywhere, it's
             | somewhere in the bowels of Xerox. I say that based on
             | observing that:
             | 
             | 1. In the paper by Ronni L. Roseniberg at
             | https://apps.dtic.mil/sti/pdfs/ADA209126.pdf the citation
             | to the SHARD paper changed to
             | 
             |  _Sunil Sarin, Mark DeWitt, and Ronni Rosenberg, "Overview
             | of SHARD: A System for Highly Available Replicated Data,"
             | Technical Report 162, Xerox Advanced Information Technology
             | (May 1988)._
             | 
             | 2. Per Wikipedia[1] _Crowntek sold Computer Corporation of
             | America 's Advanced Information Technology division to
             | Xerox Corporation in 1988._
             | 
             | To me this suggests that it was the "Advanced Information
             | Technology division" specifically which would have had the
             | paper in question, and that bit of CCA wound up with Xerox.
             | 
             | That said, it can't hurt to reach out anybody connected to
             | this in any way. You never know who will wind up "knowing a
             | guy who knows a gal, who knows a ..." or whatever.
             | 
             | [1]: https://en.wikipedia.org/wiki/Computer_Corporation_of_
             | Americ...
        
             | mindcrime wrote:
             | I also sent a note to another former Computer Corporation
             | of America employee that I found on LinkedIn. I don't know
             | them personally, but we live near each other and have some
             | common connections, so maybe they will at least receive my
             | unsolicited message with some favor.
        
         | alexb_ wrote:
         | This is wild if true. Surely _someone_ has to have a copy of
         | this. How is it even being referenced if it is non existent?
        
           | antegamisou wrote:
           | I don't know why parent comment is stirring up drama but:
           | 
           | 1. Not available online doesn't mean the paper's existence is
           | made up. It's a very bold claim to make for the authors that
           | they cite work that is fabricated.
           | 
           | From the available information, this looks like a technical
           | report by a, probably now defunct, company back in the 80s.
           | If this was its only form of publication, and not on some
           | conference proceedings for example, it would be only found
           | available on select university libraries as a physical copy.
           | But most important,
           | 
           | 2. This isn't even as an impactful paper as the parent
           | comment states. Or if its proposed concept is, the original
           | idea is probably derived from some other paper that is indeed
           | the one that is highly cited and most definitely available
           | online.
           | 
           | Accumulative citations number from Google Scholar and
           | IEEEXplore doesn't exceed fifteen for the particular paper
           | though.
           | 
           | https://scholar.google.com/scholar?cites=1491448744595502026.
           | ..
        
             | mindcrime wrote:
             | _Not available online doesn 't mean the paper's existence
             | is made up._
             | 
             | True, but note that the post you're referring to does say:
             | 
             |  _I 've emailed dozens of academics, libraries, and
             | archives - none of them have a copy._
             | 
             | So this isn't somebody just saying "I couldn't find it with
             | Google, therefore it doesn't exist."
             | 
             |  _From the available information, this looks like a
             | technical report by a, probably now defunct, company back
             | in the 80s._
             | 
             | Yeah, I think that's the key point. An internal technical
             | memo from a private company, from that far back, isn't
             | likely to be easy to find. It's quite possible that it's
             | never been digitized and put on the 'net, and it it wasn't
             | published in a journal, it may never have been archived by
             | any university libraries or such-like.
             | 
             | That said, I'd be a little surprised if a copy didn't turn
             | up _somewhere_ , even if it means a former employee of CCA
             | finding a copy in a desk drawer and providing it. But who
             | knows?
        
         | CamperBob2 wrote:
         | 1988? As far as anyone can tell, the use of the term "shard" in
         | the context of database replication originated with Ultima
         | Online, which was released in 1997, and which used the term in
         | connection with its underlying mythos (the idea of representing
         | world instances as shards of Mondain's shattered gem).
         | 
         | So a documented reference to sharding that's earlier than that
         | would be interesting to see.
         | 
         | (Disagree? Instead of downvoting, consider posting a citation
         | that actually resolves to a real paper.)
        
           | [deleted]
        
           | andyjohnson0 wrote:
           | (I didn't downvote your comment)
           | 
           | "SHARD" is the name of the software - it was common back then
           | to name systems using acronyms. It's not clear whether the
           | paper/report actually uses the term "shard" in the sense that
           | it is now used in distributed systems, or even whether it
           | uses it at all.
        
             | mindcrime wrote:
             | One of the related papers I stumbled across, while not
             | _the_ SHARD paper, does go into a fair amount of detail
             | about SHARD and the problem they were trying to address.
             | One bit of verbiage here might be illuminating:
             | 
             |  _The new SHARD) (System for Highly Available Replicated
             | Data) system under development at Computer Corporation of
             | America (CCA) is designed to address the problems described
             | above. It provides highly available distributed data
             | processing in the face of communication failures (including
             | network partitions). It does not guarantee serializability,
             | nor does it preserve integrity constraints, but it does
             | guarantee many practical and interesting properties of the
             | database._
             | 
             |  _The reader is referred to [SBKJ for a detailed
             | description of the architecture of the SHARI) system.
             | Briefly the main ideas are as follows. The network consists
             | of a collection of nodes, each of which has a copy of the
             | complete database. (Full replication is a simplifying
             | assumption we have used for our initial prototype, many of
             | our ideas seem extendible to the case of partial
             | replication, but this extension remains to be made.)
             | Replication allows transactions to be processed locally,
             | thus reducing communication costs and delays, and providing
             | high availability._
             | 
             | So it sounds to me like their main concern was availability
             | through replication, and not so much horizontal scalability
             | (which seems to be more the "point" of modern day
             | "sharding"). Yet I would probably claim that there is
             | enough conceptual overlap to say that SHARD does relate to
             | the modern use of sharding in some sense. Although it's
             | hard to be sure without that original paper.
        
           | w-m wrote:
           | You can find reports from before 1988 mentioning SHARD being
           | in development, like the one from June 1986 linked in this
           | sister comment: https://news.ycombinator.com/item?id=36849634
        
             | mindcrime wrote:
             | Some more, from 1989[1][2][3]. Which again, reference the
             | missing "SHARD" paper, but contain enough detail to make it
             | clear that the _idea_ of SHARD existed, regardless of the
             | status of that particular document.
             | 
             | [1]: https://apps.dtic.mil/sti/tr/pdf/ADA214478.pdf
             | 
             | [2]: https://apps.dtic.mil/sti/tr/pdf/ADA216523.pdf
             | 
             | [3]: https://apps.dtic.mil/sti/tr/pdf/ADA209437.pdf
        
         | foobarian wrote:
         | I guess I'm not too surprised, this seems like a corporate tech
         | report. Some companies were good at having public archives of
         | these (like Bell Labs) but I'm sure it takes a lot of resources
         | to keep that up. It's essentially some company's internal Wiki
         | page.
        
         | pavo-etc wrote:
         | Wow this is fascinating. Are any of these authors still alive?
        
         | mindcrime wrote:
         | Here's something that seems related. Maybe one of these authors
         | would have a copy of the other paper? Not sure if they would be
         | among the set of folks you've already tried or not...
         | 
         | https://apps.dtic.mil/sti/pdfs/ADA171427.pdf
        
           | CamperBob2 wrote:
           | Interesting, thanks. Wonder if it was a coincidence that UO
           | adopted that terminology, or if somebody there knew about
           | this.
        
             | mindcrime wrote:
             | It's a good question. I mean, it's not entirely out of the
             | question that the UO folks independently developed the same
             | term without being aware of the SHARD research. But OTOH,
             | it's entirely possible they were aware. Without talking to
             | somebody that was there, I doubt we'll ever know for sure.
        
               | morelisp wrote:
               | Raph Koster claims parallel invention of the term "shard"
               | for separate worlds in MMOs.
               | https://www.raphkoster.com/2009/01/08/database-sharding-
               | came...
        
         | leonheld wrote:
         | I think this is your blog, so I'm going to post it here because
         | it is so great to look at!
         | 
         | https://shkspr.mobi/blog/2021/06/where-is-the-original-overv...
        
       | Silamoth wrote:
       | I recently read "Enabling tabular deep learning when d [?] n with
       | an auxiliary knowledge graph"
       | (https://arxiv.org/pdf/2306.04766.pdf) for one of my graduate
       | classes. Essentially, when there are significantly more data
       | points than features (n >> d), machine learning usually works
       | fine (assuming data quality, an underlying relationship, etc.).
       | But, for sparse datasets where there are fewer data points than
       | features (d >> n), most machine learning methods fail. There's
       | just not enough data to learn all the relationships. This paper
       | builds a knowledge graph based on relationships and other pre-
       | existing knowledge of data features to improve model performance
       | in this case. It's really interesting - I hadn't realized there
       | were ways to get better performance in this case.
        
       | bitxbitxbitcoin wrote:
       | "A classification of endangered high-THC cannabis (Cannabis
       | sativa subsp. indica) domesticates and their wild relatives"
       | 
       | By McPartland and Small.
       | 
       | Moving on from cannabis sativa indica and cannabis sativa sativa
       | to cannabis sativa indica Himalayansis and cannabis sativa indica
       | asperrima depending on distribution from the original location of
       | the extinct ancient cannabis wildtype.
       | 
       | Following this new classification, I believe there's a third
       | undocumented variety in North East Asia.
       | 
       | If anyone else has noticed the samesameification of cannabis
       | strains and is wondering what the path forward is, this may be
       | illuminating.
       | 
       | https://phytokeys.pensoft.net/article/46700/
        
       ___________________________________________________________________
       (page generated 2023-07-24 23:02 UTC)