Newsgroups: comp.ai.neural-nets
Path: utzoo!utgpu!jarvis.csri.toronto.edu!neat.ai.toronto.edu!bradb
From: bradb@ai.toronto.edu (Brad Brown)
Subject: Refs/opinions wanted -- Neural nets & approximate reasoning
Message-ID: <88Nov18.011810est.6198@neat.ai.toronto.edu>
Organization: Department of Computer Science, University of Toronto
Date: Fri, 18 Nov 88 01:18:08 EST

   I am  working on  a paper which compares symbolic and neural
   network approaches to approximate reasoning, including fuzzy
   sets, probabilities  logic,  and  approximate  solutions  to
   problems.   I would  very  much  appreciate  references  and
   personal comments.
 
   Given current  hardware technology  and current  neural  net
   (NN) learning  algorithms, NNs  seem to  have some desirable
   properties that  symbolic systems  do not,  but suffer  from
   implementation problems  that prevent them from being useful
   or efficient in many cases.  A summary of my thinking, which
   includes  many   generalizations  and  omits  justification,
   follows.
 
   Neural network-based  systems have  advantages over symbolic
   systems for the following reasons.
 
   (1)  For some  classes of  problems, NN  learning algorithms
        are known.  In these cases, "programming" a NN is often
        a matter of presenting it with training information and
        letting it learn.
 
        Symbolic systems  have more known algorithms and can be
        applied to  more problems  than NNs,  but  constructing
        symbolic programs  is labour  intensive.  The resulting
        programs are typically problem-specific.
 
   (2)  Neural nets  can adapt to changes in their environment.
        For instance,  a financial expert system implemented as
        a  NN   could  use   new  information   to  modify  its
        performance  over   time  to  reflect  changing  market
        conditions.  Symbolic systems are usually either static
        or require re-training on a substantial fraction of the
        dataset to adapt to new data.
 
        Neural nets  are forgiving  in their response to input.
        Inputs that  are similar  are treated  similarly.    In
        symbolic systems  it is  very  difficult  to  give  the
        system a  notion of  what constitutes a "similar" input
        so input  errors or  input noise  are big  problems for
        symbolic systems.
 
   (3)  NNs are  good  at  constraint  problems  and  have  the
        desirable property  of finding  good compromises when a
        single best solution does not exist.
 
   (4)  NNs can deal with multiple sources of information.  For
        instance, a financial system could consider inputs from
        both stock  market  information  and  internal  company
        sales information, which are not causally related.  The
        learning procedure can be expected to find weights that
        "weigh"  different   kinds  of   evidence   and   judge
        accordingly.  Symbolic systems require extensive manual
        tuning  to   be  able   to  effectively   use  multiple
        orthogonal sources of information.
 
   On the other hand, practical applications of NNs are held
   back by
 
   (1)  Lack of  well-understood training  algorithms for  many
        tasks.   Many interesting tasks simply cannot be solved
        with NNs because no one knows how to train them.
 
   (2)  Difficulty  in  running  neural  nets  on  commercially
        available hardware.   Neural  net  simulations  require
        vast CPU  and memory resources so NN systems may not be
        cost effective compared to equivalent symbolic systems.
 
   (3)  Absence  of   an  ability   to  easily  explain  why  a
        particular result  was achieved.   Because knowledge is
        distributed throughout  the network  and  there  is  no
        concept of  the network  as a whole proceeding stepwise
        toward a solution, explaining results is difficult.
 
   All things considered, I am a believer in neural networks.
   I see them as the "natural" way to make big advances towards
   "human-level" intelligence, but the field is too new to be
   applied to many practical applications right now.  Symbolic
   approaches draw on a more mature and complete base of
   experience.  Nevertheless, it is very difficult to get
   symbolic systems to show some of the nice traits seen in
   neural networks, like an ability to deal with noise and
   approximate inputs and to produce good compromise solutions.
   An interesting compromise would be the integration of neural
   networks into symbolic reasoning systems, which has been
   tried with some success by at least one expert system group.
 
   -----------------------------------------------------------
 
   
 
   Comments and criticisms on these thoughts would be greatly
   appreciated.  References to current work on neural networks
   for approximate reasoning and comparisons between neural
   networks and symbolic processing systems would also be very
   much appreciated.  Thank you very much for your time and
   thoughts.
 
   
 
                                 (-:  Brad Brown  :-)
 
                                 bradb@ai.toronto.edu

