Subj : Re: Change Patterns (was: Polymorphism sucks)
To   : comp.programming,comp.object
From : topmind
Date : Tue Jun 28 2005 05:50 pm

>
> >>>> Now, you COULD have a data processor that reads from and writes to a
> >>>> database...but you still have to develop the individual data handlers,
> >>>> so there's no code benefit, PLUS now you have to deal with an external
> >>>> system, deal with potential related errors and have a data transport
> >>>> layer to handle the IO.
> >>>
> >>> I don't consider the DB an "external system", but rather a
> >>> useful partner tool in the process, along with the
> >>> app programming language. Partners to divide-and-conquer.
> >>
> >> Regardless it IS still an external system,
> >
> > External to what?
>
> The running program, of course.  Using a DBS requires a DBS (rule #1 in a
> gun fight: bring a gun).  Even if you're lucky enough to have something
> that runs in RAM, you have a choice between bringing an entire DBS into
> your application or using IPC to something out of process.  I would imagine
> that anything with any muscle has gotta run out of process and store data
> on disk.


Back in the 286 days, small semi-relational DB's fit in about 200K.
That is lint on a flee's booger by today's standards. I agree that the
small-end DB choices available today are generally limited. Part of it
was caused by OO hype making DB tools fall out of favor. It is a
self-fulfilling prophecy in many ways: "You should use OO tools because
OO hype killed the alternatives".

(Well, part of the blame is also SQL: SQL is a bloated, verbose
relational language. The small DB engines I used to use did not use
SQL.)


>
> Unless there's a compelling need for that kind of overhead, it's not an
> appealing solution.

200k, boo hoo hoo.

>
> > What if a language/library allows the DB/table engine to be compiled into
> > the EXE?
>
> Unless there's a compelling need for that kind of overhead, it's not an
> appealing solution.
>

Compelling: roll-your-own-spaghetti data structures composed of lists
of arrays of lists of pointers suck.

>
> >> Your method requires shipping and installing an *external* *system*
> >> to the primary program.  To justify this requires, well, justification.
> >
> > Why do you keep saying "external"? Is the GUI library "external". Is
> > the file handling library "external"?
>
> If I have to provide separate installable drivers, yes.
> If I have to install a database on the client system, yes.
> If the client has to run software in addition to my program, yes.
>
> With GUI and File libraries, the actual answers are usually no, no and no.
> With DBSes, the actual answers are usually probably, yes and maybe.

You are obsessed with hardware and installation issues like C
programmers trying to defend writing everything in C. So 80's.

>
>
> >>> Further, I don't encounter much need to swap the input/output source
> >>> in real-time.
> >>
> >> "Swap the input/output source"?  You apparently don't understand what
> >> I told you.  If you can't even understand such a simple example, what
> >> makes your opinion worth anything?
> >
> > Of course the problem is with me and not your bad writing.
>
> If you don't understand an example, especially one provided in a fair
> amount of detail, you should ask for clarification.  It's not just
> simple courtesy, you're likely to get called on it later.

I could find several examples of your diplomacy taking a dump. I
generally mirror the attitude others display toward me. The "Jesus"
thing to do would be to ignore insulting behavior instead of mirror it,
but I ain't no Jesus.

>
> I'll summarize the example for you.
>
> Consider a common software component: a "blackbox" function that takes
> a data stream of some sort, does something with/about that data, and
> emits an output datastream.  You like business examples, consider a case
> where the component reads an XML stream of POS (Point Of Sale) info and
> emits a report summarizing that data (this is not an imaginary example).


Reports quite often cannot easily be produced in the *same order* that
they receive input. It is good decoupling to make the report production
independent of the data source ordering. If your data comes in with an
ABCD order, the output quite possibly could be in BDCA order.

Thus, some kind of middle-man data structure or database is probably
going to be needed anyhow. I doubt one could make a general-purpose
stream processor API that can handle any order without using an
intermediate data structure. Might as well use a DB for that instead of
arrays of lists of arrays of pointers to arrays of pointers to lists.


>
> From an engineering and maintenance point of view, not to mention from
> an elegance and simplicity point of view, you'd like to completely
> decouple the component from the data provider as well as from the data
> consumer.


What about decoupling from input order?


>
> To use your own example, you might like to emit these reports to emails
> to managers, HTML to a website, and a database for archival.  On the input
> side, you might normally get data from a web service taking SOAP messages,
> but you'd also like to get data from the disk for test and validation
> purposes.
>
> What your component does, in an OOP polymorphic solution, is take two
> input objects that are interfaces: an "Input" interface and an "Output"
> interface.  You design your component to read from the Input and write
> to the Output.
>
> Now you can pass your component any pair of objects, so long as one
> supports the Input interface and the other supports the Output.  If you
> want the ability to write to multiple outputs *simultaneously*, you design
> your component to take a list (or array or vector or collection or ...) of
> Output objects and just iterate over each for any output.
>
> Now here's the crucial point: In OOPLs, this is trivially easy to implement
> and is native to the language.  No libraries, no IPC, no databases.  Very
> low overhead, easy to implement, native to the language, fast running....
> that's a hard combo to beat.
>

I will give you speed for the sake of argument. Other than that, I see
no problem with using a DB to store the info and then making the output
a separate consideration, especially if one is already using a DB for
other purposes.

I don't have a need to swap the output format that often anyhow.

The probability of having to have a wide variety of output "devices"
AND not being able to use a DB as an intermediate holding spot is quite
small. If the chances of one is 10% and the chance of the second is
also 10%, then the chance of both of them happening at the same time is
about 1%.

I agree that once in a *blue moon* poly may be the better alternative.
But since one cannot usually know in advanced, I'll play the odds as I
have observed them over the years.


> >>> Expensive? Go open-source. Try MySQL or Postgre-SQL.
> >>
> >> Not talking dollars.  Talking cost of implementing, maintaining, etc.
> >
> > Oracle is not the only model of how a DB can be.
>
> Doesn't matter who.  All DBSes have a cost of implementing and maintaining.
> With some, the cost is huge.  With others, it's less painful.  It's still
> always there, however.


Well, they are a common tool. If you have it for one need you might as
well use it for another instead of arrays of lists of arrays of
pointers to ....


>
> >>> It is essentially the same thing: sub-type classifications.
> >>
> >> I don't agree that a class that implements an interface is a sub-type.
> >
> > In practice it tends to be so, at least for domain-centric classes.
>
> Practice is affected by many factors beyond correctness.  When talking
> about what a thing *is*, practice doesn't really apply.  The fact is
> that implementing an interface does not *require* sub-typing.

I am not sure what your point is here. Technically they are indeed not
identical, but my argument did not depend on them being identical.

>
>
> >>> (Not all polymorphism is related to sub-typing, but most is in
> >>> practice, at least at the domain level.)
> >>
> >> I disagree.  I do a lot of VB programming.  VB supports interface
> >> "inheritance" but has no concept of implementation inheritance.
> >> My polymorphic VB objects are not sub-types.
> >
> > Maybe not from the language's perspective, but conceptually
> > they are usually subtypes.
>
> The irony is that, to the extent that's true, it reflects either the innate
> hierarchical nature of the problem being modeled or a hierarchical way of
> breaking it down.

Innate, eh?

Quote: "[Even though...] it has been known since 1847 that
classifications are dependent on the purpose of the classification,
people continue to believe that it is possible to create a
classification system that is context-independent. (Haim Kilov on
comp.object, 6/01. Note that I consider sub-types to be
"classifications".)"

>
>
> >> If the code needs to decide what sort of object is at hand in many
> >> places, then you have many case statements you must first find and then
> >> change.  And in my experience, you will have a lot of places where the
> >> code needs to think, "Gee, is this a Savings Account, Checking Account,
> >> or that new fangled Combo Account."  EACH ONE OF THOSE must be located
> >> and changed.
> >
> > Most case lists are not mirror duplicates of each other in my
> > experience, at least over the longer run.
>
> Which makes the problem that much worse.  If they WERE close or duplicates,
> they'd be easier for search engines to find (among all the other case
> statements).


Yes, but it also means that conversion to polymorphism will not
be a clean simple tree.


>
> > If there are lots of them, it is often a sign that tables should be
> > used instead.
>
> So now, every time you dispatch a method on an object, you need to go do
> a DBLookup() function of some sort?
>
> Eewwww.

Better than traversing arrays of lists of arrays of....

>
> >> Compare this to a system using a generic Account object.  If you add a
> >> new sub-type, that's it, you're basically done.
> >
> > Yes, that is the most common change scenario promoted by OO books.
> > But it is relatively improbable in my experience.
>
> IT'S YOUR OWN "REAL LIFE" EXAMPLE!!!!  In the other thread, you've been
> trying to get them to accept it as a reality.  Now you're disowning it?

I am not sure what "other thread" you are referring to. Unfortunately,
I wrote a lot of messages. If you mean adding a hybrid "type", I don't
consider that a natural type anymore than any other a-la-carte
combination of factors is a "type".

>
> To quote your own words about a dozen lines later:
>
> > I've heard it with my own ears. No matter what taxonomy I present,
> > you can always claim it may be fake.
>
> No, see, I took your ball an ran with it.  Now you've turned face.


Well, its anecdotes versus anecdotes.  At least a reader can see that a
bank account can potentially be both checking and savings. You have not
provided any evidence or business-logic argument of a "force" that
would prevent such from existing. The priest asks if anybody objects to
the holy union, and you did not raise your hand. Thus, they now on
honeymoon.

A-La-Carte feature selection is the best future-proofing bet. There is
no inherent tree cop or tree herder in the biz world that I have found.
There indeed are a fair amount half-@ss tree patterns to be found, but
sets handle these better than hacked-up trees. Sets just plain
"degenerate better".

>
>
> > Then present some stable business hierarchies.
>
> The hierarchy of positions in my company hasn't changed in decades
> (CEO, Senior VPs, VPs, etc on down the line). Nor has the hierarchy
> of business units changed significantly.

I already gave an anecdote of such breaking down during a budgeting
summary app. I actually spent a couple of weeks removing the tree
system and replacing it with a set-based system. It was not a
light-weight decision. The accountant and I went over and over it
before the set-based change was agreed upon and made.
Honest-to-God/Buddha/Allah/etc; or may I be struck down by a huge bolt
of polymorphic lightning (as far as IIRC the details).

>
> BOMs are nicely handled hierarchically (and, in fact, would be very
> unwieldy if handled as flat lists)

I never proposed a "flat list". (How does a flat list differ from a
non-flat list?) And tree BOM's have long known to have rough spots. For
example, accounting may want to search and sort by multiple factors;
factors probably different than what a material specialist would want
to see. A material specialist may not really care how stuff is
connected. There is NO One Right Taxonomy for non-trivial stuff.

>
> CAD/CAM hierarchies are, well, hierarchical, and thankfully so.

If bolt C holds part A and part B together, does C belong to part A or
part B? If you answer "both", you just broke away from a pure tree.
Congratulations!


>
> Hierarchies and fractals have a lot in common in that there are self-
> similar *levels* and in the concept of "drilling down" (or "zooming
> back").  The tree structure--as much as you try to find fault with
> it--IS a fundamental, natural relationship.


Wrong. It is a "useful lie" on a small scale, a barely tolerable lie on
a medium scale, and boat heading for Disaster Island on a large scale.


>
> Databases fail to efficiently model trees, because trees are "triangular"
> in nature.  Databases are tabular, or "rectangular", in nature.  These
> are mutually exclusive natures.  In places where one shines, the other
> usually does not, and vice versa.

RDB's are whatever shape you want. If you supply the right attributes,
you can get just about any output or view format you want. The main
purpose of a DB is to represent facts and relationships between facts.
What you do with those facts is up to your imagination. RDB's can
represent trees also, but trees can't do non-trees very easily.

Trees hard-wire potentially faulty assumptions about the shape of
things.

>
>
> >> The "Family of Man" is a huge hierarchy.
> >
> > Wrong. Human genetics is not a pure tree. A directed
> > graph may be a better description.
>
> If you mean some branches re-connect, that doesn't stop it from being
> a tree (just stops it from being certain kinds of tree).

A tree with duplicate nodes. Genetic copy-and-paste.

>
> There are all sorts of trees.  And on the small scale, such as when you
> graph the genealogy of a family, it is entirely treelike.

No, it is not. It may appear tree-like on a small scale, but as you add
more nodes it is less and less tree-like. It is a directed graph, not a
tree.

>
>
> >> The files on your disk are
> >> arranged in a hierarchy for very good reason (what would you recommend?
> >> putting them all in an RDB and searching for the ones you want?).
> >
> >
> > Yes! Hierarchal file systems suck!!!! They are fine when you have 200
> > files, but don't scale.
>
> I have MANY tens (if not hundreds) of thousands of files on my laptop.
> What do you mean it doesn't scale?  If the best you can do is wave your
> hands and declare, "They suck!!!!!", then you've said nothing.
>
> > I encounter times I wish I could query for files with SQL all the time.
>
> SQL is just a query language, nothing prevents it from being applied to
> any dataset you like.

In other words, ditch/ignore the tree, eh? Now you're talkin'!

> Now imagine you have all the power of SQL, PLUS
> part of your filtering ability is to restrict the search to a subset of
> records (verses the entire dataset).
>
> Wouldn't you call that a win?
>

Please clarify. Query languages *already* provide filtering ability.

>
> >> Companies and armies are hierarchically structured.  Etc. Etc. Etc.
> >
> > Not really.
>
> More hand waving saying nothing.
>
> Yes, really.  I can trace a unique path from me to the CEO.  It's 100%
> tree shaped.  As for armies, have you ever heard the phrase "chain of
> command"?  What do you think that involves, if not a tree?
>


Most personnel org representations are in RDBMS in practice. Why?
Because many of the searches and processes are not by tree paths even
if some are. The Hedge Factor again. If you are looking at who to
blame, a tree VIEW makes sense. But if you are trying to figure out if
there is a copper processing expert in Cleveland, the tree is mostly
moot.


>
> > One of my first exposures to the problems of trees happened when I was
> > building a budget summary app that presented budget allocation in a
> > hierarchical fashion (among other things).
> >
> > They kept bastardizing the structure such that a given office would
> > appear in *multiple categories*.
>
> So you mistook the original problem as being tree shaped, but it wasn't.
> So what?  Not EVERYTHING is a tree, obviously.
>

I and others KEPT making that mistake in other things. I have finally
learned to hedge with sets (usually via relational). After getting
burned by trees so many times, I stopped touching the stove.

In fact, it keeps happening. Some places dictate (Darth Tree) that I
use the FuseBox module organization methodology, which generally
encourages hierarchical taxonomies and/or mutually-exclusive
categories. But, I keep running into the same tree-busting problems.

>
> >>>> One thing I think is true is that OOD is harder, and does require
> >>>> a higher skill set, than simpler forms of programming.
> >>>
> >>> That is because OOD is ill-defined, ill-documented, ill-evidenced,
> >>
> >> That's nonsense.  It's because OOD is more sophisticated and abstract.
> >
> > Yeah, too "sophisticated and abstract" to document and science-atize.
>
> That's true of software construction in general.  It's an extremely complex
> task, intellectually on par, I believe, with law and medicine.  All three
> fields, in their daily practice, break new ground (like researchers), but
> also need to produce approximately immediate useful results (not like
> researchers!).  And all three fields deal with a large network of inter-
> connected data that often reacts holistically.

In that case, why not use a tool that handles
unexpected patterns better than trees?

>
> Software development has rightly been called, "an exercise in managing
> complexity."  There IS a lot of science that informs law, medicine and
> computer, um, science, but as a real world practice, there's a fair bit
> of art and craft to it, too.
>
>
> >> I'm coming to understand you're an evangelist.  I don't do evangelists.
> >
> > Well, you sure did a damned good *impression* of one.
>
> If you pay attention, you'll see I'm standing here in the middle of the
> road.  You're the one over there in the ditch trying to baptize everyone.

No, you are not. You are a tree evangelist. A tree hugger, so to speak.
  
-T-

.