https://groups.google.com/g/erlang-programming/c/LKLesmrss2k [logo_group]Groups Conversations All groups and messages [Search conversations][ ] Send feedback to Google Help Training # Sign in Groups Erlang Programming Conversations About Privacy * Terms Groups keyboard shortcuts have been updated Dismiss See shortcuts [erlang-questions] Why do we need modules at all? 16,724 views Skip to first unread message Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 4:06:19 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Why do we need modules at all? This is a brain-dump-stream-of-consciousness-thing. I've been thinking about this for a while. I'm proposing a slightly different way of programming here The basic idea is - do away with modules - all functions have unique distinct names - all functions have (lots of) meta data - all functions go into a global (searchable) Key-value database - we need letrec - contribution to open source can be as simple as contributing a single function - there are no "open source projects" - only "the open source Key-Value database of all functions" - Content is peer reviewed These are discussed in no particular order below: Why does Erlang have modules? There's a good an bad side to modules: Good: Provides a unit of compilation, a unit of code distribution. unit of code replacement Bad: It's very difficult to decide which module to put an individual function in. Break encapsulation (see later) Aside: lib_misc.erl When I'm programming I often get to the point were I say there should a function foo/2 in lists.erl but their isn't. There should be but there isn't - foo/2 is a small self contained thing. Why should it be in lists.erl because it "feels right". Strings are lists, so why do we have two modules lists.erl and string.erl how should I decide in which module my new string/list processing function should go. To avoid all mental anguish when I need a small function that should be somewhere else and isn't I stick it in a module elib1_misc.erl. My elib1_misc exports the following: added_files/2 make_challenge/0 as_bits/1 make_response/2 as_bits_test/0 make_response_test/0 bdump/2 make_test_strings/1 bin2hex/1 make_test_strings_test/0 bin2hex_test/0 make_tmp_filename/2 check_io_list/1 merge_kv/1 collect_atom/1 merge_kv_test/0 collect_atom_test/0 mini_shell/0 collect_int/1 module_info/0 collect_int_test/0 module_info/1 collect_string/1 ndots/1 collect_string_test/0 nibble_to_hex_char/1 collect_word/1 nibble_to_hex_char_test/0 complete/2 odd/1 complete_test/0 on_exit/2 dos2unix/1 out_of_date/2 downcase_char/1 outfile/2 dump/2 padd/2 dump_tmp/2 perms/1 duplicates/1 perms_test/0 ensure_started/2 pmap/2 eval_file/1 pmap1/2 eval_file_test/0 pmap1_test/0 eval_string/1 pmap_test/0 eval_string_test/0 priority_receive/0 every/3 random_seed/0 expand_env_vars/1 random_string/1 expand_file_template/3 random_string/2 expand_string_template/2 read_at_most_n_lines/2 expand_tabs/1 read_at_most_n_lines_test/0 expand_tabs_test/0 remove_duplicates/1 expand_template/2 remove_duplicates_test/0 extract_attribute/2 remove_leading_and_trailing_whitespace/ 1 extract_attribute_test/0 remove_leading_and_trailing_whitespace_test/0 extract_prefix/2 remove_leading_whitespace/1 fetch/2 remove_prefix/2 fetch_test/0 remove_prefix_test/0 file2lines/1 remove_trailing_whitespace/1 file2lines_test/0 replace/3 file2md5/1 root_dir/0 file2numberedlines/1 rpc/2 file2numberedlines_test/0 safe/1 file2paras/1 show_loaded/1 file2stream/1 signed_byte_to_hex_string/1 file2string/1 signed_byte_to_hex_string_test/0 file2template/1 skip_blanks/1 file2term/1 skip_blanks_test/0 file_size_and_type/1 skip_to_nl/1 find_src/1 skip_to_nl_test/0 first/1 sleep/1 flatten_io_list/1 spawn_monitor/3 flush_buffer/0 split_at_char/2 for/3 split_at_char_test/0 force/1 split_list/2 foreach_chunk_in_file/3 split_list_test/0 foreach_word_in_file/2 string2exprs/1 foreach_word_in_string/2 string2exprs_test/0 forever/0 string2html/1 get_erl_section/2 string2latex/1 get_line/1 string2lines/1 get_line/2 string2lines_test/0 have_common_prefix/1 string2stream/1 have_common_prefix_test/0 string2stream_test/0 hex2bin/1 string2template/1 hex2bin_test/0 string2template_test/0 hex2list/1 string2term/1 hex2list_test/0 string2term_test/0 hex_nibble2int/1 string2toks/1 hex_nibble2int_test/0 string2toks_test/0 id/1 sub_binary/3 include_dir/0 template2file/3 include_file/1 term2file/2 interleave/2 term2string/1 is_alphanum/1 test/0 is_blank_line/1 test1_test/0 is_prefix/2 test_function_over_substrings/2 is_prefix_test/0 tex2pdf/1 is_response_correct/3 time_fun/2 keep_alive/2 time_stamp/0 lines2para/1 to_lower/1 list2frequency_distribution/1 to_lower_test/0 list2frequency_distribution_tetrim/1 longest_common_prefix/1 trim_test/0 longest_common_prefix_test/0 unconsult/2 lookup/2 unsigned_byte_to_hex_string/1 lorem/1 unsigned_byte_to_hex_string_test/0 ls/1 which/1 which_added/1 Now I find this very convenient when I write a new small utility function I stick in in elib1_misc.erl - no mental anguish in choosing a module name is involved. The observation that I find this very-convenient is telling me something about modules - I like my elib1_misc it feels right. (aside - It seems many development projects have their own private lib_miscs ...) Which brings me to the point of my question. Do we need module's at all? Erlang programs are composed of lots of small functions, the only place where modules seem useful is to hide a letrec. The classic example is fibonacci. We want to expose fib/1 but hide the helper function fib/3. Using modules we say -module(math). -export([fib/1]). fib(N) -> fib(N, 1, 0). fib(N, A, B) when N < 2 -> A; fib(N, A, B) -> fib(N-1, A+B, A). The downside is we have had to *invent* one module name math - whose *only* purpose is to hide the definition of fib/3 which we don't want to be made callable. If we put a second function into the module math, then this second function could call fib/3 which breaks the encapsulation of fib/3. We could say: let fib = fun(N) -> fib(N, 1, 0) end in fib(N, A, B) when N < 2 -> A; fib(N, A, B) -> fib(N-1, A+B, A). end. I hardly dare suggest a syntax for this since I've been following another thread in this forum where syntax discussion seem to encourage much comment. ** Please do suggest alternative syntax's here - but do not comment on other peoples suggestions ... I would like to just talk about why we have modules. Another question: Does the idea of a module come from the idea that functions have to be stored somewhere, so we store them in a file, and we slurp the file (as a unit) into the system, so the file becomes a module? If all the files were store by themselves in a database would this change things. I am thinking more and more that if would be nice to have *all* functions in a key_value database with unique names. lookup(foo,2) would get the definition foo foo/2 from a database. The unique names bit is interesting - is this a good idea. Qualified names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good idea but but when I'm programming I have to invent the xxx or the a.b.c which is very difficult. It also involves the "decision problem" if the namespaces xxx and a.b.c already exist I have to *choose* which to put my new function in. I think there might be a case for alises here joe:foo/2 could be used while developing "joe" would expand to a horrible random local string the real name being ab123aZwerasch123123_foo/2 but I would not be able to publish my code or make it available to a third_part before I had chosen a sensible name. (( managing namespaces seems really tricky, a lot of peoople seem to thing that the problem goes away by adding "." 's to the name but managing a namespace with namees like foo.bar.baz.z is just as complex as managing a namespace with names like foo_bar_baz_z or names like 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic name like www.a.b to a reference like 123.45.23.12 - but how do we discover the initial name www.a.b? - there are two answers - a) we are given the name (ie we click on a link) - we do not know the name but we search fo it )) When programs are small we can live with "just the code" in "a few modules" the ratio of code to meta data is high. When programs are large we need a lot of meta-data to understand them. I would like to see all functions with all meta-data in a data base. I'd like to say: lookup(foo,2,Attribute) when Attribute = code|source|documentation|type signatures|revision history| authors|... The more I think about it the more I think program development should viewed as changing the state of a Key-Value database. So I imagine: 1) all functions have unique names 2) there are no modules 3) we discover the name of a function by searching metadata describing the function in a database 4) all public functions (think open source) are in the same database We could make a system to do this. I think this would make open-source projects easier, since the granularity of contribution goes down. You could contribute a single function - not an entire application. (( A problem with GUT style open source projects is there is not one database of functions, I often what one function from this project, another function from another project -- the granularity of reusable parts should be the individual function. functions are really easy to reuse modules are more difficult to reuse entire applications are very difficult to reuse (Unless there are isolated through a communication channel)) Possible extensions. 1) Voting for promotion 2) A review process Given a raw database will *all* functions in it - we could derive an "approved" functions database. Popular functions could be moved to the approved database - the review process would need to be discussed - so kind of peer-review/ wiki stuff. Comments? Volunteers? /Joe Ralf P. Gans's profile photo Ralf P. Gans unread, May 24, 2011, 4:11:32 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Hi, the ability to replace code in a running environment depends, to my knowledge, on modules. So modules prevent downtimes and allow 7/24 operation. cheers, Ralf > _______________________________________________ > erlang-questions mailing list > erlang-q...@erlang.org > http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-q...@erlang.org http://erlang.org/mailman/listinfo/erlang-questions Amy Lear's profile photo Amy Lear unread, May 24, 2011, 4:19:55 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Ralf P. Gans, Erlang On Tue, May 24, 2011 at 1:11 AM, Ralf P. Gans wrote: Hi, the ability to replace code in a running environment depends, to my knowledge, on modules. So modules prevent downtimes and allow 7/24 operation. cheers, Ralf This is only true because that's the implementation we have. The VM's ability to maintain a couple of versions of a particular function could still be maintained on that granular a level.. ... Provided there's a way to, with atomicity, ensure that nothing calls the new version of a function without all of the related changes in associated functions rolling into the VM at the same time, since you may be moving quite a bit around if you find yourself refactoring a few functions to add new capabilities. And that IS something that is easier with modules as a complete unit of dependency. Is there a way in a clean semantic manner track cross-function dependencies (potentially through some type of version tags?) in such a code database? Mazen Harake's profile photo Mazen Harake unread, May 24, 2011, 4:33:36 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Aren't you just redefining the problem? Your next problem is going to be to name more generic type of functions. Imagine you have 3 different search functions which do different things in different modules. Your solution would instead of: foo:search(), bar:search(), baz:search() give foo_search(), bar_search(), baz_search() now you want to add a function that does replace; but replace what? say you call it replace() now I want to make one because it is specific to my problem so I have a decision problem foo_replace() ? bar_replace() ? so I will end up defining a prefix so that my versions of functions (where they collide with the already existing ones) always are unique thus: mazen_replace() now everyone starts creating functions with prefixes, now they want to start extending other functions so they build a prefix before the those: mazen_foo_replace() and before you know it you will have 20 functions ending with "search ()" where search means what ever the prefix hints at... So you are back to square -1 because now it is worse, you don't have modules and you don't have applications beacuse an application would become what modules was before it. In fact, this is already a problem, all my opensource applications (and non-opensource as well) have a prefix on the modules because I can't call all my server type modules "server" (there is no namespace). My 2 cents. /M Dmitrii Dimandt's profile photo Dmitrii Dimandt unread, May 24, 2011, 4:39:06 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Questions > > So I imagine: > > 1) all functions have unique names > 2) there are no modules > 3) we discover the name of a function by searching metadata > describing the function in a database > 4) all public functions (think open source) are in the same > database > > We could make a system to do this. Hm... you'd still need *some* kind of modules. For instance, in Webmachine each resource is expected to have some, or all, of predefined public/exported functions: to_html, resource_exists, allowed_methods etc. So, each resource will have functions with the same name. If we move all functions to a global namespace, then how do we differentiate between these functions? Probably by naming them resource_name_function_name? Furthermore, what do we do with behaviours? A behaviour is encapsulated within a module, also with a predefined set of exported funcions (and any number of custom exported functions). Gordon Guthrie's profile photo Gordon Guthrie unread, May 24, 2011, 4:48:09 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang There are two good things about modules: * they are a higher level of abstraction * they support good working practices Lets take a module that I use a little bit - and which a colleague introduced me to: http://erldocs.com/R14B02/stdlib/digraph.html?i=2&search=digra# get_digraph/1 I can use the couple of exposed functions of that that I know, confident that if it comes to it I can come back and master the rest of the interface later. I have confidence that there is a rubustness there that makes code using it maintainable. (my idea of maintainable is to be able to ignore most stuff, most of the time...). I can learn about the code at this higher level of abstraction. We too have a module with loads of random functions in it, well we have three (hn_util, util, util2), and each of them is just library functions. In the utility modules the ratio of exported to not-exported fns is maybe 7 to 1. In functional modules the exported/not ratio goes from 1-1 (in modules that define apis) to 1-3 against. If we had the (promised) -export-to() directive the number of pure exported fns would drop. Given that we build our testing strategy around -export() this separation is critical... Gordon > _______________________________________________ > erlang-questions mailing list > erlang-q...@erlang.org > http://erlang.org/mailman/listinfo/erlang-questions > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 Josh Johnston's profile photo Josh Johnston unread, May 24, 2011, 4:49:00 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Questions On 24/05/2011, at 6:39 PM, Dmitrii Dimandt wrote: So I imagine: 1) all functions have unique names 2) there are no modules 3) we discover the name of a function by searching metadata describing the function in a database 4) all public functions (think open source) are in the same database We could make a system to do this. Hm... you'd still need *some* kind of modules. For instance, in Webmachine each resource is expected to have some, or all, of predefined public/exported functions: to_html, resource_exists, allowed_methods etc. So, each resource will have functions with the same name. If we move all functions to a global namespace, then how do we differentiate between these functions? Probably by naming them resource_name_function_name? In a normal language this would be a problem, but in erlang we can get the desired behaviour by using pattern matching, right? So instead of to_html() resource_exists() etc We have resource(to_html, ...) resource(exists, ...) etc I'm not disagreeing with you - we do need some good way to do this. But I don't think modules are necessary in this case. Josh. Gordon Guthrie's profile photo Gordon Guthrie unread, May 24, 2011, 4:49:30 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang ...and also... following on from what Dimitri said while I was writing me e-mail, we hang unit test onto modules and design them at that level of abstraction Kresten Krab Thorup's profile photo Kresten Krab Thorup unread, May 24, 2011, 4:53:34 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Modules have many uses, but they are often mixed up. Yesterday I made this drawing: PastedGraphic-1.pdf ATT00001..htm Vlad Dumitrescu's profile photo Vlad Dumitrescu unread, May 24, 2011, 5:00:23 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Hi, On Tue, May 24, 2011 at 10:06, Joe Armstrong wrote: Do we need module's at all? Erlang programs are composed of lots of small functions, the only place where modules seem useful is to hide a letrec. The classic example is fibonacci. We want to expose fib/1 but hide the helper function fib/3. Using modules we say -module(math). -export([fib/1]). fib(N) -> fib(N, 1, 0). fib(N, A, B) when N < 2 -> A; fib(N, A, B) -> fib(N-1, A+B, A). The downside is we have had to *invent* one module name math - whose *only* purpose is to hide the definition of fib/3 which we don't want to be made callable. If we put a second function into the module math, then this second function could call fib/3 which breaks the encapsulation of fib/3. We could say: let fib = fun(N) -> fib(N, 1, 0) end in fib(N, A, B) when N < 2 -> A; fib(N, A, B) -> fib(N-1, A+B, A). end. One reason where letrec might not be a good enough replacement for a helper function that is private to a module, is if this helper function is called from many places. I wouldn't want to have to repeat its definition every time I need it... I also feel like Mazen that unless there is a big shift in the way we think about the programs, the problem will still be present at the prefix level. If there is to be a big shift in the paradigms, then the question is whether the result will still be Erlang... regards, Vlad Kresten Krab Thorup's profile photo Kresten Krab Thorup unread, May 24, 2011, 5:05:03 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Sorry, my mail got strangled in flight. Here we go again... in plain text. The gen_server module is used for three different things: - as a collection of API function (left) - a place to put the code that controls the gen_server process (middle) - a place to describe the abstract notion of a gen_server behavior (right). Ideally, I think those three usages should be separate language constructs, but probably not in separate files (or other kinds of units). When using a gen_server, it would be nice if you did not have to expose the behavior api to the "user" of your module. Currently, the module is the unit of code loading. I think we could use a bette mechanism, perhaps one that supports an STM-like thing in the VM to control the loading of a set of modules. I don't have much deep experience with code loading in Erlang (but do have it on Java), and it just seems to me that being able to control atomic loading of .. say a new version of an Erlang application would be good to have supported in the VM. Gilad Bracha (who's working on Newspeak) has an excellent blog on new notions of modues that you should read. You can start here http:// gbracha.blogspot.com/2009/06/ban-on-imports.html One of the things he's arguing is that modules should be VALUES. A "module" in newspeak is kind of a template (function), which take other module instances as arguments. So you hook up a system by applying such templates (possibly using letrec). If we had a visual programming environment, then many things don't need a name. Self is an example of this (Self, the video: http:// video.google.com/videoplay?docid=5776880551404953752#) Kresten Xiaopong Tran's profile photo Xiaopong Tran unread, May 24, 2011, 5:31:18 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to erlang-q...@erlang.org On 05/24/2011 04:49 PM, Josh Johnston wrote: > > > In a normal language this would be a problem, but in erlang we can get > the desired behaviour by using pattern matching, right? > > So instead of > > > to_html() > resource_exists() > etc > > > We have > > resource(to_html, ...) > resource(exists, ...) > etc > > I'm not disagreeing with you - we do need some good way to do this. But > I don't think modules are necessary in this case. > Pattern matching might work here, but do away with modules and using pattern matching would a heavier load on the programmer. Modules or not, it does not have much effect on the VM, we can just make the VM work without modules. But I think programmers would have to work harder, distinguishing which function is which. Although I don't really like the Erlang module, but I think it's better with it than without. Xiaopong Jesper Louis Andersen's profile photo Jesper Louis Andersen unread, May 24, 2011, 5:46:08 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Kresten Krab Thorup, Erlang On Tue, May 24, 2011 at 11:05, Kresten Krab Thorup < kr...@trifork.com> wrote: > > One of the things he's arguing is that modules should be VALUES. A "module" in newspeak is kind of a template (function), which take other module instances as arguments. So you hook up a system by applying such templates (possibly using letrec). > This is a great idea. Why? Because it is the path of existential types :-) Notice that in Erlang, a module is a Value. It is possible to call Mod:Fun(..). Second, notice that the parameterized module extension allows us to parameterize a module over some other module. Specifically we could have -module(RBTree, [OrderMod]). where OrderMod is a module which has a function -opaque t() :: ..... -export_type([t/0]). -spec order(t(), t()) -> lt | eq | gt. and so we have generalized our RedBlack trees to be able to use any ordering relation, not simply the one that is built it. The style Gilad is advocating is the "Fully functorial style" from the Standard ML community. A Functor[1] in SML terminology is a function from module to module ("functor" stems from category theory where it designates a "function"[2] one level up). In the style, you build up programs by stitching together smaller modularized parts building larger and larger modules in the process. Example: The gen_server is a functor from a module (which obeys the gen_server behaviour and has handle_* functions) to a specific instance of a server. It also gives you quite the path towards the concept of OOP, as can be observed from the parameterized module extension and its uses in various places. In languages with static types, you can of course enforce that functor applications are type-correct. If you allow modules as values in such languages you get existential types which is a way to get something OOP-like into those languages[3]. So after a little tour around: Erlang already has this :P Modules are an essential part of building large systems. The very notion that you can pack up functions in a module and ship them as a unit is quite important. Also the notion you can replace a module-component at once. I feel that the problem of function placement in the right module is less of a problem compared to the hell it would be with one large global namespace. And it is much more than just a "bunch of functions thrown together". Specifically, modules allows me to have an *abstract* view of a data structure. In the above, I have no idea what t() is. I am only allowed to compare two t()'s to each other with OrderMod:compare(T1, T2). That is, the only algebraic operation I am allowed to do is this. Other parts of the program may have a different view of t() and be able to do more with it, but to me, it is an opaque term. It matters because someone can go replace t(), and my code does not have to be changed at all. This is impossible if t() leaks. And I am grateful the dialyzer can find places where it is leaking. Likewise, a module can hide the fact it is implemented with a gen_server. So I can go replace it with a gen_fsm, and nobody has to alter their code because the API i exported was exactly the API I will keep stable. Processes componentize the heap. Modules componentize the code. [1] Beware of connotating the word functor in programming with something specific in computer science. The meaning is different in Standard ML, in Haskell and in C++. [2] I am lying. The basic notion is a morphism of which the function is a special case when considering the category of Sets. [3] I still don't get why you can't use static typing in Erlang. The conservative approach is simply to use polymorphic variants for all message passing and it would essentially work right out of the bat. -- J. Tim Carpenter's profile photo Tim Carpenter unread, May 24, 2011, 6:03:53 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Hi all, I am sticking to the request to not comment on other solutions but present my own thoughts... 1. Some units of code only make sense as a collection. who would use part of gen_server in isolation? Well, you could,but to have it separate exposes it to modification that could distort the focused purpose of the unit. 2. some functions have intentionally hidden aspects, private calls etc and these again are best left hidden or considered as a logical whole otherwise side effects or bloat may occur. 3. I see no reason why we cannot logically group functions into modules but not have the code in there...i.e. the module contains the references to functions that make it up. this allows logical handling of groups of functions for code management and can allow one function to appear in multiple modules without code duplication. it also allows one to add/delete/replace a function in a module knowing that the source code for the other functions have not been touched! 4. naming conventions should be given great thought for simplicity and intuitiveness. functions could be in n sets, so a strict hierarchy may not suit. 5. if functions are named as being in modules, the need to provide abstraction to the full name may be advantageous, so modulename.sort/ 3 can be used and the module definition translates this to a unique longhand which can change over time. The programmer should also be able to use the longhand at will using the module as a convenient bulk import/compile vehicle that will break at compile time (one would hope). Some form of 'expects...' notation could allow a form of protection against inadvertent swapping of functions in shared modules altering behaviour. 6. we may want 3 elements to naming of functions: shorthand, longhand, and version, so one can use the latest version of a longhand named function, lock to a specific version of a longhand named function or use the shorthand latest version or other such combinations. 7. n- tier modules - modules of modules? possibly. Tim Tim Carpenter Amphibian Ltd London ENGLAND +44(0)7900 888826 amphib...@gmail.com Bengt Kleberg's profile photo Bengt Kleberg unread, May 24, 2011, 6:39:18 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Greetings, It would be very nice to have module renaming/namespacing that allowed me to do the following: The modules "misc_lib" and "misc_lib_helper" from developer A should be able to call each other with their original names. The modules "misc_lib" and "misc_lib_helper" from developer B should be able to call each other with their original names. I should be able to call all 4 at the same time with names that I have create/renamed. So, I need to load modules as a set/application/??? that renames them outside of that set (it would be ok to have the same prefix added to all of them), and still allows the files in the set/application/??? to keep using to the old names. When it comes to improving stdlibs I suggest that we create new, better modules. Keep the old ones (for ever?) but print a big "deprecated" all over the man pages. bengt On Tue, 2011-05-24 at 12:26 +0200, Jesper Louis Andersen wrote: > On Tue, May 24, 2011 at 10:06, Joe Armstrong wrote: > > Why do we need modules at all? > > I think we need modules, but this post sparked a thought I have had > for some time now. > > I'd like to have module renaming/namespacing! > > Specifically, I want a mechanism by which I can clean up the naming > mess in the stdlib without affecting old programs. That is, I want to > be able to, at an application/module level, rename function calls in a > way such a way that can clean up parameter orders and so on. I don't > care about how the solution is. I am after the goal: Cleaning up the > stdlib mess. > > You may argue this is a messy thing, but I think it is a necessity if > you want to move the standard library on: > > * We can't alter existing functions because people rely on them. > * Existing functions are not consistent. > > This is a social deadlock I want to break. In fact it is a deadlock I > think we can only break if we allow the coexistence of more than one > module with the same name (but stemming from different namespaces). I > don't want nesting. Just two levels, so I can say > > -use_namespace(v2_modules). > > in modules which use version two of the stdlib in which things are > sensibly named. At some point you can then flip around and compile old > code with a compatibility layer. Gary Hai's profile photo Gary Hai unread, May 24, 2011, 6:19:59 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang I am working on state-oriented programming model, and I have the same question as you. In the view of SOP, erlang process is a FSM, and suitable granularity of functions as states. The finite set of states of a FSM just like erlang module, but it always cross modules boundaries. So in SOP model, the module concept has been very vague, just for unique distinct name of function. The model of a state activity is a function as bellow: act(Entity, Input) -> {ok, Output} | {ok, Output, UpdatedEntity} | {error, Error} | {stop, Output, Entity} The output of the state may be {NextState, Output}. The NextState may be the next state activity function or just a directive of next state. Then FSM engine matches the directive to a state activity then transfers the previous state to the new state with the previous output as input. The state activity function may be from anywhere, function in a module, binary loaded from database, code dynamically generated by JIT. -------- Gary Hai Jesper Louis Andersen's profile photo Jesper Louis Andersen unread, May 24, 2011, 6:26:51 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang On Tue, May 24, 2011 at 10:06, Joe Armstrong wrote: > Why do we need modules at all? I think we need modules, but this post sparked a thought I have had for some time now. I'd like to have module renaming/namespacing! Specifically, I want a mechanism by which I can clean up the naming mess in the stdlib without affecting old programs. That is, I want to be able to, at an application/module level, rename function calls in a way such a way that can clean up parameter orders and so on. I don't care about how the solution is. I am after the goal: Cleaning up the stdlib mess. You may argue this is a messy thing, but I think it is a necessity if you want to move the standard library on: * We can't alter existing functions because people rely on them. * Existing functions are not consistent. This is a social deadlock I want to break. In fact it is a deadlock I think we can only break if we allow the coexistence of more than one module with the same name (but stemming from different namespaces). I don't want nesting. Just two levels, so I can say -use_namespace(v2_modules). in modules which use version two of the stdlib in which things are sensibly named. At some point you can then flip around and compile old code with a compatibility layer. -- J. Tim Watson's profile photo Tim Watson unread, May 24, 2011, 7:19:27 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Jesper Louis Andersen, Erlang My 2 pennies worth: I don't think getting rid of modules is a good plan. Having global (mangled) names is a path to misery. In the languages that do mess with names (python, C++, etc) it creates a bloody nightmare. I would steer clear of this. I like the idea of having a global database of code, but not all of it floating in some etherial top level namespace. Most of what's cool about http://www.haskell.org/hoogle/ and http://hackage.haskell.org/packages/hackage.html is made possible by the combination of good standardised build tools and hosting repositories, so I'd be more included to focus on improving (and extending) things like rebar and agner/erlware to provide the same kind of developer-experience without changing the language so much. Introducing letrec is a bloody marvelous suggestion. Also making it easier to do partial application and function composition would simplify a lot of the re-use cases without messing with the module system. I'm not going to stray off too far into alternative-solution land, but given the OTP team reads this list, I don't want to miss the opportunity to mention some minor things about the module system that could do with an overhaul. - Fully supported parameterised modules Not because I want to use them as a state bag, but rather that they make it easy to implement something akin to a functor without having to reside in a process. This means that all the tools need to support this concept and it is documented, etc. Extending the module system so that you can -extend a module to create a new concrete implementation would be useful: %% finder.erl -module(finder). -functor([M]). -compile(export_all). find() -> walk(fun M:match/3). %% rx_finder.erl -module(rx_finder). -extends(finder, [re]). Perhaps the syntax/approach there isn't quite right, but the point I'm getting at is that what with parameterised modules and -extends, you should be able to compose stuff easily. - Better support for *import* As the quantity of code you're dealing with increases, you need to be able to import discrete units of it in simpler and more intuitive ways. I'm not exactly a fan of Java, but you can see in that world that the quantity of libraries (often open source ones) being used on any project means you simply cannot avoid having to import things (packages, classes, static methods/functions) without going mad. Although Erlang is still (thankfully) simple enough not to have this problem, I suspect as the number of projects blossoms it will eventually approach the critical mass. Also being able to support something like -import_alias(name, mod) would be quite handy, although I realise this is fairly trivial with a parse_transform. - Fully supporting packages Personally I think the __ convention that most of us follow is fine, but it does get rather tedious. Package, namespaces or nested modules are all solution that are known to work and are widely adopted by other languages. The package system in erlang works fine, but not all the tools properly support it today (cover, reltool and a few others are broken). Tim Watson's profile photo Tim Watson unread, May 24, 2011, 7:21:51 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to bengt....@ericsson.com, Erlang On 24 May 2011 11:39, Bengt Kleberg wrote: > Greetings, > > It would be very nice to have module renaming/namespacing that allowed > me to do the following: > The modules "misc_lib" and "misc_lib_helper" from developer A should be > able to call each other with their original names. > The modules "misc_lib" and "misc_lib_helper" from developer B should be > able to call each other with their original names. > I should be able to call all 4 at the same time with names that I have > create/renamed. > > So, I need to load modules as a set/application/??? that renames them > outside of that set (it would be ok to have the same prefix added to all > of them), and still allows the files in the set/application/??? to keep > using to the old names. > > I was thinking about this in terms of some kind of special -import or -alias directive. I think it's possible to do just with a parse_transform, as long as the set can be fully determined at compile time. That last point though - therein lies the rub. App-upgrades change things, and those things can include modules. Dragons. Nuff-said. Masklinn's profile photo Masklinn unread, May 24, 2011, 7:24:25 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Watson, Erlang On 2011-05-24, at 13:19 , Tim Watson wrote: > In the languages that do mess with names (python [...] As far as I know, Python never "messes with names" unless asked explicitly via the "__" prefix, which is rarely used and generally only there to avoid naming conflicts in inheritance cases. Do you have instances, besides this quite explicit one, where Python performs any name mangling? Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 7:25:52 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Jesper Louis Andersen, Erlang If we think of modules as containers of functions, then a module is a "set of functions" - my problem is with granularity. The smallest unit of reuse is the module. Suppose my application want just one function from a particular module - I'm forced to load the entire module. This hampers reuse - I *often* read library code and find a single function and cut-and-paste it into my new code. It also discourages sharing - since the smallest unit I can share is a module. The unit of distribution is an application - a set of modules - why this could not be a list of functions I do not know. I think I'd like to view an application as something that offered a message based service that internally was constructed from a set of functions rather than a set of modules. /Joe Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 7:28:53 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Jesper Louis Andersen, Erlang On Tue, May 24, 2011 at 12:26 PM, Jesper Louis Andersen < jesper.lou...@gmail.com> wrote: On Tue, May 24, 2011 at 10:06, Joe Armstrong wrote: > Why do we need modules at all? I think we need modules, but this post sparked a thought I have had for some time now. I'd like to have module renaming/namespacing! I agree - I still like the good 'ol Knuthian way - start with source file, then add patch files. You could make a language could make a language construct for this. new_foo = alias foo/2 Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 7:46:17 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Watson, Erlang On Tue, May 24, 2011 at 1:21 PM, Tim Watson wrote: On 24 May 2011 11:39, Bengt Kleberg wrote: > Greetings, > > It would be very nice to have module renaming/namespacing that allowed > me to do the following: > The modules "misc_lib" and "misc_lib_helper" from developer A should be > able to call each other with their original names. > The modules "misc_lib" and "misc_lib_helper" from developer B should be > able to call each other with their original names. > I should be able to call all 4 at the same time with names that I have > create/renamed. > > So, I need to load modules as a set/application/??? that renames them > outside of that set (it would be ok to have the same prefix added to all > of them), and still allows the files in the set/application/??? to keep > using to the old names. > > I was thinking about this in terms of some kind of special -import or -alias directive. I think it's possible to do just with a parse_transform, as long as the set can be fully determined at compile time. That last point though - therein lies the rub. App-upgrades change things, and those things can include modules. Dragons. Nuff-said. It depends upon what -import means. -import(foo, [bar/2 ]). Where are two interpretations a) "when you *run* this program and find a call to foo:bar/2 go and see if you can find the code for foo, load it then call the function bar/2" b ) "When you *compile* this program find the code for foo:bar/2 and extract the code for bar/2 and include it in my code" - in this case you resolve the name at compile time. I guess when developing a) is useful but when you deploy code it should be b) /Joe Tim Watson's profile photo Tim Watson unread, May 24, 2011, 7:50:03 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang > Now I find this very convenient when I write a new small utility function > I stick in in elib1_misc.erl - no mental anguish in choosing a module > name is involved. But it's not very helpful to someone else who's going to have to mentally parse your lib_misc library to figure out what they're looking for. It does sound a bit like a dumping ground. > > The observation that I find this very-convenient is telling me something > about modules - I like my elib1_misc it feels right. > > (aside - It seems many development projects have their own private > lib_miscs ...) Perhaps if those libraries were broken into their constituent parts and then those into their constituent projects, they would be easier to re-use. I see the same thing on many projects - a _config.erl that wraps a gen_server with maybe some calls to application:get_env and a _log.erl and so on. I nearly wrote a erl-config package on github until I realised gproc would be a better choice for my needs. > > Which brings me to the point of my question. > > Do we need module's at all? Erlang programs are composed of lots of small > functions, the only place where modules seem useful is to hide a letrec. > I think that's an oversimplification. Compilation units (e.g., putting code in different files) is beneficial for humans first and foremost. I think that categorising functions based on subject-area and/or the process they relate too is very useful. Once you do that categorising, putting all the categories in once place is very sensible. OCaml and Haskell programmers (of which I'm only experiences with the former), use letrec (and similar concepts such as Haskell's `where`) heavily, but would not dream of doing away with our module systems just because we can encapsulate things in this way. Modules are a useful categorisation for the human mind to remember things by. Like packages, they help us remember where to look for things. They're also useful for hiding - breaking encapsulation is a choice, not something you cannot help when designing a module. > The downside is we have had to *invent* one module name math - whose *only* > purpose is to hide the definition of fib/3 which we don't want to be made > callable. > Actually if you put all your mathematical functions in this module, it would be more useful (and easier to remember). > If we put a second function into the module math, then this second function > could call fib/3 which breaks the encapsulation of fib/3. Not sure that's such a terrible thing really. > > We could say: > > let fib = fun(N) -> fib(N, 1, 0) end > in > fib(N, A, B) when N < 2 -> A; > fib(N, A, B) -> fib(N-1, A+B, A). > end. > Having letrec (or where) is an awesome idea. Don't actually care much about the choice of syntax - just another one to learn. > Does the idea of a module come from the idea that functions have to be > stored somewhere, so we store them in a file, and we slurp the > file (as a unit) into the system, so the file becomes a module? Probably, yes. It is convenient and familiar to do so. > > If all the files were store by themselves in a database would this > change things. > Not really, as long as the database is always available and I *never ever* run into a situation where I can't work because I'm not connected. > I am thinking more and more that if would be nice to have *all* functions in > a key_value database with unique names. I love the idea of a massive code repo/server, but having everything in one namespace sounds like a nightmare. > The unique names bit is interesting - is this a good idea. Qualified > names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good > idea but but when I'm programming I have to invent the xxx or the > a.b.c which is very difficult. It also involves the "decision problem" > if the namespaces xxx and a.b.c already exist I have to *choose* which > to put my new function in. Still I think it's the right thing to do. > > I think there might be a case for alises here joe:foo/2 could be used > while developing "joe" would expand to a horrible random local string the > real name being ab123aZwerasch123123_foo/2 but I would not be able to > publish my code or make it available to a third_part before I had > chosen a sensible name. Yuk. ;) > > (( managing namespaces seems really tricky, a lot of peoople seem > to thing that the problem goes away by adding "." 's to the name > but managing a namespace with namees like foo.bar.baz.z is just as complex > as managing a namespace with names like foo_bar_baz_z or names like > 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic > name like www.a.b to a reference like 123.45.23.12 - but how do we discover > the initial name www.a.b? - there are two answers - a) we are given the name > (ie we click on a link) - we do not know the name but we search fo it )) You're right it is hard, but it is possible. > > > When programs are small we can live with "just the code" in "a few > modules" the ratio of code to meta data is high. > > When programs are large we need a lot of meta-data to understand them. > > I would like to see all functions with all meta-data in a data base. > > I'd like to say: > > lookup(foo,2,Attribute) when Attribute = > > code|source|documentation|type signatures|revision history| authors|... > I like the idea of finding a function based on a wider range of attributes such as these, but not choosing the route of dispatch based on a loose predicate. Lookup/3 would have to guarantee that it will resolve to a single callee every time, which I can't see being possible. Tim Watson's profile photo Tim Watson unread, May 24, 2011, 7:54:44 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang > Where are two interpretations > > a) "when you *run* this program and find a call to foo:bar/2 go and see if > you can find the code > for foo, load it then call the function bar/2" > > b ) "When you *compile* this program find the code for foo:bar/2 and > extract the code for bar/2 and include it in my code" - in this case you > resolve the name at compile time. > > I guess when developing a) is useful but when you deploy code it should be > b) > I completely agree. I'm also quite willing to loose the convenience of (a) providing the compiler is fast enough. Max Lapshin's profile photo Max Lapshin unread, May 24, 2011, 7:56:18 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Very strange topic for me. I'd like to know if there will be hierarchial modules in Erlang, because tree of packages is a rather good idea: erlyvideo.mpegts.encoder erlyvideo.rtp.encoder But plain module namespace is also ok. It would be impossible for me to work with 30K LOC with plain function namespace. Tim Watson's profile photo Tim Watson unread, May 24, 2011, 8:05:36 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang > But plain module namespace is also ok. It would be impossible for me > to work with 30K LOC with plain function namespace. I think what Joe is getting at - he'll correct me if I'm wrong about this - is to refer to functions not only by name, but by other metadata as well. I'm not sure what this looks like empirically, as like you I can't see how using only function names you could work with a flat namespace. Looking up functions by other (additional) metadata such as type signature, author, date, company, category (mutiples?), pre/post conditions, safe guarantees, etc - that would work. In fact if you think about a process engine (such as BPMS solutions and the like) then process (i.e., function) is often selected dynamically based on many attributes, not just name. Just how this will work in practise as a replacement for existing (shall we say normal) way of referring to a function that is in a module, I could not say. But it is an interesting concept. Perhaps using a URI as the name and thereby being able to have multiple URIs representing the various different category-based paths to identify the function - the "aliases" could then redirect to the direct, uniquely named resource which would in turn return the function metadata (and binary code presumably). Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 8:45:29 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang On Tue, May 24, 2011 at 1:56 PM, Max Lapshin wrote: Very strange topic for me. I'd like to know if there will be hierarchial modules in Erlang, because tree of packages is a rather good idea: No it's not - this has been the subject of long and heated discussion and is why packages are NOT in Erlang - many people - myself included - dislike the idea of hierarchical namespaces. The *dot* in the name has no semantics it's just a separator. The name could equally well be encoders.mpg.erlyvideo or mpg.applications.erlvideo.encoder - there is no logical way to organise the package name and it does not scale - erlyvideo.mpegts.encoder erlyvideo.rtp.encoder But plain module namespace is also ok. It would be impossible for me to work with 30K LOC with plain function namespace. The English language has a flat namespace. I'd like a drink.alcoholic.beer with my food.unhealthy.hamburger and my food.unhealthy.national.french.fries I have no problem with flat beer and chips. /Joe Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 8:53:18 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Watson, Erlang On Tue, May 24, 2011 at 2:05 PM, Tim Watson wrote: > But plain module namespace is also ok. It would be impossible for me > to work with 30K LOC with plain function namespace. I think what Joe is getting at - he'll correct me if I'm wrong about this - is to refer to functions not only by name, but by other metadata as well. I'm not sure what this looks like empirically, as like you I can't see how using only function names you could work with a flat namespace. Looking up functions by other (additional) metadata such as type signature, author, date, company, category (mutiples?), pre/post conditions, safe guarantees, etc - that would work. In fact if you think about a process engine (such as BPMS solutions and the like) then process (i.e., function) is often selected dynamically based on many attributes, not just name. Yes - how this would work I don't know - my Haskell friends say that searching for function by type signatures is good. What do I do today when I suspect that somebody has written some code ? 1) will it take me < ten minutes to write the code? yes - write it 2) google or ask a friend 3) this gets me to git or sowhere 4) search 5) cut-and-paste Not a good method - and this is *best practise* Just how this will work in practise as a replacement for existing (shall we say normal) way of referring to a function that is in a module, I could not say. But it is an interesting concept. Perhaps using a URI as the name and thereby being able to have multiple URIs representing the various different category-based paths to identify the function - the "aliases" could then redirect to the direct, uniquely named resource which would in turn return the function metadata (and binary code presumably). Something like this. Actually for development it would be great to say -import("http://a.b.c/modname", [foo/2]). this would just make an rcp to foo/2 on the remote host and save all the pain of locally installing the necessary code on my machine. /Joe Bengt Kleberg's profile photo Bengt Kleberg unread, May 24, 2011, 9:07:08 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Greetings, Perhaps it is possible to do parse_transform when loading, but I have only heard about this at compile time. I meant a way to load modules, not compile them. bengt Frederic Trottier-Hebert's profile photo Frederic Trottier-Hebert unread, May 24, 2011, 9:10:17 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang The problem of Erlang with regards to namespaces and modules goes further than just module names. In Erlang, the following units can be sharing/clashing namespaces: the process registry, ETS tables, module names, application names, shared records, etc. They're currently entirely disjoint and only held together by convention. However, any change to one of the solutions needs to somehow be reflected into the others, even if it's still by convention. IMO the basic hierarchy of Erlang seems to begin with the OTP release. The name of an OTP application will decide the prefixes used in module names, process names, table names, shared record names, etc. There is nothing to check that application names are unique except the programmer looking out for that, while xref and the compiler can help a bit with the rest. I think we can generalise OTP to have this given kind of organisation: release -> OTP applications -> [Modules, Tables, Shared Records, Processes] An application itself wouldn't have to worry about namespaces (if we forget included applications). The namespace problem comes at the release stage or if we're developing entirely outside of OTP, still using applications or their equivalent. To me, this makes it feel like namespace management/giving aliases and replacement of values should be managed from the release point of view, but I can't offer any suggestions as to how this could take place without relying on file renaming and strict use of ? NAMESPACE-like macros throughout the code. Nothing easily backwards compatible, and nothing to help people who are not using releases though. This in no way would solve Joe's problem with being undecided as to where functions should go, but I think getting rid of modules altogether in Erlang wouldn't be doable because of the other dependencies. In places like Javascript (client-side) or schemes without good module functionality, you pretty much just include the files and people then tend to wrap everything in functions or objects to provide an ad-hoc equivalent to namespaces or modules. I can see this behaviour emerging if Erlang were to go that way. -- Fred Hebert http://www.erlang-solutions.com Attila Rajmund Nohl's profile photo Attila Rajmund Nohl unread, May 24, 2011, 9:11:59 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to erlang-questions 2011/5/24, Joe Armstrong : [...] > What do I do today when I suspect that somebody has written some code ? > > > 1) will it take me < ten minutes to write the code? > yes - write it > 2) google or ask a friend > 3) this gets me to git or sowhere > 4) search > 5) cut-and-paste > > Not a good method - and this is *best practise* This only works if the function in itself useful. Somehow I don't think this is the case the most time. If I need not a function, but a functionality, there's a chance there is an ets table behind or a process, some configuration file setup - then the whole "import a single function" doesn't help, but just adds complications that even the last helper function has to have a globally unique name... Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 9:16:24 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Frederic Trottier-Hebert, Erlang 2011/5/24 Frederic Trottier-Hebert The problem of Erlang with regards to namespaces and modules goes further than just module names. In Erlang, the following units can be sharing/clashing namespaces: the process registry, ETS tables, module names, application names, shared records, etc. They're currently entirely disjoint and only held together by convention. Absolutely - and inconsistently We can say Pid = spawn(fun() -> ... end) Pid ! M register(name, Pid), name ! Pid But not Mod = load([ .... lists of funs ...]) Mod:reverse(...) register(lists, Mod) lists:reverse(..) same comment for ets tables etc. However, any change to one of the solutions needs to somehow be reflected into the others, even if it's still by convention. IMO the basic hierarchy of Erlang seems to begin with the OTP release. The name of an OTP application will decide the prefixes used in module names, process names, table names, shared record names, etc. There is nothing to check that application names are unique except the programmer looking out for that, while xref and the compiler can help a bit with the rest. I think we can generalise OTP to have this given kind of organisation: release -> OTP applications -> [Modules, Tables, Shared Records, Processes] An application itself wouldn't have to worry about namespaces (if we forget included applications). The namespace problem comes at the release stage or if we're developing entirely outside of OTP, still using applications or their equivalent. To me, this makes it feel like namespace management/giving aliases and replacement of values should be managed from the release point of view, but I can't offer any suggestions as to how this could take place without relying on file renaming and strict use of ?NAMESPACE-like macros throughout the code. Nothing easily backwards compatible, and nothing to help people who are not using releases though. This in no way would solve Joe's problem with being undecided as to where functions should go, but I think getting rid of modules altogether in Erlang wouldn't be doable because of the other dependencies. In places like Javascript (client-side) or schemes without good module functionality, you pretty much just include the files and people then tend to wrap everything in functions or objects to provide an ad-hoc equivalent to namespaces or modules. I can see this behaviour emerging if Erlang were to go that way. I agree - it's not just "getting rid of modules" there's a whole load of other stuff. Tim Carpenter's profile photo Tim Carpenter unread, May 24, 2011, 9:33:35 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang (am sending again as this did not seem to get through) Hi all, I am sticking to the request to not comment on other solutions but present my own thoughts... 1. Some units of code only make sense as a collection. who would use part of gen_server in isolation? Well, you could,but to have it separate exposes it to modification that could distort the focused purpose of the unit. 2. some functions have intentionally hidden aspects, private calls etc and these again are best left hidden or considered as a logical whole otherwise side effects or bloat may occur. 3. I see no reason why we cannot logically group functions into modules but not have the code in there...i.e. the module contains references to functions that make it up. this allows logical handling of groups of functions for code management and can allow one function to appear in multiple modules without code duplication. it also allows one to add/delete/replace a function in a module knowing that the source code for the other functions have not been touched! 4. naming conventions should be given great thought for simplicity and intuitiveness. functions could be in n sets, so a strict hierarchy may not suit. 5. if functions are named as being in modules, the need to provide abstraction to the full name may be advantageous, so modulename.sort/ 3 can be used and the module definition translates this to a unique longhand which can change over time. The programmer should also be able to use the longhand at will using the module as a convenient bulk import/compile vehicle that will break at compile time (one would hope). Some form of 'expects...' notation could allow a form of protection against inadvertent swapping of functions in shared modules altering behaviour. 6. we may want 3 elements to naming of functions: shorthand, longhand, and version, so one can use the latest version of a longhand named function, lock to a specific version of a longhand named function or use the shorthand latest version or other such combinations. 7. n- tier modules - modules of modules? possibly. Tim Tim Carpenter Amphibian Ltd London ENGLAND +44(0)7900 888826 amphib...@gmail.com _______________________________________________ Tim Watson's profile photo Tim Watson unread, May 24, 2011, 9:36:36 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang > Yes - how this would work I don't know - my Haskell friends say that > searching for > function by type signatures is good. I've only used OCaml commercially, but what little Haskell I've played with this is a very useful feature (hoogle and hackage combined with cabal) for finding and using stuff. > > What do I do today when I suspect that somebody has written some code ? > > > 1) will it take me < ten minutes to write the code? > yes - write it > 2) google or ask a friend > 3) this gets me to git or sowhere > 4) search > 5) cut-and-paste > > Not a good method - and this is *best practise* Not quite. At point (3) you probably get to github, bitbucket or some such. At that point, you *may* be able to reuse the library and/or application code in your project, in which case you install it with epm/sutro/agner or pull it in as a dependency using rebar. Only when the code is hidden away in the module and not reusable do you have to copy-paste - I think it's this (latter) case that you're not happy with and I agree it's poor show. By way of example, I'm working on a node/cluster monitoring tool (primarily a web application with web sockets notifications). I built the library code (for monitoring) by re-using the eper (performance monitoring) library and a couple of other things (zero-conf for network discovery and whatnot). I built the web application on misultin for its websocket support. I also have some common (to the project) library code which uses the esl/parse_trans library from github to support better use of records and things like that. For the database application, I used the esl/setup library from github to separate the mnesia database/schema setup from the general purpose use cases and to generate (run once) setup scripts and the like. None of this is copy and paste re-use. > Actually for development it would be great to say > > -import("http://a.b.c/modname", [foo/2]). > > this would just make an rcp to foo/2 on the remote host and save all the > pain of locally installing > the necessary code on my machine. I think it would be worth distinguishing between two disparate concerns here. 1. Where does the (binary or other representation of AST) code come from 2. Where does the/any (runtime) state reside I don't always want to RPC just to run a library routine, especially if that call is happing frequently in a tight "loop" and the code needs to run fast. Much as I love Erlang's location transparency, one cannot just assume that the network isn't there. Things I think are potentially awesome here are: 1. Importing from a URI 2. Having the binary code in a repository that is searchable, versioned, etc In fact this is hardly an unfamiliar model - consider couchdb and/or riak. When you define a map-reduce function as say javascript, you distribute/upload this function definition to the server for later use. I think my two major concerns (that are by no means unsurmountable) are: 1. wanting to be able to work when I'm not connected to the code server 2. wanting to be able to define the import based on more than just the name Point (2) is telling. In your example, you imported from http://a.b.c/modname - you still haven't gotten rid of the module. I know it was a flippant example and besides the point. Question is, can you define the search in such a way as to determine exactly where the dispatch must go? The Haskell folks say searching by type is immensely useful, and they're right. But they don't compile based on this - they find what they're looking for and then reference it (explicitly) by name. Tim Watson's profile photo Tim Watson unread, May 24, 2011, 9:37:47 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to bengt....@ericsson.com, Erlang On 24 May 2011 14:07, Bengt Kleberg wrote: > Greetings, > > Perhaps it is possible to do parse_transform when loading, but I have > only heard about this at compile time. I meant a way to load modules, > not compile them. > You could hack something together to do it at runtime if you're importing/loading from an AST rather than a blob. I take your point though - I did misunderstand slightly. Max Lapshin's profile photo Max Lapshin unread, May 24, 2011, 9:38:19 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Watson, Erlang I don't understand what are you speaking about. How do you imagine this magic function namespace in terms of plain files? Tim Watson's profile photo Tim Watson unread, May 24, 2011, 9:44:10 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang On 24 May 2011 14:38, Max Lapshin wrote: > I don't understand what are you speaking about. > > How do you imagine this magic function namespace in terms of plain files? > You forget about files. You're code editor (IDE) is now a browser based application. You search for things using a sidebar that ultimately constructs a query to the code server to look for things based on name/version/type-signature/author/safety/etc which becomes a URL. You import things based on URLs. Your code is compiled and hosted on the server. The local file system doesn't have to come into it per se. This last point is partly one of my concerns though. I don't always have an internet connection and something that doesn't work without one is potentially going to cause me problems. Perhaps if it was easy for me to copy the entire repository and run it locally, synchronising when I'm reconnected - that would do nicely. Then the service would be providing something akin to source version control as well. My head now hurts. :) Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 10:07:01 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang On Tue, May 24, 2011 at 3:38 PM, Max Lapshin wrote: I don't understand what are you speaking about. How do you imagine this magic function namespace in terms of plain files? There are no plain files. The functions that you edit and manipulate are stored in a database. When you need to call a library function you query the database it helps you find the function you need from the data in the database - you never know the real name of the function or anything about the namespace it's in. The problem with plain files is that they become unmanageable when there are lots of them. A few years ago I had problem with storage - disks weren't big enough - now I have TBs of raid storage and the next problem arises - how to find stuff. Finding my own code is difficult - it's just a matter of searching 43 K erlang modules on my local disk - finding other peoples code is worse - Google and "ask a friend" is the best solution I know of. As systems get very large the notion of files and modules seems to break down, to be replaced by "search-able stuff in a database" - I'm suggesting that the smallest unit that should be searchable/reusable/discoverable should be the function. And that to make it searchable we need to add a lot of meta data to the functions. Barry Skidmore's profile photo Barry Skidmore unread, May 24, 2011, 10:14:25 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang This conversation is reminiscent of the teapot/teacup architecture from TCL 8.4 http://wiki.tcl.tk/7579 The description of that architecture may be on target? On May 24, 2011, at 9:10 AM, Max Lapshin wrote: > I can't imagine myself working with system, that uses non-plain files > for storing source code. > Nobody will fix for me Textmate, git or ack to work with this database. > > Modules give your clear structure of functions. Without them you will > end with mpegts_encode, mpegts_decode, > and will break your head when there will be 3 or 4 underscores in name. > _______________________________________________ > erlang-questions mailing list > erlang-q...@erlang.org > http://erlang.org/mailman/listinfo/erlang-questions Thanks, Barry Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 10:15:34 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Carpenter, Erlang 2011/5/24 Tim Carpenter (am sending again as this did not seem to get through) Hi all, I am sticking to the request to not comment on other solutions but present my own thoughts... 1. Some units of code only make sense as a collection. who would use part of gen_server in isolation? Well, you could,but to have it separate exposes it to modification that could distort the focused purpose of the unit. 2. some functions have intentionally hidden aspects, private calls etc and these again are best left hidden or considered as a logical whole otherwise side effects or bloat may occur. 3. I see no reason why we cannot logically group functions into modules but not have the code in there...i.e. the module contains references to functions that make it up. this allows logical handling of groups of functions for code management and can allow one function to appear in multiple modules without code duplication. it also allows one to add/delete/replace a function in a module knowing that the source code for the other functions have not been touched! I checked some numbers. The OTP release has 1793 modules and in these there are 56317 functions with different names. 11887 functions are defined in only one module, ie the for 11887 functions the name alone uniquely identifies the module. The distribution of functions over modules has a long tail with strange spikes, ie most function names are in very small number of modules. But some function names like handle_call etc and in a very large number of modules. Searching for a function using the function name as a key would only lead to a small number of candidate module if we exclude names like handle_call. it would be interesting to do this with type signatures ... 4. naming conventions should be given great thought for simplicity and intuitiveness. functions could be in n sets, so a strict hierarchy may not suit. 5. if functions are named as being in modules, the need to provide abstraction to the full name may be advantageous, so modulename.sort/3 can be used and the module definition translates this to a unique longhand which can change over time. The programmer should also be able to use the longhand at will using the module as a convenient bulk import/compile vehicle that will break at compile time (one would hope). Some form of 'expects...' notation could allow a form of protection against inadvertent swapping of functions in shared modules altering behaviour. 6. we may want 3 elements to naming of functions: shorthand, longhand, and version, so one can use the latest version of a longhand named function, lock to a specific version of a longhand named function or use the shorthand latest version or other such combinations. I've been thinking about this for a while - sometimes we need a very precise name (including an MD/SHA checksum) othertimes a sloppy name - it depends upon the context. 7. n- tier modules - modules of modules? possibly. I think these are "views" not names - we have functions and then different ways to view them, in collections of different kinds Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 10:22:25 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang On Tue, May 24, 2011 at 4:10 PM, Max Lapshin wrote: I can't imagine myself working with system, that uses non-plain files for storing source code. Nobody will fix for me Textmate, git or ack to work with this database. Why not? - a plain text file is an abstraction - in reality its a collection of magnetised zones on a disk, or electron spin states or something, this gets turned into a collection of bits, the bits are rendered in a editor. Plain text doesn't exist - it's just an abstraction. The underlying representation of the data you see in a browser may come from a file or as the result of a datebase query, you will never know which it was. Modules give your clear structure of functions. Without them you will end with mpegts_encode, mpegts_decode, and will break your head when there will be 3 or 4 underscores in name. or should the name be encode_mpegts and decode_mpegts the problem with structured names is choosing the name - not the fact there is structure in the name. Imagine the www was just one big structured name and there were no search engines how would you ever find anything? /Joe Cyryl Plotnicki-Chudyk's profile photo Cyryl Plotnicki-Chudyk unread, May 24, 2011, 10:30:23 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang On Tue, May 24, 2011 at 4:22 PM, Joe Armstrong wrote: > > Why not? - a plain text file is an abstraction - in reality its a collection > of magnetised zones > on a disk, or electron spin states or something, this gets turned into a > collection of > bits, the bits are rendered in a editor. Plain text doesn't exist - it's > just an abstraction. > How about storing the code in the old school plain text files but with additional, also plain text, index/metadata files ? That would satisfy both the searchability and legacy compatibility -- cyryl Anthony Ramine's profile photo Anthony Ramine unread, May 24, 2011, 10:31:43 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Le 24 mai 2011 a 14:45, Joe Armstrong a ecrit : > On Tue, May 24, 2011 at 1:56 PM, Max Lapshin wrote: > Very strange topic for me. > > I'd like to know if there will be hierarchial modules in Erlang, > because tree of packages is a rather good idea: > > No it's not - this has been the subject of long and heated discussion and is > why packages are NOT in Erlang - many people - myself included - dislike > the idea of hierarchical namespaces. The *dot* in the name has no semantics > it's just a separator. The name could equally well be encoders.mpg.erlyvideo > or mpg.applications.erlvideo.encoder - there is no logical way to organise the > package name and it does not scale - packages are NOT in Erlang? Then the related code should be removed because erl.lang.number:plus(1, 1) definitely works. Also, I think the Haskell guys would disagree about packages not scaling. -- Anthony Ramine Dev:Extend http://dev-extend.eu Jack Moffitt's profile photo Jack Moffitt unread, May 24, 2011, 10:32:37 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Max Lapshin, Erlang > I can't imagine myself working with system, that uses non-plain files > for storing source code. > Nobody will fix for me Textmate, git or ack to work with this database. I remember when Zope worked like this. I hated it, and to my knowledge, none of the editors picked up support for it despite it being pretty popular at the time. I believe they later changed to uisng the normal filesystem, but it's been a long time since I paid any attention to that project. jack. Robert Raschke's profile photo Robert Raschke unread, May 24, 2011, 10:41:28 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang For some reason this reminds me of a cross between Smalltalk and Eiffel. Robby Jesper Louis Andersen's profile photo Jesper Louis Andersen unread, May 24, 2011, 10:43:54 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Anthony Ramine, Erlang On Tue, May 24, 2011 at 16:31, Anthony Ramine wrote: > > packages are NOT in Erlang? Then the related code should be removed because > erl.lang.number:plus(1, 1) definitely works. > The problem is that a package hierarchy enforces a tree-like structure. And we may not want the tree. You might want the same module to reside in multiple packages in the hierarchy at the same time. We are past 2005 now and we know what a tag cloud is. About time we began thinking of how that would fit into a programming lang. :-) The only real advantage of a package tree is that you can: * Shove some packages underneath a common notifier. This limits the packages currently in scope. * Shortcut reference to package if they nest. I think both would be better served by a system different from hierarchy. -- J. Frederic Trottier-Hebert's profile photo Frederic Trottier-Hebert unread, May 24, 2011, 10:44:42 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang One issue is that while plain files are abstractions, they are a very common one, compatible with most systems you will see in use, and on which many tools have been developed, including source control, IDEs, e-mail attachments, documentation and conversation. I do not see the idea of a database as inherently bad -- it could be a good supplement to the developer, much like module_info is for modules. Using a DB for everything (with unique IDs) could solve the problem of nested dependencies, where library A needs library B 1.1.2 while library C needs B 1.2.0 and both can not be loaded at once. But the social problems it creates -- how to communicate about functions, how to regroup collections of them in an exportable way, how to version them, how to patch them, how to reload them, etc. are not worth the change, I believe. The code server or the VM could likely use any representation it likes for these, but the developers are used to one way of doing things, including existing projects. This isn't only a question of compatibility with older code, but also with developers. I think I could welcome a good function/module database in Erlang, something that could be as responsive as erldocs.com, but could be integrated straight to the shell or something that makes the developer's life easier and maintenance of code easier. I doubt switching to database would simplify the issues at hand. To me it sounds like we'd solve a problem, but create a bunch more that were already considered solved. It might be that everything was built on faulty assumptions, but then this sounds like the whole thing would need to be re-worked from scratch, not just the current module system. You might want to look at the way PLaneT is doing things for racket, too (http://docs.racket-lang.org/planet/Using_PLaneT.html?q=planet) You can basically import Racket/Scheme module from a server, either at run time or at build time. -- Fred Hebert http://www.erlang-solutions.com Max Lapshin's profile photo Max Lapshin unread, May 24, 2011, 10:10:27 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang I can't imagine myself working with system, that uses non-plain files for storing source code. Nobody will fix for me Textmate, git or ack to work with this database. Modules give your clear structure of functions. Without them you will end with mpegts_encode, mpegts_decode, and will break your head when there will be 3 or 4 underscores in name. Jack Moffitt's profile photo Jack Moffitt unread, May 24, 2011, 10:47:16 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Jesper Louis Andersen, Anthony Ramine, Erlang > The problem is that a package hierarchy enforces a tree-like > structure. And we may not want the tree. You might want the same > module to reside in multiple packages in the hierarchy at the same > time. We are past 2005 now and we know what a tag cloud is. About time > we began thinking of how that would fit into a programming lang. :-) Wouldn't you just use a DAG? Even with plain tags, you'd need to do something about cycles. jack. Vlad Dumitrescu's profile photo Vlad Dumitrescu unread, May 24, 2011, 10:50:09 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Frederic Trottier-Hebert, Erlang 2011/5/24 Frederic Trottier-Hebert One issue is that while plain files are abstractions, they are a very common one, compatible with most systems you will see in use, and on which many tools have been developed, including source control, IDEs, e-mail attachments, documentation and conversation. I do not see the idea of a database as inherently bad -- it could be a good supplement to the developer, much like module_info is for modules. Using a DB for everything (with unique IDs) could solve the problem of nested dependencies, where library A needs library B 1.1.2 while library C needs B 1.2.0 and both can not be loaded at once. But the social problems it creates -- how to communicate about functions, how to regroup collections of them in an exportable way, how to version them, how to patch them, how to reload them, etc. are not worth the change, I believe. One can always create a virtual filesystem on top of the database. One can also implement a virtual database on top of the filesystem. regards, Vlad Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 10:50:34 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Anthony Ramine, Erlang On Tue, May 24, 2011 at 4:31 PM, Anthony Ramine wrote: Le 24 mai 2011 a 14:45, Joe Armstrong a ecrit : > On Tue, May 24, 2011 at 1:56 PM, Max Lapshin < max.l...@gmail.com> wrote: > Very strange topic for me. > > I'd like to know if there will be hierarchial modules in Erlang, > because tree of packages is a rather good idea: > > No it's not - this has been the subject of long and heated discussion and is > why packages are NOT in Erlang - many people - myself included - dislike > the idea of hierarchical namespaces. The *dot* in the name has no semantics > it's just a separator. The name could equally well be encoders.mpg.erlyvideo > or mpg.applications.erlvideo.encoder - there is no logical way to organise the > package name and it does not scale - packages are NOT in Erlang? Then the related code should be removed because erl.lang.number:plus(1, 1) definitely works. Yes - this is an experiment unapproved feature which could be removed at any time. This is how we do things - we add experimental and undocumented features and reserve the right to remove them at any time in the future. Packages are not officially documented and thus not officially supported. Vlad Dumitrescu's profile photo Vlad Dumitrescu unread, May 24, 2011, 10:55:30 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang On Tue, May 24, 2011 at 16:15, Joe Armstrong wrote: The OTP release has 1793 modules and in these there are 56317 functions with different names. 11887 functions are defined in only one module, ie the for 11887 functions the name alone uniquely identifies the module. The distribution of functions over modules has a long tail with strange spikes, ie most function names are in very small number of modules. But some function names like handle_call etc and in a very large number of modules. Searching for a function using the function name as a key would only lead to a small number of candidate module if we exclude names like handle_call. it would be interesting to do this with type signatures ... If I understand correctly, it would then be necessary to have a database over all types, too. And a way to detect when types are identical, and a way to search types based on some meta-data associated with them... It might work, but it's a *lot* of work (pun intended) and I'm not sure how it would mix with existing codebases. More thinking required. regards, Vlad Anthony Ramine's profile photo Anthony Ramine unread, May 24, 2011, 10:58:57 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Le 24 mai 2011 a 16:50, Joe Armstrong a ecrit : > > > On Tue, May 24, 2011 at 4:31 PM, Anthony Ramine < n...@dev-extend.eu> wrote: > > Le 24 mai 2011 a 14:45, Joe Armstrong a ecrit : > > > On Tue, May 24, 2011 at 1:56 PM, Max Lapshin wrote: > > Very strange topic for me. > > > > I'd like to know if there will be hierarchial modules in Erlang, > > because tree of packages is a rather good idea: > > > > No it's not - this has been the subject of long and heated discussion and is > > why packages are NOT in Erlang - many people - myself included - dislike > > the idea of hierarchical namespaces. The *dot* in the name has no semantics > > it's just a separator. The name could equally well be encoders.mpg.erlyvideo > > or mpg.applications.erlvideo.encoder - there is no logical way to organise the > > package name and it does not scale - > > packages are NOT in Erlang? Then the related code should be removed because > erl.lang.number:plus(1, 1) definitely works. > > Yes - this is an experiment unapproved feature which could be removed at any time. > > This is how we do things - we add experimental and undocumented features > and reserve the right to remove them at any time in the future. Packages are not > officially documented and thus not officially supported. Well then, forgive me for this little offtopic but you seem to know what is unapproved and what is unfinished. Are structs - not your proper structs but the commented-out production found in erl_parse.yrl struct -> atom tuple - an unapproved feature or an unfinished feature? Regards, -- Anthony Ramine Dev:Extend http://dev-extend.eu Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 11:05:34 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Anthony Ramine, Erlang Not guilty - I've no idea actually - probably an unfinished experiment ... /Joe Parnell Springmeyer's profile photo Parnell Springmeyer unread, May 24, 2011, 11:31:05 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I actually came here to suggest Python's implementation of packages and modules - it confuses a lot of people at first, but it's actually quite powerful and I wish Erlang would do something more like it. Packages are a group of modules (modules are files in python, just like erlang) that can be sub-divided into sub-packages within that package using the filesystem, __init__.py files and naming. My apologies if this is already understood. The language I would point at as an example of a "mess" would be something like PHP. To comment on your suggestion: I much like the idea being able to inherit modules, to do it the "Erlang way", I think that may be one of the better and least painful-to-change routes. - -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJN2886AAoJEPvtlbpI1POLZn0IAL2f62A5CrSroCi5XVwVACRX Ecbq0Dzesqls61wK9gtX2lpEA16DEuxubz2m5FrGoKB18h4olrUi7fF7JCHcyEl6 +tc/rLwsGtvkE3bHrSRjbB7KTOEP1Dfa76i9xNvyPJXMZ8qdA3TBwKwiIhAx72a/ V1XkzEy4Z30UL7gWpysLkMc48p1LpoPnaTA20A5AXgcFd3SkzcuARRheWoEnuT1u L6N3FfR3glELYaNjQYiBxbOOuR8vS4kxVxk78NLgOGY87t4JGdNvmcXt/64lhxkN 3zNJnclr8zA+JLyernUPNpGDWV9fAuzpzkDExI0+57v3MN4TpnN1GEblj8l65Ao= =XGb2 -----END PGP SIGNATURE----- Parnell Springmeyer's profile photo Parnell Springmeyer unread, May 24, 2011, 11:56:10 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I really like the idea of packages - I'm not quite sure why everyone is against them... Even Racket has extended the syntax of Scheme to the point where they support packages and modules now; it's a very natural abstraction for the human mind IMHO. I would, personally, stop using (or fork) the language if I were forced to program connected to a DB. If it were a local ets-like DB that started when the Erlang VM started, cool. But not an over-the-internet type of thing, this is a particularly bad idea too because some applications really do need to be network efficient; if my language adds overhead to my network because it needs to request the definition of a function over the internet, that is A Bad Thing. I don't like the idea loading everything when the VM starts. I much prefer (and wish Erlang did it this way) the concept of calling an import statement. I like being able to see what packages, modules, and functions another module is referencing. Additionally this solves the issue of pre-loading the entire namespace when the VM starts; you only import when you /need/ it. I am intrigued, however, by the idea of type signatures and finding functions based on their metadata - cool idea. In practice though, I would really hate maintaining the metadata for every function I've wrote! I much rather group common functions into a module and then annotate the module with metadata. Anthony Ramine writes: - -- Parnell "ixmatus" Springmeyer (http://ixmat.us) -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJN29UaAAoJEPvtlbpI1POLpSAH/ijpOgfq1F+AF140/aXZke6C EXrWaiFBOgna0lkKu4lmwusOebLbyoiPs9D9vstlVff7npZ4Zdw32fp9LRuBjwfY rgGesMBoUGNMtpzXuuduCH7CkZNCIrBXj8uvKOBlDxt5csOz8FCpxvKDOXZg/z0I bFMP0HTYj1pzC2ixk2RmlUdKdbMLCCriIU+n0R5f4lIEck1FcoDm233ZHv1Sv9QC 9PlQ4kF91H/CavPAdD/JX4sDp6NRTCjV/VWCUzDTOyK/11/fl4R7nAB61TXzkVmz tv3y5z5RCkfjlatgvGSQ3d9vRbSn5fQneguxMX54JHI0WTEcWZ3xCQZZ8QKPuhY= =Faf+ -----END PGP SIGNATURE----- Tim Watson's profile photo Tim Watson unread, May 24, 2011, 11:57:01 AM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Robert Raschke, Erlang > For some reason this reminds me of a cross between Smalltalk and Eiffel. Yes, smalltalk had/has everything bootstrapped into the one environment, including the editor, stdlib, object system, etc. Tim Watson's profile photo Tim Watson unread, May 24, 2011, 12:03:51 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Parnell Springmeyer, Erlang On 24 May 2011 16:56, Parnell Springmeyer wrote: > I really like the idea of packages - I'm not quite sure why everyone is > against them... Even Racket has extended the syntax of Scheme to the > point where they support packages and modules now; it's a very natural > abstraction for the human mind IMHO. I've tried using them, but the lack of support makes it difficult. The bugs in the cover tool you can get around by doing -import(ets) in every module that needs to be cover compiled and as a work-around that fixes it. But reltool doesn't work so beam files in package sub-directories beneath ebin do not get copied into the release. There are probably other bugs too. > I don't like the idea loading everything when the VM starts. I much > prefer (and wish Erlang did it this way) the concept of calling an > import statement. I like being able to see what packages, modules, and > functions another module is referencing. You can already (today) choose whether everything loads on startup or on demand (i.e., when you call Mod:Func(Args)) by choosing to run the VM as an embedded system or not. To see what packages, modules and functions your modules are referencing, you can use the xref tool which comes with the standard distribution and is supported by tools like rebar. > I am intrigued, however, by the idea of type signatures and finding > functions based on their metadata - cool idea. Yes, this works particularly well for haskell (e.g., hoogle) and combined with cabal/hackage makes it easy to find what you need based mainly on type signatures. Sadly type signatures are often *all* the documentation you get of a Haskell library/application, which unless you're a type guru can be just as frustrating as write-yourself/cut+paste and often more so than google=>github=>rebar-deps. > > In practice though, I would really hate maintaining the metadata for > every function I've wrote! I much rather group common functions into a > module and then annotate the module with metadata. > If extra metadata were to be attached to functions, this should/would be generated by the compiler anyway. Personally, I think a step in the right direction would be to preserve type signatures at runtime - something that has been discussed elsewhere on this list not that long ago IIRC. Dave Challis's profile photo Dave Challis unread, May 24, 2011, 12:05:01 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to erlang-q...@erlang.org One thing I'd argue is that modules go some way to indicating the intent of functions within (though maybe this can be replaced with good function metadata). Let's say we've got a sets module which happens to implement set data structures as lists. This sets module might define some generally useful list manipulation functions, which others want to use. However, the intent of those functions is to serve the sets module. If the module owner wants to change one of the functions to be faster for certain types of list and slower for others, it would make sense to do so if it improved performance of the sets module as a whole. If that function was in a global namespace of functions, it would need metadata which indicated something like: "Although this is a general list processing function, it is intended to support set data structures, and as such may change internally to best serve this set of functions here." I guess what I'm trying to say is that I can see the need for code duplication when you've got: X, a function intended to do X Y, a function intended to do Y, which happens to be implemented as X Though the code for them might look identical, there's the implied contract that X will always do X, whereas Y could change at any time. Modules (or namespaces in other languages) seem to be a reasonable way of describing this intent/contract (albeit in vague terms, though most modules seem to have an understandable purpose or theme). -- Dave Challis d...@ecs.soton.ac.uk Tim Watson's profile photo Tim Watson unread, May 24, 2011, 12:05:42 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Parnell Springmeyer, Erlang > > Personally, I think a step in the right direction would be to preserve > type signatures at runtime - something that has been discussed > elsewhere on this list not that long ago IIRC. > Sorry - that wasn't what I meant to say. Rather, I think that it would be good to preserve type specifications in the beam code for a compiled module, which doesn't happen at present. Tom Murphy's profile photo Tom Murphy unread, May 24, 2011, 12:54:50 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Parnell Springmeyer, Erlang On 5/24/11, Parnell Springmeyer wrote: > In practice though, I would really hate maintaining the metadata for > every function I've wrote! I much rather group common functions into a > module and then annotate the module with metadata. Ideally, it wouldn't just be just you maintaining the metadata - if the functions were in the centralized DB, others could suggest or make changes, wikipedia-style. Tom Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 2:14:29 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Parnell Springmeyer, Erlang On Tue, May 24, 2011 at 5:56 PM, Parnell Springmeyer < ixm...@gmail.com> wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I really like the idea of packages - I'm not quite sure why everyone is against them... Even Racket has extended the syntax of Scheme to the point where they support packages and modules now; it's a very natural abstraction for the human mind IMHO. I would, personally, stop using (or fork) the language if I were forced to program connected to a DB. Why? Who cares how the data is stored - the requirements on storage would be never-loose-anything fast latency access form any machine I'd imaging a fast local db, that synced with some cloud storage and where you could work on-or off-line with periods of no net access. If your machine were struck by lightening you should be ably to resume on any other machine, possibly loosing a very small ammount of data. I imagine a local copy of couchDB with off-site replicas would be ok for this. If it were a local ets-like DB that started when the Erlang VM started, cool. Ets is too fast - we don't need to be ets fast :-) But not an over-the-internet type of thing, this is a particularly bad idea too because some applications really do need to be network efficient; if my language adds overhead to my network because it needs to request the definition of a function over the internet, that is A Bad Thing. It's a trade off - if you always store modified functions in the net the most you can loose is one function. If you don't you might loose a lot of work if your machine suddenly disintegrates. Umm let's think - in a good day I might write 500 lines of code say 500 * 50 bytes in 86400 seconds that's 0.3 bits/second I might look at ten times as much code - but I'd cache that locally. /Joe Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 2:16:20 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tom Murphy, Erlang On Tue, May 24, 2011 at 6:54 PM, Tom Murphy wrote: On 5/24/11, Parnell Springmeyer wrote: > In practice though, I would really hate maintaining the metadata for > every function I've wrote! I much rather group common functions into a > module and then annotate the module with metadata. Ideally, it wouldn't just be just you maintaining the metadata - if the functions were in the centralized DB, others could suggest or make changes, wikipedia-style. This did occur to me - the wikipedia allows large numbers of people to make small contributions. Think of what I want as a wiki where each entry is a single erlang function. Anthony Molinaro's profile photo Anthony Molinaro unread, May 24, 2011, 2:33:58 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang I'm wondering if a wiki, and an interface to construct a module (similar to those available for javascript libs like YUI/JQuery), wouldn't get you most of what you want. So I imagine being able to share individual functions from your own misc lib, and then constructing misc libs with the set of functions you want, maybe when the addition of a module name. This means you can still use the module name as a method of encapsulation but you can construct a module from a collection of functions. You might even make it such that you can start to define modules in a local file via a description language or set of compiler directives as well as providing a web interface for constructing these modules. So maybe you could do something like -module(lists2). -remote("http://erlfshare.com/?author=joe&func=remove_duplicates", dedup/2). -remote("http://erlfshare.com/?author=joe&func= list2frequency_distribution", freq/1). And that would be the definition of the lists2 module in your codebase, it could pull the functions from the shared repo at compile time. If you want to be safe you could version everything and pull in particular versions or hide the url construction, and instead have a proplist you set values in. At the very least this allows you to share functions and work with pre-existing code. Just a thought, -Anthony -- ------------------------------------------------------------------------ Anthony Molinaro Tim Watson's profile photo Tim Watson unread, May 24, 2011, 2:36:37 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang I'd like to propose a "step in the right direction", which won't go the whole hog in terms of what Joe is proposing, but would certainly move us in (what IMHO is) a good direction and potentially towards Joe's vision if he wants to go that far. I know it isn't the whole hog, but maybe it's a useful increment towards where we'd like to be. Why don't we start by building a search engine for finding Erlang "stuff" such as OTP applications/libraries, modules and functions. I would propose that we allow users to search for things by - kind (e.g., process/gen_thing, modules, functions, whole applications) - category (set of arbitrary tags?) - type signature (i.e., spec) I'm guessing this will require patching the compiler to preserve the type specs in the beam code. That is something I would need help with, but may be trivial for others. The other things I think you might want are (in no particular order): 1. binary (code) artefact repository (presumably with a REST/web interface?) 2. an means of uploading artefacts (tarballs? .ez archives?) to the repository 3. library code for introspecting module metadata (e.g., pulling type specs from beam code) 4. some means of annotating packages/modules/functions with metadata outside of the beam code 5. search (indexing, etc) capabilities 6. a way to obtain the *things* that you've identified as being useful to you Not everything will need to be built from scratch. Binary (code) artefact repositories already exist (to some extent) in the form of CEAN (which has version 2.0 coming out soon IIRC) and the Erlware repository. Those tools may not be immediately fit for re-use as they come with baggage, but they may be a good base upon which to build something for starters. Other things/ideas might be: - Integration with build tools (such as rebar) would be useful. - Integration with existing search/package-mgmt tools such as agner You could easily write a plugin that uses epm/agner/sutro to search github for something that isn't in the repo, pulls it, builds it and adds it. So many possibilities. I do have other projects on the go, and a job, a family and even a social life! Nonetheless, I would be happy to contribute to an initiative such as this (or something similarly incremental). Tim Watson's profile photo Tim Watson unread, May 24, 2011, 2:40:49 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang On this note Joe, in terms of moving the right direction... My thoughts were that once you've got all that beam code indexed and/or in memory, it's not a huge step to do what you want because the underlying infrastructure is already there. Also I see the approach you're proposing as being a complimentary and/or alternative to the current status quo, so folks wanting to carry on as-is will simply make use of the search+download capabilities without utilising the *edit my code like its a wiki page* features and so on. Tim Carpenter's profile photo Tim Carpenter unread, May 24, 2011, 2:53:13 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Anthony Molinaro, Erlang The module from a collection of functions is basically along the lines I was thinking, but with the ability to expose to the users of that module simpler, unchanging or changing names depending on the way the module is used. Sent from my iPhone Vlad Dumitrescu's profile photo Vlad Dumitrescu unread, May 24, 2011, 4:02:00 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang One thought: modules as grouping of functions and as a unit of loading are a way to ensure that a set of functions are consistent with each other, relative to some data the they can work on. This is important. Whatever replaces them should still fullfill that requirement. regards, Vlad Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 4:08:40 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Watson, Erlang On Tue, May 24, 2011 at 8:36 PM, Tim Watson wrote: I'd like to propose a "step in the right direction", which won't go the whole hog in terms of what Joe is proposing, but would certainly move us in (what IMHO is) a good direction and potentially towards Joe's vision if he wants to go that far. I know it isn't the whole hog, but maybe it's a useful increment towards where we'd like to be. Why don't we start by building a search engine for finding Erlang "stuff" such as OTP applications/libraries, modules and functions. YES - I've made a start on this - the first program is a segmenter that attempts to destruct a module. I've just bough a synology DS411 slim with 3TB of RAID 5 storage so am (finally) writing a backup program, this has first priority in the hobby dept. Then I have to attack my legacy collection of 40K erlang modules :-) Unfortunately information about individual functions is scattered all over the place. I want the source, the type signature, the documentation. These are available in a variety of different formats in different places. Documentation is in untagged comments in @tagged edoc format or in XML in a completely different file. I'd like this in one place with defined abstract types for everything Volunteers .... Silly design question: if multiple people can edit (say) a wiki of functions what formatting standards should be enforced? - the last person who edited the text - some "moderator" (who) - by a pretty printer (is there a *really good* pp? - with color syntax marking) I would propose that we allow users to search for things by - kind (e.g., process/gen_thing, modules, functions, whole applications) - category (set of arbitrary tags?) - type signature (i.e., spec) I'm guessing this will require patching the compiler to preserve the type specs in the beam code. That is something I would need help with, but may be trivial for others. Easy - it is there already (I think) poke around in beam_lib.erl this analyses the .beam files. The other things I think you might want are (in no particular order): 1. binary (code) artefact repository (presumably with a REST/web interface?) 2. an means of uploading artefacts (tarballs? .ez archives?) to the repository 3. library code for introspecting module metadata (e.g., pulling type specs from beam code) 4. some means of annotating packages/modules/functions with metadata outside of the beam code 5. search (indexing, etc) capabilities 6. a way to obtain the *things* that you've identified as being useful to you Not everything will need to be built from scratch. Binary (code) artefact repositories already exist (to some extent) in the form of CEAN (which has version 2.0 coming out soon IIRC) and the Erlware repository. Those tools may not be immediately fit for re-use as they come with baggage, but they may be a good base upon which to build something for starters. Other things/ideas might be: - Integration with build tools (such as rebar) would be useful. - Integration with existing search/package-mgmt tools such as agner You could easily write a plugin that uses epm/agner/sutro to search github for something that isn't in the repo, pulls it, builds it and adds it. So many possibilities. I do have other projects on the go, and a job, a family and even a social life! Nonetheless, I would be happy to contribute to an initiative such as this (or something similarly incremental). Great I'll be back ... /Joe Joe Armstrong's profile photo Joe Armstrong unread, May 24, 2011, 4:23:38 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Vlad Dumitrescu, Erlang Absolutly. I'm working on the idea of flows. A flow is a list of flows or grains A grain is an atomic thing A module is basically a flow where the grains are functions. A novel is a flow of " chapter flows" a "chapter flow" is a flow of paragraph grains etc. An image gallery is a flow of image grains. Everything is a flow or a grain. I'm wriiting a folding editor in javascript to support editing/ organising flows I made a weak minded attempt at this see https://github.com/joearms/elib1/tree/master/unsupported/folding In this directory are some rudimentary folding editors test_js6.html and experiment.erl are fun. (( If anybody reading this is an ace javascript hacker and wants to help please mail me -- I know *exactly* what help I need)) My first goal is to segement modules into flows and put them into a web based flow editor. All the flows will be in a global database. One thing I missed saying - associated with each function is *research* (NOT documentation) I want to store all reserach notes necessary to write the function along with the function in a hidden fold this should contain text, PDFs images etc. - the lot. Often the research to write 100 lines of codes takes weeks and we throw all this away - this must be stored with the code (NOT linked, store a copy, the link might be dead years later) /Joe Raoul Duke's profile photo Raoul Duke unread, May 24, 2011, 4:42:28 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang On Tue, May 24, 2011 at 1:06 AM, Joe Armstrong wrote: > When I'm programming I often get to the point were I say there should > a function foo/2 in lists.erl but their isn't. There should be but > there isn't - foo/2 is a small self contained thing. Why should it be > in lists.erl because it "feels right". that Open-Closed thing, and suck elk, is just never that easy no matter what the paradigm? sincerely. jm's profile photo jm unread, May 24, 2011, 8:58:46 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to erlang-q...@erlang.org On 24/05/11 9:25 PM, Joe Armstrong wrote: > > If we think of modules as containers of functions, then a module is a > "set of functions" - > my problem is with granularity. The smallest unit of reuse is the > module. Suppose my application > want just one function from a particular module - I'm forced to load > the entire module. > > This hampers reuse - I *often* read library code and find a single > function and cut-and-paste it into > my new code. It also discourages sharing - since the smallest unit I > can share is a module. > > The unit of distribution is an application - a set of modules - why > this could not be a list of functions > I do not know. > > I think I'd like to view an application as something that offered a > message based service that internally > was constructed from a set of functions rather than a set of modules. > Here's a "mad hatter" suggestion. Introduce the concept of a "shadow" module. For example, say your using the list module and your after a lists:randomise/1. In this proposal you'd have your own lists module, say joes_lists, -module(joes_lists) randomise(List) -> %% code to randomise the order of the list In you code you would then say, -shadows(joes_lists, lists). then use lists:randomise(AList) in your code. The compiler would first check the lists module and if the required function didn't exist it would use your modules. This would avoid you accidentially overriding the main modules definitions which may cause confusion. In such a case, the clash should be treated as a compilation error. The second part would be distribution and release management. There's several ways this could be done one would be to have a script which would build a set of "patch modules" which would be the cut down versions of your own development modules containing only the code and associated documentation your using in the current project. Actually, it might be better to include this as part of the project development: execute a script which scans your code repository and the -shadow/2 compiler directives and generates the shadow modules as needed. This could mean that your custom modules could be kept in a git or mercurial repository, on a web server, or in a key-value store. The URLs would then be part of the configuration for this script along with version tags, etc. Yes, I going to ignore how the modules get in there in the first place. Jeff. Edmond Begumisa's profile photo Edmond Begumisa unread, May 24, 2011, 11:34:17 PM5/24/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Hello Joe, I've personally wanted an easy way to load modules from different places. If you started with making modules easy to load from different places, then later provide support for functions, I imagine this would be easier to implement immediately and easier for people to make use of immediately ... where "people" refers to me :) I've got two suggestions (mostly inspired by the Mozilla Framework)... 1. How about letting the programmer choose where the functions come from (kv store, fs, web-server, etc)... For instance, a module could start... -import(["file://misc/collect_int", "http://erlang.org/lists/*", "mnesia://db1/misc/merge_kv"]). By default, the code loader would know what to do when it encounters "file://", "http://", "https://", "ftp://", "sftp://" and "mnesia://" URI schemes. Programmers could be allowed define their own URI schemes by providing callbacks for loading code. For instance, one would be able to define "ubf://" and have the code loader call his/her code when it encounters "ubf://foo/bar" 2. Expanding on this: letting the programmer centralise the possible locations of modules/functions and use this to define namespaces... One possible way of doing this is to steal from the "chrome registration" system that the Mozilla Framework uses. A programmer could define one-level deep namespaces in a configuration file which points to targets that could come from anywhere, then use that in her modules. A module could then start... -import(["erlang://misc/collect_int", "erlang://couch/mapreduce"]). Where "misc" and "couch" are namespaces and "erlang://" is a special URI scheme telling the code loader to refer to the configuration file "erlang.manifest" which would contain... [{"misc", "file://./ebin/misc.beam"}, {"couch", "http://localhost:7777/ebin/"}] Where "couch" points to a yaws server serving couch beams, or something along those lines. Then she can easy switch between different code sources using one file. A code-signing system could also be introduced. - Edmond - On Tue, 24 May 2011 18:06:19 +1000, Joe Armstrong wrote: > Why do we need modules at all? > > This is a brain-dump-stream-of-consciousness-thing. I've been > thinking about this for a while. > > I'm proposing a slightly different way of programming here > The basic idea is > > - do away with modules > - all functions have unique distinct names > - all functions have (lots of) meta data > - all functions go into a global (searchable) Key-value database > - we need letrec > - contribution to open source can be as simple as > contributing a single function > - there are no "open source projects" - only "the open source > Key-Value database of all functions" > - Content is peer reviewed > > These are discussed in no particular order below: > > Why does Erlang have modules? > > There's a good an bad side to modules: > > Good: Provides a unit of compilation, a unit of code > distribution. unit of code replacement > > Bad: It's very difficult to decide which module to put an individual > function in. Break encapsulation (see later) > > Aside: lib_misc.erl > > When I'm programming I often get to the point were I say there should > a function foo/2 in lists.erl but their isn't. There should be but > there isn't - foo/2 is a small self contained thing. Why should it be > in lists.erl because it "feels right". > > Strings are lists, so why do we have two modules lists.erl and > string.erl how should I decide in which module my new string/list > processing function should go. > > To avoid all mental anguish when I need a small function that > should be somewhere else and isn't I stick it in > a module elib1_misc.erl. > > My elib1_misc exports the following: > > added_files/2 make_challenge/0 > as_bits/1 make_response/2 > as_bits_test/0 make_response_test/0 > bdump/2 make_test_strings/1 > bin2hex/1 make_test_strings_test/0 > bin2hex_test/0 make_tmp_filename/2 > check_io_list/1 merge_kv/1 > collect_atom/1 merge_kv_test/0 > collect_atom_test/0 mini_shell/0 > collect_int/1 module_info/0 > collect_int_test/0 module_info/1 > collect_string/1 ndots/1 > collect_string_test/0 nibble_to_hex_char/1 > collect_word/1 nibble_to_hex_char_test/0 > complete/2 odd/1 > complete_test/0 on_exit/2 > dos2unix/1 out_of_date/2 > downcase_char/1 outfile/2 > dump/2 padd/2 > dump_tmp/2 perms/1 > duplicates/1 perms_test/0 > ensure_started/2 pmap/2 > eval_file/1 pmap1/2 > eval_file_test/0 pmap1_test/0 > eval_string/1 pmap_test/0 > eval_string_test/0 priority_receive/0 > every/3 random_seed/0 > expand_env_vars/1 random_string/1 > expand_file_template/3 random_string/2 > expand_string_template/2 read_at_most_n_lines/2 > expand_tabs/1 read_at_most_n_lines_test/0 > expand_tabs_test/0 remove_duplicates/1 > expand_template/2 remove_duplicates_test/0 > extract_attribute/2 remove_leading_and_trailing_whitespace/1 > extract_attribute_test/0 > remove_leading_and_trailing_whitespace_test/0 > extract_prefix/2 remove_leading_whitespace/1 > fetch/2 remove_prefix/2 > fetch_test/0 remove_prefix_test/0 > file2lines/1 remove_trailing_whitespace/1 > file2lines_test/0 replace/3 > file2md5/1 root_dir/0 > file2numberedlines/1 rpc/2 > file2numberedlines_test/0 safe/1 > file2paras/1 show_loaded/1 > file2stream/1 signed_byte_to_hex_string/1 > file2string/1 signed_byte_to_hex_string_test/0 > file2template/1 skip_blanks/1 > file2term/1 skip_blanks_test/0 > file_size_and_type/1 skip_to_nl/1 > find_src/1 skip_to_nl_test/0 > first/1 sleep/1 > flatten_io_list/1 spawn_monitor/3 > flush_buffer/0 split_at_char/2 > for/3 split_at_char_test/0 > force/1 split_list/2 > foreach_chunk_in_file/3 split_list_test/0 > foreach_word_in_file/2 string2exprs/1 > foreach_word_in_string/2 string2exprs_test/0 > forever/0 string2html/1 > get_erl_section/2 string2latex/1 > get_line/1 string2lines/1 > get_line/2 string2lines_test/0 > have_common_prefix/1 string2stream/1 > have_common_prefix_test/0 string2stream_test/0 > hex2bin/1 string2template/1 > hex2bin_test/0 string2template_test/0 > hex2list/1 string2term/1 > hex2list_test/0 string2term_test/0 > hex_nibble2int/1 string2toks/1 > hex_nibble2int_test/0 string2toks_test/0 > id/1 sub_binary/3 > include_dir/0 template2file/3 > include_file/1 term2file/2 > interleave/2 term2string/1 > is_alphanum/1 test/0 > is_blank_line/1 test1_test/0 > is_prefix/2 test_function_over_substrings/2 > is_prefix_test/0 tex2pdf/1 > is_response_correct/3 time_fun/2 > keep_alive/2 time_stamp/0 > lines2para/1 to_lower/1 > list2frequency_distribution/1 to_lower_test/0 > list2frequency_distribution_tetrim/1 > longest_common_prefix/1 trim_test/0 > longest_common_prefix_test/0 unconsult/2 > lookup/2 unsigned_byte_to_hex_string/1 > lorem/1 unsigned_byte_to_hex_string_test/0 > ls/1 which/1 > which_added/1 > > Now I find this very convenient when I write a new small utility function > I stick in in elib1_misc.erl - no mental anguish in choosing a module > name is involved. > > The observation that I find this very-convenient is telling me something > about modules - I like my elib1_misc it feels right. > > (aside - It seems many development projects have their own private > lib_miscs ...) > > Which brings me to the point of my question. > > Do we need module's at all? Erlang programs are composed of lots of small > functions, the only place where modules seem useful is to hide a letrec. > > The classic example is fibonacci. We want to expose fib/1 but hide the > helper function fib/3. Using modules we say > > -module(math). > -export([fib/1]). > > fib(N) -> > fib(N, 1, 0). > > fib(N, A, B) when N < 2 -> A; > fib(N, A, B) -> fib(N-1, A+B, A). > > The downside is we have had to *invent* one module name math - whose > *only* > purpose is to hide the definition of fib/3 which we don't want to be made > callable. > > If we put a second function into the module math, then this second > function > could call fib/3 which breaks the encapsulation of fib/3. > > We could say: > > let fib = fun(N) -> fib(N, 1, 0) end > in > fib(N, A, B) when N < 2 -> A; > fib(N, A, B) -> fib(N-1, A+B, A). > end. > > I hardly dare suggest a syntax for this since I've been following > another thread in this forum where syntax discussion seem to encourage > much comment. > > ** Please do suggest alternative syntax's here - but do not comment on > other peoples suggestions ... > > I would like to just talk about why we have modules. > > Another question: > > Does the idea of a module come from the idea that functions have to be > stored somewhere, so we store them in a file, and we slurp the > file (as a unit) into the system, so the file becomes a module? > > If all the files were store by themselves in a database would this > change things. > > I am thinking more and more that if would be nice to have *all* > functions in > a key_value database with unique names. > > lookup(foo,2) would get the definition foo foo/2 from a database. > > The unique names bit is interesting - is this a good idea. Qualified > names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good > idea but but when I'm programming I have to invent the xxx or the > a.b.c which is very difficult. It also involves the "decision problem" > if the namespaces xxx and a.b.c already exist I have to *choose* which > to put my new function in. > > I think there might be a case for alises here joe:foo/2 could be used > while developing "joe" would expand to a horrible random local string the > real name being ab123aZwerasch123123_foo/2 but I would not be able to > publish my code or make it available to a third_part before I had > chosen a sensible name. > > (( managing namespaces seems really tricky, a lot of peoople seem > to thing that the problem goes away by adding "." 's to the name > but managing a namespace with namees like foo.bar.baz.z is just as > complex > as managing a namespace with names like foo_bar_baz_z or names like > 0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic > name like www.a.b to a reference like 123.45.23.12 - but how do we > discover > the initial name www.a.b? - there are two answers - a) we are given the > name > (ie we click on a link) - we do not know the name but we search fo it )) > > > When programs are small we can live with "just the code" in "a few > modules" the ratio of code to meta data is high. > > When programs are large we need a lot of meta-data to understand them. > > I would like to see all functions with all meta-data in a data base. > > I'd like to say: > > lookup(foo,2,Attribute) when Attribute = > > code|source|documentation|type signatures|revision > history|authors|... > > The more I think about it the more I think program development should > viewed as changing the state of a Key-Value database. > > So I imagine: > > 1) all functions have unique names > 2) there are no modules > 3) we discover the name of a function by searching metadata > describing the function in a database > 4) all public functions (think open source) are in the same > database > > We could make a system to do this. > > I think this would make open-source projects easier, since the > granularity of contribution goes down. You could contribute > a single function - not an entire application. > > (( A problem with GUT style open source projects is there is > not one database of functions, I often what one function from > this project, another function from another project -- the > granularity of reusable parts should be the individual function. > > functions are really easy to reuse > modules are more difficult to reuse > entire applications are very difficult to reuse > (Unless there are isolated through a communication channel)) > > Possible extensions. > > 1) Voting for promotion > 2) A review process > > Given a raw database will *all* functions in it - we could derive an > "approved" functions database. > > Popular functions could be moved to the approved database - the > review process would need to be discussed - so kind of peer-review/ wiki > stuff. > > Comments? > > Volunteers? > > /Joe -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 12:29:16 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang On 24/05/2011, at 8:06 PM, Joe Armstrong wrote: > Why do we need modules at all? To hide data types, and to provide short names (we can import a function and not have to keep on repeating a long long name for it). I note, without recommendation either way, the existence of local in end in Standard ML, where the second lot of declarations defines things visible outside but the first lot defines things visible only inside this form. > fib(N) -> > fib(N, 1, 0). > > fib(N, A, B) when N < 2 -> A; > fib(N, A, B) -> fib(N-1, A+B, A). could be local fun fib_aux n a b = if n < 2 then a else fib_aux (n-1) (a+b) a in fun fib n = fib_aux n 1 0 end in SML. I also note that this is NOT something that SML uses instead of modules. >Does the idea of a module come from the idea that functions have to be > > stored somewhere, so we store them in a file, and we slurp the > file (as a unit) into the system, so the file becomes a module? Probably not. Smalltalk has classes, but the unit of compilation is a single method, and the unit of storage is classically a "change set" (which may involve any number of classes and methods, and may include revisions to existing classes). I also note that in the days when Interlisp-D was just that and did not include Xerox Common Lisp, you slurped a file (as a unit) into the system, but there were no modules. There are plenty of languages where a file may contain many modules, possibly nested. (SML, Ada, to name just two.) > If all the files were store by themselves in a database would this > change things. > > I am thinking more and more that if would be nice to have *all* functions in > a key_value database with unique names. In Erlang as it stands, a source file is the natural unit of version control and documentation. The idea of coping with a sea of functions all being busily revised by hundreds if not thousands of programmers acting asynchronously troubles me. Haskell has a large package library, and people can update things often, and I keep on seeing version issues in the Haskell-cafe mailing list. Can - scope of names - grain of compilation and reloading - version control - documentation - units of (program) storage be teased apart? Smalltalk seems to suggest that something along these lines might be doable. (Smalltalk methods are version controlled individually.) However, Smalltalk documentation, especially internal documentation, varies between woeful and dire. > When programs are large we need a lot of meta-data to understand them. The Xerox slogan: a program is a data base, not a listing. Interlisp-D did not strictly speaking _have_ source files. There were files with source code in them, but they were randomly accessed data bases. Thanks to macros, this did not work as well as one might hope, but it did work after a fashion. Hey, maybe this could be a way to get rid of the preprocessor! Joe, you've invented one great language, this is crazy enough to be the concept for a second one. Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 12:38:57 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Dmitrii Dimandt, Erlang Questions On 24/05/2011, at 8:39 PM, Dmitrii Dimandt wrote: >> >> So I imagine: >> >> 1) all functions have unique names >> 2) there are no modules >> 3) we discover the name of a function by searching metadata >> describing the function in a database >> 4) all public functions (think open source) are in the same >> database >> >> We could make a system to do this. > > Hm... you'd still need *some* kind of modules. > > For instance, in Webmachine each resource is expected to have some, or all, of predefined public/exported functions: to_html, resource_exists, allowed_methods etc. > > So, each resource will have functions with the same name. If we move all functions to a global namespace, then how do we differentiate between these functions? Probably by naming them resource_name_function_name? Meta-data. You say "I need a function about with a ping cousin about and a pong friend about " > > Furthermore, what do we do with behaviours? A behaviour is encapsulated within a module, also with a predefined set of exported funcions (and any number of custom exported functions). In SML, a behaviour would be a functor. In some other languages, it would be a single function that took a record of anonymous functions &c and returned a record of anonymous functions. Inventing an ad hoc syntax: local <{start ~ Start, stop ~ Stop, send ~ Send, explode ~ Explode}> = someY=wonderfulY=newY=behaviour(<{ initial_state ~ 42, shut_down ~ fun (State) -> ... end, restart_policy ~ ad_infinitum, update_state ~ fun (State, Info) -> ... end }>) in what you please using Start, Stop, Send, Explode end The trick is to combine this with hot loading, and I have too many things to do today to think of something. That's what Joe is for... Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 12:51:16 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Gordon Guthrie, Erlang On 24/05/2011, at 8:49 PM, Gordon Guthrie wrote: > ...and also... > > following on from what Dimitri said while I was writing me e-mail, we > hang unit test onto modules and design them at that level of > abstraction > Right now that is clear a Really Good Thing to do. The question is whether it has to be that way, or whether it is possible to devise something better. Do the unit of documentation and the unit of testing and the unit of loading *have* to be the same thing? How would a revised system *work* in a distributed world? Back in the early days of the language Pop, the people at the Edinburgh AI department had a game. The whole machine they were using was programmed in Pop, on top of bare metal. The game goes like this: how much of the system can you zap before you have to reboot the machine? How much of a shared sea-of-functions can you update before having to revert to backups? Michael Turner's profile photo Michael Turner unread, May 25, 2011, 12:54:18 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Richard O'Keefe, Erlang If someone had told me in 1981 (when I got a taste of both Prolog and Smalltalk) that, in 2011, we'd still be editing files of code with text editors, with a strong coincidence between "module" and "code-file/header-file pair", I would have been surprised. But here we are. Is it just cultural inertia? Or is it that this way of doing things isn't really broken, just the best compromise in the best of all programming worlds? By some coincidence, Richard and I are discussing non-contiguous clauses in named functions as something that was nice in Prolog, and would be nice in Erlang. Which would suggest an even finer granularity for the database: as long as you can maintain a (partial) order of clauses for a function, clauses, not functions, might be the minimal units. I think it's worth looking very closely at how other systems tried to solve the same or similar problems (even decades ago, e.g., Smalltalk and Interlisp-D) and what problems they ran up against, and at what kind of scaffolding would make the architecture of any Erlang solution backward compatible with (most) existing Erlang development. The ideal (probably unachievable) would be that nobody would need to change anything about how they work now, while anybody who wants to take fuller advantage of the system would face no obstacles that were rationalized in the name of backward compatibility. A tall order. -michael turner Vance Shipley's profile photo Vance Shipley unread, May 25, 2011, 1:58:50 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Joe, I love the idea of ditching modules files and storing functions in a data store. What I'm picturing though is that the functions are stored in their abstract syntax form (i.e. data) so that the issue of formatting standards is stripped away. You need tools which translate it in and out so that we can continue to use the editors we are familiar with but you can edit in emacs using it's bizarre indentation and I can use vim and indent with tabs. ROK can put seperators at the begging of lines, I'll see them at the end. I have long wanted to put a toolchain together which allowed me to easily store abstract syntax in the source code control system repository. The check in and check out processes would convert (pretty print) it to my personal prefernce settings. The problem with that plan seems to be with the preprocessor. It seems to me though that your plan suffers the same problem. If macros are defined at a module level, or in an include file, you'll have to deal with them when you pull the functions out of the module and stuff it into the data store. The extended syntax tree used in the erl_syntax application might be the answer. How do you deal with record definitions? -- -Vance On Tue, May 24, 2011 at 10:08:40PM +0200, Joe Armstrong wrote: } Silly design question: if multiple people can edit (say) a wiki } of functions what formatting standards should be enforced? } } - the last person who edited the text } - some "moderator" (who) } - by a pretty printer } (is there a *really good* pp? - with color syntax marking) -- -Vance Edmond Begumisa's profile photo Edmond Begumisa unread, May 25, 2011, 2:07:57 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Richard O'Keefe, erlang-q...@erlang.org On Wed, 25 May 2011 14:29:16 +1000, Richard O'Keefe < o...@cs.otago.ac.nz> wrote: > On 24/05/2011, at 8:06 PM, Joe Armstrong wrote: > >> Why do we need modules at all? > > To hide data types, What are the possible ways of dealing with this? When you break modules up into individual functions? I presume such a system would therefore need a way of importing types (complete with versioning) in addition to functions. Which would then introduce additional complexity for the programmer than just importing a whole module (the C header-file mess). Maybe the proposed system could support both -- modules and individual functions? - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ jm's profile photo jm unread, May 25, 2011, 2:16:48 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to erlang-q...@erlang.org On 25/05/11 2:51 PM, Richard O'Keefe wrote: > > Back in the early days of the language Pop, the people at > the Edinburgh AI department had a game. The whole machine > they were using was programmed in Pop, on top of bare metal. > The game goes like this: > how much of the system can you zap before you have > to reboot the machine? > > How much of a shared sea-of-functions can you update before > having to revert to backups? > This is an aside, but one of those things I've been wondering about. Is there a language in existence which scales from "here's a piece of bare metal" and scales up to declarative distributed programming of the likes of Erlang? At the moment, it seems that the accepted way is to change from assembler to C to something else. Is there a language that progresses from the low level machine code basics up by adding language features and improving the runtime environment which is written in itself, ie turtles all the way down, and is more than an AST written in ascii (I'm looking at you lisp)? Jeff. Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 2:18:19 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Jesper Louis Andersen, Erlang On 24/05/2011, at 10:26 PM, Jesper Louis Andersen wrote: > On Tue, May 24, 2011 at 10:06, Joe Armstrong wrote: >> Why do we need modules at all? > > I think we need modules, but this post sparked a thought I have had > for some time now. > > I'd like to have module renaming/namespacing! This has been proposed in some detail. Can't remember who by, but I think his initials were R.A.O'K. (:-) That proposal was inspired by Eiffel's LACE configuration language. Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 2:33:12 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang On 25/05/2011, at 12:45 AM, Joe Armstrong wrote: > > > On Tue, May 24, 2011 at 1:56 PM, Max Lapshin wrote: > Very strange topic for me. > > I'd like to know if there will be hierarchial modules in Erlang, > because tree of packages is a rather good idea: > > No it's not - this has been the subject of long and heated discussion and is > why packages are NOT in Erlang - many people - myself included - dislike > the idea of hierarchical namespaces. The *dot* in the name has no semantics > it's just a separator. The name could equally well be encoders.mpg.erlyvideo > or mpg.applications.erlvideo.encoder - there is no logical way to organise the > package name and it does not scale - I should point out that Haskell has dotted names, uses them to indicate semantics, and has a *managed* hierarchical name space; Data.* and Control.* and Net.* and so on are tolerably clear. I should also admit that there are occasional complaints about it not being clear where things have been put, and other indexing and searching tools are often used to find things. Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 3:07:04 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Edmond Begumisa, Erlang On 25/05/2011, at 3:34 PM, Edmond Begumisa wrote: > Hello Joe, > > I've personally wanted an easy way to load modules from different places. > > If you started with making modules easy to load from different places, then later provide support for functions, I imagine this would be easier to implement immediately and easier for people to make use of immediately ... where "people" refers to me :) > > I've got two suggestions (mostly inspired by the Mozilla Framework)... > > 1. How about letting the programmer choose where the functions come from (kv store, fs, web-server, etc)... > > For instance, a module could start... > > -import(["file://misc/collect_int", "http://erlang.org/lists/*", "mnesia://db1/misc/merge_kv"]). Why does this have to be done inside the module? Why cannot the mapping from module names to sources be *outside*, like an SGML (or XML) catalogue, or a .lace file? Bengt Kleberg's profile photo Bengt Kleberg unread, May 25, 2011, 3:11:48 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Erlang Moreover, to solve the problem this should work for two different beam modules, having the same name, being used at the same time. Since they are already compiled we must do the mapping outside of the modules. bengt Edmond Begumisa's profile photo Edmond Begumisa unread, May 25, 2011, 3:18:09 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Richard O'Keefe, Erlang That's precisely what I was suggesting in suggestion 2... > ... 2. Expanding on this: letting the programmer centralise the possible > locations ... That "erlang.manfest" contains is a prop_list that maps "erlang://" URIs to actual locations. Sorry it wasn't clear enough. - Edmond - On Wed, 25 May 2011 17:07:04 +1000, Richard O'Keefe < o...@cs.otago.ac.nz> wrote: > > On 25/05/2011, at 3:34 PM, Edmond Begumisa wrote: > >> Hello Joe, >> >> I've personally wanted an easy way to load modules from different >> places. >> >> If you started with making modules easy to load from different places, >> then later provide support for functions, I imagine this would be >> easier to implement immediately and easier for people to make use of >> immediately ... where "people" refers to me :) >> >> I've got two suggestions (mostly inspired by the Mozilla Framework)... >> >> 1. How about letting the programmer choose where the functions come >> from (kv store, fs, web-server, etc)... >> >> For instance, a module could start... >> >> -import(["file://misc/collect_int", "http://erlang.org/lists/*", >> "mnesia://db1/misc/merge_kv"]). > > Why does this have to be done inside the module? > Why cannot the mapping from module names to sources > be *outside*, like an SGML (or XML) catalogue, > or a .lace file? > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ Michael Truog's profile photo Michael Truog unread, May 25, 2011, 3:25:48 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Richard O'Keefe, Erlang On 05/24/2011 11:33 PM, Richard O'Keefe wrote: > On 25/05/2011, at 12:45 AM, Joe Armstrong wrote: >> >> On Tue, May 24, 2011 at 1:56 PM, Max Lapshin wrote: >> Very strange topic for me. >> >> I'd like to know if there will be hierarchial modules in Erlang, >> because tree of packages is a rather good idea: >> >> No it's not - this has been the subject of long and heated discussion and is >> why packages are NOT in Erlang - many people - myself included - dislike >> the idea of hierarchical namespaces. The *dot* in the name has no semantics >> it's just a separator. The name could equally well be encoders.mpg.erlyvideo >> or mpg.applications.erlvideo.encoder - there is no logical way to organise the >> package name and it does not scale - > I should point out that Haskell has dotted names, > uses them to indicate semantics, > and has a *managed* hierarchical name space; > Data.* and Control.* and Net.* and so on are > tolerably clear. > > I should also admit that there are occasional complaints > about it not being clear where things have been put, and > other indexing and searching tools are often used to find > things. Anthropologists supposedly think that wanting a taxonomy is very natural (http://en.wikipedia.org/wiki/Taxonomy# Taxonomy_and_mental_classification). A hierarchy may not be formed in a way that everyone would expect because we lack a collective mind. However, I think that source code requires organization to be manageable. To make a hierarchy flexible, pointers could be used to represent different interpretations of what the hierarchy could be. If a large collaborative effort was formed to generate source code, it is important that code is reused to reduce complexity. A hierarchy would be our most natural way of determining source code that can be referenced to build larger systems, and that would make the dependency more likely to receive maintenance. That would helps us avoid the current situation with things like jungerl. I don't think it is important whether we use a '.' character or a '_' character to create a hierarchy with the names in our source code. However, the '.' character used by the undocumented Erlang package functionality could be used to provide better support for unmanageable hierarchies, perhaps with search and documentation benefits. The other part to imposing order on a large collaborative effort seems to be determining the quality of the code for reuse. A rating system could be used and doesn't require a hierarchy. However, with many different common segments of source code available, a hierarchy can provide a way of focusing on relevant source code and should promote code reuse. Edmond Begumisa's profile photo Edmond Begumisa unread, May 25, 2011, 3:28:51 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Bengt Kleberg, Erlang On Wed, 25 May 2011 17:11:48 +1000, Bengt Kleberg wrote: > Moreover, to solve the problem this should work for two different beam > modules, having the same name, being used at the same time. Since they > are already compiled we must do the mapping outside of the modules. > > > bengt > Yes, in my second suggestion the mappings would be outside the modules, in a single "manifest" file. This idea comes from the way "chrome.manifest" and the "chrome://" URI works in Mozilla (those who've written Firefox/Thunderbird extensions would be familiar with it.) The code loader would have to consult the manifest file for the "real" locations of the modules. That's what I was trying to describe here... > 2. Expanding on this: letting the programmer centralise the possible > locations of modules/functions and use this to define namespaces... > > One possible way of doing this is to steal from the "chrome > registration" system that the Mozilla Framework uses. A programmer could > define one-level deep namespaces in a configuration file which points to > targets that could come from anywhere, then use that in her modules. A > module could then start... > > -import(["erlang://misc/collect_int", "erlang://couch/mapreduce"]). > > Where "misc" and "couch" are namespaces and "erlang://" is a special URI > scheme telling the code loader to refer to the configuration file > "erlang.manifest" which would contain... > > [{"misc", "file://./ebin/misc.beam"}, > {"couch", "http://localhost:7777/ebin/"}] > > Where "couch" points to a yaws server serving couch beams, or something > along those lines. Then she can easy switch between different code > sources using one file. > > A code-signing system could also be introduced. > - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ Richard O'Keefe's profile photo Richard O'Keefe unread, May 25, 2011, 3:30:44 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to jm, erlang-q...@erlang.org On 25/05/2011, at 6:16 PM, jm wrote: > > Is there a language in existence which scales from "here's a piece of bare metal" and scales up to declarative distributed programming of the likes of Erlang? At the moment, it seems that the accepted way is to change from assembler to C to something else. Is there a language that progresses from the low level machine code basics up by adding language features and improving the runtime environment which is written in itself, ie turtles all the way down, and is more than an AST written in ascii (I'm looking at you lisp)? The original Smalltalk ran on bare metal. There is or was a "SqueakNOS" (no operating system) port that ran on bare metal, but with a glue layer in C. Interlisp-D ran on bare metal. There were some low level primitives, but you quickly got into Lisp, and everything from the garbage collector to the network and file system stacks as in Lisp. Lisp machines ditto. In both the MIT and Xerox lisp machines, you could twiddle bits in registers or work on natural language translation. Lisp is definitely more than an AST written in ASCII, and there are several Algol-ish front ends for it. Reduce has (had?) RLisp and Macsyma used CGOL, I think. The Poplog system ran under VMS, and because they were serving lots of undergraduates, they really wanted a rock solid time sharing system underneath. But there's a language layer called Syspop that gets you quite low, with Pop11, Prolog, Common Lisp, and ML layered on top. My experience with Interlisp-D was that you really want to hide the bottom level pretty quickly. Joe Armstrong's profile photo Joe Armstrong unread, May 25, 2011, 4:37:39 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Tim Watson, Erlang On Wed, May 25, 2011 at 7:46 AM, Tim Watson wrote: > Unfortunately information about individual functions is > scattered all over the place. I want the source, the type signature, > the documentation. These are available in a variety of different > formats in different places. Documentation is in untagged comments > in @tagged edoc format or in XML in a completely different file. Yes, this is a really important part of it. Being able to slurp in Erlang projects from the file system (and github, etc), access all this information and provide it in a searchable fashion would *already* put us way ahead of the crowd in terms of discovery. > Silly design question: if multiple people can edit (say) a wiki > of functions what formatting standards should be enforced? > > - the last person who edited the text > - some "moderator" (who) > - by a pretty printer > (is there a *really good* pp? - with color syntax marking) I think there are reasonable ones around. Take a look at http://tidier.softlab.ntua.gr:20000/tidier/getstarted. I would also propose that you let the viewer configure the pp to their own preferences. Given that you're actually storing abstract code (or AST or something like that) rather than lines of source code anyway, this is just human readable representation after all. Let people view with whatever (potentially hideous) whitespace &c convention they like. Very interesting idea - there was once a pascal pretty printer that analysed your code and inferred the style that you preferred - "Umm this guy likes semi-colons at the end of line ..." they in applied it to code you hadn't written. >> I'm guessing this will require patching the compiler to preserve the >> type specs in the beam code. That is something I would need help with, >> but may be trivial for others. > > Easy - it is there already (I think) poke around in beam_lib.erl > this analyses the .beam files. > I was pretty sure this wasn't the case. I'll dig in to beam_lib at lunch time and have another look. Joe Armstrong's profile photo Joe Armstrong unread, May 25, 2011, 4:40:46 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Vance Shipley, Joe Armstrong, Erlang On Wed, May 25, 2011 at 7:58 AM, Vance Shipley wrote: Joe, I love the idea of ditching modules files and storing functions in a data store. What I'm picturing though is that the functions are stored in their abstract syntax form (i.e. data) so that the issue of formatting standards is stripped away. You need tools which translate it in and out so that we can continue to use the editors we are familiar with but you can edit in emacs using it's bizarre indentation and I can use vim and indent with tabs. ROK can put seperators at the begging of lines, I'll see them at the end. I have long wanted to put a toolchain together which allowed me to easily store abstract syntax in the source code control system repository. The check in and check out processes would convert (pretty print) it to my personal prefernce settings. YES The problem with that plan seems to be with the preprocessor. It seems to me though that your plan suffers the same problem. If macros are defined at a module level, or in an include file, you'll have to deal with them when you pull the functions out of the module and stuff it into the data store. The extended syntax tree used in the erl_syntax application might be the answer. YES How do you deal with record definitions? YES :-) - compile their hash into the function? Edmond Begumisa's profile photo Edmond Begumisa unread, May 25, 2011, 4:54:43 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Bengt Kleberg, Richard O'Keefe, erlang-q...@erlang.org Motivation: If I can easily load modules or Joe's functions from different locations, then I can deploy my Erlang applications similarly to web-applications. I'm using Erlang at both the client and the server. Currently, reltool and appup I've found a too tedious. web-app style deployment would have big advantages... * In corporate environments: clients can have a bare-bones installation, launch that, then get most of the Erlang code from a central server. Upgrading everyone is replacing a few files on the server. I could write some simple code to upgrade modules already loaded by clients. * On the web: With code-signing supported, I can safely deploy all applications on my startup's web-server. I get all users (using different completely applications) to download and install the *same* single bare-bones installation. Erlang code for the different apps comes from my web-server. Users whose licenses run dry can't access newer code. - Edmond - Vlad Dumitrescu's profile photo Vlad Dumitrescu unread, May 25, 2011, 4:55:21 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Vance Shipley, Joe Armstrong, Erlang On Wed, May 25, 2011 at 07:58, Vance Shipley wrote: I love the idea of ditching modules files and storing functions in a data store. What I'm picturing though is that the functions are stored in their abstract syntax form (i.e. data) so that the issue of formatting standards is stripped away. The problem with that plan seems to be with the preprocessor. It seems to me though that your plan suffers the same problem. If macros are defined at a module level, or in an include file, you'll have to deal with them when you pull the functions out of the module and stuff it into the data store. The extended syntax tree used in the erl_syntax application might be the answer. There are more things that need to be preserved by the parser, which make existing pretty-printers less than good. For example: - integer values can be shown using different bases, from case to case, so one can't use a generic setting. So the parser needs to keep track of that. - comments have to be anchored somewhere in the code. erl_syntax guesses only almost right. Of course, if there is going to be a more advanced metadata mechanism, the comments belong there too. Also, I didn't understand where/how are record and macro definitions going to be stored and identified. As somebody else said it too, the best solution would be to get rid of them altogether and it's starting to look more and more like a different language :-) regards, Vlad Joe Armstrong's profile photo Joe Armstrong unread, May 25, 2011, 4:55:53 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to jm, erlang-q...@erlang.org On Wed, May 25, 2011 at 2:58 AM, jm wrote: On 24/05/11 9:25 PM, Joe Armstrong wrote: If we think of modules as containers of functions, then a module is a "set of functions" - my problem is with granularity. The smallest unit of reuse is the module. Suppose my application want just one function from a particular module - I'm forced to load the entire module. This hampers reuse - I *often* read library code and find a single function and cut-and-paste it into my new code. It also discourages sharing - since the smallest unit I can share is a module. The unit of distribution is an application - a set of modules - why this could not be a list of functions I do not know. I think I'd like to view an application as something that offered a message based service that internally was constructed from a set of functions rather than a set of modules. Here's a "mad hatter" suggestion. Introduce the concept of a "shadow" module. For example, say your using the list module and your after a lists:randomise/1. In this proposal you'd have your own lists module, say joes_lists, -module(joes_lists) randomise(List) -> %% code to randomise the order of the list In you code you would then say, -shadows(joes_lists, lists). then use lists:randomise(AList) in your code. The compiler would first check the lists module and if the required function didn't exist it would use your modules. This would avoid you accidentially overriding the main modules definitions which may cause confusion. In such a case, the clash should be treated as a compilation error. The second part would be distribution and release management. There's several ways this could be done one would be to have a script which would build a set of "patch modules" which would be the cut down versions of your own development modules containing only the code and associated documentation your using in the current project. This is not mad hatter :-) Knuth mad a patch system years ago. You start off with a file - freeze it so it's read only then write patches (replace lines N1,N2 with ...) - he did this with no tool support. One problems today is how we work. We make file1 and edit it to file2, of course we can compute a patch P that transforms file1 to file2. P works like this: P = diff(file1, file2) such that patch(file, P) => file2 The problem with P is that is bears no meaningful semantic information - there is no intentionality. many years ago I thought we shuld have thingks like this: -module(foo). -export([bar/2], ...). ... This defines foo.erl (as usual) Now we define a patch bar.patch -patch(bar). -insert(foo). -insert(baz). -delete(foo:bar/2). -delete(baz:foo/4). -rename(baz:zxy/2, new/2). code as before ... A preprocessor takes bar.patch and makes bar.erl from the patch instructions. This defines a compile time editing operation - insert actually inserts code from foo.erl, delete deletes code ... There are just actually simple refactoring commands. The *nice* thing about bar.patch is we can preserves intentionality (if there is such a word) - the patches produced by diff are usless - this is why a revision control system want you to comment every diff. Here the place for comments is in the patch file. The above is really trivial to implement it would be relatively easy to generate a list of such commands from my (proposed) folding editor -insert(foo, "http://.....") would be very nice /Joe Joe Armstrong's profile photo Joe Armstrong unread, May 25, 2011, 4:57:15 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Edmond Begumisa, Erlang On Wed, May 25, 2011 at 5:34 AM, Edmond Begumisa < ebeg...@hysteria-tech.com> wrote: Hello Joe, I've personally wanted an easy way to load modules from different places. If you started with making modules easy to load from different places, then later provide support for functions, I imagine this would be easier to implement immediately and easier for people to make use of immediately ... where "people" refers to me :) I've got two suggestions (mostly inspired by the Mozilla Framework)... 1. How about letting the programmer choose where the functions come from (kv store, fs, web-server, etc)... For instance, a module could start... -import(["file://misc/collect_int", "http://erlang.org/lists/*", "mnesia://db1/misc/merge_kv"]). By default, the code loader would know what to do when it encounters "file://", "http://", "https://", "ftp://", "sftp://" and "mnesia://" URI schemes. Programmers could be allowed define their own URI schemes by providing callbacks for loading code. For instance, one would be able to define "ubf://" and have the code loader call his/her code when it encounters "ubf://foo/bar" 2. Expanding on this: letting the programmer centralise the possible locations of modules/functions and use this to define namespaces... One possible way of doing this is to steal from the "chrome registration" system that the Mozilla Framework uses. A programmer could define one-level deep namespaces in a configuration file which points to targets that could come from anywhere, then use that in her modules. A module could then start... -import(["erlang://misc/collect_int", "erlang://couch/ mapreduce"]). Where "misc" and "couch" are namespaces and "erlang://" is a special URI scheme telling the code loader to refer to the configuration file "erlang.manifest" which would contain... [{"misc", "file://./ebin/misc.beam"}, {"couch", "http://localhost:7777/ebin/"}] Where "couch" points to a yaws server serving couch beams, or something along those lines. Then she can easy switch between different code sources using one file. A code-signing system could also be introduced. Did this a while back - you can also xform everything - all private modules can have their names changed to MD5SUM(Content) of the module IFYSWIM /Joe Joe Armstrong's profile photo Joe Armstrong unread, May 25, 2011, 5:01:03 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Vlad Dumitrescu, Erlang On Wed, May 25, 2011 at 10:55 AM, Vlad Dumitrescu wrote: On Wed, May 25, 2011 at 07:58, Vance Shipley wrote: I love the idea of ditching modules files and storing functions in a data store. What I'm picturing though is that the functions are stored in their abstract syntax form (i.e. data) so that the issue of formatting standards is stripped away. The problem with that plan seems to be with the preprocessor. It seems to me though that your plan suffers the same problem. If macros are defined at a module level, or in an include file, you'll have to deal with them when you pull the functions out of the module and stuff it into the data store. The extended syntax tree used in the erl_syntax application might be the answer. There are more things that need to be preserved by the parser, which make existing pretty-printers less than good. For example: - integer values can be shown using different bases, from case to case, so one can't use a generic setting. So the parser needs to keep track of that. The new parser/tokeniser does I'm not sure if its worked its way in the pretty printers - I do have some unpublished code that does exact colorization of the source program Joe Armstrong's profile photo Joe Armstrong unread, May 25, 2011, 5:09:04 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Richard O'Keefe, Erlang How about public Functions etc. private Functions etc. end. Functions etc is just a regular sequence of function defs macros, records etc. The use of the words public and private would attract hoards of programmers to our humble little language... /Joe Vlad Dumitrescu's profile photo Vlad Dumitrescu unread, May 25, 2011, 5:19:34 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang Hi, On Wed, May 25, 2011 at 11:09, Joe Armstrong wrote: On Wed, May 25, 2011 at 6:29 AM, Richard O'Keefe < o...@cs.otago.ac.nz> wrote: On 24/05/2011, at 8:06 PM, Joe Armstrong wrote: > Why do we need modules at all? To hide data types, and to provide short names (we can import a function and not have to keep on repeating a long long name for it). I note, without recommendation either way, the existence of local in end in Standard ML, where the second lot of declarations defines things visible outside but the first lot defines things visible only inside this form. How about public Functions etc. private Functions etc. end. Functions etc is just a regular sequence of function defs macros, records etc. Where would this declaration be located? This would in effect become a module definition, if I understand correctly? Will it point out functions from the "sea of functions" out there and group them together? What happens if some other such declaration says that my private functions are public? I think I need to think more about this. I would start by trying to describe a user that sits down and wants to write a program. What would he need to do? What support is needed from the tools? The use of the words public and private would attract hoards of programmers to our humble little language... :-) /Vlad Tim Watson's profile photo Tim Watson unread, May 25, 2011, 5:26:17 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Joe Armstrong, Erlang > > Very interesting idea - there was once a pascal pretty printer that analysed > your code > and inferred the style that you preferred - "Umm this guy likes semi-colons > at the end of > line ..." they in applied it to code you hadn't written. >> >> I'm guessing this will require patching the compiler to preserve the >> >> type specs in the beam code. That is something I would need help with, >> >> but may be trivial for others. >> > >> > Easy - it is there already (I think) poke around in beam_lib.erl >> > this analyses the .beam files. >> > >> >> I was pretty sure this wasn't the case. I'll dig in to beam_lib at >> lunch time and have another look. > > So you're sort of correct. The specs are present in the abstract_code chunk. I'm guessing this will only be the case if (a) you've compiled with debug_info and/or (b) you've used the abstract_code parse_transform (from github?) to ensure it's present either way. You would need to make sure it is preserved in order to get the type info by inspecting the beam. Perhaps slurping dialyzers generated plt file(s) would be a better (more general) approach? Maybe we should talk to Kostis - I suspect the idea of a type-based search engine might whet his appetite somewhat. Robert Virding's profile photo Robert Virding unread, May 25, 2011, 5:47:34 AM5/25/11 Reply to author Sign in to reply to author Forward Sign in to forward Delete You do not have permission to delete messages in this group Copy link Report message Show original message Either email addresses are anonymous for this group or you need the view member email addresses permission to view the original message to Anthony Ramine, Erlang ----- "Anthony Ramine" wrote: > > Well then, forgive me for this little offtopic but you seem to know what is unapproved > and what is unfinished. Are structs - not your proper structs but the commented-out > production found in erl_parse.yrl struct -> atom tuple - an unapproved feature or an > unfinished feature? > It is the remains of an unfinished feature which no one ever removed. There are more commented out remains in erl_expand_records, erl_lint and erl_pp. They should really all be removed as if/when the new structs/frames are implemented they will be wrong and in the way. Robert It is loading more messages. 0 new messages Search Clear search Close search Google apps Main menu