[HN Gopher] Help us collect modern Python NumPy code solutions
___________________________________________________________________
Help us collect modern Python NumPy code solutions
Author : nonunicorn
Score : 89 points
Date : 2022-03-28 10:18 UTC (12 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| johndough wrote:
| For almost all issues, you can simply search for
| site:stackoverflow.com + the title of the issue and get a larger
| variety of solutions ranked by upvotes, so this repository seems
| strictly worse.
|
| Is there any advantage that this repository brings over just
| searching for the question on StackOverflow?
| charlieyu1 wrote:
| A lot of times the best solution is buried in the third or
| fourth solution in the 5th or 6th search result. I can't even
| find the solution that I know I have googled before.
| jmmcd wrote:
| The README explains that they want a single curated answer, not
| a list of 15 answers with 25 comments. This does seem
| worthwhile, at least for some types of questions.
| johndough wrote:
| Isn't the "curated" answer on Stack Overflow the answer with
| most votes?
|
| Sure, sometimes the top voted answer is not the answer to my
| particular problem at hand, but if it isn't, I'd rather have
| more alternative answers than nothing.
| hackernewds wrote:
| To a beginner they're not really able to make that
| determination, neither would have the patience to stay
| interested checking SO hundreds of times
| danuker wrote:
| Even better, search the stackoverflow.com site directly. It
| seems Google no longer indexes everything (if it ever did).
| JoBrad wrote:
| I didn't review all of the code, but the examples I saw seemed
| to have some basic explanations with them. Additionally, they
| have samples for a lot of different technologies. A curated
| repo seems much nicer than scoring several StackOverflow
| answers. Depends on the code quality, of course.
| johndough wrote:
| The code quality seems questionable. A few examples:
|
| Repository claims modern code, but uses Python 2 (end of life
| January 1, 2020) https://github.com/Onelinerhub/onelinerhub/b
| lob/main/python/... Copied from
| https://wiki.python.org/moin/Powerful%20Python%20One-Liners
|
| Incorrect indentation causing unnecessarily large memory
| usage https://github.com/Onelinerhub/onelinerhub/blob/main/py
| thon/...
|
| Undefined behavior for i = 31 https://github.com/Onelinerhub/
| onelinerhub/blob/main/c/bits_...
|
| Printing of uninitialized memory https://github.com/Onelinerh
| ub/onelinerhub/blob/main/c/set_v...
|
| Undefined behavior https://github.com/Onelinerhub/onelinerhub
| /blob/main/c/swap_...
| Beltiras wrote:
| Fixed some of those. Simple case of just pushing the edit
| button and sending a PR.
| johndough wrote:
| The suggested solution https://github.com/Onelinerhub/one
| linerhub/blob/34467e427cc6... still runs out of memory
| with many files.
|
| I believe the original intention behind using mode="a"
| was to append to the output file while reading the input
| files at the same time. This way, there is no need for an
| ever-growing string array.
|
| But there are still many other issues like using default
| platform string encoding instead of detecting it properly
| or at least using utf-8, checking for ".txt" anywhere in
| the path instead of at the end, and not closing the input
| files with a context manager like the output file, which
| suggests that this code is just pierced together from
| various sources.
|
| A robust solution would require many more lines.
| mynameismon wrote:
| Another memory issue (although unlikely) could also be
| the file itself being larger than the RAM size, which can
| be avoided by chunking the textfiles.
| hermitdev wrote:
| For bonus points: how about properly handling UTF-8 byte
| order marks (BOM), too? The way this code is written if
| the files contain a BOM, you're going to pass the BOM
| right into the destination file. Fine for the first file,
| not so fine for any other file. Dropping a BOM mark in
| the middle of a file isn't going to result in valid
| UTF-8...
| danbruc wrote:
| I looked at the C# ones - the code is not great and neither
| are the explanations, both show a lack of experience in C#
| and .NET. How to reverse the characters in a
| string
| String.Join("",("1234Simple").Select(c=>c).Reverse());
| - String.Join("", - concatenate one or more characters in a
| single list or array, by using empty "" string literal
| separator - .Select(c=>c) - select each and every other
| character in a string into an array of characters -
| .Reverse() - reverse each and every one elements in an array
| or list*
|
| Calling Select(c => c) does nothing and can be omitted. The
| use of String.Join() feels wrong, it is just used to
| concatenate the characters and not to join them with some
| separator in between. Using the String(Char[]) constructor
| seems more appropriate. new
| String("1234Simple".Reverse().ToArray())
|
| But that is still bad code, it will rip apart and reverse
| grapheme clusters, a proper implementation should use
| System.Globalization.StringInfo. Not sure if it would still
| fit into one line.
|
| Also the explanation are not very accurate and show a lack of
| understanding of the .NET collection type architecture and
| how array, list and enumeration types fit together.
| How to reverse the words in a phrase ("This tests
| that").Split(' ').Aggregate((a,b) => b + " " + a);
| - .Split(' ') - splits a string by using a space ' '
| separator - .Aggregate - gather every one other element
| a, in an element list to a target element b - (a,b) =>
| b + " " + a - arrow function taking input arguments (a,b)
| used to concatenate b gathered character argument in the list
| in reverse with " " space separator to the targeted character
| argument a
|
| Here I would say IEnumerable<T>.Reverse() paired with
| String.Join() is the better solution as it is much easier to
| unterstand than using IEnumerable<T>.Aggregate().
| String.Join(" ", "This tests that".Split(' ').Reverse())
|
| Now nobody has to figure out that (a, b) are the accumulated
| result and the current list element in this order which is
| even with the explanation not to obvious - better variable
| names could really help here. How to retrieve
| numeric fibonacci list up to 10 List<long> Fib =
| new List<long>(); Enumerable.Range(0,10).ToList().ForEach(n
| => (Fib).Add(n <= 1 ? 1 : Fib[n-2] + Fib[n-1])); -
| new List(); - create an empty list of large integers -
| Enumerable.Range(0,10) - get an enumerable list populated
| with integers from 0 to 10 - .ToList() - convert the
| enumerable list to a standard list - .ForEach - apply a
| specific arrow function for each and every one item in the
| list - n => (Fib).Add - arrow function used to add an
| item n to enumerable list Fib based on a condition for every
| one item in the list - n <= 1 ? 1 : Fib[n-2] + Fib[n-1]
| - check if n less or equal to 1 then return 1, if n is larger
| then return addition of elements [n-2] and [n-1] in Fib list
| for every n
|
| This is really abusing LINQ to express a for loop - create an
| IEnumerbale<T>, materialize it as a List<T> only to be able
| to use List<T>.ForEach() which is - for better or worse - not
| available for IEnumerable<T> and then manipulate a different
| list created with a second statement in the same line. If you
| desperately want a one line solution for Fibonacci numbers,
| maybe consider using a closed-form expression, the recursive
| definition just doesn't work well in C# and a single line.
| This will however run into precision issues for big enough
| numbers while working with integers will work fine until
| Int64 overflows. Enumerable.Range(0,
| 10).Select(n => (Int64)(0.5 + Math.Pow((1 + Math.Sqrt(5)) /
| 2, n) / Math.Sqrt(5)))
|
| Or with _using static System.Math_ even shorter and more
| readable. Enumerable.Range(0, 10).Select(n =>
| (Int64)(0.5 + Pow((1 + Sqrt(5)) / 2, n) / Sqrt(5)))
|
| The explanation has again issues with the collection types,
| Enumerable.Range(0, 10) generates 10 values starting from 0,
| i.e. 0 to 9, but maybe they wanted to say from 0 (inclusive)
| to 10 (exclusive). Also arrow functions are called lambda
| functions or lambda expressions in C#.
| xnorswap wrote:
| Indeed, their example is terrible, because it's also the
| kind of 'toy' problem that just doesn't have good real
| world uses too.
|
| How often do you want to reverse a string.
|
| I think your solution is fine and isn't "bad code" at all.
| Certainly a real explanation warrants talking about the
| IEnumerable interface.
|
| I'd also consider the simpler (and ever so slightly better
| performing): var buffer =
| "1234simple".ToArray(); Array.Reverse(buffer);
| new string(buffer);
|
| edit: The following is dramatically (x10) even better
| performing: var buffer =
| "1234simple".ToCharArray(); Array.Reverse(buffer);
| new string(buffer);
|
| But again with the caveat of that not respecting unicode
| sequences.
|
| But to re-iterate and to add to your criticism, I can't
| understand the use-case of needing to do this outside of
| toy problems.
| kingaillas wrote:
| > I can't understand the use-case of needing to do this
| outside of toy problems.
|
| Not sure, not my domain, but perhaps bioinformatics has
| the equivalent of the convolution operation on sequences?
|
| EDIT: there is the "reversal" operation. Peeking ahead
| from the Rosalind problems, I'm not quite up to Reversal
| Distance (https://rosalind.info/problems/rear/).
| danbruc wrote:
| But would they represent their data as character strings
| or would they pack four nucleotides with two bits each
| into one byte? Or would they be working on the amino acid
| level or something else all together? And would they
| really physically reverse potentially very long strings
| or just use a decreasing index into the original string?
| I am sure reversing strings has some use cases, maybe
| even in bioinformatics, but I would agree with the other
| comment, it seems a pretty niche operation, otherwise
| there would probably be a String.Reverse() method.
| omarhaneef wrote:
| Just had a quick look at some of the responders who collectively
| say:
|
| 1 What is the use of this curated list over just accepting the
| popularly voted answer?
|
| 2 I found some errors in the default answer and here is a
| detailed explanation of what was wrong with the answer.
|
| And I hope they can see that #2 shows the issues with #1.
|
| To put it another way if I were looking for the best solution
| (which I often am), and I had a choice between a one line stack
| overflow answer and one of these detailed solutions, I would pick
| the detailed, curated solution every time.
|
| edit: And the best stack overflow answers are often detailed
| curated solutions, so I am not saying they don't wind their way
| up there.
| m00dy wrote:
| #write a function that zips numpy arrays
|
| import numpy as np def zip_arrays(arrays):
| return np.array([np.concatenate(arr) for arr in zip(*arrays)])
| PartiallyTyped wrote:
| >>> arrays = [np.arange(1000)]*10
|
| >>> zip_arrays(arrays)
|
| Traceback (most recent call last): File
| "<stdin>", line 1, in <module> File "<stdin>", line
| 2, in zip_arrays2 File "<stdin>", line 2, in
| <listcomp> File "<__array_function__ internals>",
| line 180, in concatenate
|
| ValueError: zero-dimensional arrays cannot be concatenated
|
| Try this:
|
| def zip_arrays(arrays): return
| np.concatenate([array[:l,...,None] for l in [min(array.shape[0]
| for array in arrays)] for array in arrays], axis=-1)
|
| Or if we are trying to be more tidy instead of a one liner
|
| def zip_arrays(arrays): l = min(a.shape[0]
| for a in arrays) arrays = [a[:l, ..., None] for a
| in arrays] return np.concatenate(arrays)
| m00dy wrote:
| well, I used the code from Copilot :)
| johndough wrote:
| I think a more Pythonic solution would be
| np.array(arrays).T
|
| or maybe np.copy(arrays).T
|
| for one less character. The shortest solution is probably
| np.c_[arrays].T
|
| but likely causes a Google search on first read.
| kzrdude wrote:
| I'm used to numpy and I can't understand np.c_ even with
| the docs. It says "Translates slice objects to
| concatenation along the second axis." but the examples pass
| in a list of arrays and use no slice objects (the builtin
| slice, I'm assuming)?
|
| If there was an example using just `np.c_[1:5, 11:15]` it
| would make sense. But clearly np.c_'s meaning has been
| extended beyond slices in some way. (Maybe I'm coming
| closer to understanding, but still miffed about the
| misleading doc.)
| PartiallyTyped wrote:
| That does not zip though. In fact, it creates a
| 1-dimensional array of objects because the arrays do not
| have the same number of items. >>> a =
| [np.arange(i) for i in range(1, 11)] >>>
| np.array(a).T array([array([0]), array([0,
| 1]), array([0, 1, 2]), array([0, 1, 2, 3]),
| array([0, 1, 2, 3, 4]), array([0, 1, 2, 3, 4, 5]),
| array([0, 1, 2, 3, 4, 5, 6]), array([0, 1, 2, 3, 4, 5, 6,
| 7]), array([0, 1, 2, 3, 4, 5, 6, 7, 8]),
| array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])], dtype=object)
|
| In fact, if we are to zip them, it should yield:
|
| >>> list(zip(*a))
|
| [(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)]
|
| zip_arrays otoh does give the same result.
|
| >>> zip_arrays(a)
|
| array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])
| johndough wrote:
| You are completely right. Nevertheless, when I zip over
| arrays of different length, it is usually a bug instead
| of being intentional.
|
| A code-golfed version for different array lengths:
| np.c_[[a[:min(map(len,arrays))].T for a in arrays]].T
|
| The trick to get a local variable within a list
| comprehension statement with for l in
| [foo()]
|
| from your zip_array function is very cool by the way, but
| I sacrificed it in favor of shorter code.
| PartiallyTyped wrote:
| I like your your solution, I think using
| `a[0:min(map(len,arrays))]` (ie with 0) is faster because
| it returns a slice and not a copy of the array.
|
| I am glad you liked the trick!
| TrackerFF wrote:
| Some of these problems don't really have much information /
| specs.
|
| "Generate random array" (python-numpy)"
|
| Ok - but what dimension? size/length? what range? sample from
| what distribution? only unique values? etc.
|
| Many of these problems have "native" one-liner numpy solutions
| which are trivial to find (basically just read the
| documentation).
| denisrosset wrote:
| hoosieree wrote:
| I learned J before NumPy and I'm glad I did, because it's much
| easier to see and learn the algorithms when they're short
| symbols.
|
| Compare np.sqrt(sum(np.square(x)))
|
| versus %: +/ *: x
|
| So if you can translate one of the array languages to NumPy, you
| can tap into the wealth of "idioms" collected over the years.
|
| For example: J Phrases[0] (organized by category), BQNcrate[1]
| and APLcart[2] (searchable).
|
| [0]: https://www.jsoftware.com/help/phrases/sums_means.htm [1]:
| https://mlochbaum.github.io/bqncrate [2]: https://aplcart.info/
| kzrdude wrote:
| Interesting suggestion since %: +/ *: x
|
| Is impossible to read of course without learning the J
| language's main constructs. While the python function
| application is easier to guess from other languages.
| hoosieree wrote:
| I know English, so I'll just guess at what those non-English
| people are saying. What could go wrong?
| kzrdude wrote:
| I know spanish so I understand some french sentences. It's
| actually quite easy to see what I don't understand. It
| doesn't usually go wrong.
| johndough wrote:
| I think a fairer comparison would be
| np.linalg.norm(x)
|
| or even norm(x)
|
| if you import numpy.linalg.norm as norm
|
| beforehand. This comes with the advantage that you can look up
| the function by name to know what it does instead of having to
| look up chains of symbols on several websites.
| RodgerTheGreat wrote:
| In practice, if you're reading a J expression which contains
| unfamiliar primitives, there's only one place you need to
| look: https://code.jsoftware.com/wiki/NuVoc
|
| Idioms are composed from primitives, but many of them are
| shorter than a descriptive name would be, and since the set
| of primitives is small (compared to the standard library in
| most languages), it doesn't take that long to memorize most
| or all of the symbols.
| bdrum wrote:
| I've added a comment to the issue How to plot a data frame
| https://github.com/Onelinerhub/onelinerhub/issues/1378#issue...
|
| Actually concerning visualization tasks I don't see a sense in
| one line solution because as I think you should understand how
| you want to represent your data and this has strong influence to
| calling signatures.
| ledauphin wrote:
| I would love to see something like this for `pandas`. Also
| SQLAlchemy. EDIT: apparently there's a very limited set of
| oneliners for `pandas` already.
|
| Why do so few libraries seem to understand that good
| documentation starts by providing meaningful examples of simple
| tasks with semantic labels that can be found via search?
| Reference-style documentation assumes that you already know the
| name of the thing you're looking for (you don't), or that you
| have time to read for several days and build up an entire mental
| model of the system in one go before getting anything productive
| done (also no).
| muggermuch wrote:
| +1 absolutely - it's been one of my pet peeves. There really
| needs to be some thought put into "doc UX".
| na85 wrote:
| The worst is when some project or library is extolled as being
| extensively documented and you find out it's just doxygen
| autogens.
|
| Awful.
| kwertzzz wrote:
| Would it be useful to add a Julia section?
| infinityio wrote:
| Just had a quick look at a few of the python examples:
|
| - it might be neat, but advocating for `eval(input())` [0] might
| not be the safest solution for this problem, especially without
| explaining the dangers of `eval` (assuming this site is partially
| aimed at beginners?)
|
| - for an article titled 'how to terminate a script', the
| suggested method (`quit()`) [1] is specifically described in the
| official python docs [2] as code that "should not be used in
| programs".
|
| [0]: https://onelinerhub.com/python/calculator [1]:
| https://onelinerhub.com/python/how-to-terminate-script [2]:
| https://docs.python.org/3/library/constants.html#constants-a...
| kortex wrote:
| I agree eval is pretty gross and folks should not be
| encouraging its use since it can lead to bad habits.
|
| Quit/exit are pretty innocuous though. I actually just thought
| exit was an alias for sys.exit. I've never had a problem using
| plain exit() in programs.
___________________________________________________________________
(page generated 2022-03-28 23:02 UTC)