[HN Gopher] Show HN: Csvbox.io - CSV import button for a web app...
___________________________________________________________________
Show HN: Csvbox.io - CSV import button for a web app, SaaS or API
Author : spreadr
Score : 135 points
Date : 2021-05-12 10:10 UTC (12 hours ago)
(HTM) web link (csvbox.io)
(TXT) w3m dump (csvbox.io)
| devops000 wrote:
| Very nice!
| spreadr wrote:
| Thank you!
| scottoreilly wrote:
| Typo in headline: should be "validate data" instead of "valid
| data". Otherwise, looking good!
| ericcholis wrote:
| Another typo in the javascript in "Code" tab when editing a
| sheet
|
| "There was a porblem uploading the sheet"
| spreadr wrote:
| Corrected. Thank you :)
| imiric wrote:
| Congrats on the launch!
|
| The service looks interesting, but I'm wondering if there's a
| technical reason why it's a SaaS. CSVs could be parsed entirely
| on the client, and it should be possible for .XLS/.XLSX as well.
| If not with pure JavaScript+Web Workers then surely with
| WebAssembly.
|
| Doing this traditionally on the backend is fine of course, but as
| others have mentioned, trusting you with potentially sensitive
| data is a hurdle for many companies and individuals.
| spreadr wrote:
| Thank you!
|
| To be honest, we haven't thought about the possibility to work
| everything client side. Its an idea worth exploring.
|
| Our aim is to integrate the importer with multiple destinations
| such as databases (MySQL, PostgreSQL, S3 etc), APIs (custom,
| Zapier, GSheets, Airtable) and others (FTP, Email etc). Coding
| server side would be easier and faster.
|
| Moreover we plan to introduce on-premise deployment options in
| the future.
| pm90 wrote:
| Client side would be pretty amazing, especially considering if
| this is sensitive data.
|
| Then again, enterprises that deal with sensitive data don't
| just allow employees to access that data via a web browser (or
| should not) and especially not through a modern one. So it
| would be a huge pain to figure out which browsers are capable
| of even supporting Wasm and other modern tech.
| flippyhead wrote:
| This seems like a great idea. It's a hassle to do this and for a
| bunch of use cases I can see outsourcing it like this as very
| useful.
| spreadr wrote:
| Thank you! I am already using it in 3 of my other apps.
| onion2k wrote:
| There are a lot of assumptions being made about the shape of the
| data in the client side validation rules, and seemingly no way to
| control them. For example, in the example.csv file there's an
| "Age" column that has to be completed by the user in order to
| submit the data, and no way to turn that off. I don't think that
| would work at all for any production system; often you just don't
| have the data available. This would force you to put dummy
| placeholder data in.
| spreadr wrote:
| Thank you for your feedback.
|
| The validation rules for any sheet can be controlled via the
| admin dashboard. You can choose to mark any column as
| required/not required.
|
| To demonstrate the validation capability we enforced 'required'
| validation on the 'Age' column in the demo.
| gregsadetsky wrote:
| Some notes:
|
| - in your embed code snippet, you have `There was some porblem
| uploading the sheet` -> the word "problem" is misspelled
|
| - the embed code includes https://js.csvbox.io/script.js which
| has two console.log debugging calls which could be removed in
| production
|
| - when adding columns to a new sheet, it would be neat to be able
| to paste the header line from a CSV file and for the columns to
| be created (this wouldn't help with the Column Type or Required
| options, but at least the Column Name would be set/wouldn't have
| to be typed one by one)
|
| - also, when adding columns, it would be great to press the enter
| key after typing a Column Name to save/close the modal
|
| I agree with most other commenters' feedback re: pricing,
| especially https://news.ycombinator.com/item?id=27133717 i.e.
| making the Sandbox much more generous. I was surprised to see
| only 5 rows imported initially, and thought it was a bug on your
| side (i.e. I thought only the first 5 rows were parsed)
|
| Congrats on the launch!
| spreadr wrote:
| Thanks for your feedback and actually trying out the app in the
| first place.
|
| I will get the bugs fixed.
|
| I have also noted down the suggestions on column name and
| Keypress. We will implement them soon.
|
| Pricing restructure is also on the cards.
|
| Once again, thank you!!!
| gregsadetsky wrote:
| More feedback :-)
|
| How are you dealing with character encoding? I uploaded a CSV
| file that has an utf-8 character in the name of a column, and
| in your import flow, under "File Columns", the column name
| appeared incorrectly encoded (i.e. it's been latin-1'ized)
|
| My file is most probably missing the UTF-8 BOM. I'm not sure
| what a good option is here.
|
| Have an advanced setting for Sheets (on your site) and pre-
| set the uploaded files' encoding?
| spreadr wrote:
| We have a preset of utf-8 and will probably have to find a
| way for managing encoding.
| enahs-sf wrote:
| I think this product is cool, but honestly you should charge
| more. The lowest tier after the free should be at least $19.
| Maybe more. The folks who are purchasing this product are likely
| not paying for it with their own money (business expense) and
| therefore can justify a higher cost.
| spreadr wrote:
| Thank you for the feedback.
|
| This is our introductory pricing that will evolve over time.
| Here is our dilemma - on one hand we want to target indie
| developers, side projects, bootstrapped startups (that are cost
| conscious) and on the other hand we want to maximize the
| earning potential and also not look like a low value product.
| config_yml wrote:
| Congrats on building this! I've had the exact same idea and pain
| for many of the projects I've worked on, so happy to see this out
| there and I hope you'll be successful with it!
| spreadr wrote:
| Thank you!
| [deleted]
| admissionsguy wrote:
| Oh dear, I had the exact same idea near the top of my list. Glad
| to see it is being well received, and nicely done, too. Good
| speed to you!
|
| When I use the test file, it makes all fields required. It should
| probably accept empty/unknown values, no? (I just saw you
| commented on that elsewhere)
|
| In my experience, the manual validation feature is not that
| useful for most use cases, where the sheets have lots of rows. My
| clients have always preferred to have a list of validation errors
| / row numbers, modify the source spreadsheet itself using all the
| power of Excel, and reupload.
|
| I think the true value of this tool will be in how well it deals
| with messy data. A common problem is character encoding. At the
| moment, when I put in Polish characters in the file "EACASZAECA",
| it gets displayed as "EACAAA1/2AAAA" - need to fix that.
|
| The validation could be more robust. Common data types (age,
| first names) could easily have their own specific rules. At the
| moment, you can put negative or out of range values for the age,
| or special characters for the name. And what would be really
| valuable is detection of outlier values - to tell the user if a
| value is three times longer than other values in the same column,
| contains unusual characters, etc.
|
| As for features, I would like a PDF import (you can just take one
| of the open source pdf2excel libraries, then write a bit of smart
| code to detect&strip the headers and footers - took me 3 days to
| implement last time and the client loved it ;) Doesn't need to
| cover all cases to be very useful.
|
| And of course, file upload is a must, ideally allowing multiple
| columns IMAGE1, IMAGE2, etc.
| spreadr wrote:
| Thank you for the awesome suggestions!
|
| You can configure the validation criteria for each column via
| the app admin. Simply mark the column as 'not required' to
| accept empty values.
|
| I have noted down all your feedback for fast implementation.
|
| Cant thank you enough!
| PeterWhittaker wrote:
| How does this compare with some of the available FOSS
| alternatives, e.g., https://github.com/mholt/PapaParse ?
| spreadr wrote:
| PapaParse is a great library for parsing csv files. Whereas
| csvbox.io is a no code widget that can directly we installed on
| your site to help you get started in minutes.
|
| PapaParse has no built is validations. With csvbox you can
| define a data model in the admin. The users can match columns,
| validate data and submit the file without you having to write a
| single line of code.
| fyp1 wrote:
| basic bootstrap UI? no thank you!
| spreadr wrote:
| Its a bootstrapped startup :)
|
| Your point is well taken. We will upgrade in the future.
| tunaisnotfree wrote:
| The highest tier limit of 100K rows is unfortunate as I've a
| number of imports that often range closer to 400k which would
| cause me to have to pass on this. I like the idea though!
| spreadr wrote:
| Thank you for your feedback. We are already testing for higher
| limits and will soon introduce/update plans accordingly.
| jhunter1016 wrote:
| This is fantastic! Building an importer is not fun at all. Really
| smart product.
| spreadr wrote:
| Absolutely, I hated building importers and more so, maintaining
| them!
| granshaw wrote:
| One feature I've needed to build into all my csv parsers is to be
| tolerant of some "junk" rows at top, and automatically find the
| row where the actual headers are, which might not be the first
| row. Do you handle this case?
|
| Eg Mac Numbers app is really bad about this where it will
| sometimes insert the file name as a "title row"
| spreadr wrote:
| Thats great feedback. We do not yet have this option. I am
| adding it to your to-do list.
|
| Currently, once the users upload the file, they have the option
| to delete any rows that they do not want, by clicking the
| remove button in the widget itself. Once the data is clean they
| can submit the file.
| Raed667 wrote:
| I built complex CSV/Excel importers for 3 companies so far. The
| business really needed and benefited from it, but it was some of
| the most boring coding I have ever done.
|
| Great idea to make it its own product!
|
| Best of luck!
| spreadr wrote:
| Thank you!
| nwhatt wrote:
| Very cool idea.
|
| I recently gave one pass at a Jira CSV import and went with the
| API instead.
|
| Excel is the ultimate no-code solution.
| yunyu wrote:
| Congrats on the launch! How do you differentiate yourself from
| hosted competitors like https://flatfile.io/?
| spreadr wrote:
| Flatfile is a fantastic service. In fact, I was one of their
| early users.
|
| 1. We are focused on indie developers and startups. Our pricing
| is reflective of that. 2. csvbox is a no code widget. The
| entire column model and validation requirements can be defined
| in the app admin without the need to write a single line of
| code.
| mchusma wrote:
| I can say I looked at flat file and I think their pricing is
| just out of whack for what it does. Just being a cheaper
| version seems like a good market to me.
|
| P.S. others promote self hosted, but I wouldn't bother if
| we're you. I did esignatures and lots of people asked for it,
| so we built it, but no indie/startup dev ever bought it. The
| only people who only ever truly implemented self hosting were
| enterprises. My 2 cents.
| deedubaya wrote:
| I want this. I want this so bad. But I'm not giving your systems
| my customer data -- sorry. I'd pay $XXX/year for a self hosted
| version.
| spreadr wrote:
| Thanks for writing in the first place.
|
| I completely understand your position. Self hosted version is
| in our long term development pipeline.
|
| Data security is important to us. We have the option to get the
| data deleted as soon as it is pushed to your app. Please check
| our note on user data - https://help.csvbox.io/legal/data-
| policy
| richardkmichael wrote:
| I think this is a great idea. I've had to solve this on almost
| every web app I've worked on.
|
| I've just gone though your `example.csv`, and have this feedback:
|
| - the (row) "Remove" action seemed like data column to me,
| because it is placed adjacent to the data. I would separate the
| actions completely from the data. I'd probably avoid the "X"
| within the column, as again, it looks like data. Admittedly,
| users probably know the content of the CSV they are uploading, so
| this might not be too big a concern. But the UX/semantics of
| styling the "actions" the same as "data" seems like it will lead
| to confusion. At the moment, the only action is "Remove", so I
| might drop the Remove "column", just put a button labelled
| "Remove"; after all, the "X" is already a "button" (as a link).
| That said, bulk actions a chore to implement and so I suspect
| getting this working nicely will probably be a big draw for your
| target market. Whether it's checkbox boxes with a single select
| drop down to apply an "action", or a select drop down on each row
| (painful UX, I'd think). Hopefully you already have a revision on
| your roadmap.
|
| - The wording "File has 1 invalid rows. Please resolve the errors
| before uploading." is occurring after the user has already
| uploaded the CSV, so I think it should be adjusted. At this point
| in the workflow, I'd probably stop referring to "File" (the user
| is only concerned with the data at this point). I'd suggest more
| succinct wording, perhaps: "1 invalid row must be resolve before
| continuing."; or "...before you may continue.", if you prefer
| using the second person in the app messages.
|
| - The full screen modal for a small number of rows forces the
| user to hunt for the buttons and mouse a far distance. I realize
| for most data, the modal is _likely_ to be the entire screen (and
| multi-page), but nevertheless, I would probably make the modal
| shrink to the data. As a consideration for your market: I 've
| written apps which use CSV for only ~25 rows, and I would still
| consider using this because the user interaction for sanity and
| clean-up was still code I would've prefer to skip writing.
|
| - I would increase the number of rows permitted at the lower
| tiers. Maybe you are analyzing the imports and have a lot of
| information about the pricing breakpoints and segmentation, but
| my hunch would be that fewer imports with more rows might entice
| people. I can think of apps which might have only 5 or 10 imports
| per month, but need more than 50K rows per import, but it's a
| pretty big jump to Basic, so you might miss those customers.
|
| - I would make the corrected/edited CSV available for download
| (by the customer) at the end of the import. If anyone needs to
| re-import, it will surely be annoying to re-correct it during
| import; or even to remember all the corrections they made during
| import and go back and correct their original spreadsheet (or
| other data source).
|
| - Would be nice to see PostgreSQL on your destination roadmap :-)
|
| Again, this seems like a really good idea to me. I wish you
| success!
| spreadr wrote:
| Fantastic! I cannot thank you enough for this feedback.
|
| I am adding all of the 6 points that you have raised in our
| roadmap to be developed in the near term.
| richardkmichael wrote:
| You're welcome!
|
| I just thought of one more comment: have you tested the UI/UX
| when there are enough columns so that the modal must scroll?
| In that case, the actions adjacent to the data will also
| probably not work very well; i.e., I would not make the user
| horizontal scroll to access the actions. Meaning, I think
| you'll definitely want the actions separated from the data,
| so that the data scrolls while the actions remain fixed
| position.
|
| Sorry I don't have more time to experiment with actual data
| types. I'm sure you can do a lot here too. I once dealt with
| an app that imported a CSV with geographic coords [lat,long].
| During import validation, we showed a map to allow for
| correction / precise placement. That kind of richness would
| be great. To be validated with your market, of course.
|
| Good luck!
| spreadr wrote:
| We have tested with 30 columns. The horizontal scrolling
| works alright, but as you have rightly pointed out, the
| actions column is right at the end. We will have to work
| something out.
|
| We plan to add advanced validation capabilities as and when
| we get use cases demanded by real customers. Its a huge
| project in itself!
|
| Thanks once again!
| bottled_poe wrote:
| I'm speculating, but feel that your pricing model should be more
| strictly based on number of rows processed. Not sure why a 100k
| row limit is imposed on the top tier. I suspect this model would
| better reflect where your costs are and the customer would have a
| precise way to estimate costs. I wonder why the pricing model is
| more complicated than this?
| spreadr wrote:
| Thank you for your feedback. Our costs are a function of number
| of rows processed and the total imports. We factored that in
| the pricing.
|
| We will soon be introducing plans with higher row limits as
| well.
| fareesh wrote:
| Maybe sell a lifetime subscription with the caveat that you give
| a self-hosted version if you stop offering the service.
| stevoski wrote:
| My experience is, don't sell lifetime subscriptions, and don't
| sell self-hosted versions.
|
| Neither of them work out, business-wise.
| spreadr wrote:
| Interesting idea. It will inject cash flow at the initial
| stage.
| spreadr wrote:
| Hello All, I'm Tejas Sangoi.
|
| I have been developing Shopify apps for the past 5 years.
| Currently working on a new project csvbox.io to solve a common
| problem that I have faced many times.
|
| There have been many applications where we needed to accept
| spreadsheets from the end customers. Building an importer from
| scratch is time consuming and maintaining it is another headache.
| Moreover, the import process for the customers is messy because
| they have to read the docs and follow the upload templates. In
| many cases, after long support email threads we ultimately have
| to manually accept, clean and input the data.
|
| As a solution I built the csvbox.io import widget. Its a no code,
| drop-in widget that allows you to accept files in minutes. Users
| can upload files, match columns, validate data all in a few
| clicks. You receive ready to use data in your app. A better
| experience for your customers, fewer headaches for your team!
|
| I launched it only a few days ago. Happy to receive feedback on
| everything - UI/UX, features, pricing etc.
| adamhp wrote:
| Typo on the main page: Featues? :)
| spreadr wrote:
| Corrected now :)
| futhey wrote:
| Looks nice! Seems like you've thought of all the basics.
|
| Easy to miss the drop target though (testing this at a bit past
| midnight) - seems like it would be easy to make that larger (or
| just make the whole window a drop target for the CSV)
| spreadr wrote:
| Thank you for the feedback.
|
| Making the entire window a drop target is a neat idea. I am
| noting it down for implementation.
| airblade wrote:
| This is a nifty idea for a product.
|
| Does / will your system import images in spreadsheets?
| spreadr wrote:
| Thank you!
|
| Currently, the widget does not support images. Honestly, we
| havent yet thought about it. Now that you have raised this
| point, we will surely discuss it.
| xmpirate wrote:
| It would be nice to have the ability to send data to API in
| batches instead of sending all data in bulk. This is really
| helpful when sending huge csv/excel files.
| xmpirate wrote:
| Or an ability to stream the data rows
| spreadr wrote:
| If you configure chunk size = 1 then the data will be
| pushed one row at a time.
| spreadr wrote:
| Thanks for writing. We do have that option. You can even
| configure the chunk size that defines the number of rows to
| be pushed in each POST request.
| taf2 wrote:
| Ah very cool I built something like this for our application
| and used a web worker to offload validation etc... did you do
| something similar?
| hndl wrote:
| It looks neat! Are you SOC2 compliant?
| spreadr wrote:
| Thank you! Not SOC2 complaint yet.
|
| We'll take on this once we hit a few milestones.
| dividuum wrote:
| Having built a CSV import pipeline handling a few thousand
| different CSV files from almost the same number of different
| providers: I've probably seen almost everything one can mess up
| while writing out data: Invalid or missing escaping, double or
| per column string encoding, truncated columns, BOMs, EANs in E
| notation, Month names instead of numbers and so on. Out of
| curiosity: do you handle any of that?
| spreadr wrote:
| Thats a great question. We handle most of it, except:
|
| 1. Escaping in csv 2. Month names will be treated as string.
|
| This has got us thinking for more such scenarios. Thank you!
| stephenhuey wrote:
| Pretty cool how you offer this as a widget. 18 years ago I
| built a handy importer for CSV and tab-delimited files in a
| logistics company's traditional Java web app with no
| JavaScript/AJAX, and it enabled the users to map columns in a
| multi-page workflow. CSV was new to me and of course all the
| edge cases were surprising, but on top of that all the data
| consisted of shipping addresses, and I was amazed at how many
| different ways address fields would be presented! So it
| definitely felt like a double whammy of CSV inconsistencies
| and address inconsistencies!
| simlan wrote:
| Same in the sciences. Every instrument manufacturer invents
| their own esoteric format. Simple x,y,y2,y3 is the
| exception. Sometimes they call it tsv, csv but no commas or
| tabs were intended.
|
| Multiple datasets in the same csv but instead of adding
| extra columns put in some breaks in one long column. So you
| start scanning for empties, skip some metadata and start
| reading again... Sometimes it is wild.
| codeulike wrote:
| Great point, there are so many gotchas with CSV.
|
| Could the app handle this one? "First
| Name","Last Name","Address","Town","Postcode"
| David,O'Leary,"12 Acacia Avenue",London,NW5 3DF
| June,Robinson,"14, Abbey Court","Putney",SW6 4FG
| Greg,Hampton,"",, Stephen,James,"""Dunroamin"" 45
| Bridge Street",Bristol,BS2 6TG Daisy,Smith,"Flat 6,
| Porchester Court, 24 Dragon Road",Newcastle,NE3 4FG
|
| (i.e. some fields enclosed in speechmarks and some not, commas
| in text, blank values, escaped speechmarks, carriage returns)
|
| edit: I tried it with the csvbox.io demo - it seemed to deal
| with it ok!
| spreadr wrote:
| It will work. I have setup a demo for you for this data.
| Simply create a csv file and upload here - https://app.csvbox
| .io/upload/HdPOFwRMeU73O3hLMqM98lOB2ABllS?...
| stevoski wrote:
| Great idea.
|
| Charge more. Get rid of the $9/plan.
|
| Saying this as a SaaS operator who could use something like this,
| and who pays much more for other apps.
| unknown_error wrote:
| TLDR, IMHO: Get rid of the Small plan, make the Sandbox tier a
| much more generous free tier (1k/10k with logo), and charge
| more for the bigger plans. "First one is always free" kinda
| thing, so that devs can use your service for small projects
| (that don't cost you much) but then upsell it for broader
| enterprise-wide adoption on bigger projects.
|
| As a dev, I have the opposite opinion: The low-end pricing is a
| barrier to adoption. $9/mo isn't much, but it's the difference
| between "Oh cool, I can immediately use this in some of our
| side projects!" and "Oh, I have to request budget approval for
| this, explain what it is to superiors who might not be in the
| IT dept, answer questions about why we don't do just build it
| ourselves, answer questions about vendor support and lock-in,
| etc."
|
| It would cost more time for me to jump through all those hoops
| than to just use a standard CSV parsing library -- even though
| the workflow is vastly inferior -- because it's much less
| bureaucratic.
|
| Other services often provide a more generous free tier so that
| devs (the first ones to try something, often) can use it in a
| real-world project, fall in love with it, and then sell the
| paid version (which could be much more than $99) to their
| superiors only when the prototypes have proven valuable enough
| to make a solid sales pitch.
|
| I'm not a business consultant by any stretch, just letting you
| know that as a dev, I'd probably never bother using this
| because of the bureaucratic headaches involved with your
| pricing structure. Just my 2c.
| unknown_error wrote:
| Forgot to say: If the computing resources don't cost you that
| much, consider making higher business/enterprise plans where
| you sell an SLA and/or support contract and/or custom feature
| development/integrations, etc. rather than making it usage-
| based.
| spreadr wrote:
| All of this makes a lot of sense.
|
| We will surely have a relook at the pricing model.
| spreadr wrote:
| Thank you! I just got a very similar suggestion.
|
| What according to you should be the minimum pricing tier?
| stevoski wrote:
| A good place to start with pricing is tiers at $49/$99/$249.
|
| (As recommended in Jason Cohen's classic and awesome talk
| "Designing the Ideal Bootstrapped Business":
| https://youtu.be/otbnC2zE2rw)
|
| Customers at very low price points (such as $9/month) tend to
| churn quickly, and, counterintuitively, send more customer
| support interactions per customer than at higher points.
| You'll never earn enough from them to cover your customer
| acquisition costs.
|
| Getting rid of the cheap plan also removes the, shall we say,
| more difficult customer.
| spreadr wrote:
| Thank you so much for this!
| mushufasa wrote:
| Neat! Is there a FOSS or self-hosted version of this, for
| environments that have strict data compliance requirements?
| spreadr wrote:
| We do not yet have a self-hosted version. Its in the
| development pipeline for sure.
| robbiejs wrote:
| Looks like a nice tool! Does it also allow editing-in-place, like
| https://editcsvonline.com?
| spreadr wrote:
| Thanks. It does allow editing the cell data before file is
| submitted.
|
| The app validates the spreadsheet data against the data model
| rules configured by the developer and shows the errors to the
| users. The users can edit the data in the widget itself to
| resolve the validation errors before submitting the file. This
| ensures that the submitted data is clean and ready-to-use.
___________________________________________________________________
(page generated 2021-05-12 23:02 UTC)