A deep dive into Paperless-ngx reddit / selfhosted ---------------------------------------------------------------------- KillerTic · 2023-10-06 23:36 I am back already, with a new article I wrote about my experience with Paperless-ngx. I have been using Paperless for years and really enjoy it very much. I wanted to share with everyone how I have choosen to set it up (the article includes my docker compose and explenation of why it is done that way), as well as a review of my configuration of paperless (the tags I use, document types, ...). Also a general view of, why everyone should be going digital and start ditching their paper based solutions. The feedback on my last post was amazing. I originaly didn't want to post a new article (and on here) so quickly again, but I couldn't help myself. I really hope this article helps people out their. Might it be deciding to go digital, helping them to organise their paperless install or use my code to spin up their instance. [https://nerdyarticles.com/a-clutter-free-life-with-paperless- ngx/](https://nerdyarticles.com/a-clutter-free-life-with-paperless- ngx/) ---------------------------------------------------------------------- starbuck93 · 2023-10-07 00:35 Thanks for sharing. I'll be adding some of those tags and doc types to my instance next time I sit down to scan my papers piling up on my desk. ---------------------------------------------------------------------- KillerTic · 2023-10-07 00:39 Mine are always piling on the scanner 😂 ---------------------------------------------------------------------- spelwomendge · 2023-10-07 02:43 This got me to finally try it out! Thanks for the write up! ---------------------------------------------------------------------- matthewdavis · 2023-10-07 02:52 Great article. Getting a logical tagging system was my biggest hurdle to convert to paperless. For me I have the following * Health Record * Receipts * Health Record * Service Records * Tax Forms The I use tagging that further categorizes the document. Like * Name1 * Name2 * Name3 * year * taxform-number like 1099-INT And so on. Getting a high level categorization system was my brain block. I need to be better about putting more into it. Plus I bind mount the PDF directory that gets automatically backed up on my NAS. ---------------------------------------------------------------------- KillerTic · 2023-10-07 04:48 Happy to hear that. Exactly the effect I was looking for 😊 ---------------------------------------------------------------------- KillerTic · 2023-10-07 04:51 Interesting! Thanks for sharing. But why do you tag something unique like the tax form number? Would the search of Paperless be enough for this? Don’t get me wrong, just trying to understand! ---------------------------------------------------------------------- matthewdavis · 2023-10-07 05:24 A few reasons. * to help mentally recognize what forms are needed or not * Explicit categorization. Some of the Tax Forms I receive are multiple forms in a single PDF and have zero dollar entries for some areas. So while a search term will return true for that specific form ID, I don't need to "file" it away for that useless form * I may be a bit tag happy. But I keep categories very high level, but tags are kinda willy nilly. ---------------------------------------------------------------------- sowhatidoit · 2023-10-07 05:29 /u/KillerTic - What a timely article! Literally sat down to configure Paperless-NGX. Apart from the obvious how-to guide, you article provides great insight on how I might consider organization at a higher level. Thank you! ---------------------------------------------------------------------- KillerTic · 2023-10-07 05:30 Thanks for sharing! In the end of the day, it has to work for you! Personally I like to keep it very light, that way I am also likely to not forget about a tag and I use the search quite often. Anyway, Paperless is great 🫶🏼 ---------------------------------------------------------------------- KillerTic · 2023-10-07 05:33 I am glad it hit the spot of timely delivery for you! I was also struggling to find more out there with people sharing their configuration and reasoning behind it. One reason I really wanted to include that part. It also helped me refine and tidy up a bit. In the process of writing I actually deleted quite a few document types and a few tags 👍🏼 ---------------------------------------------------------------------- sowhatidoit · 2023-10-07 05:37 I love it when the writing process presents the opportunity to self reflect. Less is more! ---------------------------------------------------------------------- wall-e29 · 2023-10-07 06:19 Cool article - I still remember the struggles I had when setting this up ... took quite a while. I am running a Python script as cronjob to look into a Google drive folder every minute, download this file and move it into the consumer folder of paperless 😊 with that I can even when on the go just upload the file there and do not need to send it via (insecure) email (e.g. for paychecks etc) ---------------------------------------------------------------------- KillerTic · 2023-10-07 06:25 That is a good idea as well! I should move my watch folder to be one of my Owncloud folders! Endless possibilities 😊 ---------------------------------------------------------------------- Simplixt · 2023-10-07 06:27 Instant bookmark, thank you! Just started using paperless, so it's great to have some insipiration. I must admit, I'm using a more traditional approach in the moment, so working mainly with "folder paths", e.g. "Shopping/{created\_year}/{correspondent}/{document\_type}/{created\_y ear}-{created\_month}-{created\_day} - {document\_type} - {correspondent} - {title} I have a differnet **Folder-Paths** for: \- Shopping (e.g. Amazon, etc.) \- Finance (e.g. Banks, Crypto, etc) \- Job \- Household \- Health \- Real Estate - Object 1, Real Estate - Object 2, ... \- etc. **Correspondents** and **Document Types** I'm not restricting. So correspondent is always the sender (or receiver if I'm the sender). So even every Shop gets a new entry. Document types are flexible extended, so can be "contract", "cancelation", "confirmation of the cancellation" etc. The most general I'm using is "correspondence" if it's not a recurring document . The **title** is a summary of the content. For a bill just something like "HP Printer". The main goal is to get a folder (and file name) structure, that is working independently of paperless, so even if I stop using it at some point, that's not a problem at all. But I really love your idea of "Who is affected" and "Tax year" **tags** \- will copy this :D ---------------------------------------------------------------------- senectus · 2023-10-07 06:30 I've just bought a brother MFC-L3770CDW to use with paperless. I'm looking forward to using it, but am unsure if I can get it to connect directly with paperless or if I'll have to just dump it into an ftp share. (On my synology nas where paperless is hosted in a docker instance) ---------------------------------------------------------------------- KillerTic · 2023-10-07 06:31 I guess you could sense my dislike for folder structure. To one dimensional for me. All hail the tags and search😂 But I honestly mean it, what ever works for you, there are always reasons for a different approach. Happy I could inspire you with some aspects of my solution 👍🏼 ---------------------------------------------------------------------- KillerTic · 2023-10-07 06:33 Directly connecting it?! Interesting! If my scanner could do ftp I would opt for that. Keeping it simple, so I have less things I need to fix 😂 ---------------------------------------------------------------------- senectus · 2023-10-07 06:42 Yeah there is the possibly too insert via a websocket/api I believe ---------------------------------------------------------------------- TheNewAndy · 2023-10-07 06:44 You are in luck - I have recently done exactly this. An MFC-L3770CDW that scans directly to an SMB mount on an RS1221+. Paperless-NGX runs on the RS1221+ and watches this directory and it all works pretty fine. I have a few shortcuts set up on the printer for doing the scanning - this is the fiddliest part and it seems that you need to edit them using the web interface not using the touch screen. I have 4 shortcuts - "Single Sided Paperless", "Double Sided Paperless", "Single Sided No Paperless", "Double Sided No Paperless". The double sided ones I have configured to scan instantly, and the single sided ones let you adjust options (e.g. DPI) before it scans. There is a separate share on the NAS for non-paperless so if I want to scan something and not put it in paperless, then that is easy enough. Last night I set up an email address for it too, but I am yet to actually use this or even test it works. ---------------------------------------------------------------------- senectus · 2023-10-07 06:47 OMG fantastic this is exactly my dream. I may be poking your inbox ;-) ---------------------------------------------------------------------- KillerTic · 2023-10-07 07:03 Cool! Good luck, I hope it isn't a hustle to setup. ---------------------------------------------------------------------- Mean_Einstein · 2023-10-07 07:11 I added a dedicated syncthing instance to my stack as a consume shipper. So I just have a folder on every device called paperless- consume, whatever I drop in there gets pushed to paperless, I don't need to worry about being in the home network, starting a VPN or anything, it just works. Also I only scan from my phone which produces remarkably good results, no need for a pile of paper to work through. Basically I open the mail box, take some pictures and throw the paper away. ---------------------------------------------------------------------- KillerTic · 2023-10-07 07:16 Oh cool! I really love this community, everyone has really nice ways of solving issues. ---------------------------------------------------------------------- ElevenNotes · 2023-10-07 07:37 Now do the whole thing with podman, not docker. ---------------------------------------------------------------------- KillerTic · 2023-10-07 08:03 last time I looked into it, it felt like a too daunting task. Also when researching I didn't not have the fealing, I would easily be able to debug all the problems I am going to run into. On the other hand, I had a k3s cluster up and running for a short time. Maybe I should give it a go again ---------------------------------------------------------------------- ElevenNotes · 2023-10-07 08:22 The reason why you feel that way is because podman is rootless and the default paperless image does not work rootless. ---------------------------------------------------------------------- KillerTic · 2023-10-07 08:23 Oh I didn't even start doing it or started looking at specific images. Just the general overview on how I could start migrating my docker stack and keep the convienience I am used to. Just what you mentioned about paperless not beeing rootless, makes me fear a world of pain I am not ready for ---------------------------------------------------------------------- moontear · 2023-10-07 08:24 Oh nice! About document types I always have problems distinguishing a couple of things: Recurring Insurance statements: „your insurance is worth 10$“ or „we have paid your 10$ health bill“, „your new price for insurance is +10$“ Bank statements: „you have 10$ in your account“, „your received 10$ of interest last month“ What document types would you categorize these bad boys under? I currently switch between „Information“, „statement“ and „invoice“ but I am not sure. ---------------------------------------------------------------------- Fungled · 2023-10-07 08:38 I’d be interested to hear an honest comparison between Paperless and Mayan EDMS. I’ve been using Mayan for a long time, and more or less like it, but it’s very overpowered for my needs and is also easily my flakiest service. It would be tough to switch, but I might consider it if there were a simpler solution ---------------------------------------------------------------------- KillerTic · 2023-10-07 08:59 I consider those insurance statements "letters" As per the bank statements. I don't get any and would ask my bank to stop sending them, as they go straight into the shredder :D But to answer your question. I consider those "letters" as well. Maybe try to think about it from a different angel. What is the purpose of the document type. What benefit do you havem whenm you select statement and then see the documents assigned to it. I don't care what "offical" category a piece of paper belongs into, it is about me having filters and information at my finger tips. (hope that makes sense) ---------------------------------------------------------------------- KillerTic · 2023-10-07 09:05 The reason I did not do a comparision is, that it is \~4 years ago I looked at the alternatives, before going with paperless and I have NEVER regretted it. I did look at Mayan EDMS 4 years ago and came to the exact same conclusion you did. Far to overpowerd for my needs. It was in my point of view a large enterprise ready solution, which means for me it is too much, a lot of configuration, to bulky, ... I also looked at Teedy back then. I can't really remember I prefered paperless more. Could have been as simple as the design. Go give Paperless a try with like 20-50 documents to start with and see what you think afterwards. For me it really is a service I never have to touch (apart from putting documents in and searching for them) ---------------------------------------------------------------------- imjerry · 2023-10-07 09:10 That's great 👍 I will try it soon, I have been meaning to for a while. ---------------------------------------------------------------------- Bruceflix · 2023-10-07 09:49 Paperless-ngx is one of my favourite apps. It just works. Does what it does and does it well. ---------------------------------------------------------------------- Manauer · 2023-10-07 10:01 I tried it two weeks ago, but it just wont accept some of my pdfs. As soon as it can not do OCR for whatever reason it also denies uploading it. As long as this is the behaviour, i have to rely on paper unfortunately. ---------------------------------------------------------------------- Fungled · 2023-10-07 10:01 It is indeed pretty weighty! At least the latest 4.5 version appears to be sorting out some long standing issues (finally) Tempting to try out paperless, but I’ve got >1500 docs in Mayan, so that would be a pain! ---------------------------------------------------------------------- KillerTic · 2023-10-07 10:10 Give it a small test spin and decide if it is worth it. Another user also just commented, that he loves paperless, as it is soooo hustle free. I really can not remember having issues with it. Even did a postgres db upgrade from 13-16 yesterday and it just went up afterwards! ---------------------------------------------------------------------- KillerTic · 2023-10-07 10:12 Hmm… sorry to hear that. I never had any problems. Did you check their documentation, if there are any settings to let you upload even when OCR fails? ---------------------------------------------------------------------- Fungled · 2023-10-07 10:32 I have indeed fired it up. It does look very nice! I’ll have to have a play and consider if it’s worth doing a migration. Perhaps someone has made a migration tool?… otherwise I’ll have to fire up some api-to-api solution. That’ll be fun 🤯 ---------------------------------------------------------------------- katrinatransfem · 2023-10-07 10:34 The answer is to think about when you are going to want to refer back to them, and in that situation, how are you going to find them, without also finding too much other stuff that isn't relevant. I personally would classify the insurance stuff as "insurance", and separate categories within insurance for home, car and travel; and the bank stuff as "bank". But maybe you need to classify the interest documents separately because you need to refer back to them when completing a tax return? ---------------------------------------------------------------------- Manauer · 2023-10-07 10:57 I think so, but did not found anything. This is the link to the GitHub Issue from someone else who has the same problem: https://github.com/paperless-ngx/paperless- ngx/discussions/4145 ---------------------------------------------------------------------- moontear · 2023-10-07 10:59 Oh I don’t „get“ them either, those bank statement. I simply download them from their website as a backup. I have had inquiries for statements as far back as 6 years and not all banks keep data that long (or at least not easily accessible via their web ui) ---------------------------------------------------------------------- KillerTic · 2023-10-07 11:10 They do have to keep them that long, so I keep the problem with them, if I ever need them :D But I also never had a request for my bank statements to be honest. ---------------------------------------------------------------------- KillerTic · 2023-10-07 11:14 I wish you the best of luck! ---------------------------------------------------------------------- KillerTic · 2023-10-07 11:16 What language have you set for PAPERLESS\_OCR\_LANGUAGE? I see the person with the issue on GitHub only has one language set. It is a complete shot in the dark tbh... ---------------------------------------------------------------------- Fungled · 2023-10-07 11:18 Thanks for the inspiration! ---------------------------------------------------------------------- Manauer · 2023-10-07 11:50 its not it. i experimented with that variable. its some incompatibility with some proprietary pdf standards i guess. All the pdfs that i get from my electricity provider are incompatible with paperless-ngx ---------------------------------------------------------------------- Fungled · 2023-10-07 12:01 Quick question: any opinions on using Mariadb vs Postgres? I can use either, but I’m more comfortable with Mariadb. I see there are a couple of minor caveats with Mariadb, but looks like no big deal to me. But i don’t want to get into a position where I have to switch later ---------------------------------------------------------------------- KillerTic · 2023-10-07 12:05 I honestly can not tell you what is better. I prefer PostgreSQL, for one very simple reason: The docker compose environment variables let me set the user straight to what I want, I do not need a root and a user password. It’s a absolutely stupid reason I know 😂 If you don’t mind, I would go with their default, if you are more happy with MariaDB, then go for it. ---------------------------------------------------------------------- Fungled · 2023-10-07 12:07 Cool thanks. They at least appear to be equally supported. Mayan is supposed to support both, but the MySQL support had quite a few hidden issues ---------------------------------------------------------------------- KillerTic · 2023-10-07 12:10 I just read the caveat. Oh man what a deal breaker… I can not have a tag family & FAMILY?! 🤣 ---------------------------------------------------------------------- Fungled · 2023-10-07 12:14 Indeed! Case insensitive is definitely fine with me ---------------------------------------------------------------------- agent_kater · 2023-10-07 15:11 You should probably mention how to make backups. (Since Paperless uses Postgres you can't simply do a file backup, but Paperless has a builtin export specifically for backups.) ---------------------------------------------------------------------- KillerTic · 2023-10-07 15:25 You got a point. As I copy my whole docker dir (expect the NAS files, which includes the PDFs) twice daily to an offsite backup), I never worry about backups. The NAS files get backuped up every night. I did restore Paperless before and everything was fine. I am aware to just copy the docker database files, while it is running has some risk involved, but I keep those the last 2 hourly, 6 daily, 3 weekly and 1 monthly. But let me have a quick look into the paperless backup solution and add it to the article. Thanks for pointing that out! ---------------------------------------------------------------------- BleepsSweepsNCreeps · 2023-10-07 15:32 I've been looking at getting a Paperless instance set up but I have one fairly significant detail to overcome. I have my scanner at home but currently my homelab setup is at my MIL's house. My printer supports Scan to Computer, Scan to Email, Scan to SharePoint, and Scan to Network Folder. I don't have SharePoint and the Network Folder only supports the physical network so no FTP or anything like that. I need to figure out how to get it to scan to my remote server share. I know now that Paperless will scan an email inbox so Scan to Email could be an option but I don't necessarily want to add more stuff to my inbox. However, having them in an inbox could also serve as an extra backup solution. Another option I was considering was connecting a RPi via USB and either running something like Syncthing or setting a WireGuard client configuration on it then just set up an SMB share or something. Has anyone else run into this situation where your printer and homelab are in two different locations and what have you found to be the best solution? Thanks! ---------------------------------------------------------------------- KillerTic · 2023-10-07 15:47 Are you concerned about your documents being added to a mail box out of privacy reasons? If not, I would probably just create a new mailbox for this purpose. Send all documents I want to import to that inbox and configure the mailbox like I did (check the screenshot in my article). This will delete the mail, after the document was imported. Another user also wrote about using syncthing on all his devices, which then will push the files to the paperless consume folder. Also a need solution. ---------------------------------------------------------------------- BleepsSweepsNCreeps · 2023-10-07 15:53 Not so much a privacy concern. I just didn't particularly want to clutter up my inbox with more stuff. I hadn't thought about setting up a separate email for it but even that I'm kind of on the fence about as I don't really want to have another email with another set of credentials to keep track of. I'm not saying I wouldn't go the email route but I was hoping to see all the options people have used and what seems to work best. If email was the most efficient solution, I'd end up using that despite not being my first choice right off the bat. Thanks for the input! ---------------------------------------------------------------------- KillerTic · 2023-10-07 15:56 No worries. Maybe there will be more ideas! You could also use your mailbox, have a rule, which moves all your scanner documents to one particular folder and point paperless at that folder. It also is capable of doing that. Then leave them there or delete after import. But let's see what other ideas there are. In regards of not having more credentials. Setup Vaultwarden, if you haven't yet and then forget about these credentials, as once you have your scanner and paperless working, you will never see that inbox again :D ---------------------------------------------------------------------- KillerTic · 2023-10-07 16:23 Thanks man. Learned something new, implemented the backup and updated the article! Do you know how to stop paperless inside docker, without stopping the container?! Supposetly we should be stopping paperless before the backup, but I could not find anything on how to do that, as I need the container up to run the backup. ---------------------------------------------------------------------- Aluhut · 2023-10-07 17:09 > Regardless of how the documents are imported into Paperless, the ToDo tag is set. How did you manage to do that? ---------------------------------------------------------------------- agent_kater · 2023-10-07 18:23 Are you using the [document exporter](https://docs.paperless- ngx.com/administration/#exporter)? Then you don't need to (actually I think you must not) stop Paperless while you do the export. If you want to do a file backup while Postgres is running, you can, but(!) you have to take an atomic file system snapshot, for example with LVM. Simply rsyncing the file system including the database will often appear to work when the database is idle during the copying, but you're really risking your data. If you must go down this route, make sure you have generational backups, so you can use an older one when the most recent one is broken. Note that you won't notice the brokenness until you read the whole database, so do a pgdump after a restore to check the database. Or just do it properly. ---------------------------------------------------------------------- KillerTic · 2023-10-07 18:30 Oh man. Good spot, I will update the article in a minute! When you create your 'ToDo' tag (or what every you want to call it, there is a setting to make it your inbox tag. That is all you need. There also is no need to set the tag in your mail rule. I for the life of me could not figure out, how it worked and spun up a new instance to see what I am missing. Man I didn't even know how to get into a fresh install. I also need to update that part :D Thanks, helped me to make the guide more complete! ---------------------------------------------------------------------- Aluhut · 2023-10-07 18:33 Aaah danke dir. I missed that because I've set them up along the process going from file to file. Other than that, the post is good. I just used it to set it up :D ---------------------------------------------------------------------- KillerTic · 2023-10-07 18:39 Well the documentation is not that clear on the document\_exportert, but yeah just set it up. Will be fine. I use restic for file based backups and I do keep enough versions (2x hourly, 6x daily, 3x weekly, 1x monthy) + pretty much the same for my VM backups, only that those only run once a night and not twice a day like restic. ---------------------------------------------------------------------- KillerTic · 2023-10-07 18:44 I have now updated it, with also adding way you can actually log in... :D Thanks! Happy to hear you like it. I am sooo amazed how this and the last article are blowing up. Would never have imagined anything close ---------------------------------------------------------------------- stevie-tv · 2023-10-07 19:03 would love to hear what scanner you use and if you'd recommend it. ---------------------------------------------------------------------- KillerTic · 2023-10-07 19:17 I use a HP ENVY Pro 6400 Would I recommend it... For normal houshold use, sure. Do I like it. Not really... I don't like to HP Smart software (the mobile app is all right) I am forced to use and I should have really selected a scanner, that can write to a network share/FTP/SFTP or just send an email to a programed adress. Bonus points for being able to scan double sided documents. Let's hear what people use and are actually happy with for the purpose of digitalising their life. ---------------------------------------------------------------------- stevie-tv · 2023-10-07 19:20 Thanks! I was considering the Brother ADS-1200, but before I make the plunge was hoping to hear others inputs ---------------------------------------------------------------------- KillerTic · 2023-10-07 19:25 It says it can only scan-to-pc. I would really love to be able to throw documents on it, press a button (or two) and then check paperless, when I feel like it and ensure the documents are preset correctly. ---------------------------------------------------------------------- stevie-tv · 2023-10-07 19:28 oh shoot you're right. I confused it with the ADS-1200W! ---------------------------------------------------------------------- KillerTic · 2023-10-07 19:46 hmm... brother needs to work or their website, as they list none of those capabilities... ---------------------------------------------------------------------- xX__M_E_K__Xx · 2023-10-07 19:48 I was using this script to make an export : source : https://skerritt.blog/how-i-store-physical-documents/ script to export paperless-ngx ```bash # export to zip file docker exec paperless_ngx-webserver-1 document_exporter /usr/src/paperless/export --zip ``` But your script is way way way nicer. I'm switching to yours. Thanks! ---------------------------------------------------------------------- deano_southafrican · 2023-10-07 22:13 This was a great article and I've stolen your backup idea as I've been fairly lazy. I migrated from another server the really bad and lazy way but now that I've found a permanent home for it and am actually using it I've been needing to setup backups. Great article and I've added you to my listy of regular reads! ---------------------------------------------------------------------- KillerTic · 2023-10-08 03:21 Happy to hear that you liked it and that you could copy the backup from the article. The exact reason I started it all! ---------------------------------------------------------------------- pheellprice · 2023-10-08 06:41 If it’s an hp printer it’s possible to run a node docker instance that drops in a consumer folder. https://hub.docker.com/r/manuc66/node-hp-scan-to ---------------------------------------------------------------------- pheellprice · 2023-10-08 06:44 Could you open them and print to pdf? Or use Stirling-pdf to convert? ---------------------------------------------------------------------- KillerTic · 2023-10-08 07:12 nice project! Unfortunately my printer does not support scan-to-pc... :( ---------------------------------------------------------------------- Manauer · 2023-10-08 10:48 That is indeed a workaround i have not thought of. Thank you, I will try that. Paperless still should allow uploads without successful ocr. ---------------------------------------------------------------------- reddit_lanre · 2023-10-08 12:30 Nice write-up. I've already got it set up, but it was useful to run through your article to confirm some of my set-up decisions (where they aligned). Thanks for the post! ---------------------------------------------------------------------- agent_kater · 2023-10-08 16:11 Yeah, that's the setup I'm using too. Use the document exporter into an intermediate directory, then use restic to sync it off-site. The document exporter can keep the directory updated by the way, no need to delete in-between, reduces wear on the drive. Pretty important on a Raspberry Pi for example. Another tip... if you have services that use SQLite as database, you can call `flock /path/to/sqlite.file restic ...` to keep the SQLite database locked for the duration of the backup. Otherwise you have the exact same problem as with Postgres, ---------------------------------------------------------------------- KillerTic · 2023-10-08 16:22 What do you mean by not having to delete it? Do you mean the ‚-d‘ part of the backup command? Thanks for the tip! But that would mean I have to do that for every sqlite file. I am too lazy fort that 🤣 I feel pretty save with the VM backups, twice daily via restic and keeping quite a few iterations. Never had any problems with a corruption and even if I have one, the chances are near 0 that all are corrupt. ---------------------------------------------------------------------- ohuf · 2023-10-08 18:15 Hi Henning, Thanks for sharing in such a detailed way. Lots of food for thought, indeed. I'm looking forward to your next blog postings. BTW: how do you manage your Docker infrastructure? Just by hand using files, or do you use a GUI á la podman or portainer? ---------------------------------------------------------------------- KillerTic · 2023-10-08 18:24 Thanks! I am amazed at all the feedback! I always have everything in one docker compose. So it is just a matter of docker compose pull, docker compose up -d and docker system prune -af —volumes Furthermore I have an Ansible playbook, which updates my whole infrastructure including the docker deployments. I will for sure write about it soon! ---------------------------------------------------------------------- ohuf · 2023-10-08 18:27 Thanks!! ---------------------------------------------------------------------- KillerTic · 2023-10-08 18:28 Thanks for the sub by the way! ---------------------------------------------------------------------- agent_kater · 2023-10-08 20:26 > What do you mean by not having to delete it? Do you mean the ‚-d‘ part of the backup command? Yes and no. I just meant that you can run the exporter again with the same destination directory and it will update it. One might assume it had to be run against an empty directory. And you are correct, in that case you should run with `-d` so that you don't end up with a backup that is cluttered with old files. > Thanks for the tip! But that would mean I have to do that for every sqlite file. I am too lazy fort that 🤣 I feel pretty save with the VM backups, twice daily via restic and keeping quite a few iterations. Never had any problems with a corruption and even if I have one, the chances are near 0 that all are corrupt. Uhm, are you saying you do have SQLite databases and you just back them up by copying them while they are in use? Sorry, but that's just reckless. Or do you stop the services during the backup? That's ok of course, if you can live with the downtime. You can chain multiple flock commands: `flock /first_database.sqlite flock /second_database.sqlite restic ...` ---------------------------------------------------------------------- KillerTic · 2023-10-08 21:27 Yeah, just set ‚-d‘ to make sure the directory stays tidy. Yes I like to live on the edge with my sqlite DBs :D Stringing the command is not a problem, but I have to adjust my backup script every time a sqlite DBs joins my stack. I also always try to use a proper databases. But as I said I keep quite a few of backup versions and so far never had any problems. But it is a good remark, I will make sure to include it in the article about backing up, once I come around to writing it. Thank you! ---------------------------------------------------------------------- agent_kater · 2023-10-09 04:13 > but I have to adjust my backup script every time a sqlite DBs joins my stack Sounds like you really need LVM (or Btrfs or ZFS) snapshots. ---------------------------------------------------------------------- fredflintstone88 · 2023-10-10 02:55 Is there an iOS app? ---------------------------------------------------------------------- KillerTic · 2023-10-10 05:59 There are a few different apps, but I can tell you anything about them. I habe never used paperless with an app ---------------------------------------------------------------------- fredflintstone88 · 2023-10-10 10:43 Okay, so how are you typically scanning physical documents? Just use the web browser on phone? ---------------------------------------------------------------------- KillerTic · 2023-10-10 10:56 A normal scanner/printer. Very old school ---------------------------------------------------------------------- Fungled · 2023-10-10 14:33 Reporting back on this, since I'm interested in migrating away from Mayan EDMS. I fired up the image at the weekend, and liked what I saw. So decided to give it more of a spin. Here's some info about what I did for the interested: * Imported my Mayan docs more-or-less directly from its storage by copying into the consumer directory * Did a bunch of API processing to transfer over: * Original added/created dates from Mayan (very important!) * Mayan cabinets as new Paperless tags * Converted some of those tags to Paperless correspondents * Other cleaning up So far, great! I'm really liking: * My preferred import method is working great: "Inbox" folder in Nextcloud that's mounted as the Paperless consumer directory. Anything dragged/uploaded there is imported no probs (currently broken in Mayan...) * Checksum-based detection of duplicate documents! This has been broken in Mayan for a LONG time * Automatic tagging, particularly of correspondents. Even already it appears to be working great * Same-page search results ---------------------------------------------------------------------- KillerTic · 2023-10-10 15:06 Great to hear and happy for you, that it seems to be a good fit ---------------------------------------------------------------------- dnt_pnc · 2023-10-14 15:37 Great write up! Thank you. I set up paperless-ngx this week as it seemed pretty easy using that docker script in the paperless git. ---------------------------------------------------------------------- KillerTic · 2023-10-14 15:38 Cool! Have fun digitalising. Personally not a fan of scripts, as I like to habe full control, but that is me ---------------------------------------------------------------------- Sacmanxman4 · 2023-10-25 15:32 I tried paperless a while back but got hung up on the first step: Getting paper into it. I even bought one of the Brother scanners to make it as easy as possible, but the problem was that some of the paper I wanted to scan was weird shapes or orientations. If the paper was scanned sideways it wouldn't work. How fast and easy are the phone apps to use? Taking photos of stuff like that can be tedious. I want as easy as possible of a solution. ---------------------------------------------------------------------- KillerTic · 2023-10-25 16:35 Sounds more like a scanner problem to get the orientation right. There surely are apps out there, which can help you in those occasions. I never had to use them though. ---------------------------------------------------------------------- kru89 · 2023-10-26 20:14 Thank you for the great write up. I have paperless installed on nas container. I scanned and uploaded couple of pdfs (eg passport )and it couldn’t OCR. Can you please recommend any tips or ticks for that ? ---------------------------------------------------------------------- KillerTic · 2023-10-26 20:23 I can’t say I have I can’t say I have had any problems with OCR before. I also haven’t tried importing a passport. It does take some time though, if you don’t habe much processing power. Did you give it some time to process? Did you also set the right languages? How are other documents like invoices? Does the log throw any errors? ---------------------------------------------------------------------- 007craft · 2023-11-11 05:49 I've been using Paperless for a few years now, but I found a workaround for its destructive nature. Before Paperless, I had my files all organized in nested folders like this: Documents -----Manuals -----Waranty Information -----Tax stuff *____------ Tax Year 2022* ____------ Tax Year 2023 -----Doctors notes -----etc etc I liked the idea of paperless, but didnt want to destroy my original folder structure or original files (This one is the most important). Unfortunately when Paperless Auto imports from its consume folder, it does just that, and destroys your files as it uses its own database. It also means you're now reliant on paperless to view your files. The solution for me was to setup an instance of Syncthing (But you can use any file syncing software really). Now I have me nice organized documents folder, and as soon as I add a new document, Syncthing will do a one way, read only "sync" to the document and copy it over to a folder elsewhere I've called "Paperless Consumed Documents". Then I point Paperless to consume documents in this folder. That way new documents are scanned in, and only the copies are destroyed by paperless, not the originals. I really wish Paperless was updated to simply not destroy original documents in it's consume folder but from what I gathered from other people complaining about this, is that the developers WANT that behaviour to happen, so I doubt it will ever change. This solution does however let those of use use the software who want a non destructive document management system. Now I can use Paperless if I ever need to search for a document based off internal text or document title, ect. But my organization (Which will always trump and AIs), remains ---------------------------------------------------------------------- hawkinsst7 · 2023-12-17 05:59 Paperless does support Storage Paths https://docs.paperless-ngx.com/advanced_usage/#storage-paths ---------------------------------------------------------------------- Bill_Guarnere · 2023-12-20 22:32 Thank you for your feedback. I just started to use Paperless-ngx on a RPi4, I have a question. Maybe I'm using it in the wrong way, but I find quite tricky to define a storage path for my documents. Let's say I want to define my storage path in this way ├── car │   ├── ford │   │   ├── insurance │   │   └── maintenance │   └── renault │   ├── insurance │   └── maintenance └── motorcycle └── tenere700 ├── insurance ├── maintenance └── misc For each one of these levels I have to create a new storage path (car/ford/insurance, car/ford/maintenance, car/renault/insurance, etc etc...) and Paperless-ngx shows them as a list instead of a tree, which makes a mess if you have a lot of directories and subdirectories. Is there any way to visually create and manage storage paths and be able to navigate throught storage paths just like a filesystem manager? ---------------------------------------------------------------------- KillerTic · 2023-12-20 22:41 Hey. I don’t use storage path, as they came along a long time after I had everything setup. I don’t really get them, but haven’t spent a lot of time with them. So better for someone else to dive deeper on them. But… I would challenge, why you are making it hard in yourself. In the beginning of the article I talk about tags vs. folders. Keep the folder as simple as possible and use the tags for your categorisation needs. Try to free yourself from those limiting folder structures 😂 I habe my filename set to: PAPERLESS_FILENAME_FORMAT: "{created_year}/{correspondent}/{created} {title}" Resulting in folders for year in which I habe the corespondent as a folder. But I never use the folders. I use the interface and the tags. Obviously here is no right or wrong!!! ---------------------------------------------------------------------- Bill_Guarnere · 2023-12-21 22:43 Absolutely, I understand your way of using tags but for me folders are much more simple and flexible. I should create tons of tags, one for each subdirectory and every time I have to find something I should filter by so many tags, it would be a mess. Ok (almost) every document is searchable thanks to OCR, but imho the search engine is not the answer, it should be the last thing to use. I always thought that If you have to use a search engine it means that something is wrong with the application usability or in the way documents are archived. Using tags means also that I should create tags also for years and then months, if not days. Using folders means that under a folder (for example /cars/ford/insurance) I can create how many folders I want with a date prefix (for example /cars/ford/insurance/20231221-insurance-company1, /cars/ford/insurance/20231221-insurance-company2, etc etc...). In this way a simple browse of the folder ordered by name means also it's ordered by date (file modification date is not the most reliable data to find when the document was created, a simple copy without preserving file metadata will change it) ---------------------------------------------------------------------- KillerTic · 2023-12-21 23:16 I do disagree with you on a couple of things, but hey… the beauty is, that the system is flexible 👍🏼 Hope you find a good and easy way for how you want to implement it! Habe a nice christmas time