Paperless-ngx Pre-Consume Script
reddit / selfhosted

----------------------------------------------------------------------
AwarePerformance6812 · 2023-10-16 16:28

Hey there,

I've got paperless-ngx running in a docker container on an ODROIDHC4
running OpenMediaVault. I have some PDF files (bills) that, when
uploaded, would fail with error "File type application/octet-stream
not supported". I did a bit of digging and found that if I ran `qpdf
filename --replace-input` the file would then successfully upload to
paperless. So my solution was going to be to run that as a pre-consume
script, so that any improperly formatted PDFs would be corrected
before being consumed. So I wrote a short bash script following the
[advanced topics](https://docs.paperless-ngx.com/advanced_usage/#pre-
consume-script) page of the paperless docs

    #!/usr/bin/env bash
    qpdf ${DOCUMENT_WORKING_PATH} --replace-input

I stored this in my paperless user's home directory in a scripts
folder and mapped that to a scripts folder in the docker container.

        volumes:
          - data:/usr/src/paperless/data
          - media:/usr/src/paperless/media
          - ./export:/usr/src/paperless/export
          - /srv/dev-disk-by-uuid-ba97f10c-0a74-4e97-a563-
e235115c012b/home/paperless/consume:/usr/src/paperless/consume
          - /srv/dev-disk-by-uuid-ba97f10c-0a74-4e97-a563-
e235115c012b/home/paperless/scripts:/usr/local/bin/scripts
    ...
        environment:
            PAPERLESS_PRE_CONSUME_SCRIPT:
${PAPERLESS_PRE_CONSUME_SCRIPT}

    .env
PAPERLESS_PRE_CONSUME_SCRIPT="/usr/local/bin/scripts/paperless-pre-
consume-script.sh"

 If I `qpdf` one of the input PDF files locally and upload it through
the web interface I receive the following in the docker-compose log
for the paperless-webserver:

    paperless-webserver-1  | [2023-10-16 12:07:41,653] [INFO]
[celery.worker.strategy] Task
documents.tasks.consume_file[bd91de0b-cb8f-4035-8f1b-ed903e835193]
received
    paperless-webserver-1  | [2023-10-16 12:07:42,010] [INFO]
[paperless.consumer] Consuming 2023-10-15_good-file-test2.pdf
    paperless-webserver-1  | [2023-10-16 12:07:42,046] [INFO]
[paperless.consumer] Executing pre-consume script
/usr/local/bin/scripts/paperless-pre-consume-script.sh
    paperless-webserver-1  | [2023-10-16 12:07:42,168] [INFO]
[paperless.consumer] /usr/local/bin/scripts/paperless-pre-consume-
script.sh exited 0
    paperless-webserver-1  | [2023-10-16 12:07:42,170] [INFO]
[paperless.consumer] Script stdout:
    paperless-webserver-1  | [2023-10-16 12:07:42,172] [INFO]
[paperless.consumer] /tmp/paperless/paperless-
ngx4wmukdan/2023-10-15_good-file-test2.pdf
    paperless-webserver-1  | [2023-10-16 12:07:42,174] [WARNING]
[paperless.consumer] Script stderr:
    paperless-webserver-1  | [2023-10-16 12:07:42,175] [WARNING]
[paperless.consumer] tee: /usr/src/paperless/scripts/pre-consume-
example.log: No such file or directory
    paperless-webserver-1  | [2023-10-16 12:07:46,114] [INFO]
[ocrmypdf._sync] Start processing 2 pages concurrently
    paperless-webserver-1  | [2023-10-16 12:07:46,167] [INFO]
[ocrmypdf._pipeline] skipping all processing on this page
    paperless-webserver-1  | [2023-10-16 12:07:46,169] [INFO]
[ocrmypdf._pipeline] skipping all processing on this page
    paperless-webserver-1  | [2023-10-16 12:07:46,206] [INFO]
[ocrmypdf._sync] Postprocessing...
    paperless-webserver-1  | [2023-10-16 12:07:47,466] [INFO]
[ocrmypdf._pipeline] Image optimization ratio: 1.00 savings: 0.2%
    paperless-webserver-1  | [2023-10-16 12:07:47,466] [INFO]
[ocrmypdf._pipeline] Total file size ratio: 8.83 savings: 88.7%
    paperless-webserver-1  | [2023-10-16 12:07:47,485] [INFO]
[ocrmypdf._sync] Output file is a PDF/A-2B (as expected)

However, when I upload the file prior to qpdf, I receive the following
output in the docker-compose log:

    paperless-webserver-1  | [2023-10-16 11:57:58,776] [WARNING]
[django.request] Bad Request: /api/documents/post_document/

And the "File type application/octet-stream not supported" in the
paperless web GUI. The output from the good file seems to suggest the
consumer becomes aware of the file and then runs the pre-consumption
script. Is something happening between those two activities that is
causing it to error out? I appreciate the help!

----------------------------------------------------------------------
thekrautboy · 2023-10-16 16:45

https://github.com/paperless-ngx/paperless-ngx/discussions

https://github.com/paperless-ngx/paperless-ngx/issues

https://matrix.to/#/#paperless:adnidor.de

----------------------------------------------------------------------
AwarePerformance6812 · 2023-10-16 16:56

Thanks! I had the mistaken impression from searching through the sub
that questions on specific applications were permissible.