Skip to main content

Treat an Equipment Document Archive as an Untrusted Processing Job

Updated 4 min readMarkdown

An equipment enquiry accepts one ZIP containing drawings, a quotation and manuals. The upload fits the HTTP size limit, so the application sends it straight to an extraction worker. That limit says little about the work the archive can cause after decompression.

For a procurement portal, the trust boundary is the processing job, not merely the upload request.

This is a defensive design for a hypothetical portal. File names, budgets and code are illustrative; this is not a description of Zenith Eclipse software or a reported incident.

Define what the business actually needs

Zenith Eclipse's machinery and equipment sourcing scope asks for specifications, drawings or photographs and identifies support requirements as part of the offer. A document workflow should preserve which enquiry and equipment revision each submission supports.

That need does not imply accepting every format or automatically unpacking every container. Where individual files meet the workflow, avoid archives altogether. If bundles are necessary, document supported member formats and the treatment of nested, encrypted or unsupported entries.

OWASP's file-upload guidance recommends layered controls including authorization, generated filenames, type validation, isolated storage and size limits. No single extension or signature check establishes safety.

Use quarantine before review

Store the original upload under an application-generated identifier in private quarantine. Tie it to an authorized enquiry and tenant. The uploader's filename is display metadata, never an output path.

Record a state such as received, inspecting, rejected or ready for authorized review. Do not mark the whole submission accepted because one member extracted successfully. Keep the processing outcome separate from technical acceptance of the equipment.

Use a low-privilege worker with bounded memory, disk, CPU and elapsed runtime. It should not inherit production credentials or unrestricted network access. Concurrency and per-tenant quotas matter too: individually bounded jobs can still overwhelm a shared queue.

Budget expanded bytes, not only uploaded bytes

Set limits on compressed upload size, member count, individual expanded size and aggregate expanded size. Inspect metadata early, but enforce limits while reading as well.

This helper illustrates only the streaming byte budget. The source is an already-open member stream; the sink is a private staging object whose key was generated by the application.

def copy_bounded(source, sink, member_limit, budget):
    used = 0
    while True:
        room = min(member_limit - used, budget["remaining"])
        block = source.read(min(65536, room + 1))
        if not block:
            return used
        if len(block) > room:
            raise ValueError("Expanded-byte budget exceeded")
        written = sink.write(block)
        if written != len(block):
            raise OSError("Incomplete staging write")
        used += len(block)
        budget["remaining"] -= len(block)

The caller supplies nonnegative integer limits and one shared budget for the entire job. It must abandon the job on an exception and remove all partial staging objects. The extra byte distinguishes end-of-stream at the limit from an over-limit member without writing the excess.

This helper does not bound CPU spent inside the decompressor or memory spent parsing archive metadata. Those limits belong to the worker environment and parser policy. Nor does it implement scanning, content validation or authorization.

Do not extract into supplier-controlled paths

Python's zipfile documentation describes member inspection and stream access. Prefer mapping accepted regular-file entries to generated storage keys over recreating an archive's directory tree.

Reject unsupported entry types under the declared policy. Keep an explicit policy for duplicate names and normalization collisions: two entries named “specification.pdf” must not silently overwrite one another. A manifest can assign each accepted member its own identifier and retain the original name for display.

Do not recursively unpack nested archives by default. If nested processing is a real requirement, apply a depth cap and the same aggregate budget across the whole traversal, not a new allowance for every level.

Separate safe processing from trustworthy evidence

After inspection, scanning and any approved format transformation, retain the relationship between the original upload and review derivatives. A checksum identifies bytes; it does not prove the supplier authored them or that a capacity claim is true.

Serve approved documents through authorized retrieval rather than exposing quarantine paths. Treat preview rendering as another parser boundary with its own controls.

Test small fixtures for duplicate names, unsupported types, a member one byte over budget, aggregate exhaustion and a write failure. Confirm that rejection leaves no partially approved bundle. Security testing should also exercise worker termination and cleanup.

The objective is a controlled handoff: a supplier submission becomes a bounded processing job, then a reviewable document set. It never becomes trusted equipment evidence merely because decompression completed.


About the publisher: Zenith Eclipse Co works across agricultural commodities, international logistics and machinery sourcing.

No comments yet. Be the first to comment.