Skip to content

Implement file import XML and log file output to CERNBox #19

@PascalEgn

Description

@PascalEgn

Description

Once MARCXML files and matching log files are generated, they need to be uploaded back to CERNBox. This issue covers implementing an upload step that takes the output directory (or temp directory from the XML generation step) and pushes all relevant files to a given CERNBox URL using credentials from environment variables (if needed).

Work involved

  • Implement an upload_to_cernbox(local_path: str, cernbox_url: str) function (reusing the same CERNBox access method established in the fetch issue)
  • Upload all generated XML files (per-Boite + combined) and the matching log file
  • Read credentials from environment variables (same pattern as the fetch step)
  • Log each uploaded file and raise a clear error on failure

Acceptance criteria

  • All XML files and log files produced by the pipeline are uploaded to the specified CERNBox URL
  • Credentials are read from environment variables
  • Failed uploads raise a clear error with the affected file name
  • End-to-end pipeline (fetch Boite files → match → generate XML → upload)

Screenshots(Optional)

Metadata

Metadata

Assignees

Labels

File Import ProjectThis task is related to the file import project of digitization

Type

No type
No fields configured for issues without a type.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions