Storage backend how-tos

As described here, CsvPath Framework stores files in three protected spaces:

• Source file staging (a.k.a. the named-files area)

• The csvpaths area (a.k.a. the named-paths area)

• The archive

Each of these areas can live in any of the five supported backends:

• File system

• SFTP

• AWS S3

• Azure

• Google Cloud Storage

CsvPath Framework generates events during staging named-files, loading named-paths, and running named-paths groups against named-files to populate the archive. Events are stored in the file backends as manifest.json files. In addition to your main file backend options, you can also choose to store events in a SQL database and/or send them to an observability system that supports OpenTelemetry or OpenLineage.