Storage backend how-tos
As described here, CsvPath Framework stores files in three protected spaces:
Source file staging (a.k.a. the named-files area)
The csvpaths area (a.k.a. the named-paths area)
The archive
Each of these areas can live in any of the five supported backends:
File system
CsvPath Framework generates events during staging named-files, loading named-paths, and running named-paths groups against named-files to populate the archive. Events are stored in the file backends as manifest.json files. In addition to your main file backend options, you can also choose to store events in a SQL database and/or send them to an observability system that supports OpenTelemetry or OpenLineage.
Last updated