CSV, Excel and JSONL Ingestion

Automate CSV, Excel, and JSONL/NDJSON

End Manual Data Validation

CsvPath Framework data preboarding registers, versions, validates, upgrades, and stages CSV, Excel, JSONL files and data frames from data partners before they break your pipelines.

CsvPath Framework is the open source way to shift-left data quality. Enforce data contracts as data enters the organization with less manual effort, fewer ingestion failures, and more agile development using consistent data preboarding tools you can try in minutes.

Get FlightPath Data, the cross-platform app frontend

FlightPath Data is a powerful new frontend to CsvPath Framework. Go beyond CsvPath Framework's built-in CLI. Get up and running faster with a purpose-built preboarding development and operations console. FlightPath Data gives you all the help and examples you need move quickly.

FlightPath Data is bundled with FlightPath Server, the no-code/low-code automation REST API connecting your existing infrastructure to data preboarding.

Available as a free download from the Microsoft Store and the Apple MacOS Store.

The Architecture For Efficient Data File Feed Ingestion

CsvPath Framework implements the Collect, Store, Validate Publish architectural pattern. Ingestion goes faster, is more cost-efficient, and more effective with a preboarding stage.

CsvPath Framework was built to fill the blindspot between MFT (managed file transfer) and the data lake with a simple path to provably correct data.

This data preboarding blindspot is a big deal. Think about it. If even 1 in 30 companies depends heavily on CSV or Excel data, the lack of delimited file preboarding is a trillion-dollar problem.

A data flow diagram showing how CSV, Excel and other tabular data come into the organization through a preboarding process that acts as a Trusted Publisher to the data lake and applications.

Why roll your own preboarding? CsvPath Framework is a purpose-built off-the-shelf solution you can rollout now.

Powerful CSV and Excel Validation

CSV, Excel, NDJSON, and JSONL validation is core to the Framework. CsvPath Validation Language is simple, easy to integrate, and flexible enough to handle the unexpected. Inspired by Schematron, XPath, and SQL, CsvPath Validation Language brings powerful data validation to tabular and JSON structured data. Start here.

Together CsvPath Framework and FlightPath Data can help you build leadership's confidence that your data governance doesn't turn a blind eye to your most unruly data.

Integrated With Your Existing Tools

Parquet / Sqlite / S3 / OpenLineage / OpenTelemetry / Slack / Postgres / MySQL / Azure / Google Cloud Storage / Webhooks / JSONPath / XPath / Airflow and more.

Give CsvPath Framework a Try

For more background on preboarding and the CsvPath and FlightPath architecture check out the CsvPath blog.

CsvPath - Data Onboarding Simplified.pdf

PDF · 1MB

GitHub - csvpath/csvpath: Data Engineering infrastructure that controls inbound tabular data file feeds and validates data contractsGitHub

NextDATA PREBOARDING