# AWS S3

<figure><img src="/files/NrrZu3bzP187nxJa2H2F" alt=""><figcaption></figcaption></figure>

{% hint style="info" %}
All five backends work exactly the same. The main difference is in the protocol part of the URI:

* azure://
* sftp\://
* s3://
* gs\://
* / or c:\\

*(HTTP is read-only, so doesn't count as a storage backend. It is only for loading files.)*

Each backend has its own way of storing credentials, of course. In the case of SFTP there is a form in the config panel's integrations tab. The three cloud stores use env vars that you can set in the OS, in FlightPath Data, or in a FlightPath Server project. &#x20;
{% endhint %}

There are three places CsvPath keeps files:&#x20;

* [The archive](/topics/practical-stuff/namespacing-with-the-archive.md) — the place where results live
* [Inputs](/topics/higher-level-topics/data-validation-files-and-storage/named_files_and_paths.md):
  * Named files — all the source Excel and CSV data files
  * Named paths — your named groups of csvpaths

Each of these can be independently placed. By default Archive is the folder at `./archive`. Named-files and named-paths default to:

* `./inputs/named_files`
* `./inputs/named_paths`

All of these locations and names can be changed. Keep in mind that the Archive takes its name from the last part of its path. That means that if you put your archive at `./this/is/my/stuff` your archive will be named `stuff`. In most cases that doesn't matter, but when we're tying into other systems, such as the CKAN or Marquez integrations, the archive name is meaningful.

How do you set up the three file storage locations? Easy, just change [your settings](/topics/how-tos/config.ini.md). There are three settings in `config.ini`. By default config.ini is in `./config/config.ini`. The settings you need are in the `[results]` and `[inputs]` sections. Archive is set under `[results]` with the archive key. Named-files and named-paths are set under `[inputs]` using the `files` and `csvpaths` keys, respectively.

<figure><img src="/files/6Gbir8pOpaD0kP74nuwc" alt="" width="375"><figcaption></figcaption></figure>

If you'd like your results to go to an archive in S3 all you need to do is set the `archive` key to an S3 URI like:&#x20;

```ini
archive = s3://csvpath-example-1/archive
```

&#x20;Using this setting would send all results to the `csvpath-example-1` bucket with paths beginning with `archive`. (S3 doesn't truly have directories, but in effect, everything goes into the `archive` directory).

Likewise, to store your source data in S3 you would set a key like:&#x20;

```ini
files = s3://csvpath-example-1/inputs/named_files
```

As usual with AWS you will need to authenticate at the command line. This can be as simple as exporting your SK and AK as env vars. See AWS's docs.

Bear in mind that as soon as you separate your data and compute you incur a network latency cost, as well as actual dollars and cents costs. There are ways to mitigate the latency and moving your compute to AWS along with your data is likely to be a big help. All-in-all, using S3 is sweet, but as with any work you do using CsvPath, try, test, automate, and iterate.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://www.csvpath.org/topics/how-tos/storage-backends/aws-s3.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.