Doing rewind / replay, part 2

Can we achieve the same goals using only the CLI? Yes!

While the Python we used to drive Rewind / replay, part 1 was not complicated, using the CLI would be even quicker and in a triage situation might be a better option. So let's try that.

We'll use the same data and csvpaths for this second example. Only our use of the CLI is different. Because it is the same activity we'll do it quickly and let you refer back to part 1 for the background information.

Here are our files. Two sets of three csvpaths and one data file.

183B

people.csv

918B

sourcemode.csvpaths

924B

sourcemode2.csvpaths

You stage the data and load the csvpaths like this:

Next, run the original from sourcemode.csvpaths. We're using the version where the source2 csvpath has the word working. Here are the steps:

The result is a bit messy, but if you look closely those printouts tell us the csvpaths worked. Anyway, the print() statements in the csvpaths can be removed when you want to show off this trick to your friends.

Running a rewind/replay in the CLI is easy. But remember to reload your csvpaths when you make changes. We have a second csvpaths file that has our changes, so load it now. Use the same named-paths name as you did the first time.

Now we're ready to rewind and replay using the same data generated by the source1 csvpath with the new csvpaths. The new csvpaths file is only different in the source2 and source3 csvpaths, so we're going to start with source2:

And there again we did the substitution of thinking for working.

You can see that rewind/replay based on immutable data is straightforward — using the CLI or Python. If you need to redo any part of your process and you don't want to start at the very beginning, you have an easy way to do that. And because the data is immutable and the processing is idempotent, there is little risk in iterating on a solution. Using the CLI, if you prefer that approach, is just icing on the cake.

Here's a quick video to show the steps we took. Try it yourself.