Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Using this folder structure, it should be possible to algorithmically recreate a snapshot of a provider at a particular time; for instance, by retrieving the most recent Avro file of mapped records from a particular date (perhaps by walking backwards until a date is found), and then enriching those and indexing the results. In a case like this, a new Plan provenance file (manifest) could be created that points to both the historical mapped-records Avro file and the output enriched-records Avro file.

The Plan files (in the "plan" folder and for an activity) are recommended but optional. Our software should be designed so that you can run an activity like a mapping either automatically by passing it the name of a Plan file, or by passing it the name of a generator activity's file directly.

...