The Map Metaphor
Pipeline systems are fairly common these days.
Most if not all attempt to provide a pipeline execution model;
in other words, they are the pipeline.
In contrast to that, PipeGadget provides a visualization of the pipeline.
This is our overall metaphor:
Your pipeline is like roads and bridges; your data is like cars and trucks.
PipeGadget is the map.
PipeGadget wants to know the roads and bridges,
and how cars and trucks behave,
but not the specific instances of cars and trucks.
Your production management system is like the central office at a trucking company.
They know that
a truckload of widgets is due at the factory next Tuesday,
and the truck is being driven by Fred.
They do not know the turn-by-turn directions, namely:
exit number 42B, left onto the service road,
second right into the driveway.
For that, they need a mapping system.
Your asset management system is like the trucking company garage.
It provides safe parking/storage for trucks, and supplies trucks as needed.
Thus PipeGadget diagrams have nodes for applications and programs.
(These are the roads in our metaphor).
In addition, PipeGadget diagrams have nodes for files and data, but only in template form.
(These represent how cars behave but not the specific instances of cars).
See more on how the map metaphor influences the pipeline diagrams.
Maps are Useful
Maps are super useful for many things, including
Where are the roads now? What do they connect to?
Urban Planning —
Where will we put the new airport? How does that impact the current roads?
Route Mapping —
How do I get from SF to Tahoe? I need a map.
In pipeline terms,
how do I get this asset from modeling through to rendering? I need a map.
Traffic Routing —
Turn this light red, that one green. Open or close these lanes.
In pipeline terms,
this means changing the filenames or metadata contracts.
Traffic Monitoring —
Where are the accidents, where is traffic flowing freely?
In pipeline terms, this is Analytics.
The node graph is the right shape to accumulate data
— there is a node for each thing you care about.
Are there any coffee shops nearby? What are the hours? menu? wifi policy?
In pipeline terms, this is Metadata.
Again, the node graph is the right shape to store all the metadata contracts.
(More on that below).