A decade of growth as a software as service company led to continual change and some dramatic reorganizations. We collect various mentions of the work we've done observing software through the metadata produced throughout its creation and operation.
Our approach to viewing our own work has been positively influenced by employing a graph database as a point of aggregation from many sources.
The system follows the data warehousing strategy of extract-transform-load. We chose the Neo4j graph database for downstream analysis and built a web app to perform and present queries we found useful.
We query the graph by navigating a landscape of scenarios finding the nearest next query to makes use of accumulated facts chosen from previous results. Each query result set is presented in tabular form, with column values hyperlinked when their choice would advance to a recognized next query.
Where query results describe a network, with columns such as "source" and "sink", we generate a vector sketch of this network with clickable nodes and arcs leading to obvious next queries, often drilling into details.
We've leveraged open software and expertise generally made available and now reciprocate. Link to follow when I resolve the link posting error.
Our work feels a lot like test-driven development felt twenty years ago. New technology made tdd practical then. So too with our work today.
Like tdd, this work pays back in scale. That is, expect integration of your first few sources to be no more rewarding than your first few tests. The reward comes later when a sense of knowing replaces concern for the unknowns that compound with growth.
