In addition to advanced algorithms or automation solutions, designing dataflows between internal and external data sources is a significant challenge. The organizational #dataflow models are often dictated by the enterprise system architecture or the nature of computational requirements. A significant change in existing dataflows is not a small task. The authors discuss challenges in optimization along all dimensions of correctness, latency, and cost for various implementations. This excellent paper is a little dated, but it outlines several fundamental models concerning concepts of #scalability, #unbounded and #bounded #datasets, #streaming, #batching, time considerations, and incremental processing. #data #architecture
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.