Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- 3 Application development – the basics
- 4 Application development – data flow programming
- 5 Large-scale development – modularity, extensibility, and distribution
- 6 Visualization and debugging
- Part III System architecture
- Part IV Application design and analytics
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
4 - Application development – data flow programming
from Part II - Application development
Published online by Cambridge University Press: 05 March 2014
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- 3 Application development – the basics
- 4 Application development – data flow programming
- 5 Large-scale development – modularity, extensibility, and distribution
- 6 Visualization and debugging
- Part III System architecture
- Part IV Application design and analytics
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
Summary
Overview
In this chapter we study data flow programming, including flow composition and flow manipulation. Flow composition focuses on techniques used for creating the topology associated with the flow graph for an application, while flow manipulation covers the use of operators to perform transformations on these flows.
We start by introducing different forms of flow composition in Section 4.2. In Section 4.3, we discuss flow manipulation and the properties of stream processing operators, including their internal state, selectivity, and arity, as well as parameterization, output assignments and functions, punctuations, and windowing configuration used to perform such manipulation.
Flow composition
Flow composition patterns fall into three main categories: static, dynamic, and nested composition.
Static composition is used to create the parts of the application topology that are known at development time. For instance, consider an application that has a source operator consuming data from a specific Internet source, for example, a Twitter [1] feed. Let's assume that the stream generated by the source is to be processed by a specific operator that analyzes this data, for example, a sentiment analysis operator [2] that probes the messages for positive or negative tone. In this case, the connection between the source operator and the analysis operator is known at development time and thus can be explicitly created by connecting the output port of the source operator to the input port of the analysis operator.
- Type
- Chapter
- Information
- Fundamentals of Stream ProcessingApplication Design, Systems, and Analytics, pp. 106 - 147Publisher: Cambridge University PressPrint publication year: 2014