Six years after dataflows I built flows. A dataflow shapes a table and waits to be asked for it. A flow decides when things happen: which data source reloads, in which order, on what trigger, and what runs afterwards. It is the part of a board that keeps working while nobody is looking at the screen.
The problem with intervals
Every data source had its own reload interval. Set it to thirty seconds and it reloads every thirty seconds, independently of everything else on the board. That is a reasonable default and it holds up as long as the sources have nothing to do with each other.
They rarely have nothing to do with each other. One list has to be loaded before the dataflow that joins it, or the join runs against yesterday’s rows. A nightly export must not start while the source it reads is still refreshing. Something has to happen at six in the morning and not every thirty seconds. A shift change has to trigger a recalculation, and shifts do not align to an interval.
None of that fits into a number of seconds per source. What people did instead was write a script with a timer in it, keep track of what had already finished, and hope the order held. It usually did, until a source got slow.
The model
A flow is a trigger and an ordered list of steps.
Triggers come in four kinds: periodic, at a fixed time of day, on a data change in a list, or a function call with parameters. The last one is what makes a flow callable from a button or from another script.
Steps do one of three things: reload a data source or a dataflow, run a function, or write a result into a list, either appending to it or replacing what is there.
A step that runs a function decides how the flow continues: always, or only when the function returned true, or only when it returned false. That is the whole branching model. On top of that, a step can name a second function that runs when the step itself fails.
Keeping the model that small was deliberate, and it was the part I argued about most. A flow runs unattended for months on a device nobody logs into. Every construct I add is one more way for it to sit there stuck at three in the morning, and one more thing that has to be explained to somebody who is not a developer. Loops, nested conditions and parallel branches all got proposed and all stayed out. If a flow needs that much logic, the logic belongs in a function, where it can be tested.
Hub Flows and a project without screens
The second half of the work was getting flows off the boards.
A flow that reloads a list and writes a result somewhere does not need a screen. It does not need a device on a wall either. But until then every flow lived inside a board package, so the only way to run one was to have a board running it, and the same nightly job had to be maintained on every box that needed it.
So the designer got a second project type. A normal Peakboard project is a UI project: screens, controls, the design surface, and data behind it. The new one is a data project. It has data sources, dataflows, flows, variables and functions, and no screens at all. It is not a board with the screens left empty, it is a different kind of package with its own file type, and it gets uploaded to the Hub rather than deployed to a device. There it runs centrally, on the server, without a display.
Building that was more work than it sounds, and almost none of it was the flow engine. The designer had grown up around the assumption that a project has a design surface. The package explorer, the toolbox, the preview, the deployment dialog, the validation, the start screen: all of it asked what the current screen was. Teaching the application that a project can be legitimately screenless touched far more of the UI than the feature itself.
Hub Flows are currently part of the on-premise edition of the Hub.
That’s it.
If you have questions feel free to contact me.
Links:
Flows documentation →
Your first steps with Dataflows →
Peakboard Hub →