In 2018 I developed Dataflows for Peakboard. A dataflow takes a data source and runs an ordered list of steps on it. Each step changes the table a little, for instance it filters rows or renames a column. The result is a new list that behaves like any other data source, without a line of Lua.
Data rarely arrives in the shape a board needs. A machine reports one column too many. A database returns the article number but not the article name, and the name is in a different table. Before Dataflows, the answer was a Lua script that reads the list, loops over it and builds a new table.
The screenshots in this post are from a current version of Peakboard Designer, not from 2018. The first one shows the Add dataflow dialog. The Steps list on the left starts with the base data source, here Products. The next entries are the steps Change column data type, Rename column, Remove columns, Sort and Filter. The Preview on the right shows the data up to the selected step, so you see the effect of a step right after you add it. The Add step button opens the Add Dataflow step dialog with all steps.
Each step has a name. You can rename, disable, move or delete a step at any time. The Settings entry at the top left holds the Data source name and the Reload state. The Reload state sets when the dataflow reloads, by default together with its source.
The next screenshot shows the Add Dataflow step dialog with the steps in seven groups:
- Edit columns and table with Add column, Rename column, Reorder columns, Change column data type, Switch rows/columns and Remove columns
- Manipulate column data with Update column, Replace column text and the steps for number, date and text formats
- Filter and sort with Filter, Sort, Filter by date/time columns and Remove duplicates
- Unite tables with Join, Union and Add lookup column
- Limit rows with Select top, Select range and Select last row
- Calculations with Aggregate and Accumulate
- Parse table with Parse table from JSON and Parse table from XML
The Demo area on the right shows a small example for the selected step.
Join supports the four usual types, inner, full, left and right. Add lookup column adds a column with readable text for an ID from a second table, for instance the article name for an article number. A dataflow can also use another dataflow as its base, so a long chain can be split into smaller parts.
That’s it.
If you have questions feel free to contact me.
Links:
Peakboard.com →
Your first steps with Dataflows →
Link table-like datasets using joins →