From 7c595300cc878e646f8d6271ae987cb824bcf05a Mon Sep 17 00:00:00 2001 From: Christophe Favergeon Date: Fri, 9 Sep 2022 11:04:03 +0200 Subject: [PATCH] Updated documentation due to the renaming of SDF. Corrected small bug introduced by duplicate feature in SDF. --- ComputeGraph/README.md | 32 ++++++++++++--------- ComputeGraph/documentation/example1.md | 14 ++++----- ComputeGraph/documentation/example2.md | 2 +- ComputeGraph/documentation/example3.md | 8 +++--- ComputeGraph/documentation/example4.md | 2 +- ComputeGraph/examples/example4/debug.py | 2 +- ComputeGraph/examples/example7/custom.py | 2 +- README.md | 8 +++--- cmsisdsp/cg/static/scheduler/description.py | 3 +- 9 files changed, 39 insertions(+), 34 deletions(-) diff --git a/ComputeGraph/README.md b/ComputeGraph/README.md index 3f04ca48..625f0dd8 100644 --- a/ComputeGraph/README.md +++ b/ComputeGraph/README.md @@ -1,8 +1,8 @@ -# Synchronous Dataflow for CMSIS-DSP +# Compute Graph for streaming with CMSIS-DSP ## Introduction -A dataflow graph is a representation of how compute blocks are connected to implement a processing. +A dataflow graph is a representation of how compute blocks are connected to implement a streaming processing. Here is an example with 3 nodes: @@ -26,14 +26,16 @@ When the processing is applied to a stream of samples then the problem to solve The general problem can be very difficult. But, if some constraints are applied to the graph then some algorithms can compute a static schedule. -When the following constraints are satisfied we say we have a Synchronous Dataflow Graph (SDF): +When the following constraints are satisfied we say we have a Synchronous Dataflow Graph: - Static graph : graph topology is not changing - Each node is always consuming and producing the same number of samples -The CMSIS-DSP SDF Tools are a set of Python scripts and C++ classes with following features: +In CMSIS-DSP, we are naming this a static flow. -- A SDF can be described in Python +The CMSIS-DSP Compute Graph Tools are a set of Python scripts and C++ classes with following features: + +- A compute graph and its static flow can be described in Python - The Python script will compute a static schedule and the FIFOs size - A static schedule is: - A periodic sequence of functions calls @@ -53,7 +55,7 @@ Without any scheduling tool for a dataflow graph, there is a problem of modulari - You may need to change how many times the predecessor blocks must run - You may have to change the FIFOs sizes -With the CMSIS-DSP SDF Tools you don't have to think about those details while you are still experimenting with your data processing pipeline. It makes it easier to experiment, add or remove blocks, change their parameters. +With the CMSIS-DSP Compute Graph (CG) Tools you don't have to think about those details while you are still experimenting with your data processing pipeline. It makes it easier to experiment, add or remove blocks, change their parameters. The tools will generate a schedule and the FIFOs. Even if you don't use this at the end for a final implementation, the information could be useful : is the schedule too long ? Are the FIFOs too big ? @@ -61,7 +63,7 @@ Let's look at an (artificial) example: graph1 -Without a tool, the user would probably try to modify the sample values so that the number of sample produced is equal to the number of samples consumed. With the SDF Tools we know that such a graph can be scheduled and that the FIFO sizes need to be 11 and 5. +Without a tool, the user would probably try to modify the sample values so that the number of sample produced is equal to the number of samples consumed. With the CG Tools we know that such a graph can be scheduled and that the FIFO sizes need to be 11 and 5. The periodic schedule generated for this graph has a length of 19. It is big for such a small graph and it is because, indeed 5 and 7 are not very well chosen values. But, it is working even with those values. @@ -87,7 +89,7 @@ The schedule is (the size of the FIFOs after the execution of the node displayed At the end, both FIFOs are empty so the schedule can be run again : it is periodic ! -## How to use the Synchronous Data Flow (SDF) +## How to use the static scheduler generator First, you must install the `CMSIS-DSP` PythonWrapper: @@ -100,14 +102,14 @@ The script inside the cmsisdsp wrapper can be used to describe and generate the You can create a `graph.py` and include : ```python -from cmsisdsp.sdf.scheduler import * +from cmsisdsp.cg.static.scheduler import * ``` -Then you can describe the blocks that you need in the compute graph if they are not provided by the SDF. +You can describe new type of blocks that you need in the compute graph if they are not provided by the python package by default. Finally, you can execute `graph.py` to generate the C++ files. -The generated files need to include the `sdf/src/GenericNodes.h` and the nodes used in the graph and which can be found in `sdf/nodes/cpp`. +The generated files need to include the `ComputeGraph/cg/static/src/GenericNodes.h` and the nodes used in the graph and which can be found in `cg/static/nodes/cpp`. If you have declared new nodes in `graph.py` then you'll need to provide an implementation. @@ -120,7 +122,7 @@ More details and explanations can be found in the documentation for the examples ### How to build the examples -In folder `SDFTools/example/build`, type the `cmake` command: +In folder `ComputeGraph/example/build`, type the `cmake` command: ```bash cmake -DHOST=YES \ @@ -150,7 +152,7 @@ To build the C examples: * CMSIS-DSP must be built, * the .cpp file contained in the example must be built -* the include folder `sdf/src` must be added +* the include folder `cg/static/src` must be added For `example3` which is using an input file, `cmake` should have copied the input test pattern `input_example3.txt` inside the build folder. The output file will also be generated in the build folder. @@ -169,7 +171,7 @@ The first line is generating the schedule in Python. The second line is executin It is a first version and there are lots of limitations and probably bugs: -- The code generation is using [Jinja](https://jinja.palletsprojects.com/en/3.0.x/) template in `sdf/templates`. They must be cleaned to be more readable. You can modify the templates according to your needs ; +- The code generation is using [Jinja](https://jinja.palletsprojects.com/en/3.0.x/) template in `cg/static/templates`. They must be cleaned to be more readable. You can modify the templates according to your needs ; - CMSIS-DSP integration must be improved to make it easier - Some optimizations are missing - Some checks are missing : for instance you can connect several nodes to the same io port. And io port must be connected to only one other io port. It is not checked by the script. @@ -217,3 +219,5 @@ Here is a list of the nodes supported by default. More can be easily added: Examples 5 and 6 are showing how to use the CMSIS-DSP MFCC with a synchronous data flow. Example 7 is communicating with OpenModelica. The Modelica model (PythonTest) in the example is implementing a Larsen effect. + +Example 8 is showing how to define a new custom datatype for the IOs of the nodes. Example 8 is also demonstrating a new feature where an IO can be connected up to 3 inputs and the static scheduler will automatically generate duplicate nodes. diff --git a/ComputeGraph/documentation/example1.md b/ComputeGraph/documentation/example1.md index 15014105..38d2f074 100644 --- a/ComputeGraph/documentation/example1.md +++ b/ComputeGraph/documentation/example1.md @@ -23,10 +23,10 @@ Let's analyze the file `graph.py` in the `example1` folder. This file is describ -First, we add some path so that the example can find the sdf packages when run from example1 folder. +First, we add some path so that the example can find the CG static packages when run from example1 folder. ```python -from cmsisdsp.sdf.scheduler import * +from cmsisdsp.cg.static.scheduler import * ``` @@ -93,7 +93,7 @@ class ProcessingNode(Node): We just define its type. -Once it is done, we can start creating instance of those nodes. We will also need to define the type for the samples (float32 in this example). The functions and constants are defined in `sdf.schedule.types`. +Once it is done, we can start creating instance of those nodes. We will also need to define the type for the samples (float32 in this example). The functions and constants are defined in `cg.static.types`. ```python floatType=CType(F32) @@ -379,7 +379,7 @@ The returned valued is the number of schedules fully executed when the error occ The scheduling function is starting with a definition of some variables used for debug and statistics: ```C++ -int sdfError=0; +int cgStaticError=0; uint32_t nbSchedule=0; int32_t debugCounter=1; ``` @@ -408,15 +408,15 @@ Source source(fifo0); And finally, the function is entering the scheduling loop: ```C++ - while((sdfError==0) && (debugCounter > 0)) + while((cgStaticError==0) && (debugCounter > 0)) { nbSchedule++; - sdfError = source.run(); + cgStaticError = source.run(); CHECKERROR; ``` -`CHECKERROR` is a macro defined in `Sched.h`. It is just testing if `sdfError < 0` and breaking out of the loop if it is the case. +`CHECKERROR` is a macro defined in `Sched.h`. It is just testing if `cgStaticError< 0` and breaking out of the loop if it is the case. Since an application may want to use several SDF graphs, the name of the `sched` and `customInit` functions can be customized in the `configuration` object on the Python side: diff --git a/ComputeGraph/documentation/example2.md b/ComputeGraph/documentation/example2.md index ef8f8bb6..60a01830 100644 --- a/ComputeGraph/documentation/example2.md +++ b/ComputeGraph/documentation/example2.md @@ -68,7 +68,7 @@ The code generated in `sched.cpp` will not require any C++ class, It will look l i0=fifo2.getReadBuffer(160); o2=fifo4.getWriteBuffer(160); arm_scale_f32(i0,HALF,o2,160); - sdfError = 0; + cgStaticError = 0; } ``` diff --git a/ComputeGraph/documentation/example3.md b/ComputeGraph/documentation/example3.md index 93d3a186..9ab12e24 100644 --- a/ComputeGraph/documentation/example3.md +++ b/ComputeGraph/documentation/example3.md @@ -94,13 +94,13 @@ It is verbose but not difficult. The constructor is initializing the CMSIS-DSP F -The run function is applying the arm_cfft_f32. Since this function is modifying the input buffer, there is a memcpy. It is not really needed here. The read buffer can be modified by the CFFT. It will just make it more difficult to debug if you'd like to inspect the content of the FIFOs. +The run function is applying the `arm_cfft_f32`. Since this function is modifying the input buffer, there is a `memcpy`. It is not really needed here. The read buffer can be modified by the CFFT. It will just make it more difficult to debug if you'd like to inspect the content of the FIFOs. -This node is provided in sdf/nodes/cpp so no need to define it. You can just use it by including the right headers. +This node is provided in `cg/static/nodes/cpp` so no need to define it. You can just use it by including the right headers. -It can be used by just doing in your AppNodes.h file : +It can be used by just doing in your `AppNodes.h` file : ```c++ #include "CFFT.h" @@ -109,6 +109,6 @@ It can be used by just doing in your AppNodes.h file : From Python side it would be: ```python -from cmsisdsp.sdf.nodes.CFFT import * +from cmsisdsp.cg.static.nodes.CFFT import * ``` diff --git a/ComputeGraph/documentation/example4.md b/ComputeGraph/documentation/example4.md index 45a154aa..19ec9fb9 100644 --- a/ComputeGraph/documentation/example4.md +++ b/ComputeGraph/documentation/example4.md @@ -30,7 +30,7 @@ This file is defining the new nodes which were used in `graph.py`. In `graph.py` In `appnodes.py` we including new kind of nodes for simulation purpose: ```python -from cmsisdsp.sdf.nodes.CFFT import * +from cmsisdsp.cg.static.nodes.CFFT import * ``` diff --git a/ComputeGraph/examples/example4/debug.py b/ComputeGraph/examples/example4/debug.py index 36ef2d1b..becd1195 100644 --- a/ComputeGraph/examples/example4/debug.py +++ b/ComputeGraph/examples/example4/debug.py @@ -1,5 +1,5 @@ import numpy as np -from cmsisdsp.sdf.nodes.simu import * +from cmsisdsp.cg.static.nodes.simu import * a=np.zeros(10) f=FIFO(10,a) diff --git a/ComputeGraph/examples/example7/custom.py b/ComputeGraph/examples/example7/custom.py index b0d7305f..2a5a31bd 100644 --- a/ComputeGraph/examples/example7/custom.py +++ b/ComputeGraph/examples/example7/custom.py @@ -1,4 +1,4 @@ -from cmsisdsp.sdf.nodes.simu import * +from cmsisdsp.cg.static.nodes.simu import * import numpy as np import cmsisdsp as dsp diff --git a/README.md b/README.md index e56982b1..bf12d6d1 100755 --- a/README.md +++ b/README.md @@ -34,15 +34,15 @@ With this wrapper you can design your algorithm in Python using an API as close The goal is to make it easier to move from a design to a final implementation in C. -### Synchronous Data Flow (SDF) +### Compute Graph -CMSIS-DSP is also providing an experimental [synchronous data flow scheduler](SDFTools/README.md): +CMSIS-DSP is also providing an experimental [static scheduler for compute graph](ComputeGraph/README.md) to describe streaming solutions: * You define your compute graph in Python -* A static schedule (computed by the Python script) is generated +* A static and deterministic schedule (computed by the Python script) is generated * The static schedule can be run on the device with very low overhead -The Python scripts for the synchronous data flow (SDF) are part of the CMSIS-DSP Python wrapper. +The Python scripts for the static scheduler generator are part of the CMSIS-DSP Python wrapper. The header files are part of the CMSIS-DSP pack (version 1.10.2 and above). diff --git a/cmsisdsp/cg/static/scheduler/description.py b/cmsisdsp/cg/static/scheduler/description.py index 0b0d0e30..1c2ec761 100644 --- a/cmsisdsp/cg/static/scheduler/description.py +++ b/cmsisdsp/cg/static/scheduler/description.py @@ -265,7 +265,8 @@ class Graph(): # Instead we have our own graph with self._edges # (This script will have to be rewritten in a much # cleaner way) - self._g.remove_edge(nodea.owner,nodeb.owner) + if self._g.has_edge(nodea.owner,nodeb.owner): + self._g.remove_edge(nodea.owner,nodeb.owner) del self._edges[(nodea,nodeb)] if (nodea,nodeb) in self._delays: del self._delays[(nodea,nodeb)]