Painting a Program (instead of writing)

Flow-based programming is a way of writing programs by drawing pictures of graphs. It’s a very intuitive and friendly way of working. Today I’d like to talk about my adventures exploring this space. I hope that you’ve had similar ideas and would like to join me on this journey.

If you’ve learned to code a little bit, you’re probably familiar with code that looks like this, a method that adds two numbers together:

class Math {
   * Returns a+b
   * @returns a+b
  double add(double a, double b) {
    return a+b;

If you play a lot of video games you might have seen designs like this assembler combining two inputs to produce an output in Factorio:

Adding two things together in Factorio

And people born after 2000 are probably familiar with the Scratch programming language from MIT:

Adding two things together in Scratch 3

I contend that there is a level of abstraction where these are all the same thing. There is a magic box that does a job, some connection to inputs, some connection where to send the results, and visual indicators showing the state of the various elements. In Flow-Based Programming (FBP), Packages of data flow through Nodes. You may have also seen similar models in Unity, Blender, Nuke, NodeRed, and other editing programs. The idea has been kicking around since the 1960s.

Philosophically, I feel that a good use of a maker’s time is to make making easier. The benefits compound! One of the ways FBP makes programming easier is that there are no syntax errors, only logical errors. I can’t forget a pair of braces or a semi-colon. Also values are visible all the time, so no digging into the debugger to find the element I want… the list goes on. Strangely, we still write code as long form text in drab color documents with weird ideas like ‘linking’ and ‘package management’ that should have died in the Cambrian era. Point being I’m highly motivated to use a FBP system.

The system has to be like all modern editors: there has to be a way to extend or plug in or mod new features. It must be easy for everyone to extend in a collaborative way – not just jailed to Unity or Blender, but some where all nodes of all types thrive together. It’s got to be as beautiful and intuitive (and fun) as Factorio. To this end I started an editor called Donatello.

(1) The very first image of Donatello

I built a simple Graph that has Nodes linked through their ReceivingDock and ShippingDock by Connections. All data in the system is stored in the Docks. When a ShippingDock is changed it is flagged at dirty. After all nodes are processed the Graph looks for dirty ShippingDocks and copies their values to connected ReceivingDocks, regardless of whatever may be there. When a ReceivingDock is dirty it causes the Node to run on the next cycle. Dirty Docks are marked with red text. After a few days it started to look not awful:

(2) A machine that counts up forever. It can be done with only one Node!

To repeat: Graphs have Connections between Nodes. Nodes have AbstractDocks, some of which are ShippingDocks and some are ReceivingDocks. When a node runs, it might poke the ShippingDock. AbstractDocks might also have one ConnectionPoint each that make it easier to add and remove Connections.

When you want to make a new Node, you have to pick a name, declare the Docks, and write the update() method to read the inputs, do a thing, and probably poke an output.

Part of the motivation comes from making line art for plotter robots like Makelangelo 5. Having written several picture-to-art projects I started to notice similarities. I thought “Wouldn’t it be great if these similar bits were LEGO blocks and anyone could play with them?”

So far I had

  • A core NodeGraphCore library defining Graph/Node/Dock/Connection and a Service that plugins can use to advertise their Nodes.
  • A GUI built using NodeGraphCore called Donatello. It uses Java ServiceLoader to find valid plugins and load the Nodes.
  • A set of Nodes in Makelangelo-software to give me access to all the line art manipulation tools. I put Makelangelo-software in the ~/Donatello/extensions/ folder and then Donatello can do stuff with lines and art.

I put them all together to make some art and test the practical limits of big graphs. Note in the video (3) I am tweaking the program while it is running, tho I could pause it at any time.

(3) Making art in Donatello with nodes from Makelangelo-software

What worked

Making new nodes is so easy, as is registering them with a NodeFactory so they show up in the GUI. Using the mouse to move items around, draw and erase connections, and alter behavior was also very intuitive. It was easy to save graphs, load graphs, and step through the program. I have built unit tests without visuals. I can copy/paste/delete/edit nodes and their connections; I can select two nodes far apart and run a tool that selects everything between them in the shortest down-stream path. Undo/Redo work great. I can graph a group of Nodes and Fold them down to a single Node. Put another way, a Node might represent an entire Graph. Folded Nodes can be Unfolded and should have the ability to be saved to a separate file, but it hasn’t been a priority.

What needs work

My first way of drawing connections was straight lines. The second implementation of connections uses Bezier curves. It’s okay…? but now I dream of a third way with right angle lines like I’m used to seeing in a circuit diagram or Factorio.

(4) KiCAD circuit diagram

Also like both circuits and Factorio it would be great if I could rotate Nodes for more intuitive graph layouts.

There’s little visual indication of what connection point can legally attach to another connection point. At least there is some checking! In circuits and Factorio you can connect anything to anything and no one will stop you. Will it run? No. but it also won’t complain, either.

I cannot yet easily make blueprints/templates of popular design patterns as in fig 5. I bet as program grow a lot of patterns will emerge and it would be great to have those ready-to-go.

There’s no reason Nodes need to be drawn in 2D. They could be done in 3D just as easily. This is neither good nor bad and I’m sure there’s interesting tradeoffs. I’m not sure I’d want to work with a VR helmet on 8h a day.

Updates to NodeGraphCore mean Makelangelo-software HAS to be rebuilt or the provided Nodes will not load in updated Donatello. Put another way, the version of all three has to match all the time. This is kinda sucky but there’s more! When a plug-in is out of date Java does not give a clear warning – it silently refuses to load and the user is left wondering what went wrong.

What doesn’t work

In classical FBP connections are more like the Factorio conveyor belts: packets of data queue on the connection until the FBP Node (assembling machine) is ready to process them. The Nodes don’t hold on to anything, ever. Resetting the program is as easy as cleaning out all the connections. This is hard in Donatello because there is no distinction between a Node’s current value and its default value. I get around it by saving the state of the graph in the starting state and ONLY the starting state. Yes, I have saved the graph in the wrong state and had to clean it up.

In Donatello nodes contain all data and immediately transmit the results down-stream when they finish a calculation. Also once any input to a node has changed that node will run on the next cycle. This was easy to implement but it has drawbacks later. In a big graph some parts run faster than others. So if a group of nodes A runs twice as fast as some other group B then Add might run A1+0, A1+B1, A2+B1, A2+B2, and so on. You can see it below (8) when TransformTurtle and AddTurtle create unintended noise in the results.

Partial solutions

If the system is set up to run like Factorio or FBP then there has to be one input on each Connection before the TransformTurtle will run. Another way would be an activation trigger – most inputs are optional and nothing runs until data arrives on the trigger input.

I’ve tried twice now to rebuild the Nodes and Connections to work more like classic FBP. There are no more isDirty flags, which means every node has its own “should I run now” test at the start of update() and every Node has to be called every step. This lets me have activation triggers.

The real problem starts when I want to initialize the graph. I need a way to put at least a number or a string of text into Connections so that they flow into the system and make everything run. I can hard-code that kind of thing easily but how do I put it in the GUI? It can’t be a Node because Nodes don’t hold data any more! It’s some kind of new class or interface … maybe Emitter or Entrance or Source. In the model above (7) I had nodes called LoadString and LoadNumber that served well for setting an initial value.

(10) Folding with Loaders

Do all Loaders become ReceivingDock and all… Unloaders? become ShippingDock when Nodes are Folded? Will there be Loaders that should not become ReceivingDock – internal constants?

If a single Loader is supposed to deliver to many Nodes, what then? I’m not going to make many identical Loaders! But I also run into problems when a single Loader is connected to many ReceivingDocks. Data types in Java cannot be cloned (copied) easily. Data types can’t be serialized easy, either. (another way to copy – save and load it into a second instance.) One workaround is to put the same instance of Packet into every Connection. As long as the Nodes treat the Packet as read-only then everything will work…

I feel inspired by Simon Lague’s Digital Logic Sim to put a fence around the entire Graph. The fence is the Node that contains this Graph. Only Loaders and Unloaders that touch the fence are exposed when Folded.

Final thoughts

I’m far from done thinking about this and I’d love to hear your ideas and see your solutions. Find me on Discord and let’s talk about it.


Friday Facts 14: All the Sixi robot arms so far (demo at end)

Discord user Wted00 said “You talk about Sixi 3 a lot. What ever happened to Sixi 1 and 2?” Great question, long answer. This video covers robot arms before Sixi and then all three models of the current brand, along with interesting things learned along the way. Plus stick around for movement demos at the end.


Friday Facts 13: How to find DH parameters for your robot

So you’ve got Marlin firmware installed, your models ready to load into Robot Overlord and now the struggle is figuring out the Denavit–Hartenberg parameters. We’ll cover where to find critical dimensions, some of the pitfalls along the way, and compare with a few different models to help clear up any misunderstandings. It is assumed that you already absorbed an understanding of D-H parameters from places like the wikipedia link previously mentioned.

Start with the data sheet

Commercial electronics products should have a data sheet or user manual. This is true for electronics components and it’s true for robot arms. The data sheet will have operating limits, physical limits, dimensions, use instructions, and more.

This is the dimension drawing for the 6-axis Sixi 3 robot.

some values have been rounded and may no exactly match the D-H parameters in the open source code.

D-H parameters in Robot Overlord

Here are the D-H parameters from Robot Overlord for Sixi3 6-axis model. Dimensions here are in centimeters. thetaMax and thetaMin are the rotation limits on a given joint.

		// name d r alpha theta thetaMax thetaMin modelFile
		addBone(new RobotArmBone("X", 7.974,     0,270,  0,170,-170,"/Sixi3b/j1.obj"));
		addBone(new RobotArmBone("Y", 9.131,17.889,  0,270,370, 170,"/Sixi3b/j2.obj"));
		addBone(new RobotArmBone("Z",     0,12.435,  0,  0,150,-150,"/Sixi3b/j3.obj"));
		addBone(new RobotArmBone("U",     0,     0,270,270,440, 100,"/Sixi3b/j4-6.obj"));
		addBone(new RobotArmBone("V",15.616,     0, 90, 90,90+180,90-180,"/Sixi3b/j5-6.obj"));
		addBone(new RobotArmBone("W",  5.15,     0,  0, 180,360,   0,"/Sixi3b/j6-6.obj"));


It’s good to remember that the physical point of rotation (PoR) is not necessarily the same as the mathematical PoR. It’s tempting to think of the first PoR being where the shoulder meets the base, about 40mm up from the origin. Actually it is at the origin! Here is what it looks like illustrated by the Robot Overlord simulation

Sixi3 in Robot Overlord with “show lineage” turned on

At each PoR there is a 3×3 matrix. Each matrix has red/green/blue lines meaning x/y/z axies, respectively. Notice that the axis of rotation is always the blue z axis. The first joint has a blue line pointing up, as does the 5th. The 7th matrix on the face of the hand is the attachment point for a tool.

The math model and the physical model are slightly different like that. It is important that the axis of rotation is in the right place. It is not crucial that the point of rotation be, say, right between two moving parts. Typically the PoR is constrained by the location of the previous and the next PoR. Each of them is also constrained by the D and R values – you cannot “move” from one link to the next along the green Y axis…ever.

Compare and contrast

Here’s an image of a meca500 I found in their user manual.

Can you identify all 6 PoR? If the first joint (0) is at the origin, where are joint 3, 4, and 5?

Final thoughts

Work out the mathematical model of your robot before you do the rest of the design. Nobody wants to be stuck with a model that is incompatible D-H parameters.

Add your arm to Robot Overlord and share it with us. We want to celebrate your greatness and collaborate together.


Friday Facts 12: How to use Marlin in a Robot Arm

Building a robot arm is one thing, but what about writing the code to make it run? Some people want to learn the fine points of precision stepper motor control, forward and inverse kinematics, and then debug all that stuff. For the rest, working together gets the job done faster. For those people the Marlin 3D printer firmware is a great option. Today I’m going to show how I tweaked it to run in the Sixi 3 robot arm. Please share your experience with us so we can improve this post.


Marlin 3D printer firmware is the code in the brain of a very large number of printers. It is very flexible with a few changes. Most people might think of printers as having four motors – one for each direction and one for the extruder. But recent changes mean that Marlin can run up to six motors. That’s great for us, because most robot arms are 6 or less.

With Marlin installed you’ll be able to control the angle of each motor by sending gcode commands and even drive them simultaneously. With Marlin’s homing routines you could locate position, and new options coming in the near future will give real time feed back (more on that later)

What needs to be tweaked

Pour yourself a drink and settle in. This list will touch at least two files and take some time… OR you can use the sixi3 branch I maintain and adjust it for your speeds and gear ratios.

I keep trying new ways to make this list less dry. What do you think?


Old valueNew Value
#define STRING_CONFIG_H_AUTHOR “(none, default config)”#define STRING_CONFIG_H_AUTHOR “(Sixi3, Marginally Clever Robots)”
//#define CUSTOM_MACHINE_NAME “3D Printer”#define CUSTOM_MACHINE_NAME “Robot Arm”
//#define LINEAR_AXES 3#define LINEAR_AXES 6
#define AXIS4_NAME ‘A’#define AXIS4_NAME ‘U’
#define AXIS5_NAME ‘B’#define AXIS5_NAME ‘V’
#define AXIS6_NAME ‘C’#define AXIS6_NAME ‘W’
#define EXTRUDERS 1define EXTRUDERS 0
//#define USE_XMIN_PLUG
//#define USE_YMIN_PLUG
//#define USE_ZMIN_PLUG
//#define I_DRIVER_TYPE A4988
//#define J_DRIVER_TYPE A4988
//#define K_DRIVER_TYPE A4988
#define E0_DRIVER_TYPE A4988
#define I_DRIVER_TYPE A4988
#define J_DRIVER_TYPE A4988
#define K_DRIVER_TYPE A4988
//#define E0_DRIVER_TYPE A4988
#define DEFAULT_AXIS_STEPS_PER_UNIT { 80, 80, 400, 500 }#define DEFAULT_AXIS_STEPS_PER_UNIT { 105, 105, 105, 105, 105, 105 }
#define DEFAULT_MAX_FEEDRATE { 300, 300, 5, 25 }#define DEFAULT_MAX_FEEDRATE { 5, 5, 5, 5, 5, 5 }
#define DEFAULT_MAX_ACCELERATION { 3000, 3000, 100, 10000 }#define DEFAULT_MAX_ACCELERATION { 10, 10, 10, 10, 10, 10 }
#define E_ENABLE_ON 0 // For all extruders
//#define I_ENABLE_ON 0
//#define J_ENABLE_ON 0
//#define K_ENABLE_ON 0
//#define E_ENABLE_ON 0 // For all extruders
#define I_ENABLE_ON 0
#define J_ENABLE_ON 0
#define K_ENABLE_ON 0
#define INVERT_Y_DIR true#define INVERT_Y_DIR false
//#define INVERT_I_DIR false
//#define INVERT_J_DIR false
//#define INVERT_K_DIR false
#define INVERT_I_DIR false
#define INVERT_J_DIR false
#define INVERT_K_DIR false
//#define I_HOME_DIR -1
//#define J_HOME_DIR -1
//#define K_HOME_DIR -1
#define I_HOME_DIR -1
#define J_HOME_DIR -1
#define K_HOME_DIR -1
define X_BED_SIZE 200
define Y_BED_SIZE 200
//#define X_BED_SIZE 200
//#define Y_BED_SIZE 200
#define X_MIN_POS 0
#define Y_MIN_POS 0
#define Z_MIN_POS 0
#define X_MIN_POS -360
#define Y_MIN_POS 360
#define Z_MIN_POS -360
#define X_MAX_POS 360
#define Y_MAX_POS -360
//#define I_MIN_POS 0
//#define I_MAX_POS 50
//#define J_MIN_POS 0
//#define J_MAX_POS 50
//#define K_MIN_POS 0
//#define K_MAX_POS 50
#define I_MIN_POS -360
#define I_MAX_POS 360
#define J_MIN_POS -360
#define J_MAX_POS 360
#define K_MIN_POS -360
#define K_MAX_POS 360
#define HOMING_FEEDRATE_MM_M { (50*60), (50*60), (4*60) }#define HOMING_FEEDRATE_MM_M { (4*60), (4*60), (4*60), (4*60), (4*60), (4*60) }
//#define SDSUPPORT#define SDSUPPORT


define AXIS_RELATIVE_MODES { false, false, false, false }#define AXIS_RELATIVE_MODES { false, false, false, false, false, false }
#define HOMING_BUMP_MM      { 5, 5, 2 }
#define HOMING_BUMP_DIVISOR { 2, 2, 4 }
#define HOMING_BUMP_MM      { 5, 5, 5, 5, 5, 5 }
#define HOMING_BUMP_DIVISOR { 2, 2, 2, 2, 2, 2 }


  • MOTHERBOARD is your choice of brain board. Anything Mariln supports AND has 6 axies will work.
  • DEFAULT_AXIS_STEPS_PER_UNIT is the gear ratio at the given joint. For all sixi3 gearboxes the ratio is 70:1 (harmonic) * 54:20 (timing belt) * 200/360 (for 1.8 degree stepper motors at full step) = 105.
  • Because the gear ratio is so high the motors are not physically able to exceed the DEFAULT_MAX_FEEDRATE. If you use faster motors or a faster brain board you may be able to improve on these numbers.
  • EEPROM_SETTINGS, SDSUPPORT, and REPRAP_DISCOUNT_SMART_CONTROLLER are not required. I use these to tweak settings for testing, run programs from the SD card, and to have an LCD panel on my robot.
  • Every other change is to adjust from 3 axies to 6.

Homing and Real time feedback

There are some exciting new features coming to Marlin that should make real time feedback possible. This means we’ll know the robot position without having to guess or to home. It also means we can tell when the actual position deviates from the expected position too much that a collision has occurred and that can save a lot of trouble! The new configuration options to explore are:

  • REALTIME_REPORTING_COMMANDS adds some “quick commands” that get processed before anything else in the gcode buffer of the robot. Great for emergency breaking and for requesting position information (Gcode “S000”)
  • M114_REALTIME adds “M114 R” which reports the real-time position of the robot instead of the projected position at the end of the planned moves.
  • I2C_POSITION_ENCODERS is a first pass at adding real time sensors. This will no doubt be expanded later to include other types and features.

Further Reading

The Marlin Configuration guide online

robot arms in Robot Overlord

Friday Facts 11: How to add a robot arm to Robot Overlord (2022)

Robot Overlord is going to be the inkscape, the VLC, the Steam of robot arms – the one vendor-agnostic interface everyone teaches, knows, and loves. In order to get there it has to support every robot arm under the sun. This post is for robot arm makers that want to save time by not writing all their own code.

Some of the arms already available

Did you know Robot Overlord speaks natively to Marlin 3D printer firmware? Save even more time by using the same firmware.

You will need

  • A 3D CAD model of your robot arm
  • The D-H parameters of the arm in the same pose as it appears in your CAD file
  • The ability to write Java code for the Robot Overlord project
  • Some familiarity with git (forks, commits, pull requests)

Prepare your CAD file

It is easiest to export your arm into discrete moving sections, all of which with the same origin at the bottom center of the base of the arm (see slide 1). This is the same origin as the D-H parameters.

A meca500 robot, color coded with the base and six parts. the origin would be in the red part, on the same axis of rotation as the pink part

To help Robot Overlord run smoothly and to protect your IP it is recommended that you decimate the model by removing all hidden internal structures and components. Consider leaving screw heads while removing the threaded sections to save many megabytes of file size.

Each discrete section should be exported as an OBJ or STL file. (more on this later.)

The exported files should be located in /src/main/resources/[your robot folder]/. So if your robot is named Foo then it would be /src/main/resources/foo/. It would be consistent to name them base, j0, j1, etc.

Prepare your Robot Overlord class

In Robot Overlord’s /src/main/java/com/marginallyclever/robotOverlord/robots/robotArm/implementations you will find the collection of currently supported robot arms. It may be easiest to copy one of these classes and modify it for your purposes. Here is the minimum needed to code your arm. Every instance of Foo should be replaced with your class name.

public class Foo extends RobotArmIK {
	private static final long serialVersionUID = 1L;

	public Foo() {
		setName("Foo v1");  // the name that appears to users.
	protected void loadModel() {
		setBaseShape(new Shape("Base","/Foo/j0.obj"));

		// The DH parameters and the model file, added in order from J0 ... J5.  
		// angles are degrees, distances are centimeters.
		// name d r alpha theta thetaMax thetaMin modelFile
		addBone(new RobotArmBone("X", 7.974,     0,270,  0,170,-170,"/Foo/j1.obj"));
		addBone(new RobotArmBone("Y", 9.131,17.889,  0,270,370, 170,"/Foo/j2.obj"));
		addBone(new RobotArmBone("Z",     0,12.435,  0,  0,150,-150,"/Foo/j3.obj"));
		addBone(new RobotArmBone("U",     0,     0,270,270,440, 100,"/Foo/j4-6.obj"));
		addBone(new RobotArmBone("V",15.616,     0, 90, 90,270,-90,"/Foo/j5-6.obj"));
		addBone(new RobotArmBone("W",  5.12,     0,  0, 180,360,   0,"/Foo/j6-6.obj"));


Now that the arm can be loaded by the app it needs to be on the menu of things that can be created by the user. In /src/main/java/com/marginallyclever/robotOverlord/ add your new class:

public class EntityFactory {
	private static Class<?> [] available = {
		// ...
		com.marginallyclever.robotOverlord.robots.robotArm.implementations.Foo.class,  // "Foo" must be your class name.
	// ...

Now when you run the application and open the Add menu your robot name will appear.

‘Foo v1’ will appear on this list

Model size and orientation fixes

It is possible that your model appears in Robot Overlord too large, too small, or the parts are rotated in a strange way. My solution is to use a modelling program like Blender to rotate, scale, decimate, and even texture the model.

Forward and Up will control the rotation. +Z is always up in a Robot Overlord scene.

Selection Only will simplify your exporting from Blender. In the image above it will only export J2.

Scale will control the size. For models that appear in meters instead of centimeters, choose 0.1. If your model is imperial, you’ll probably want 2.54.

Now share it with …the world!

You’ve changed the code, you’ve massaged the model, it runs on your machine. Now to share it with everyone else! A pull request from you to the Robot Overlord project will tell the dev team that your stuff is ready. This is the best way to make sure your model gets in the way you want it.

No time? Let us do it for you.

We can add your model(s) to our system. Contact us! We’re looking to collaborate and work with everyone. Writing the class is free; preparing the CAD files is specialized work we outsource and will quote.

What about URDF files?

URDF is the Unified Robot Description Format, part of ROS, the Robot Operating System. ROS is a nice system but much harder to get running – part of the reason I work on Robot Overlord. Join the Robot Overlord github project and help make it happen? Imagine what we could do together!