Architecture LinuxCNC-block-diagram - update

[zz912]

I tried to update the block diagram of the LinuxCNC architecture. I would like the image to include both the ideal state and the current unwanted state that has arisen.

I am currently observing that some problems are being solved at the frontend level, instead of being solved at the background level.

When I started with LinuxCNC, I was missing a display where the HAL is. So I drew it.

If anyone knows how other parts of LCNC should work that would be good to draw. Feel free to send them to me drawn with a pencil on paper and photographed with a mobile phone. I can then draw them in the image.

I will be happy if there is a discussion about my proposal.

[andypugh]

I am surprised to see spindles in EMCIO. I am pretty sure those are in EMCMOT.
In 2,9 there is src/emc/iotask/iocontrol.cc but that is gone in master (and doesn't do very much at all in 2.9)
764655e

So the whole EMCIO block is probably not there any more.

[zz912]

I am interested only aboat 2.10 version. I made quick modification:

If I understand correctly, here are the HAL pins of the TASK:

linuxcnc/src/emc/task/taskclass.cc

Lines 46 to 70 in 764655e

	struct iocontrol_str {
	hal_bit_t *user_enable_out; /* output, TRUE when EMC wants stop */
	hal_bit_t *emc_enable_in; /* input, TRUE on any external stop */
	hal_bit_t *user_request_enable; /* output, used to reset ENABLE latch */
	hal_bit_t *coolant_mist; /* coolant mist output pin */
	hal_bit_t *coolant_flood; /* coolant flood output pin */
	hal_bit_t *lube; /* lube output pin */
	hal_bit_t *lube_level; /* lube level input pin */
	// the following pins are needed for toolchanging
	//tool-prepare
	hal_bit_t *tool_prepare; /* output, pin that notifies HAL it needs to prepare a tool */
	hal_s32_t *tool_prep_pocket;/* output, pin that holds the pocketno for the tool table entry matching the tool to be prepared,
	only valid when tool-prepare=TRUE */
	hal_s32_t *tool_from_pocket;/* output, pin indicating pocket current load tool retrieved from*/
	hal_s32_t *tool_prep_index; /* output, pin for internal index (idx) of prepped tool above */
	hal_s32_t *tool_prep_number;/* output, pin that holds the tool number to be prepared, only valid when tool-prepare=TRUE */
	hal_s32_t *tool_number; /* output, pin that holds the tool number currently in the spindle */
	hal_bit_t *tool_prepared; /* input, pin that notifies that the tool has been prepared */
	//tool-change
	hal_bit_t *tool_change; /* output, notifies a tool-change should happen (emc should be in the tool-change position) */
	hal_bit_t *tool_changed; /* input, notifies tool has been changed */
	// note: spindle control has been moved to motion
	} * iocontrol_data; //pointer to the HAL-struct

[andypugh]

Yes, though the lube pins don't exist any more. (They didn't have any useful behaviour)

The tool-change pins are missing from the diagram. (change / changed and number for both tools and pockets). They always have been despite being the main point of emcio.

[zz912]

@c-morley disagrees with me that HALUI should not be associated with GUI

#3580 (comment)

HAL and NML are for two different uses and are not really interchangeable. NML and ZMQ are similar and could be interchanged.

So yes if NML was easier to use, I might have used it instead of ZMQ for this pr.

The reasons for having the GUI communicate to HALUI:

• It's the defacto standard way to connect HAL to a control panel.
• it does not preclude using sinething else, where as if you put it in task you are quite stuck with what we decide.
• it's small/simple enough I can see it through
• it should not break configuration that use the current way, at least for now.
• there is information that task/motion don't know, such as jog rate. Currently if you use HALUI and a GUI then there are two jog rates.

I understand the functional reasons, but I would be interested in others' opinions from the perspective of preserving the architecture.

[rmu75]

If we are going to replace this diagram I suggest to

• make it more correct than the current version and
• use a diagram generation tool like graphviz or mermaid or something the current documentation generation pipeline understands already (ar there any? can somebody in the know comment?) maybe draw.io?
• check in sources of diagram and svg / png formats

[c-morley]

HALUI should not really be in this view.
HALUI and the GUI act the same in this overview.

[zz912]

If we are going to replace this diagram I suggest to

• make it more correct than the current version and

• use a diagram generation tool like graphviz or mermaid or something the current documentation generation pipeline understands already (ar there any? can somebody in the know comment?) maybe draw.io?

• check in sources of diagram and svg / png formats

Do you have a model of what you would like the LCNC architecture to look like? Do you know of any open-source projects with architecture?

Do you prefer graphviz or mermaid?

I have no experience with software architecture, but I am convinced that we need it. At least do it in reverse.

[zz912]

HALUI should not really be in this view. HALUI and the GUI act the same in this overview.

HALUI is part of LCNC. It is an inseparable part of Gmoccapy.

We can discuss:

• where should it be in the architecture?
• how should it communicate with TASK?
• whether and how should it communicate with GUI?
• whether the GUI should also have its own HAL layer?

I am interested in both states. How it is and how it should ideally be.

I want to update this picture mainly because of HALUI. HALUI is a specific component and therefore I would like to know its current function and whether we want to continue using it this way.

The abbreviation HALUI is "HAL User Interface", but at the moment HALUI serves more as an engine for Gmoccapy.

If I removed HALUI from the architecture, I would de facto stick my head in the sand like an ostrich.

Maybe no one will want to discuss it.
Maybe the discussion will not change anything.
Or maybe it will move us forward.

[rmu75]

I didn't find the source of the architecture diagram, I guess at some point in time it originated in xfig. Some translations also translated this diagram and even colorized it. With a vector format, we could do better and make that consistent.

I played around a bit with draw.io, will attach a draft later.

[c-morley]

This diagram is to show the basic architecture of linuxcnc. It is not to show a typical setup. HALUI is equivalent to the GUI and so just adds detail that is not needed.

I would suggest making another diagram that shows a typical setup maybe including Mesa card/ spindle control/ ethernet servo? In this diagram one might show task/motion as just one box called motion controller.

If Gmoccapy requires HALUI to be useful then IMHO a undisearable limiting choice has been made. One should be able to use a different program the HALUI such as panelui or a custom program.

[andypugh]

HALUI is part of LCNC.

It is.

It is an inseparable part of Gmoccapy.

It really shouldn't be....
I was going to argue that GUIs shouldn't have HAL pins at all, but then realised that Touchy, at least, couldn't work without them as it relies on hardware buttons and a jogwheel. Similarly Gmoccapy needs HAL pins for the hardware buttons.

I am curious what Gmoccapy relies on in HALUI that isn't available to it from the Python interface? Maybe this is a throwback to the fact that Mocca was written in Pascal ( https://forum.linuxcnc.org/41-guis/1813-new-gui-for-emc-available-for-testing?limit=6&start=0#1813 ) and we have no LinuxCNC Pascal interface.

[Sigma1912]

@andypugh

There are ideas to use HALUI pins to partially replace native GUI pins.

This discussion might shed some light on the topic:
#3504

[zz912]

Similarly Gmoccapy needs HAL pins for the hardware buttons.

I agree that some of the HAL pins created in Gmoccapy make sense, for example this:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.py

Lines 6188 to 6197 in 781023e

	def _make_hal_pins(self):
	# generate the horizontal button pins
	for h_button in range(0, 10):
	pin = self.halcomp.newpin("h-button.button-{0}".format(h_button), hal.HAL_BIT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._button_pin_changed)
	# generate the vertical button pins
	for v_button in range(0, 7):
	pin = self.halcomp.newpin("v-button.button-{0}".format(v_button), hal.HAL_BIT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._button_pin_changed)

But creating pins like these seems weird to me:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.py

Lines 6271 to 6276 in 781023e

	# make pins to react to tool_offset changes
	pin = self.halcomp.newpin("tooloffset-x", hal.HAL_FLOAT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._offset_changed, "tooloffset-x")
	pin = self.halcomp.newpin("tooloffset-z", hal.HAL_FLOAT, hal.HAL_IN)
	hal_glib.GPin(pin).connect("value_changed", self._offset_changed, "tooloffset-z")
	self.halcomp.newpin("tool-diameter", hal.HAL_FLOAT, hal.HAL_OUT)

For the other pins created by Gmoccapy, I can't determine whether they are there just because of Python interface shortcomings, or whether they have a purpose.

I am curious what Gmoccapy relies on in HALUI that isn't available to it from the Python interface?

I dont know what is available to it from the Python interface.

Here are halui pins in Gmoccapy:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.py

Lines 3797 to 3812 in 781023e

	def _update_halui_pin(self):
	if self.spindle_override != self.stat.spindle[0]['override']:
	self.initialized = False
	self.widgets.spc_spindle.set_value(self.stat.spindle[0]['override'] * 100)
	self.spindle_override = self.stat.spindle[0]['override']
	self.initialized = True
	if self.feed_override != self.stat.feedrate:
	self.initialized = False
	self.widgets.spc_feed.set_value(self.stat.feedrate * 100)
	self.feed_override = self.stat.feedrate
	self.initialized = True
	if self.rapidrate != self.stat.rapidrate:
	self.initialized = False
	self.widgets.spc_rapid.set_value(self.stat.rapidrate * 100)
	self.rapidrate = self.stat.rapidrate
	self.initialized = True

I don't know if this is about using hal signals in the glade file. But they are called hal:

linuxcnc/src/emc/usr_intf/gmoccapy/gmoccapy.glade

Lines 180 to 202 in 781023e

	<object class="EMC_Action_Open" id="hal_action_open"/>
	<object class="EMC_Action_Reload" id="hal_action_reload"/>
	<object class="EMC_Action_Step" id="hal_action_step"/>
	<object class="EMC_Action_Stop" id="hal_action_stop"/>
	<object class="EMC_Stat" id="hal_status">
	<signal name="all-homed" handler="on_hal_status_all_homed" swapped="no"/>
	<signal name="hard-limits-tripped" handler="on_hal_status_limits_tripped" swapped="no"/>
	<signal name="interp-idle" handler="on_hal_status_interp_idle" swapped="no"/>
	<signal name="interp-run" handler="on_hal_status_interp_run" swapped="no"/>
	<signal name="line-changed" handler="on_hal_status_line_changed" swapped="no"/>
	<signal name="metric-mode-changed" handler="on_hal_status_metric_mode_changed" swapped="no"/>
	<signal name="mode-auto" handler="on_hal_status_mode_auto" swapped="no"/>
	<signal name="mode-manual" handler="on_hal_status_mode_manual" swapped="no"/>
	<signal name="mode-mdi" handler="on_hal_status_mode_mdi" swapped="no"/>
	<signal name="motion-mode-changed" handler="on_hal_status_motion_mode_changed" swapped="no"/>
	<signal name="not-all-homed" handler="on_hal_status_not_all_homed" swapped="no"/>
	<signal name="override-limits-changed" handler="on_hal_status_override_limits_changed" swapped="no"/>
	<signal name="state-estop" handler="on_hal_status_state_estop" swapped="no"/>
	<signal name="state-estop-reset" handler="on_hal_status_state_estop_reset" swapped="no"/>
	<signal name="state-off" handler="on_hal_status_state_off" swapped="no"/>
	<signal name="state-on" handler="on_hal_status_state_on" swapped="no"/>
	<signal name="tool-in-spindle-changed" handler="on_hal_status_tool_in_spindle_changed" swapped="no"/>
	</object>

[zz912]

It is an inseparable part of Gmoccapy.

I guess I'm wrong. Gmoccapy uses a function named "def _update_halui_pin(self):" , but it doesn't use halui pins.

Edit:
Definitely I am wrong. Gmoccapy can be loaded without HALUI.

[rmu75]

linuxcnc architecture test.drawio

[zz912]

To rmu75:
Thank you. I will explore drawio as soon as I can. I have currently dedicated more to LCNC than I can and I have a time deficit.

[satiowadahc]

My two cents, while draw.io has been around for a while mermaid or LaTeX might be preferable so it can be maintained with the source code.

[zz912]

I redrawn the diagram in drawio.

linuxcnc.architecture.drawio
LCNC_Architecture_C3 drawio.svg

[andypugh]

I don't think that there is a PID in Motion, the ones I know of are in HAL components. (there is a simple one in stepgen as well as the the standalone PID)

[andypugh]

I don't think that there is a PID in Motion,

(Possibly spindle-synch motion)

[c-morley]

Again, I believe you are adding too much information for what this diagram is for.
This diagram is in code notes, not integrator notes, nor user notes.
Keep it simple as it was and then build a different purpose built diagram for interrogators or users or what ever your target audience is.
I understand the info you are adding is helpful to interrogators but then you need to tailor it to interrogators.
eg. Interrogators do not need detail about NML or task, developers do. Users only need to know HAL and INI. etc

[c-morley]

I don't think that there is a PID in Motion,

(Possibly spindle-synch motion)

spindle sync does not use PID

[c-morley]

I must say though - the drawings do look much nicer!

[rmu75]

That really starts to look very nice -- thanks a lot!. Some comments:

• Task and motion don't talk NML with each other, it is more of a "single producer single consumer" queue between non-rt and rt domains. Status is communicated back via some structure IIRC with no explicit locking, will check that. All NML stuff is strictly in the non-realtime domain.

• PID is in hal

• I think it wouldn't hurt if you could draw an additional rectangle that delimits or groups the interface components (like mesa, gpio, lpt port) from the external stuff like buttons / contactors / ...

• There is a typo in "bus convertor".

[zz912]

Now I'm at the stage where I redrawn the image in drawio and added HALUI there. I consider adding HALUI very important because many Pull Requests are created that do not respect the original concept of HALUI and GUI. I myself was confused about this, thinking that HALUI was part of Gmoccapy. There may be a change, for example, HALUI may be canceled, or HALUI and GUI may communicate with each other, or ...... But now at least we have the current state.

I've done the simplest thing. The difficult part is just beginning.

Chris:

Again, I believe you are adding too much information for what this diagram is for.

Give me a chance to convince you that the amount of information I want to put in this diagram is right. I understand that the more information I put in this diagram, the more work I will have to maintain it. On the other hand, I believe that the maintenance of this diagram outweighs its benefits. I expect that developers will contact me and I will modify the diagram in the future. If I stop enjoying it, it will be easy to simplify the diagram.

Chris:

This diagram is in code notes, not integrator notes, nor user notes.

I agree. I made extra diagram for integrators and begginers. #3738

rmu75:

Task and motion don't talk NML with each other, it is more of a "single producer single consumer" queue between non-rt and rt domains. Status is communicated back via some structure IIRC with no explicit locking, will check that. All NML stuff is strictly in the non-realtime domain.

I would be happy if you could help me with this.

rmu75:

PID is in hal

I thought so, but I didn't solve it during the phase of redrawing the diagram into drawio.

rmu75:

I think it wouldn't hurt if you could draw an additional rectangle that delimits or groups the interface components (like mesa, gpio, lpt port) from the external stuff like buttons / contactors / ...

My goal is to make a diagram where the heart of LCNC is the TASK module and the interface is the HAL layer. This diagram should be intended mainly for developers. Moreover, the boundary from TASK to HAL is always the same, or similar. The boundary from HAL to hardware is unique for each integration. I do not want to draw the boundary from HAL to hardware here, or only very simplified as in the upper part of the figure (non-realtime hardware). I want to delete the lower part of the figure "Example of Hardware". I think that most LCNC applications do not use an encoder on the axis.

If someone wants to draw a diagram for integrators with specific hardware, I can do it, but it would be another diagram tied to this core.

[rmu75]

Task and motion don't talk NML with each other, it is more of a "single producer single consumer" queue between non-rt and rt domains. Status is communicated back via some structure IIRC with no explicit locking, will check that. All NML stuff is strictly in the non-realtime domain.

I would be happy if you could help me with this.

just remove "NML" from the arrows or replace with "FIFO" (first in first out).

I think it wouldn't hurt if you could draw an additional rectangle that delimits or groups the interface components (like mesa, gpio, lpt port) from the external stuff like buttons / contactors / ...

My goal is to make a diagram where the heart of LCNC is the TASK module and the interface is the HAL layer. This diagram should be intended mainly for developers. Moreover, the boundary from TASK to HAL is always the same, or similar. The boundary from HAL to hardware is unique for each integration. I do not want to draw the boundary from HAL to hardware here, or only very simplified as in the upper part of the figure (non-realtime hardware). I want to delete the lower part of the figure "Example of Hardware". I think that most LCNC applications do not use an encoder on the axis.

If someone wants to draw a diagram for integrators with specific hardware, I can do it, but it would be another diagram tied to this core.

don't overthink this, i just meant another rectangle just like the "JOINT 0" group around "encoder counter", "D/A converter" (notice also typo there), I would call that "INTERFACE". and perhaps add "field I/O" and connect the switches box.

I would have done that myself but the .drawio you attached seems to be my initial draft?