The search for lower latency

[msimpson99]

Once I got a couple of the THC systems I purchased up and running, I decided to do some research into the latency issues. Seems everyone has a different opinion as to what to use and how to solve the problem. There are actually several approaches you can take.

My first challenge was a way to measure the latency. While there are several kinds of latency, the one I am looking at right now I call "Command Response" IE how long it takes once the THC as fired an up or down signal before the Z axis actually responds.

To perform my tests a added a gopro to my Z axis recording some remote LEDs I hooked up to the output of the THC. It also records the Z axis movement. The GoPro is recording at about 8ms per frame so I figure my actual time resolution is about 17ms just to be safe.

_MG_7556.jpg

You can see my full writeup here:
https://www.kronosrobotics.com/searchin ... ow-latency

The results were very enlightening. I was able to several interfaces. Originally I was getting a Command Response latency of 100-200ms. The issue was the AMD PC I was using. While it was perfectly fine for CNC router operations, it would not work for plasma cutting with a THC.

Once I moved to a faster Intel PC things just kept getting better. I tested the following three interfaces on this PC using Mach3.

Mach3 Tests
UC100 USB interface
UC300 Ethernet interface
Parallel Port
Here are the averages I came up with on these earlier tests.

1st Place: Parallel Port at 21.5ms
2nd Place: UC100 at 38.7ms
3rd Place: UC300 at 55.3ms

I also test the Ethernet SmoothStepper, but had timing issues with my Anti-Dive control signal.

Next I tested the same cuts using UCCNC software.

UCCNC Tests
UC100 gave me an average of about 30ms
UC300 was the clear winner of all the tests at 16.6ms. It is obvious that the UC300 is moving the motors without going back to Windows 7 or the UCCNC software. I also suspect the results would be much better as I was down below my actual test resolution.

Now that I have some reasonable Command Response latency, I started playing with other settings.

Here is a little slow motion video I made showing THC Oscillation.
https://youtu.be/7i8XPznnA6g

I have more tests to perform and some actual projects to test my UCCNC setup. I will keep you posted.

[robertspark]

On the UC and smooth stepper you can adjust the kernel (internal) loop frequency which should improve the z axis response.

The parallel port will have variance as it's timing will not be consistent as it depends what the pc processor is doing and it's load.... Also is it running GPU as well as processor applications on the one chip using shared memory

Suggest using a complex (lot of changes in direction) gcode test file ant then trigger up / down and then check latency.

Plus try it on different pc processors too.... And you'll get different results.... The motion controllers should be consistent

Also check if a denounce setting has been set for the ESS as this will affect response too.... But it should be consistent

Interesting test results though

[msimpson99]

The parallel port was never really an option. I did it only for comparison. While I have several PC's that have them, and you can add cards to desktop machines, they are pretty much deprecated. Motion controllers are the only way to go now. Or some sort of IO card if you are going Linux.

Mach3 is pretty much a no go for THC use, unless you are cutting slow, I can cut 1/4" steel all day long using my AVHC10 and Mach3 and the UC100. Once I move to the UC300ETH, its interface is twice as slow. Where things go south fast is when cutting thin sheet at high feed rates. Mach3s latency just cant keep up. Ironically thin sheet is where you need the THC the most.

AS I stated, I did try several different PCs. Originally I was getting latencies as high as 200ms. This was an older machine running an AMD processor. All my tests improved to below 50ms or so when I moved to faster Intel processors.

The AVHC10 has the ability to stop THC operations instantly using its Anti-Dive control lead. I really like this. This give you two ways of controlling the THC. You can turn on and off the THC system as a whole, or (using the ANtiDive signal) suspend the up/down signals from the AVHC10. However the AVHC10 manual has you use the UCCNC softwares antidive feature. While it does work, I also like the flexibility using rules in SheetCam. Yes you can use the M205 and M206 macros with UCCNC to turn on/off the THC. I dont like this. As an M205 in my gcode will turn on the THC system where I did not want it to. UCCNC software can do syncronous output using the M10.X and M11.X commands. So I am using those to activate the Anti-Dive and using the M205 and M206 for global control. IE turning THC operations on and off as separate operation.

The problem I had with the ESS was I could not get a synchronous output to work. IE the M10 and M11 coomands would actually fire before they were supposed to. I really wanted the ESS to work as I could still use Mach3. For someone who has been using Mach3 from the start its hard to switch to something else. I do have some experience with UCCNC but there are things I just have not been able to get working correctly.

I am about ready to start my MiniTHC testing, but now have enough information to jump start those tests.

[beefy]

msimpson,

you may be interested in this topic on the UCCNC forum.
https://www.forum.cncdrive.com/viewtopic.php?f=4&t=1740

My user name is "beefy" on that forum.

I think servo motors with closed loop PID control and a relatively high sampling rate, is the only decent way to get accurate and fast THC response for cutting thinner metals at high speed.

"PlasmaC" which is the latest dedicated Linuxcnc plasma setup, is now supposed to have internal PID control of THC, so measuring latency in that system may not be possible. It may simply be better to use a welding lens and video the torch nozzle as it travels over some warped sheet, seeing how constant the gap stays.

And latency is not the only factor in THC responsiveness. The plasma table gantry and Z axis need excellent rigidity so the inertia of fast Z axis direction changes, and XY direction changes, do not cause "flex" anywhere that will alter the distance of the torch to the plate. My table was knocked together from a lot of scrounged bits, and I also built my gantry too high. You can actually see it flex slightly under sharp direction changes and this will not doubt affect the torch gap, causing the THC to respond accordingly. No big deal at slower cutting speeds but it wouldn't be good for thin metals.

[msimpson99]

I have three machines. Two CNCCS machines that were designed with plasma cutting in mind. While they are smaller machines, they are capable of 600IPM and an acceleration of .3Gs. Perfectly suitable of THC base plasma cutting. My other machine is a larger KRMx02 capable of 2000IPM at .3Gs as well.

The UCCNC software is probably the way to go as it can really push my machines and allows the UC300ETH to control the Z drive directly. Its the lack of quality documentation that make it a harder choice. The Plasma options are not even in the docs. I also have dual drives with dual homing switches so that I can electronically square the XY axis. Works great on MAch3 but have not been able to get it to work with UCCNC. The UCCNC software and the UC300ETH is probably what I am going to go with at least with the AVHC10. With the MiniTHC and CAP04, the vote is still out on those.

[robertspark]

To home / square two motors on the same axis with uccnc you need two home or limit switches

[robertspark]

Minithc? What's your issue

[msimpson99]

Not understanding your comment. I have no issue with the MiniTHC, at least not yet, as Its hooked up, but untested.

I have two separate switches on my Y/A axis. I'm sure the issue here is that I don't have something set correctly in the software. I'm sure I will get it working eventually. For now I just disabled the the A-axis switch and it homes fine. Just no auto squaring.

[robertspark]

Not understanding your comment. I have no issue with the MiniTHC, at least not yet, as Its hooked up, but untested.

It came from this statement

.... With the MiniTHC and CAP04, the vote is still out on those.

[robertspark]

I have two separate switches on my Y/A axis. I'm sure the issue here is that I don't have something set correctly in the software. I'm sure I will get it working eventually. For now I just disabled the the A-axis switch and it homes fine. Just no auto squaring.

This post should be of help for you

http://forum.cncdrive.com/viewtopic.php ... axis#p2472

[msimpson99]

Thanks, I got it working. For some reason my stops were a little too close to the homing positions. I backed the stops out about 1/16" and this allowed the Y/A axis to do their thing.

[msimpson99]

What I meant by the statement:

.... With the MiniTHC and CAP04, the vote is still out on those.

I have it hooked up, but not yet tested it. IE the vote is still out. Nothing good or bad...

Hook up was easy enough I just need some time to put it through its paces.

[Jgorm]

Interesting. I'm still building mine but i went with the tmc3in1 THC on the ess with Mach 4. I think it's supposed to eliminate the latency with the direct motion controller interface but i have to get it working first!

[Rodw]

I'm sorry but the whole concept of latency in a THC is so foriegn to me as a LinuxCNC user. Well in fact a THC is foreign to me as a LinuxCNC user. I've always done THC internally within Linuxcnc which responds to changes in torch voltage in one millisecond. Yes thats right, LinuxCNC adjusts torch height 1000 times a second. Torch voltage sampling, velocity antidive, void crossing, reliable ohmic sensing on a water table, its all there. But more to the point its now a complete plasma controller with so many other features to mention. I love sheetcam but I never bother with cuttng rules, everything just works out of the box with Linuxcnc becasue of the sophistication it applies to plasma control. Yeh it takes a bit to set up but its worth it in the end. BUt we are working on making it easier....

The parallel port was never really an option. I did it only for comparison. While I have several PC's that have them, and you can add cards to desktop machines, they are pretty much deprecated. Motion controllers are the only way to go now. Or some sort of IO card if you are going Linux.

With LinuxCNC, this is not true. There are a number of people having success with a parallel port setup with Linuxcnc and a USD $69 Mesa THCAD board and a software encoder input to manage the THC function internally. Some in Europe are actually building such machines for sale. Sure they won't get the performance you might get from a higher specced motion card, they still get good results because there is no latency with LinuxCNC.

Its actually pretty impressive to see a gantry running at 50 m/sec with 2.8 amp stepper motors under Linuxcnc control... but don't expect that with a parallel port.

[robertspark]

Rod,

most (from my experience) THC's also operate on a 1khz sample loop.
miniTHC https://minithc.com/
razordance http://www.razordance.co.uk/THC.htm

The neuron samples higher than this (500uSec >> 1/2 the time or double the speed of linuxCNC) and is stand alone. https://neuroncnc.com/products/002

There IS latency with linuxcnc, yes it may be small and depending upon whether you are using a mesa (motion) controller or parallel port depends whether linuxcnc uses the 25uSec loop or the 1mSec loop.
http://linuxcnc.org/docs/html/install/latency-test.html

the fastest step rate you can achieve with a parallel port and Linuxcnc is 40khz (25mSec loop) ..... and yes it DOES have latency..... afterall linuxcnc is TRYING to run loops on a PC (microprocessor with many peripherals) which is NOT a mincrocontroller which can operate on an interrupt driven loop, and therefore have NO latency. linuxcnc has "jitter" (latency) because it TRIES to run a constant loop.... but the only way that it appears to be able to do this is via a polling loop given the error.

http://wiki.linuxcnc.org/cgi-bin/wiki.pl?Latency-Test

I'm not criticising Linuxcnc.... its good at what it does and does cheaply... but it is supported by volunteers (who's enthusiasm for support may wain over time )..... for example, while I don't know the full story or the accurate story, I was lead to believe that linuxcnc benefited from when Tormach switched from (I believe) Mach3 to linuxCNC and kindly invested in (rewrote / altered / improved) the trajectory planner and kindly (it is suppose to be open source as part of the linuxcnc licence) back shared (pushed) the trajecroy planner back to linuxcnc which (apparently) has improved linuxcnc.

http://microfabricator.com/articles/vie ... -pathpilot

but I do wonder long term how things will pan out with volunteers / enthusiastic users over time..... but then main stream manufacturers shut up shop too and also can update their hardware and software and refuse to support the older versions of it or when its sold onwards to a new customer

I have digressed..... but the point being is it DOES have SOME latency but it is small and it is not quite REALTIME but very close too.... or as close as is possible on a PC which is not a microcontroller.

[Rodw]

Robert sorry, a few things wrong there.
1. The only thing that runs in the faster base thread is the step generators
2. Latency with Linuxcnc is due to the hardware and how it works with the Operating system
3. At the hardware level, hardware interrupts and timer interrupts still exist on any PC. Linuxcnc runs a timer interrupt for the servo thread. Ive used them on Windows from the DOS days. But today they are not real time and not guaranteed to run on time in Windows.
4. If the latency at the OS level causes LinuxCNC to run late on one of the threads, it will trigger an error. This should never ever happen. So there is no latency that affects those threads.
5. If you run a different board with LinuxCNC (Eg a Mesa board), there is no need to run the fast base thread so there is less chance of triggering a latency error.
6. The THC in LinuxCNC has an inate advantage to all other external hardware becasue it is so closely coupled to the motion controller. It can act on an input that very millisecond.

Yes Tormach paid one of the lead developers to improve the motion controller and they were obliged to publish that work under the open source model. I know of at least one manufacturer considering funding further improvements to the trajectory planner.

Probably the unfortunate thing for Linuxcnc is that they don't have a sponsor or a commercial revenue stream like many open source projects. BNut there semes to be a bit of a changing of the guard now So I hope the dev team becomes a bit more revitalised. Ther has actually been a number of improvements just recently that I think will be very helpful moving forward but they are buried fairly deep so not visible to the average user.