Pen Plotter 4 - InkScape/KiCAD drawing toolchain

The first test of the plotter, this one:

was done with the following software toolchain:
I draw that spiral with InkScape, saved as DXF and converted with the DXF2GCODE software from sf.net:
https://sourceforge.net/projects/dxf2gcode/
Tried to use a few gcode sender, but finally kept using the Rasberry PI/Ubuntu/Octoprint set. As I'm quite familiar with it from my 3D Printing past.
The toolhain above has some drawbacks:

• I wasn't able to produce correct scaling. I don't know it is just my inability or problem of the software used.
• The DXF2GCODE is more likely a basic CAM software and not just a plug and play conversion solution. The plotter needs much simpler gcode than a CNC Router, and the functionality of working with tools, pockets unnecessary at this point.
• There is a drawn line between the home point and the start of the spiral what isn't in the original drawing. It is not caused by the toolchain. I'll will come back to this issue later

At this point I was thinking a bit differently. The DXF is a mechanical CAD format, so not really designed for pen plotters, but the HPGL definitely the language of the HP pen plotters.
I searched for a program what is able to convert HPGL to G code. I found a few converters. None of them was fit into my needs. Most of them was not able to convert the HPGL AA arcs to G2, G3 gcodes. This conversion needs some trigonometrical knowledge so it looked hard to achieve for some programmers. After a few trials I gave up to use something I found on the net.
So I wrote one. It is in the github repository of the plotter:
https://github.com/sufzoli/suf-3D-Plotter/tree/master/SW/Hpgl2Gcode
It still has some problems like incomplete error handling and lack of path optimization, but does it's job.
Now let see, how the toolchain works
Here is the original drawing:

First all of the object in the drawing must be converted to path, otherwise it will not represented in the HPGL output:

When it's done, you can save it as HPGL:

Now the bit tricky part coming:
The InkScape HPGL save dialog default parameters are mainly setup for cutting and not drawing plotters. Therefore most of them need to be changed according to the following:

 

Now you have to convert it to G code.
I used two different settings. The reason is the G code of my plotter use M280 P0 S50 command to lift the pen and M280 P0 S0 to put it down.
The usual CNC routers use G1 command to move the Z axis. The widely used G code simulator CAMotics Doesn't understand my pen up and down commands, so for the simulation I changed them to G1 Z5 F50 and G1 Z-2 F50 respectively:

And here is the result in the simulator:

As it looked good, I made the conversion for the plotter:

As you can see, the PenUp and PenDown parameters are missing here. The reason of this that it was setup in the application config file.
After this uploaded it to the Octoprint and sent out to the plotter. The result is disaster. It drawn a 2cm dashed line from the home point towards the starting point of the drawing, then drawn the whole drawing in the air (pen lifted).

I known what is the problem, from the first moment. It is the same problem what I mentioned at the beggining of this post (line from the home point to the beggining of the spiral). The G code commands are not executed in order.

I know about the Marlin firmware that it queue the commands and some commands are executed in order and some out of order. I was reading the source code of the Marlin for a few hours to find, how to change this behaviour Unsuccessfully. I gave up at this point and asked in the Marlin forum.

The answer was much more easier than I ment. Adding an M400 command (wait for finish of the previous commands) before an out of order command solve the issue. I wasn't even need code change for this as my pen hadling commands are represented in the configuration file.

The only trick here that the handling of the multiline string in the .Net config file. So the parameters look like this now:

M400
M280 P0 S0

And finally the result:

The converter I wrote also understand the HPGL output of KiCAD. It can be used for drawing and not PCB milling. This will come later.

Graylog, Elasticsearch, Maximum number of fields, graylog_deflector

Just because I'm not a Linux guy
A few days ago, I setup our shiny new Graylog server.
I successfully added something like 10+ windows servers to collect the event log entries from.
As the second part of the task I wanted to add linux servers also.
I added the first one, it looks like the collector running and the messages are arriving, but no message shown on the web console. Weird.
Digging a bit deeper I found thousands of this message in the indexer fail:
{"type":"illegal_argument_exception","reason":"Limit of total fields [1000] in index [graylog_0] has been exceeded"}
This means we exceeded the maximum field number (I think due to the event types in the Windows).
I read through some forum posts about it. Tried to change the settings in the elasticsearch.yml file.
It didn't help. The result of my actions was a inoperable elasticsearch.
Finally I deleted the whole thing (elasticsearch) together with the indexes, and reinstalled it.
The result:
A working elasticsearch instance. The 1000 field limit kept, and in addition I got a new error. It said something like this: The graylog_deflector is an index and not an alias.
Googling around, I found the problem, but not the solution. Then I was start to think instead of googling. What I've learned:

1.  From one of the log files I learned, that the elasticsearch configuration isn't done through the config file but through the web API with JSON objects.
2. curl is your friend
3. The Graylog creates a graylog_deflector index when it can't find the graylog_deflector alias. What it unable to use. You can't do anything with it from the Graylog, so you screwed.

The solution based above:

1. In the Graylog web UI go to the System/Indices>Indices. Select the Default index set
2. In the Maintanance select the Rotate active write index. It will create a graylog_0 index (but it will not work)
3. Go to the console and stop the graylog:
   sudo service graylog-server stop
4. Handle the 1000 field problem:
   curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
     "index.mapping.total_fields.limit" : "5000"
   }'
5. Stop the graylog_deflector index:
   curl -XPOST 'localhost:9200/graylog_deflector/_close?pretty'
6. Delete the graylog_deflector index:
   curl -XDELETE 'localhost:9200/graylog_deflector?pretty'
7. Add the graylog_deflector as alias to the newly created graylog_0 index:
   curl -XPOST 'localhost:9200/_aliases?pretty' -H 'Content-Type: application/json' -d'
   {
       "actions" : [
           { "add" : { "index" : "graylog_0", "alias" : "graylog_deflector" } }
       ]
   }'
8. Restart graylog:
   sudo service graylog-server start
9. Now the graylog starts the correct reindexing process it can even take days to finish, but you can see your collected messages in the meantime.