Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.
Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).
Dear friends!
It has been a long time since we released an update for IO Ninja -- more than eight months! Such a delay happened because we were working hard on the new major release, IO Ninja 5. And I'm happy to announce that it's finally out!
The most significant change from the previous generation of IO Ninja is the introduction of accounts, capabilities, and subscriptions. Please read the intro on our website to learn more about our motivation for such a major change. The dedicated section of the user manual covers this new ecosystem in more detail.
The IO Ninja 5 also features two new plugins dedicated to helping web developers debug modern WebSocket-based applications. Please read more about these plugins on the respective pages: WebSocket Client and WebSocket Server.
IO Ninja 5 includes a critical update of the Device Monitor Service for Linux that enables running the Serial Monitor on modern Linux kernels. There's also a major clean-up of UI for the macOS version, critical bug fixes and optimizations in the logging engine, and plenty of other important updates and improvements.
Besides the software updates, with IO Ninja 5, we introduce a new e-store that seamlessly integrates with the main website and provides a much more smooth experience. And of course, we -- finally! -- opened this forum, just as so many of you suggested. Here, you can ask questions, share your suggestions on improving IO Ninja, and receive support -- both from the IO Ninja Team and other users.
We worked with passion and dedication to release this new version of IO Ninja. We are sure that IO Ninja will help you in your work, and you will enjoy using it!
Currently, IAS (in-app-script) can only control the underlying session via:
connect()
disconnect()
transmit(p, size)
Fine-tuning of the sessions configuration is currently not possible from IAS. But we plan to add this capability in the near future.
Each session class will export a public IAS interface (e.g., a serial session will have properties to query and control baud rate, parity, status lines, etc.) Then, IAS should be able to access this interface via a global constant g_session (or something like that).
g_session
Hello Bartosz,
The script for a filter to do what you want is very simple. First, you deduce the state of the CS line (from the I2cSpiTapLogRecordCode.SpiStart and SpiStop log records); then, you use this state to either hide or show the MOSI/MISO data (the log.StdRecordCode.TxRx log records).
I2cSpiTapLogRecordCode.SpiStart and SpiStop
log.StdRecordCode.TxRx
The source code for such a filter might look something like this:
import "doc_Layer.jnc" import "I2cSpiTap/I2cSpiTapLogRecordCode.jnc" class SpiCsFilterLayer: doc.Layer, log.Filter { protected: ui.EnumProperty* m_csFilterProp; // a property to choose the filtering strategy int m_cs; // the state of the CS line public: construct(doc.PluginHost* pluginHost); override bool filter( uint64_t timestamp, uint64_t recordCode, void const* p, size_t size ); } SpiCsFilterLayer.construct(doc.PluginHost* pluginHost) { basetype.construct(pluginHost); ui.EnumPropertyOption csFilterOptions[] = { { "Show always", -1 }, { "Show when CS low", 0 }, { "Show when CS high", 1 }, } m_csFilterProp = m_pluginHost.m_propertyGrid.createEnumProperty( "MOSI/MISO filter", "Show MOSI/MISO filtering criteria", csFilterOptions, countof(csFilterOptions) ); m_cs = -1; // -1 means unknown pluginHost.m_log.addFilter(this); } bool SpiCsFilterLayer.filter( uint64_t timestamp, uint64_t recordCode, void const* p, size_t size ) { bool isVisible = true; switch (recordCode) { case log.StdRecordCode.SessionStarted: m_cs = -1; // reset to unknown break; case I2cSpiTapLogRecordCode.SpiStart: m_cs = 0; break; case I2cSpiTapLogRecordCode.SpiStop: m_cs = 1; break; case log.StdRecordCode.TxRx: isVisible = m_csFilterProp.m_value == -1 || // show always m_csFilterProp.m_value == m_cs; // matches the current state of CS break; } return isVisible; }
An archive with the complete filter plugin is attached (SpiCsFilter.7z)
Usage:
SpiCsFilter.njplg
A little bit late for a follow-up, but still Which OS are you running on? Just FYI, ioninja-5.2.0 now picks up the system-wide dark theme on macOS and Linux with KDE desktops. If you running one of those, you can try the newly added dark theme support io IO Ninja. If not, stay tuned, as we do plan to put some extra effort and allow switching IO Ninja to dark mode at the user's will.
Hello,
Apologies for the delayed response. Support for njlog output directly from ioninja-hwc is on our TODO list (it was already requested by users). For the time being, you need to post-process the raw output of ioninja-hwc using any scripting language of your choice.
njlog
ioninja-hwc
The protocol of communication between ioninja-hwc and plugins such as Serial over SSH or Serial Tap over SSH is not currently documented, but the decoding process and all relevant constants and data structures can be looked up in the sources of those plugins (remember, all IO Ninja plugins are open-sourced and available at ioninja/scripts/plugins).
ioninja/scripts/plugins
In particular, all the relevant constants and packet structures for the ioninja-hwc protocol are contained in ioninja/scripts/common/io_HwcProto.jnc. The output file generated by ioninja-hwc is basically a sequence of "out" messages of this protocol. Each message starts with HwcMsgHdr followed by extra data block, the meaning of which depends on the message code. For example, HwcMsgCode.Rx is followed by the received bytes, HwcMsgCode.SerialTapCtsDsr is followed by HwcSerialStatusLines flags, etc.
ioninja/scripts/common/io_HwcProto.jnc
HwcMsgHdr
HwcMsgCode.Rx
HwcMsgCode.SerialTapCtsDsr
HwcSerialStatusLines
Try decoding the output of ioninja-hwc using any language of your choice, and feel free to let me know if you run into a stumbling block or have any other questions.
Thanks for the feedback!
Auto-baud rate could be convenient, but when used in a sniffer, there's a quirk -- detecting baud rate can't be 100% reliable without prior knowledge of the incoming data stream. Serial protocols that support auto-baud rate always use specific packet headers that allow reliable detection of the baud rate for this exact reason. So, a sniffer with auto-baud rate detection (which can't make any assumptions about the underlying data) could initially produce incorrect bytes before it deduces the actual baud rate.
All that said, yes, we do consider adding the automatic baud rate detection in the new generation of our Serial Tap devices.
Not exactly a bug but rather a missing misuse protection.
So, what's happening? The automatic calculation of Modbus TCP frames is done like this:
void updateModbusTcpLength(void* p) { size_t size = dynamic sizeof(p); if (size < sizeof(io.ModbusTcpAduHdr)) return; ((io.ModbusTcpAduHdr*)p).m_length = size - offsetof(io.ModbusTcpAduHdr.m_deviceAddress); }
Basically, m_length is set to the length of the payload (from m_deviceAddress and to the very end of the packet).
m_length
m_deviceAddress
In your case, the packet length is 17, offset of m_deviceAddress is 6, so m_length is set to 11. However, the size of Modbus TCP read should always be 12 (not 17). If you erase the trailing 5 bytes from your packet, it will work as expected.
Modbus TCP read
That should solve the issue; for more relevant details, read on.
By design, packet templates never allow creating a packet shorter than the template itself. However, it's allowed to add any suffix after the templated headers (that would be required for Modbus TCP write, for instance).
Modbus TCP write
In ioninja-5.1.0 we added the fixedSize attribute exactly to prevent unintentional misuse in case of packets that should never grow longer than the template itself.
ioninja-5.1.0
fixedSize
Modbus TCP read is exactly this kind of packet, but in the current version of IO Ninja, it's missing the fixedSize attribute. We will add it in the next release of IO Ninja, but in the meantime, you can fix it by yourself. To do so, please open ioninja/scripts/packets/ModbusTcp.jnc in any text editor, locate the definition of ModbuTcpReadPacket (in the very beginning), and modify it as such:
ioninja/scripts/packets/ModbusTcp.jnc
ModbuTcpReadPacket
[ displayName = "Modbus TCP read", fixedSize ] struct ModbusTcpReadPacket { // ...
In order to apply it, please reload this template into the Packet Template editor (Edit packet template -> Load stock script -> Modbus TCP).
Edit packet template
Load stock script
Modbus TCP
Let me know if this works for you.
Hi Massimo!
Indeed, the filter is currently implemented in the session only -- it should be transferred to the log plugin instead; this way, it will be there when you open a standalone log. We definitely will fix this in the upcoming release (scheduled for May). In the meantime, here's a workaround:
Create a fresh Ethernet Tap session and save it (let's say, in $HOME/ethernet-tap-1). This will create 3 files under $HOME/ethernet-tap-1:
$HOME/ethernet-tap-1
ethernet-tap-1.njssn
ethernet-tap-1.njlog
ethernet-tap-1.njcfg
Close the session. Then, using any file manager, copy your original log file over ethernet-tap-1.njlog. Re-open this session, and it will use the original log file. Now filtering packets will be available -- together with all other actions of the Ethernet Tap session (e.g., capturing more data and appending it to the log).
Hope this works for you.
Once again, the issue with not being able to filter standalone Ethernet Tap/Pcap Sniffer logs will fixed in the next release.
I think it would be a very helpful feature to add to IO ninja to just be able to save a log as a raw binary file for each direction.
It's very easy to achieve with a log filter/observer plugin script.
However, I can't say I find it very useful to have two raw binary files containing just the raw bytes flowing in each directing -- how would you reconstruct the mapping between requests and replies in case of some non-trivial conversation between the client and server?
I often have serial port logs that I just want to save to a bin file as-is, and I end up closing IO ninja and just going back to something like CoolTerm which supports this natively.
If it's a uni-directional log kind of thing (e.g., your device keeps dumping debug printfs to a RS232, and you want to save it to a file) -- this can be done using Session Linking. Open a Serial session, open a File Stream session in Write-only mode and point it to some file, then "link" those sessions. Everything that the Serial session reads from the device, will be redirected to your file.
it also supports recording directly to a file, in case the computer crashes or powers off during logging
Actually, IO Ninja always keeps its logs in disk files. When you "Save Log", the temporary .njlog file simply gets copied into the file you select. But if you "Save Session", those .njlog files will get stored in the specified folder, and you will know exactly where to find them in case of a crash.
.njlog
But once again, a dedicated plugin that would explicitly save RX to one file and TX to another is quite easy to implement. I don't know, maybe we should add it to one of the upcoming releases as an official plugin (or a sample)?
Try the updated analyzer:
BacNetMsTp.7z
It checks CRCs now and discards frames with broken headers (in the previous log, the decoder didn't actually stop, but because a broken header specified a very long payload size, it kept buffering data assuming it still was a payload).
Also, when you upload the log, please detach the Analyzer before saving the log. This way, we can access the original raw & unprocessed data (instead of the already decoded frames).
No prob, I've increased the upload file size limit to 8MB.
Also, you could have archived it with 7z
7z
Apparently, this was a permission issue; it doesn't apply to administrators, so I didn't even know it was there...
Anyway, I've adjusted the file upload permissions for registered users; please try again.
Tried using the upload images(right most icon) to send a 7z zip file but I don't have privileges for it.
Not "Upload image", "Upload file". Does your "Reply toolbar" have this button?
How do I send you the log that is out of sync
Hit "Reply" (not "Post quick reply"); the rightmost button on the toolbar is "Upload File". 7-zip it first so that it has the .7z extension permitted by this forum.
.7z
Removing the padding gives this trying to load the Analyzer:
Most likely, you just commented out the padding code, but forgot to add a semicolon ; at the end of the statement. In Jancy, just like in C, declarations and expression statements must be terminated with a semicolon.
;
While using this analyzer should we have been running Settings->Log Engine->Binary Data->Binary Data Merge with a 3msec threshold.
This won't affect the analyzer, but I think it's completely unnecessary now. You will see packets boundaries anyway -- and not just boundaries, decoded header fields, too.
P.S. For simplicity, just sync on 0x55; that should be good enough.
0x55
In BacNetMsTpParser.parse(...), add this:
BacNetMsTpParser.parse(...)
while (p < end) { void const* p0 = p; // 0. sync on 0x55 if (!m_buffer.m_size) { void const* preamble = memchr(p, 0x55, end - p); if (!preamble) break; p = preamble; // skip everything before the preamble } ...
Version 5.5.0 will not display the contents of BACnet headers properly (after you click [+] to expand). The pre-release .7z file is a portable archive; extract it anywhere and run ioninja-5.5.1\bin\ioninja.exe.
ioninja-5.5.1\bin\ioninja.exe
Re "stop" of the analyzer -- could you share the original .njlog with the large packet?
I suspect that the reason is the incorrect handling of padding in the script (it went out of sync immediately after the large frame). Try removing the padding:
size_t frameSize = sizeof(BacNetMsTpHdr) + // header hdr.m_length + // payload sizeof(uint16_t); // CRC // (hdr.m_length & 1); // padding <---
Also, it makes sense to wait and sync on \x55\xff before starting buffering the packet; this way, the parser will re-sync after synchronization is lost (this still can occasionally happen because RS485 does not guarantee lossless delivery).
\x55\xff
P.S. FYI, just as with any representation generated by log.representStruct(), you can click on a header field, and the corresponding data bytes will be highlighted below -- so that you can see the mapping between header fields and data bytes.
log.representStruct()