• Rxtx Download

I hesitated posting this issue, because I can't provide much in the way of help in figuring out what is the underlying cause. However, I can reproduce it consistently.

I've also posted this as a question to StackOverflow: I've copied the important parts from that post here: I recently switched from RXTX to PureJavaComm. However, I have started noticing seemingly random corrupt byte reads with PureJavaComm on linux. The corruption does not occur when using RXTX. I can switch back and forth very easily and quickly on same computer with same hardware.

Every time, I will quickly see random corrupt byte reads with PureJavaComm (although not immediately or continuously), and no issues using RXTX. I am using the 0.0.13 version of PureJavaComm, and I've been running with multiple variants of RXTX. The corruption does not seem to occur on Windows 7 (32 or 64). The test linux box is a small Zotac server running Ubuntu 12.04 32 bit connected to custom hardware via a usb-serial connector. I open the serial port like this.

String line = buffReader.readLine; // blocks until new line char received The read happens in it's own thread so that I can handle timeouts manually (because setReceiveTimeout never seemed to work consistently across platforms in RXTX). There is also an output stream, but after a few commands sent at initialization, it's not used. The corruption is evident in the line variable.

Sometimes it is a sequence of corrupt (non-ascii) characters, sometimes it is just one character in the middle of the line. I haven't yet determine if it is always the same character (such as a NUL). So far I only know it's corrupt because viewing it in vi shows a '^@' symbol where the character(s) would normally be. I would suspect a serial port timing problem.

However, that doesn't make sense in this case because RXTX seems to have no problem with it. If you have any suggestions on how I might debug this, they would be appreciated. Your thoughts on buffer overflow look potentially relevant. This particular hardware is sending lines of verbose data, but only at periodic intervals.

I'll try ideas from your notes above and report back next week. (Thanks for your pointer on BufferedReader's inefficiency for typical serial port reading. However, at 1st glance I don't see how a more efficient approach can be devised when the protocol is ASCII character oriented and it sends lines of arbitrary lengths in groups at periodic intervals. It does not send a line length value, so I never know how many bytes can be read without potentially blocking until the next interval. I would never write a serial protocol like that myself, but in this case I don't have control over it.).

Some further thoughts: Reading RXTX source code for Windows (termios.c) function readserial it looks to me (meaning I've not actually executed the code so this maybe wrong conclusion) that it ignores VMIN parameter and always reads as much as requested or timeouts. I did not have a look how this is handled at higher levels in RXTX but this seem like violation of how termios is supposed to work.

Be that as it may, this might explain why RXTX behaves 'better' here. BufferedReaderd attempts to read 8192 bytes and so in RXTX it will return from the read when that many bytes have been read or the timeout kicks in. This behavior, though it looks to me like a violation of the JavaComm spec, is much more efficient in of course, it also means that it will block forever if the data flow stops and there is no timeout. If above is correct it looks to me that your code (as one might expect) does not care weather you get the data 'as it comes in' or only after the BufferedReader internal buffer is full, because it looks like you actually get the lines only when the buffer is ful (in RXTX on Windows)l. That being the case you might be able get better performance from PJC if you enable the threshold with say for example value of 100 bytes. You might need to enable the timeout too, but if your data flow is continuos that should no make a difference. I've updated to 0.14, but have same issues.

I think you might misunderstand the nature of the protocol I'm working with, and thus why BufferedReader still makes sense for me to use, and why setting threshold should not have any effect. Trying to state it more clearly: I must read lines of ascii strings from the serial port. The lines are ASCII encoded and are terminated by CRLF characters. I do not have any indication as to length of the incoming strings, so I must read one byte at a time from the serial port, blocking until I finally find the LF character at which point I know I have an entire string and can process it.

With this in mind, I think using BufferedReader makes sense and setting threshold beyond one byte should have no effect. I did try setting threshold to '1' though, instead of just disabling it, just-in-case. If it helps, this is a sample of the lines of character data I'm receiving when the 'corruption' occurs.

In every case, it's as if a new byte gets inserted in to the stream. Once one corrupt byte gets inserted, then subsequent lines are much more likely to also have a corrupt byte. //A few hundred lines might have been read without corrupt characters before this.

On 18.02, kaliatech wrote: I've updated to 0.14, but have same issues. I think you might misunderstand the nature of the protocol I'm working with, and thus why BufferedReader still makes sense for me to use, and why setting threshold should not have any effect. I don't think I misunderstood you.

What I was saying is: I think RXTX works for you because RXTX does NOT work the way it is supposed to. BufferedReader will try to read 8192 bytes from the input stream. RXTX will read that much, although according to the JavaComm spec it should return as soon as any data (one byte) is available.

This makes BufferedReader to work more efficiently as it is not reading stuff one char by one char but in large chunks. And because it works for you with RXTX I don't think your application cares that in practice it is not receiving the lines as soon as they come but only after the buffer is full. You would not notice it the data keeps coming and you just keep reading.

Like I said, I've not verified above by running code, just looked into the RXTX code and BufferedReader code, so I maybe totally wrong. However, if above is true, then you can set the threshold to the maximum (255) and that should increase the performance hundredfold or something. So please try that. Just to further explain and verify this is not related to BufferedReader, I also modified my read code to look like this. I set threshold to 255.

It made no difference except to sometimes block temporarily, likely due to threshold not being met until next batch of bytes were sent from device. This is as I would expect. I also refactored code to use the raw inputstream returned from SerialPort rather than a Reader. It also did not seem to make a difference. I did not expect that it would.

My understanding of BufferedReader and InputStreamReader seems to differ from yours. Regardless, I don't think it's important for this issue. The issue still occurs when using the raw input stream. I agree that my last code snippet is a very inefficient way to read bytes from the serial port. However, at this point the issue is not one of efficiency, but of reliability and correctness. All variants of this code continues to work fine and reliably using PJC on Win7, or rxtx on both platforms. It generates these corrupts bytes and other weirdness only with PureJavaComm on linux.

(I say other weirdness because I have also noticed that the serial inputstream periodically seems to 'block'.which I think in the way my program loops means that inputstream.avail is constantly returning zero until all of a sudden it releases a backlog of bytes. But I'm not postive on that yet. Again, it happens only with linux/PJC.) I will soon be writing a simulator that can simulate the hardware output over virtual com ports. Once I have that, it will be interesting to see if I can still replicate the issue. I just noticed an interesting pattern in the log output whenever the issue occurs that might be helpful in determining root cause.

The pattern does repeat itself, kind-of. Hi, might be educational to just do a simple loop that receives data using nothing but InputStream.read(byte) and collect the results to a large buffer for some minutes and the dump that as hex values. Meanwhile last night I create a simple self contained test case that uses two threads to send and receive messages (200 bytes each) as fast as possible, ie one thread just sends stuff with no delay, no handshake and on other thread reads them with threshold set to message length. You can access the code at: On my 2.6 GHz MacBook Pro running both Mountain Lion and Ubuntu under Parallels virtual machine using FTDI UBS/serial dongle at 230.000 baud I get an average baud rate of 229737 (varies of course) for 500 messages, with no transmission errors. You might try that code to see if you can reproduce your problem with it.

If you can you could then use it as a base for seeing what gets lost and where by capturing the data to a buffer, as I suggest above. I had my program running for the past few days without interruption. What I noticed is that the errors will only start to occur with certain types of messages that the device sends.

(Overnight the device does not send these specific types of messages, and during that time no corrupt bytes were received.) So, the issue would seem to be device specific. The thing I can't explain is why it occurs with PJC, but not rxtx. Regardless, at this point it is not worth troubleshooting further until/unless I can replicate the issue outside of this specific device using a simpler test case. I will try your loop program above, however, I expect if there actually is an issue specific to PJC, it will be somehow dependent on the types of messages and associated timing being sent by this device.

Also, there is some risk in using a VM as you are doing to verify if the issue is repeatable. I've noticed that certain types of serial port problems do not replicate when using a VM, likely due to how the serial port is proxied between the guest and host. I can/will try in VM though to see if the issue still occurs.

If okay with you, I will close this issue for now. And then if/when I am to replicate with a simpler test case, I will create a new issue.

(It might be a while before I have time to try doing so.). I'm okey with closing this, you're welcome to re-open it anytime.

Yes, I know virtual machine is not ideal for testing though it often makes issues worse so it it is kind of worst case test, but always. But that is what I have atm.

Regarding you problem one thing that comes to mind is that it maybe that PJC initialized something differently and thus maybe in your case this makes a difference. There was one issue where PJC did not initialize some field or another and this caused issues with CR and LF being 'eaten' away from the data stream. Is all you data printable ASCII or are there control characters ( less that 20 hex)?

All data sent from the device should be printable ASCII, excluding the CR/LF characters at the end of each line. That seems to be the case when using rxtx. Also, I just used to log the raw data being sent by the device with out rxtx/PJC/java in the loop. After startup (in which I can see the device does send a sequence of NUL bytes), no NUL bytes were seen after monitoring it for a good fifteen minutes.

I then switched back to PJC and NUL bytes where again reported per the above. I think that confirms that PJC is somehow inserting NUL bytes in to the stream, or at least is directly responsible for doing something that causes NUL bytes to get inserted in to the stream. The fact that it only happens with certain types of messages though, in my specific case, is really perplexing. The only obvious difference in the message 'types', is that the problematic messages are a couple bytes longer.

A non obvious difference, and probably more important, is the timing of the messages. The problematic messages come in faster when they are being generated and there is a lot more data being sent for a given time period. So once again, it would possibly seem to be a buffer overflow type issue. However, the fact that it is periodic and happens over a length of time where no problems occur also seems perplexing. I'm getting close to having a simulator working. Once I have that, I'll be able to test a number of data flow scenarios more easily and hopefully I'll be able to replicate the issue consistently. On 8.4.2012 19.37, Alistair Bell wrote: All data sent from the device should be printable ASCII, excluding the CR/LF characters at the end of each line.

That seems to be the case when using rxtx. Also, I just used to log the raw data being sent by the device with out rxtx/PJC/java in the loop. After startup (in which I can see the device does send a sequence of NUL bytes), no NUL bytes were seen after monitoring it for a good fifteen minutes.

I then switched back to PJC and NUL bytes where again reported per the above. I think that confirms that PJC is somehow inserting NUL bytes in to the stream, or at least is directly responsible for doing something that causes NUL bytes to get inserted in to the stream.

Rxtx Download

Good to get confirmation on that though I never doubted that. The fact that it only happens with certain types of messages though, in my specific case, is really perplexing. The only obvious difference in the message 'types', is that the problematic messages are a couple bytes longer. A non obvious difference, and probably more important, is the timing of the messages.

The problematic messages come in faster when they are being generated and there is a lot more data being sent for a given time period. So once again, it would possibly seem to be a buffer overflow type issue. I agree that it must something like that. However, the fact that it is periodic and happens over a length of time where no problems occur also seems perplexing. I'm getting close to having a simulator working. Once I have that, I'll be able to test a number of data flow scenarios more easily and hopefully I'll be able to replicate the issue consistently.

Good, look forward to getting some more data to chew. One thing you could try would be to hack the PureJavaComm.InputStream.read to full fill the buffer with some known non zero value and the InputStream internal buffer with some other non zero value so that you could see if it is PJC or the OS that fills the buffer with zeros. Have you tried a pure read(byte) approach for testing, meaning getting as little machinery between your code and PJC code? One thing that comes to mind is that if PJC somehow would return wrong number of bytes (too many) then this might cause cause something either in your code or BufferedReader to 'invent' zeros into to stream. I'm closing this issue for now because:. The comments above got too lengthy to read through.

I'm still not 100% this is a PJC issue. I don't have the time to dedicate to debugging further at this time. To restate the underlying issue more simply:. It seems that PJC is inserting 0x00 (NUL) bytes in to the inputstream in certain use cases while running on linux. I really would prefer to use PJC over rxtx as it just seems like a fundamentally better approach. When I get some time, I will try generating a simple standalone test case to replicate the issue.

If successful, I'll open a new ticket.

LucidControl goes JAVA I was a bit of surprised a few weeks ago that Java has only a very limited native access to the serial comports. We use the serial ports to communicate with our. These modules have analog and digital IO functionality and they are compatible with USB CDC (Communication Device Class) which means that most operating systems provide a driver which installs a virtual comport. Microsoft Windows enumerates them as a COMxx device. Linux operating systems give the devices usually names like /dev/ttyACMxx where xx is the identifier number of the port.

I’m familiar with C# which provides the SerialPort class. This is supported by dotNet and also by Mono and runs under Linux as well. I used this class several times for portable applications running mostly under Windows and Linux. Because of this I expected easy access to the serial port under Java as well and I didn’t waste much thought on doing such a common task in Java. But in reality Java does not support them – at least under Microsoft Windows. In the following I will explain which options are available in order to access serial ports in Java.

CommAPI – Limited native support The provides an interface to access serial ports. This sounds very good, but has one significant drawback because support for Microsoft Windows was dropped years ago and it is not possible to write portable applications anymore. Because of this severe restriction I dropped this solution without further investigation. RXTX is the first option which comes one in mind when talking about alternatives of CommAPI. It is a comprehensive and reliable interface which is used in many projects. I used this in our pre-release revisions of JLucidIo Library.

Which implements classes encapsulating the functionality of our LucidControl USB IO Modules. I thought this was a good option because it is relatively easy to install and it is working reliably.

So why didn’t we go for it? This has mainly three reasons:.

RXTX is not pure java. The hardware dependent part is written in a different language. For Windows the hardware dependent layer is called rxtxSerial.dll and you can see that this part is not Java code. Maybe it was written in C, but I didn’t research this further. This makes it necessary to install different software on different computers which is not what a Java programmer prefers.

I find the API of RXTX needs getting used to. It is different to the SerialPort class in C# and the implementation has one severe drawback. In C a port can be opened by using fopen(comportName) and the comport must be handy when opening the device which is the normal way to do it. RXTX uses a different approach.

Before a port can be opened ALL ports must be enumerated and a list of all serial ports is created. This is done by calling CommPortIdentifier.getPortIdentifiers which isn’t the best idea in my eyes. Enumerating all serial ports by default is pointless when you only intend to open one specific port. And the implementation in RXTX is done a resource consuming way because it makes the operating system to create this list by querying all ports. This may take a long time especially when Bluetooth is involved. The operating system may ask all (Bluetooth) devices for their ability which can take several seconds per port! Have you counted the number Bluetooth ports of your computer?

I have to admit that this behavior was only reported for Microsoft Windows, but in my case it took approx. 5 seconds to open a comport. For my tool I couldn’t accept this. Last but not least the development of RXTX seems to be neglected since 2009.

After all these findings I continued my search and found jSSC eventually. JSSC (Java Simple Serial Connector) The is available in version 2.6.0 which is dated on June 2013. It consists only of one JAR file which has to be included to your project.

Like RXTX jSSC has also a OS dependent part but all is included in a single JAR and you do not have to care about the system you want to use your software. JSSC provides a very simple API. In my case it took less than 30 minutes the replace RXTX by jSSC and it worked out of the box under Microsoft Windows, Ubuntu and on the Raspberry Pi.

There is no more to write because it does exactly what it should – not more not less 😉 Conclusion I cannot say much about CommAPI but RXTX and jSSC are worth a look. RXTX is a good approach but because of the reasons I explained it was not the right library for our project and we went for jSSC.