Catalyst Development

Re: TelnetClient disconnects at random

Bart Brown — Tue, 06 May 2008 14:23:32 GMT

Author: Bart Brown
Post subject: Re: TelnetClient disconnects at random
Posted: Tue May 06, 2008 7:23 am

Thanks Mike,

This reduces to a flaw in the documentation about behavior when Blocking.

From the help file about the Read Method:
"If the control is in blocking mode, the program will stop until data is returned by the server or the connection is closed."

From the help file about the Blocking Property:
"If set to True, then each control action, such as sending or receiving data, will return when the operation has completed or timed-out. "

The documentation is not quite correct. Well, at least the 'when the operation has COMPLETED' is not correct. In truth, when Blocking, when as few as one character is in the buffer, processing will continue. There is no relation/correlation to the operation completing. The 'secret' to coding with Blocking is to loop after every Write operation -- checking for something in the buffer that indicates the work is complete; like the next OS prompt.

I understand now how to get this to work. Thanks again for the clarification about the control.

-Bart

Re: TelnetClient disconnects at random

Mike Stefanik — Tue, 06 May 2008 00:23:20 GMT

Author: Mike Stefanik
Post subject: Re: TelnetClient disconnects at random
Posted: Mon May 05, 2008 5:23 pm

Actually, the Read method will return immediately upon receiving any data. Although knowledge base article 100243 was written specifically for our TCP library, the same general principal applies here as well. The Read method will never guarantee that you've gotten the "complete" amount of data. It will only guarantee that it will either return at least 1 byte, or it will indicate an error and/or the connection is closed. The reason that it's working in debug mode, or when waiting for a while, is because you're waiting long enough for the buffer to accumulate with data. In other words, when you call Read asking for a specific number of bytes, you're specifying a maximum, not a minimum. Of course, with blocking mode, you're also introducing a whole host of other issues if you're trying to do things with multiple instances of the control. I'm afraid that classic VB is just not suitable for that particular scenario because if you want mutliple blocking sessions, they must be in different threads.

Re: TelnetClient disconnects at random

Bart Brown — Mon, 05 May 2008 21:49:27 GMT

Author: Bart Brown
Post subject: Re: TelnetClient disconnects at random
Posted: Mon May 05, 2008 2:49 pm

Thanks Mike,
But this does not resolve either problem:
1) blocking=true, read drops output from server
2) blocking=false, onRead does not fire consistenly

For now, consider on the 1st problem: behavior when blocking.

In debug mode stepping through the code things will work. But in run mode, things are lost. Consider the sample below. Here the task is simple, run a command (uname -r) and take action depending on the result. The server responds with "V5.1-ELS...", but the mstrBuffer contains two non-printing characters. If I run this in debug mode, mstrBuffer contains the correct value from the server. It behaves as though the Read happens before the server is done with the response ... so I only get a portion of the response. If I put a 'sleep' after the telRemote.Write, I get the full response during the Read. If I understand the documention, Read should not pull back reponse until the operation has completed or timed-out. Therein lies the rub. Why does Read not wait for the operation to complete?

telRemote.Timeout = 10
iWritten = telRemote.Write("uname -r" & Chr(13))
lstArray.AddItem "uname -r: " & iWritten
DoEvents
iRead = telRemote.Read(mstrBuffer, 4096)
If iRead > -1 Then
If (InStr(mstrBuffer, "V4.0") > 0) Then
mstrTemipVersion = "V4.0"
iWritten = telRemote.Write "cat /usr/temip_licenses/licenses.dat" & Chr(13)
ElseIf (InStr(mstrBuffer, "V5.0") > 0) Then
mstrTemipVersion = "V5.0"
iWritten = telRemote.Write "cat /var/opt/temip/licenses/licenses.dat" & Chr(13)
ElseIf (InStr(mstrBuffer, "V5.1") > 0) Then
mstrTemipVersion = "V5.1"
iWritten = telRemote.Write "cat /var/opt/temip/licenses/licenses.dat" & Chr(13)
End If
Else
MsgBox "Read error getting results from: uname -r"
End If

Re: TelnetClient disconnects at random

Mike Stefanik — Mon, 05 May 2008 21:21:59 GMT

Author: Mike Stefanik
Post subject: Re: TelnetClient disconnects at random
Posted: Mon May 05, 2008 2:21 pm

Based on what you describe, I think part of the problem is that you're expecting the Read method to buffer the data for you in some way (for example, reading the result of a command after sending it with the Write method). That isn't how it's designed to work. Basically what you need to do is have your OnRead handler read all of the data that's available, effectively moving it out of the socket's buffer space in to your own, and your application is responsible for actually managing that buffer. As you've noted, that is more work, but it's how things need to be handled with non-blocking sessions.

Also, when reading data, always check the return value from the Read method. In other words, this is wrong:

Code: Select all: Call telRemote.Read(mstrBuffer, 4096) iRead - len(mstrBuffer)

Instead, it should be:

Code: Select all: iRead = telRemote.Read(mstrBuffer, 4096) If iRead > 0 Then ' I have some data in the buffer, iRead specifies how many bytes ElseIf iRead = -1 Then ' An error has occurred End If

Right now, you're basically assuming that calls to the Read method are always successful, and that's never safe to do.

Re: TelnetClient disconnects at random

Bart Brown — Mon, 05 May 2008 18:34:59 GMT

Author: Bart Brown
Post subject: Re: TelnetClient disconnects at random
Posted: Mon May 05, 2008 11:34 am

Ok; the blocking switch would explain how the background telnetClient is losing the connection. The foreground session changes blocking momentarily to login. Given they are all running on the same socket, there is no clean work-around for classic VB.

Related: You suggest always operating in non-blocking mode, which is painful for a simple application where you want to step though a sequence on a server. Would be much simpler to do with blocking; brutally ineffecient to handle using the OnRead event with blocking=False. For example:
- Connect,
- login,
- run command, (eg: uname -r)
- check response,
- run command, (depending on output of previous command)
- check response,
- logoff

I tried pulling this off: One form, one telnetClient control, blocking=True, and am finding that the behavior is not asynchronous. Or it is, but messed up. With a debugger on, where I step through each command, it works like a charm. But operating at normal speed, rarely does the Read accurately pick up the response from the server.

iWritten = telRemote.Write("uname -r" & Chr(13))
Call telRemote.Read(mstrBuffer, 4096)
iRead - len(mstrBuffer)

Given the above sequence, iWritten is 9, iRead is 2, and mstrBuffer contains two non-printing characters. The actual response from the server should have looked like "V5.1.ELsmp". Why does this consistently fail at normal speed and succeed at debug speed. The operation should be asynchronous. The real question would be - where did the response from the server go? It is not in the buffer.

I should note that I also tried coding this with blocking=false and relying on the OnRead event. Even there I see that at debug speed things work, but at normal speed the OnRead does not even fire after the first command. A high speed, the OnRead event only reliably fires for the last execution.

Very frustrating. It sure does seam like telnetClient is not reliable with classic VB. Or can only be made reliable with a boat load of secondary coding.

-Bart

Re: TelnetClient disconnects at random

Mike Stefanik — Mon, 05 May 2008 17:20:50 GMT

Author: Mike Stefanik
Post subject: Re: TelnetClient disconnects at random
Posted: Mon May 05, 2008 10:20 am

Generally speaking, switching the clients in and out of blocking mode can be problematic for multiple connections because Windows Sockets only allows one blocking operation per thread. If you're using VB.NET, then you can isolate sessions in their own threads, but you're basically stuck with a single-threaded application using "classic" Visual Basic.

Also, if you're displaying modal dialogs or message boxes, they can interfere with the notification messages that the client is being sent whenever a network event occurs (annoyingly enough, this also happens inside the VB debugger, which can make debugging problems like this problematic). Again, this is only an issue with older versions of the language.

Because events are level-triggered, once it has been fired (even if the notification is discarded), it won't be fired again until some action is taken on the socket. This means that one thing you can try if you think a connection has "stalled" is attempt to read from it. If it returns -1, then no data is available; if there is, then it should start sending notifications again when more data is received. But I would recommend that you go to a strictly non-blocking mode of operation. It does make things a bit more complicated (particularly if you're trying to do something like automatically log the user in and issue commands), but it's a limitation of the development platform that you really can't work around unless you break out those sessions so that they're "wrapped" in their own ActiveX EXE as an out-of-process COM server.

Re: UDP Examples

Mike Stefanik — Mon, 05 May 2008 17:08:24 GMT

Author: Mike Stefanik
Post subject: Re: UDP Examples
Posted: Mon May 05, 2008 10:08 am

Those examples are in the Visual and .NET Editions, I'm afraid that there's no counterpart in the Library Edition examples.

Re: 'Unable to resynchronize with server' through tunnel

Mike Stefanik — Mon, 05 May 2008 17:03:56 GMT

Author: Mike Stefanik
Post subject: Re: 'Unable to resynchronize with server' through tunnel
Posted: Mon May 05, 2008 10:03 am

This error typically indicates that internally something when wrong with the command channel connection and it was unable to recover. Either the connection was aborted, or unexpected data was returned in response to a command (e.g.: instead of a numeric response code in the expected format, it's getting some other text or just garbage data). When this happens, the library will attempt to automatically resynchronize with the server and get things back to a "known state". The error indicates the resynchronization attempt failed because of one of the following situations:

1. It couldn't re-establish a connection to the server
2. It didn't receive the expected "welcome" banner after the connection was made
3. It wasn't able to re-authenticate the client session
4. It couldn't change the current working directory to where the client was previously

If resynchronization fails, the library considers the connection to be unrecoverable and sets the last error code to ST_ERROR_SERVER_RESYNC_FAILED. At this point, your program has no choice but to call FtpDisconnect and then attempt to establish a new connection using FtpConnect.

TelnetClient disconnects at random

Bart Brown — Mon, 05 May 2008 17:02:04 GMT

Author: Bart Brown
Post subject: TelnetClient disconnects at random
Posted: Mon May 05, 2008 10:02 am

I'm noticing the TelnetClient control randomly disconnect from the server. I've yet to find a cause or pattern. The OnDisconnect event does not fire when this happens. Rather, the control just stops communicating for some reason.

I don't notice this behavior given a VB form with only 1 telnetClient and 1 terminal control.

I'm working on a multi-pane terminal emulator. At any time, there could be 5 telnet/terminal sessions active. There is an array of telnet clients, and associate array of terminals. At any time, only one terminal is in the foreground. I see the problem start to occur when as few as two of the telnetClients are connected. There does not seam to be a pattern if both telnetClients are connected to same server, or to different servers.

During setting up a connection, Blocking is set to True during the login, and set to False once the connection is established. Blocking stays at false for the duration of the session. The timeout is set to 60 seconds. The problem will occur in as little as 2-3 seconds.

The problem always occurs when the terminal and associated telnetClient are in the background. For example: given two terminals A and B; if I submit something to A and then bring B to the foreground before A has responded then A may disconnect. This is usually the case, but even if I let A finish before bringing B to the forground, when I go back to A it has disconnected. Again, it is somewhat random. Most times if A is not done when I go to B, A does not disconnect. And the last request typed on A that leads to a disconnect is not always the same. And not always just on A. Somtimes B, C, D, or E will randomly disconnect. But the disconnect never happens when I'm looking for it, and never on the terminal in the foreground, and OnDisconnect and OnTimeout never fire when this happens.

So I'm perplexed. On the surface, it would appear there is a limitation with the number of telnectClient controls that are on a form. Or there is a limitation that the telnetClient and/or terminal controls can not be used in a control array.

Has anyone enountered this and found a work-around? Isolating this little flaw is driving me crazy; any suggestions or thoughts would be greatly appreciated.

-Bart

UDP Examples

Frigyes Nagy — Mon, 05 May 2008 09:24:06 GMT

Author: Frigyes Nagy
Post subject: UDP Examples
Posted: Mon May 05, 2008 2:24 am

Hi!

I will try some tests with the UDP connection,

I search for examples but I can't find neither the QueryIP and ServerIP nor UDPEcho sample.
(referred in other posts in the forum).

I use Sec.Lib. Version 5.0 Build 5030

TIA
regards, frigyes

'Unable to resynchronize with server' through tunnel

Kyle Alons — Sun, 04 May 2008 01:56:33 GMT

Author: Kyle Alons
Post subject: 'Unable to resynchronize with server' through tunnel
Posted: Sat May 03, 2008 6:56 pm

Our app uses SocketTools Secure Lib v4.5 (csftpapi.dll 4.50.0.4542). BTW, we do plan on upgrading to v6 when it's released, which may solve this problem, since it involves using an SSH tunnel as a partial workaround for not having SFTP capability, but we're hoping there is something we're missing that would allow v4.5 to work for now.

Since ST does not currently support SFTP, we and some of our users create an SSH tunnel (with Putty) and establish an FTP connection through the tunnel (so that authentication is encrypted; I understand that the data channel is not encrypted by default in this scenario). One of our users is able to connect to their FTP server directly using our app, but when trying to connect through the tunnel, FtpConnect() (with dwOptions of FTP_OPTION_PASSIVE) returns 'Unable to resynchronize with server'. Any ideas why and if anything can be done about it? Thanks.

Connecting to FTP server 'localhost' on port 15022
VISBUILDPRO 090836 0000 INF: started thread 0xAE0 trace on 05-02-2008 09:08:36 [4.50.4540]
VISBUILDPRO 090836 0015 INF: gethostbyname("localhost") returned 127.0.0.1 (Rog)
VISBUILDPRO 090836 0000 INF: socket(2, 1, 0) returned 1016
VISBUILDPRO 090836 0078 WRN: connect(1016, 127.0.0.1:15022, 16) returned -1 [10035]
VISBUILDPRO 090836 0000 INF: select(64, 0x0, 0x28aeb78, 0x28aec7c, 20:0) returned 1
VISBUILDPRO 090837 0688 INF: select(64, 0x28ae9b4, 0x0, 0x0, 20:0) returned 1
VISBUILDPRO 090837 0000 INF: recv(1016, 0x28aeadc, 1, 0) returned 1
0000 53 S
VISBUILDPRO 090837 0000 INF: select(64, 0x28ae9b4, 0x0, 0x0, 20:0) returned 1
VISBUILDPRO 090837 0000 INF: recv(1016, 0x28aeadd, 1, 0) returned 1
0000 53 S
VISBUILDPRO 090837 0000 INF: select(64, 0x28ae9b4, 0x0, 0x0, 20:0) returned 1
VISBUILDPRO 090837 0000 INF: recv(1016, 0x28aeade, 1, 0) returned 1
0000 48 H
VISBUILDPRO 090837 0000 INF: select(64, 0x28ae9b4, 0x0, 0x0, 20:0) returned 1
VISBUILDPRO 090837 0000 INF: recv(1016, 0x28aeadf, 1, 0) returned 1
0000 2D -
VISBUILDPRO 090837 0000 INF: closesocket(1016) returned 0

Error connecting to FTP server: Unable to resynchronize with server

Re: Incorrect HTML Formatted Email Message Body

Mike Stefanik — Thu, 01 May 2008 16:53:48 GMT

Author: Mike Stefanik
Post subject: Re: Incorrect HTML Formatted Email Message Body
Posted: Thu May 01, 2008 9:53 am

There is a known issue that we've been working on that would cause this to happen with HTML formatted emails. I'm going to email you a hotfix that we've created for testing. If further testing is successful, we'll be posting it on our website, we just need to make sure that it completely resolves the problem.

Incorrect HTML Formatted Email Message Body

Luan Ong — Wed, 30 Apr 2008 23:05:11 GMT

Author: Luan Ong
Post subject: Incorrect HTML Formatted Email Message Body
Posted: Wed Apr 30, 2008 4:05 pm

Overview:
I have an aspx web page that uses the SMTP and MIME controls (SocketTools .NET Secure edition ver. 5.00.5030).
I'm using the MIME.ComposeMessage() method and SMTP.SendMessage() method.
For the ComposeMessage() call, I use one of the override method to include both a plain text and an HTML format for the message body.
Also, I'm using VS Studio 2005 for development under Vista.

Issue:

When I ran the page using the VS integrated web server environment, the code executed successfully and the email message body displays the HTML formatted message correctly.
However, after I compiled the application and deployed it to the web server, the email displays the following:

000000000000000000000000000000000000000000000000000000000000000000000000= 00=3D=0D=0A00=3D3D=3D0D=3D0A00=3D3D3D=3D3D0D=3D3D0A00=3D3D3D3D=3D3D3D0D=3D= 3D3D0A00=3D3D3D3D3D=3D3D3D3D0D=3D3D=3D=0D=0A3D3D0A00=3D3D3D3D3D3D=3D3D3D3= D3D0D=3D3D3D=3D3D=3D0D=3D0A3D3D0A00=3D3D3D3D3D3D3D=3D3D3D3D3=3D=0D=0AD3D0= D=3D3D3D3D=3D3D3D=3D3D0D=3D3D0A3D3D0A00=3D3D3D3D3D3D3D3D=3D3D3D3D3D3=3D3D= =3D0D=3D0AD3D0=3D=0D=0AD=3D3D3D3D3D=3D3D3D3D=3D3D3D0D=3D3D3D0A3D3D0A00=3D= 3D3D3D3D3D3D3D3D=3D3D3D3D3D3D3=3D3D3D=3D=0D=0A=3D3D0D=3D3D0AD3D0=3D3D=3D0= D=3D0AD=3D3D3D3D3D3D=3D3D3D3D3D=3D3D3D3D0D=3D3D3D3D0A3D3D0A00=3D3D=3D=0D=0A=

.....

Any advise would be greatly appreciated.

Thanks in advance.

Re: Something I have been throwing around ...

Shannara — Wed, 23 Apr 2008 22:41:29 GMT

Author: Shannara
Post subject: Re: Something I have been throwing around ...
Posted: Wed Apr 23, 2008 3:41 pm

Thank you for the reply. I definitely look forward to upgrading to 6.0 for the secure library edition, and side grading (?? or another license?) for the .NET components (secure edition) in addition to the secure library component I have now. Especially if there's a possibility for a 64bit edition in the future ;) That being said, I really look forward to the changes in 6.0. Wow, lots of possibilities. I can't believe I stay up at night trying to figure out something. My main concern is not rewrite the networking system to support as many connections I can without slowing down the use of shared resources ...

I like the idea of multiple sockets per thread, I'll have to look into that.

Re: Something I have been throwing around ...

Mike Stefanik — Wed, 23 Apr 2008 22:32:42 GMT

Author: Mike Stefanik
Post subject: Re: Something I have been throwing around ...
Posted: Wed Apr 23, 2008 3:32 pm

If you're thinking about using asynchronous sockets, events aren't thread-specific; they use Windows messages to notify the client that a socket event has occurred. If you register callback functions, internally what happens is a hidden message window is created, subclassed and the socket notifications call the function that you've specified. You can handle those messages wherever you prefer, but you'll probably want to specify the INET_OPTION_FREETHREAD option so that you don't have to worry about thread affinity (just make sure that you synchronize access to a particular socket so there's no possibility that two threads would attempt to perform some action on the socket at the same time).

That all said, I'd generally recommend against using asynchronous sockets -- you're creating more overhead for yourself, because you're forcing every event notification to be routed through the Windows messaging system, and you'll have to have a message pump running. Right now, you can poll the sockets that are owned by a particular thread using InetIsReadable to check and see if there's data available to read. However, long term that tends to be less efficient for large numbers of sockets; version 6.0 is going to introduce some changes that will make it easier and a lot more efficient to deal with if you're really looking at having thousands of sockets open. You'll be able to pass an array of socket handles, specify an event type (e.g.: INET_EVENT_READ) and the thread will go to sleep until one (or more) of those sockets become readable, at which point your thread will wake up and you can start reading the data that's arrived. Internally, it'll use Windows event objects and WSAEventSelect, so it'll be a lot more efficient than using InetIsReadable on each socket.

In any case, your approach sounds reasonable. You may want to build in some flexibility where you start with 2 threads, but you only allow a certain number of sockets per thread. As more sockets are allocated, additional threads are created to manage them. I'd also suggest making that all configurable (the number of sockets per thread, the maximum number of threads, and so on).

Something I have been throwing around ...

Shannara — Wed, 23 Apr 2008 19:34:35 GMT

Author: Shannara
Post subject: Something I have been throwing around ...
Posted: Wed Apr 23, 2008 12:34 pm

Ok, below is a something I am thinking of doing, two reasons is to relieve stress on context switching between 1,000s of connections (and one thread per connection option). Etc ...

1. I create a console server (no windows ...)
2. I create a pool of 10 worker threads.
3. I create a listener thread.
4. When an incoming request comes in, I hand off handling that request to a shared queued link list with the socket id, and event type, etc.
5. That thread then binds the events for firing when theres incoming data, and disconnections. Then the thread has nothing to do with that socket ...
6. Whenever incoming data event is fired (in main thread?), an entry is placed in the shared queued link list, a worker will pick up that entry and read all data that is incoming, and writes it back to the client. (Echoing).

Does the above seem more scale-able for a large number of clients? Eventually, I will need to process the incoming data, and have queued objects for worker threads to send out data. But I am hoping this is a start. I would hate have to rewrite the network framework later on if I don't have to.

Re: TimeOut Value HTTP

Mike Stefanik — Fri, 18 Apr 2008 19:20:23 GMT

Author: Mike Stefanik
Post subject: Re: TimeOut Value HTTP
Posted: Fri Apr 18, 2008 12:20 pm

The way it works is that the timeout value affects each particular network connect, read or write operation. So, for example, if you're posting data to a web server and have the timeout value set to 20 seconds, that doesn't mean that the post operation has 20 seconds to complete. It means that for each write operation (and for large amounts of data, there may be many of them) it will wait up to 20 seconds for the socket to be flagged as writable. Likewise, when retrieving data from the server, it will wait up to the timeout period for at least 1 byte of data to arrive.

In other words, when you set the timeout period, you're really specifying how long you're willing to wait before deciding that the server has become non-responsive (by either not accepting any more data, or not sending you back any data).