[Tutor] Reading text lines from a socket

[Sandip Bhattacharya]

The socket objects only allow data to be read in by chunks (recv()). How
do I manage to read in data one line at a time where a line is defined
as a lf/cr-lf terminated test string?

In other words how can one write their own clients for reading/writing
data for text based protocols like smtp?

Thanks,

Sandip

[Lloyd Kvam]

Sockets deal with packetized data. The network protocols do not guarantee
keeping the data in line oriented chunks - even if the data starts out that way.

You need to deal with extracting lines from chunks. So long as the connection is
working properly, this is easy. The challenge occurs when the remainder of a line
never gets delivered. The best strategy depends upon the details of what you are
doing.

Post by Sandip Bhattacharya
The socket objects only allow data to be read in by chunks (recv()). How
do I manage to read in data one line at a time where a line is defined
as a lf/cr-lf terminated test string?
In other words how can one write their own clients for reading/writing
data for text based protocols like smtp?
Thanks,
Sandip

[Sandip Bhattacharya]

[Reposting to the general list too]

Post by Lloyd Kvam
Sockets deal with packetized data. The network protocols do not guarantee
keeping the data in line oriented chunks - even if the data starts out that way.
You need to deal with extracting lines from chunks. So long as the connection is
working properly, this is easy. The challenge occurs when the remainder of a line
never gets delivered. The best strategy depends upon the details of what you are
doing.

I understand that. In Python using sockets is like using sockets in C.

However, I was looking for a python equivalent of the perl idiom of
opening a socket and getting a file handle ... and then reading this
file handle as any other line oriented file using $socket->readline

I am trying to write a simple Netcraft style script which tries the HEAD
method on a webserver and reads the Server: header from the response. I
would not like to use urllib, because even a call to something like
urllib.info() reads in the complete webpage on an open().

Even if this particular problem could be done elegantly, I would be
interested to know the answer to my original question because the
feature of reading a line at a time from a socket will help me in a lot
of other places in the future.

- Sandip

[Lloyd Kvam]

Well, I've never used it, but sockets do have the makefile method. That
would seem to fit what you're trying to do.

Post by Sandip Bhattacharya
[Reposting to the general list too]

Post by Lloyd Kvam
Sockets deal with packetized data. The network protocols do not guarantee
keeping the data in line oriented chunks - even if the data starts out that way.
You need to deal with extracting lines from chunks. So long as the connection is
working properly, this is easy. The challenge occurs when the remainder of a line
never gets delivered. The best strategy depends upon the details of what you are
doing.

I understand that. In Python using sockets is like using sockets in C.
However, I was looking for a python equivalent of the perl idiom of
opening a socket and getting a file handle ... and then reading this
file handle as any other line oriented file using $socket->readline
I am trying to write a simple Netcraft style script which tries the HEAD
method on a webserver and reads the Server: header from the response. I
would not like to use urllib, because even a call to something like
urllib.info() reads in the complete webpage on an open().
Even if this particular problem could be done elegantly, I would be
interested to know the answer to my original question because the
feature of reading a line at a time from a socket will help me in a lot
of other places in the future.
- Sandip