[Tutor] TCP server/client application failing. Need some help with the basics!

[Anubhav Yadav]

I am very new to python. I have been a very mediocre programmer, but now I
have decided that I want to level up as a programmer.

I wanted to write a simple TCP client/server (where in the server acts as a
simple TCP Listener). I searched on the forums and I saw that people
advised using frameworks like twisted and sync.io for client server
applications to take advantage of asynchronous model. But since I couldn't
visualize the problems that people faced when they implemented servers as
multithreaded model, so I decided to start by implementing a multithreaded
server, and then improve as I go on facing issues.

So I started with using socket library for both client and server:

Here is my listener.py:

import socket
import threading
from time import sleep

class Serve(threading.Thread):
def __init__(self, client_socket, client_address):
threading.Thread.__init__(self)
self.socket = client_socket
self.address, self.port = client_address
def run(self):
try:
while True:
data = self.socket.recv(300)
if data:
print data
sleep(2)
else:
break
except Exception as e:
print e
finally:
self.socket.close()


if __name__ == '__main__':
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('localhost', 5555))
sock.listen(1)

servers = []

while True:
connection, client_address = sock.accept()
print "New Connection from {}".format(client_address)
server = Serve(connection, client_address)
servers.append(server)
server.start()

for server in servers:
server.join()

Here is my sender.py. It is supposed to simulate clients connecting to the
server.

import argparse
import socket
import threading
from time import sleep


parser = argparse.ArgumentParser(description="A simple TCP sender")
parser.add_argument('-H', '--host', help='host to connect to',
default='localhost')
parser.add_argument('-p', '--port', help='port of the host', type=int,
default=5555)
parser.add_argument('-w', '--workers', help='number of threads to
create', default=1000, type=int)
args = parser.parse_args()
count = args.workers

def send(id):
lock = threading.Lock()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = (args.host, args.port)
try:
sock.connect(server_address)
while True:
sock.sendall('@ABCDEF1234567890FABC#')
sleep(1)
except Exception as e:
print "thread no {} killed".format(id)
print e
lock.acquire()
count-=1
lock.release()
sleep(1)
print "Total threads are {}".format(count)
finally:
sock.close()

threads = []
if __name__ == '__main__':
for i in range(args.workers):
t = threading.Thread(target=send, args=(i,))
threads.append(t)
t.start()

for thread in threads:
thread.join()

When I run the client and server together with 1000 clients, many threads
are killed on a machine with low memory, and only 210-220 clients are
connected with the server. The killed clients gave the following error:

`[Errno 104] Connection reset by peer`

Right now I am not on the low memory machine, but I remember the error also
had "broken pipe" in it.

So I decided to run the same code with 1000 clients on my laptop with 8 GB
memory. This time almost all clients where connected but one or two clients
got killed with the same above error (I could not see broken error in the
messages this time".

Then I ran the same code with 500 clients, and this is where the code broke
with the following errors.

Exception in thread Thread-4999:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 763, in run
File "sender.py", line 17, in send
File "/usr/lib/python2.7/socket.py", line 187, in __init__
error: [Errno 24] Too many open files

Total threads are 4998

These errors came a lot, but the count said that only two threads failed. I
think my logic to calculate the count is wrong.

Then I decided to use SocketServer for the server. Here is the new Server:

import SocketServer
from threading import Thread
from time import sleep

class service(SocketServer.BaseRequestHandler):
def handle(self):
print "Client connected with ", self.client_address
while True:
try:
data = self.request.recv(300)
if data:
print data
sleep(2)
else:
print "Client exited"
self.request.close()
except Exception as e:
print e


class ThreadedTCPServer(SocketServer.ThreadingMixIn,
SocketServer.TCPServer):
pass

t = ThreadedTCPServer(('localhost',5555), service)
t.serve_forever()

I have used the same client. This time there is some durability when there
are 1000 clients, but if I disconnect the sender (send a `kill -KILL pid`
signal) the listener breaks with the following error

`[Errno 9] Bad file descriptor`

Now if I run the sender with 5000 clients, the errors are back.

Exception in thread Thread-4999:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 763, in run
File "sender.py", line 17, in send
File "/usr/lib/python2.7/socket.py", line 187, in __init__
error: [Errno 24] Too many open files

I don't even know if this is the right way of writing the servers, I also
know that possibly there is something wrong with my code, and maybe I am
not doing things as they are meant to be done.

Is it even right to create many thousands clients using threads on the same
machine? Is that what is causing problems? Should I learn some library like
twisted or zmq?

Would love to understand the details of threading and sockets! Please do
leave your comments and criticize me. Sorry for this long post.
_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Alan Gauld]

Post by Anubhav Yadav
I wanted to write a simple TCP client/server (where in the server acts as a
simple TCP Listener). I searched on the forums and I saw that people
advised using frameworks like twisted and sync.io for client server
applications to take advantage of asynchronous model.

Yes, for large scale work I'd second that recommendation.,
But when learning its often better to start small.

Post by Anubhav Yadav
multithreaded model, so I decided to start by implementing a multithreaded
server, and then improve as I go on facing issues.

But starting with a multi threaded server is not what I'd
call starting small. Its quite advanced. I'd start getting
non multi threaded servers running reliably first. Then
I'd try adding some threading. As in maybe a dozen
connections sharing some data source..

Only then would I even consider the extra issues of
scalability to thousands of connections...

Post by Anubhav Yadav
Is it even right to create many thousands clients using threads on the same
machine?

It may be theoretically possible but when I was designing
large client server/web based systems I used a rule of thumb
that said not more than 100 connections per core. So for 5000
connections I'd need 50 cores. Allowing one core for the OS to
run per box that leaves 7 cores per 8-core server, so I need
8 servers to handle 5000 connections. For resilience I'd
add another one (so called N+1 architecture).

Now those servers were all carrying significant processing
requests not simple ping-like requests, so on that basis I'd
guess that you don't need 50 cores for your example. But
it does seem like a lot to ask of a single box. And if you
intend to extend that to a network model think about the
load on your network card too. My servers at work all
had a couple of (gigabit) network cards connected.

Post by Anubhav Yadav
Would love to understand the details of threading and sockets! Please do
leave your comments and criticize me. Sorry for this long post.

Too much code for me to read at this time of night, but I
would try slimming down your testing, at least initially.
Make sure it works on lower loads then ramp up slowly
until it breaks.

The other thing I'd consider is the timing of your requests.
I didn't check but have you a decent gap between requests
or are all 5000 arriving nearly simultaneously? It may
be a network configuration issue - the socket/port queue
simply getting choked to the point where packets time out.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
http://www.amazon.com/author/alan_gauld
Follow my photo-blog on Flickr at:
http://www.flickr.com/photos/alangauldphotos


_______________________________________________
Tutor maillist - ***@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor

[Anubhav Yadav]

But starting with a multi threaded server is not what I'd

Post by Alan Gauld
call starting small. Its quite advanced. I'd start getting
non multi threaded servers running reliably first. Then
I'd try adding some threading. As in maybe a dozen
connections sharing some data source..
Only then would I even consider the extra issues of
scalability to thousands of connections...

I did implemented a small server which could only serve one client at a
time. In fact
after realizing that it is able to serve a single client at a moment, I
decided to go with multithreaded
approach. I just didn't thought of including that single threaded server
code here.

Post by Alan Gauld
It may be theoretically possible but when I was designing
large client server/web based systems I used a rule of thumb
that said not more than 100 connections per core. So for 5000 connections
I'd need 50 cores. Allowing one core for the OS to
run per box that leaves 7 cores per 8-core server, so I need
8 servers to handle 5000 connections. For resilience I'd
add another one (so called N+1 architecture).
Now those servers were all carrying significant processing
requests not simple ping-like requests, so on that basis I'd
guess that you don't need 50 cores for your example. But
it does seem like a lot to ask of a single box. And if you
intend to extend that to a network model think about the
load on your network card too. My servers at work all
had a couple of (gigabit) network cards connected.

I agree, but since I am trying to learn, I think I should limit my clients
to 100 rather than 1000 or 5000
per machine? Getting a server of that capacity is difficult for me.

Post by Alan Gauld
Too much code for me to read at this time of night, but I
would try slimming down your testing, at least initially.
Make sure it works on lower loads then ramp up slowly
until it breaks.

I can relate, I wrote this mail at 2 am in the night and immediately hit
the bed. :)

Post by Alan Gauld
The other thing I'd consider is the timing of your requests.
I didn't check but have you a decent gap between requests
or are all 5000 arriving nearly simultaneously? It may
be a network configuration issue - the socket/port queue
simply getting choked to the point where packets time out.

I have a time.sleep(1) before sending each requests, from each client.