Introduction to Network Programming with Python

1 Introduction to Network Programming with PythonNorman MatloffUniversity of California, Davisc 2003-2005, N. MatloffApril 29, 2005 Contents1 Overview22 Our Example Client/Server of the Server Program .. of the Client Program ..63 Role of the Operation .. the OS Distinguishes Between Multiple Connections ..74 Thesendall()Function75 More on the Stream Nature of , It s Just One Big Byte Stream, Not Lines .. Wonderfulmakefile()Function .. the Tail End of the Data .. 106 Nonblocking Sockets117 Advanced Methods of Polling1411 OverviewThe TCP/IP Network protocol suite is the standard method for intermachine communication. Though orig-inally integral only to the UNIX operating system, its usage spread to all OS types, and it is the basisof the entire Internet. This document will briefly introduce the subject of TCP/IP Programming usingthe Python language.

2 Matloff/Networks/ a more detailed Introduction to networks and TCP/IP application consists of a pair of programs, called aserverand aclient. If for example you use aWeb browser to , the browser is the client, and the Web server at Yahoo headquartersis the Our Example Client/Server PairAs our main illustration of client/server Programming in Python , we have modified a simple example inthe Library Reference section of the Python documentation page, Here is the server, :1# simple illustration client/server pair; client program sends a string2# to server, which echoes it back to the client (in multiple copies),3# and the latter prints to the screen45# this is the server67import socket8import sys910# create a socket11s = ( , )1213# associate the socket with a port14host = # can leave this blank on the server side15port = int( [1]) ((host, port))1718# accept "call" from (1)20conn, addr = ()21print client is at , addr2223# read string from client (assumed here to be so short that one call to24# recv() is enough), and make multiple copies (to show the need for the25# "while" loop on the client side)2627data = (1000000)28data = 10000 * data2930# wait for the go-ahead signal from the keyboard (shows that recv() at31# the client will block until server sends)32z = raw_input()3334# now (data)23637# close the ()

3 And here is the client, :1# simple illustration client/server pair; client program sends a string2# to server, which echoes it back to the client (in multiple copies),3# and the latter prints to the screen45# this is the client67import socket8import sys910# create a socket11s = ( , )1213# connect to server14host = [1] # server address15port = int( [2]) # server ((host, port)) ( [3]) # send test string1920# read echo21i = 022while(1):23data = (1000000) # read up to 1000000 bytes24i += 125if (i < 5):26print data27if not data: # if end of data, leave loop28break29print received , len(data), bytes 3031# close the ()This client/server pair doesn t do much. The client sends a test string to the server, and the server sendsback multiple copies of the string. The client then prints the earlier part of that echoed material to the user sscreen, to demonstrate that the echoing is working, and also prints the amount of data received on each read,to demonstrate the chunky nature of should run this client/server pair before reading up the server on one machine, by typingpython 2000and then start the client at another machine, by typingpython server_machine_name 2000 abc1 The source file from which this document is created, , should be available wherever you downloaded the PDF can get the client and server programs from the source file, rather than having to type them up two main points to note when you run the programs are that (a) the client will block until you providesome keyboard input at the server machine, and (b)

4 The client will receive data from the server in ratherrandom-sized Analysis of the Server ProgramNow, let s look at the 7:We import thesocketclass from Python s library; this contains all the communication methods 11:We create a socket. This is very much analogous to a file handle or file descriptor in applicationsinvolving files. Internally it is a pointer to information about our connection (not yet made) to an applicationon another machine. Again, at this point, it is merely a placeholder. We have not made any Network actionsyet. But our callsbind()etc. below will result in more and more information being added to the placepointed to by our two arguments state that we wish to the socket to be an Internet socket ( ), and that itwill use the TCP method of organizing data ( ), rather than UDP ( ).Note that the constants used in the arguments are attributes of the modulesocket, so they are preceded by socket.

5 ; in C/C++, the analog is the #include 16:We invoke thesocketclass bind()method. Say for example we specify port 2000 on the commandline when we run the server (obtained on Line 15).A port is merely an ID number, not anything physical. Since there may be many Network connections on amachine at one time, we need a way to distinguish between them. The ID number serves this purpose. Portnumbers 0-1023, the so-calledwell-known ports, are for standard services such as FTP (port 21), SSH (port22) and HTTP (port 80).2 You cannot start a server at these ports unless you are acting with root we callbind(), the operating system will first check to see whether port 2000 is already in use bysome other so, an exception will be raised, but otherwise the OS will reserve port 2000 for theserver. What that means is that from now on, whenever TCP data reaches this machine and specifies port2000, that data will be copied to our server program.

6 Note thatbind()takes a single argument consisting ofa two-element tuple, rather than two scalar 19:Thelisten()method tells the OS that if any messages come in from the Internet specifying port2000, then they should be considered to be requesting connection to this method s argument tells the OS how many connection requests from remote clients to allow to bepending at any give time for port 2000. The argument 1 here tells the OS to allow only 1 pending connectionrequest at a only care about one connection in this application, so we set the argument to 1. If we had set it to,say 5 (which is common), the OS would allow one active connection for this port, and four other pendingconnections for it. If a fifth pending request were to come it, it would be rejected, with a connectionrefused UNIX machines, a list of these is available in/ could be another invocation of our server program, or a different program entirely.

7 You could check this by hand, byrunning the UNIX netstatcommand (Windows has something similar), but it would be better to have your program do it, using is about alllisten()really term this socket to be consider it alistening socket. That means its sole purpose is to accept connectionswith clients; it is usually not used for the actual transfer of data back and forth between clients and 20:Theaccept()method tells the OS to wait for a connection request. It will block until a requestcomes in from a client at a remote will occur when the client executes aconnect()call (Line16 ). When that call occurs, the OS at the client machine will assign that client anephemeralport, which is a port number for the server to use when sending information to the client. The OS on theclient machine sends a connection request to the server machine, informing the latter as to (a) the Internetaddress of the client machine and (b) the ephemeral port of the that point, the connection has been established.

8 The OS on the server machine sets up a new socket,termed aconnected socket, which will be used in the server s communication with the remote client. Youmight wonder why there are separate listening and connected sockets . Typically a server will simultaneouslybe connected to many clients. So it needs a separate socket for communication with each this releasesaccept()from its blocking status, and it returns a two-element tuple. The first element ofthat tuple, assigned here toconn, is the connected socket. Again, this is what will be used to communicatewith the client ( on Line 35).The second item returned byaccept()tells us who the client is, the Internet address of the client, in casewe need to know 27:Therecv()method reads data from the given socket. The argument states the maximum numberof bytes we are willing to receive.

9 This depends on how much memory we are willing to have our serveruse. It is traditionally set at is absolutely crucial, though, to discuss how TCP works in this regard. Consider a connection set upbetween a client X and server Y. The entirety of data that X sends to Y is considered one giant for example X sends text data in the form of 27 lines totalling 619 characters, TCP makes no distinctionbetween one line and another; TCP simply considers it to be one 619-byte , that 619-byte message might not arrive all at once. It might, for instance, come into two pieces, oneof 402 bytes and the other of 217 bytes. And that 402-byte piece may not consist of an integer number oflines. It may, and probably would, end somewhere in the middle of a line. For that reason, we seldom seea one-time call torecv()in real production code, as we see here on Line 27.

10 Instead, the call is typicallypart of a loop, as can be seen starting on Line 22 of the client, In other words, here on Line 27 ofthe server, we have been rather sloppy, going on the assumption that the data from the client will be so shortthat it will arrive in just one piece. In a production program, we would use a 28:In order to show the need for such a loop in general, I have modified the original example bymaking the data really long. Recall that in Python , multiplying a string means duplicating it. For example:>>> 3* abc abcabcabc 4It could be used for that purpose, if our server only handles one client at a , technically, could be the same I say we, I mean we, the authors of this server program. That information may be optional for us, though obviouslyvital to the OS on the machine where the server is running.