The eMule Protocol Specification

Description

The eMule Protocol Specification
Yoram Kulbak and Danny Bickson
Email: {yorkol,daniel51}@cs.huji.ac.il
Academic supervisor: Prof. Scott Kirkpatrick
DANSS (Distributed Algorithms, Networking and Secure Systems) Lab
School of Computer Science and Engineering
The Hebrew University of Jerusalem, Jerusalem
January 17, 2005

Download pdf:
http://www.cs.huji.ac.il/labs/danss/presentations/emule.pdf

1 Introduction
1.1 Purpose and scope
eMule is a popular file sharing application which is based on the eDonkey protocol. This
report describes the network behavior of eMule and explains the basic terminology that is
needed to understand the protocol. The report also gives a full specification of the eMule network
protocol including an appendix which provides the message formats. The information
in this document is based on an open source eMule client [2]. The purpose of the following
introduction is to provide general background that will allow the reader to to read and
understand this document. An extensive information source about eMule is found in [3].
1.2 Overview
The eMule network is populated with several hundreds of eMule servers and millions of eMule
clients [1]. Clients should connect one server for getting network services, the server connection
stays open as long as the client is in the system. The servers are performing centralized
indexing services (like in Napster) and do not communicate with other servers.
Each eMule client is pre-configured with a list of servers and a list of shared files on its
local file system. A client uses a single TCP connection to an eMule server for logging into
the network, getting information about desired files and available clients. The eMule client
also uses several hundreds of TCP connections to other clients which are used to upload and
download files. Each eMule client maintains an upload queue for each of his shared files.
Downloading clients join the queue at its bottom and advance gradually until they reach
the top of the queue and begin downloading his file. A client may download the same file
from several other eMule clients, getting different fragments from each on. A client may
also upload chunks of a file which it has not yet completed downloading. Finally, eMule
extends the eDonkey capabilities and allows clients to exchange information about servers,
other clients and files. Note that both client and server communication is TCP based.
The server employs an internal database in which it stores information about clients and files.
An eMule server doesn’t store any files, it acts as a centralized index for storing information
about the location of files. An additional function of the server, which is becoming deprecated,
is to bridge between clients that connect through a firewall and are not able to accept incoming
connections. The bridging functionality increases considerably the server load. eMule employs
UDP to enhance the client’s capabilities against both the server and other clients. The
client’s ability to send and receive UDP messages is not mandatory for the client’s correct
daily operation and it would function flawlessly when a firewall prevents it from sending and
receiving UDP messages.
1.2.1 Client to server connection
Upon startup the client connects using TCP to a single eMule server. The server provides the
client with a client ID (section 1.3) which is valid only through the client-server connection’s
4
Figure 1.1: eMule high level network diagram
life time (note that when the client has high ID it will receive the same ID from all servers
until its IP address changes). Following the connection establishment the client sends the
server his list of shared files. The server stores the list in its internal database which usually
contains several hundred thousand of available files and active clients. The eMule client also
sends his download list which contains the files that it wishes to download. Section 2 provides
a detailed description of the eMule client and server TCP message exchange.
After the connection is established, the eMule server sends the client a list of other clients that
posses files which the connecting client wishes to download (these clients are called ’sources’).
From this point on, the eMule client begins to establish connections with other clients as
described in section 1.2.2 below.
Note that the client/server TCP connection is kept open during all the client’s session. After
the initial handshake transactions are triggered mainly by user activity: From time to time,
the client sends file search requests which are replied by a search results, a search transaction
is usually followed by a query for sources for a specific file, this query is replied with a list of
sources (IP and port) from which the requester can download the file from.
UDP is used for communication with servers other than the server to which the client is
5
currently connected. The purpose of UDP messages is file search enhancement, source search
enhancement and finally, keep-alive (make sure that all the eMule servers in the client’s
server list are valid). More details about client-server UDP message exchange can be found
in chapter 3.
1.2.2 Client to client connection
An eMule client connects to another eMule client (a source) in order to download a file. A
file is divided to parts which are further fragmented. A client may download the same file
from several (different) clients getting different fragments from each one.
When two clients connect they exchange capability information and then negotiate the start
of a download (or upload, depends on perspective). Each client has a download queue which
holds a list of clients that are waiting to download files. When the eMule client’s download
queue is empty a download request will most probably result in a download start (unless,
for example, if the requester is banned). When the download queue isn’t empty a download
request results in adding the requesting client to the queue. There is no attempt to serve
more than a few clients in a given moment providing a minimum bandwidth of 2.4 kbytes/sec
for each. A downloading client may be preempted by a waiting client with a higher queue
ranking than his, in the first 15 minutes of the a download session the queue ranking of the
downloading eMule client is boosted to prevent thrashing.
When a downloading client reaches the head the download queue, the uploading client initiates
a connection in order to send him his needed file parts. An eMule client may be on the waiting
queue of several other clients, registered to download the same file parts in each one. When
the waiting client actually completes downloading the parts (from one of them) it doesn’t
notify all the rest that they can remove him from their queues, it will simply reject their
upload attempt when it reaches the head of their queue.
eMule employs a credit system (see section 1.4) in order to encourage uploads, to prevent
impersonation eMule secures the credit system using RSA public-key cryptography.
Client connections may use a set of messages not defined by the eDonkey protocol, these
message are called the extended protocol. The extended protocol is used for the credit system
implementation, for general information exchange (like updates of the lists of servers and
sources) and to improve performance by sending and receiving compressed file fragments.
The eMule client connection uses UDP in a limited manner to periodically check the client’s
status on the upload queue of his peer clients while it is waiting to start downloading a file.
1.3 Client ID
The client ID is an a 4 byte identifier provided by the server at their connection handshake.
A client ID is valid only through the lifetime of a client-server TCP connection although in
case the client has a high ID it will be assigned the same ID by all servers until its IP address
changes. Client IDs are divided to low IDs and high IDs. The eMule server will typically
assigns a client with a low ID when the client can’t accept incoming connections. Having a low
ID restricts the client’s use of the eMule network and might result in the server’s rejecting
the client’s connection. A high ID is calculated on the basis of the client’s IP address as
described below. This section describes the client ID assignment and significance from the
eMule protocol point of view [3]. A high ID is given to clients that allow other clients to freely
connect to eMule’s TCP port on their host machine (the default port number is 4662). A
6
client with a high ID has no restrictions in its use of the eMule network. When the server can’t
open a TCP connection to the client’s eMule port the client is given a low ID. This happens
mainly with clients that set up a firewall on their machine denying incoming connections. A
client might also receive a low ID when it the following cases:
• When the client is connected through a NAT or proxy servers.
• When the server is to busy (causing the sever’s reconnection timer to expire).
High IDs are calculated in the following way: assuming the host IP is X.Y.Z.W the ID will
be X +28  Y +216 Z +224 W (’big endian representation’). A low ID is always lower than
16777216 (0x1000000) I could not found any clew about how its calculated, note that when
you have a low ID it differs between servers.
A low ID client has no public IP to which other clients can connect to, thus all communication
must be done through the eMule server. This increases the server’s computational overhead
and results in reluctance of servers to accept low ID clients. Also, this means that a low ID
client can’t connect to another low ID client which is not on the same server because eMule
doesn’t support tunneling the requests between servers.
To support low ID clients a callback mechanism was introduced. Using this mechanism a
high ID client can ask (through the eMule server) the low ID client to connect to it in order
to exchange files.
1.4 User ID
eMule supports a credit system in order to encourage users to share files. The more files a
user uploads to other clients, the more credit it receives and the faster it will advance in their
waiting queues [3].
The user ID is a 128 bit (16 byte) GUID created by concatenating random numbers, the
6th and 15th bytes are not randomly generated, their values are 14 and 111 respectively.
While the client ID is valid only through a client’s session with a specific server the user ID
(also called user hash) is unique and is used to identify a client across sessions (the User
ID identifies the workstation). The user ID plays an important part in the credit system,
this provides motivation for ’hackers’ to impersonate to other users in order to receive the
privileges granted by their credits. eMule supports an encryption scheme which is designed
to prevent fraud and user impersonation. The implementation is a simple challenge-response
exchange which relies on RSA public/private key encryption.
1.5 File ID
File IDs are used both to uniquely identify files in the network and for file corruption detection
and recovery. Note that eMule doesn’t rely on the file’s name in order to uniquely identify and
catalog it, a file is identified by a globally unique ID computed by hashing the file’s content.
There are two kinds of file IDs - the first is used mainly for generating the unique file ID, the
second is useful for corruption detection and recovery [3]. .
7
1.5.1 File hash
Files are uniquely identified by a 128 bit GUID hash calculated by the client and based on
the file’s contents. The GUID is calculated by applying the MD4 algorithm [4] on the file’s
data. When calculating the file ID the file is divided in parts each 9.28MB long. A GUID is
calculated separately for each part and then all the hashes are combined into the unique file
ID. When a downloading client completes downloading a file part it calculates the part hash
and compares it against the part hash sent by its peer, should the part be found corrupted,
the client will try to recover from the corruption by gradually replacing bits (180kb each) of
the part until the hash is calculated OK.
1.5.2 Root hash
The root hash is calculated for each part using the SHA1 algorithm, based on blocks sized
180kb each. It provides a higher level of reliability and fault recovery, mode details in the
official eMule website.
1.6 eMule protocol extensions
Although eMule is completely compatible with eDonkey it implements several extensions
which allow two eMule clients to provide additional functionality to their users. The extensions
are focused in the client to client communication especially in the areas of security and
UDP utilization. In this document all message flow diagram designate messages that are part
of eMule extension in gray.
1.7 Soft and hard limits
The server configuration includes two kind of limits on the number of active users - soft and
hard. The hard limit is greater equal to the soft limit. When the number of active users
reaches the soft limit the server stops accepting new low ID client connections, when the user
count reaches the hard limit the server is full and doesn’t accept any client connection.

$$ Buy "The eMule Protocol Specification" on Amazon $$

Download links for "The eMule Protocol Specification":


How to Download
You may need eMule or Bittorrent to download ebook torrents or emule links.

Report Dead Link
Please leave a comment to report dead links, so that someone else may update new links.

1. Ebooks list page : 95
2. BitTorrent org » For Developers » Protocol SpecificatioN
3. New eMule files
4. Emule Speed Hack
5. Optimize Emule Connection
6. eMule - Poradnik - Instalacja - Konfiguracja
7. [share_ebook] The Java(TM) 3D API Specification The Java(TM) 3D API Specification
8. java vm specification
9. Software Specification Methods
10. The Java Language Specification 3rd ed.
11. Software Specification Methods
12. Digital Negative (dng) Specification
13. The Java(TM) Language Specification
14. The Java 3D(TM) API Specification (2nd Edition)
15. The Java Language Specification, 3rd Edition
16. Java Language Specification, 3rd Edition