Internet and the Enterprise London, 22 - 24 June 1994 Gateways and Servers Jeroen Houttuin RARE Project Development Officer 0. Introduction The title of the conference is 'Internet and the Enterprise' and this paper is titled 'Gateways and Servers'. To keep the scope of the paper focused, it will only discuss a small, though still the most popular, subset of Internet gateways and servers, namely those for e-mail. The nature of the term 'Gateways' forces me to discuss not only the Internet, but other networks as well. It is assumed that the reader is already using some kind of (e-mail) network internally and wants to hook up to the Internet. This paper is not of any scientific nature. I just want to share some of my experiences in mail gatewaying with you. As an aid for diagonal readers, I printed all buzz-words in Italic. 1. Relevance of e-mail The days are over when you could ask yourself: 'Is e-mail beneficial for my company?' For many companies, e-mail is already as important as FAX was 5 years ago. As long as you can keep your personnel from constantly using the network to discuss private subjects (you know which..), e-mail is certainly capable of increasing your productivity. Just as with a telephone system, the relevance of e-mail for a company is roughly an exponential function of the number of other users you can reach. So far, many mail users have misunderstood this. PC-LAN based systems were installed, and since it was not being used very much, the simple conclusion often was that e-mail for this company was not a very big success. Thus the next conclusion would be that no more money was to be invested on extra equipment, external lines, gateways, etc. But what a revelation it can be to get connected to the rest of the world. For the users that is. They find out that discussion groups exist for many of the professional problems they are facing each day and that there is almost always somebody in such a group who has solved similar problems before. System administrators often see this as a worrying exponential growth in traffic (consider the exponential function mentioned earlier). Management at the same time gets frightened by the number of ECUs they will have to pay for each incoming message (if they are unlucky enough to have such an agreement for cross- company mail). As for the relevance of e-mail within the Internet: While the Internet doubles is size almost every year, the percentage of Internet bandwidth used for e-mail is decreasing. Most bandwidth today is used by World-Wide Web (WWW) applications, such as Mosaic. The conclusion of this introduction is: 'Yes, we want (at least) e- mail and we want to connect to as many other parties as possible, at a low cost.' 2. Your enterprise and the Infobahn In this paper, I will consider the existing Internet as a synonym for the so-called Infobahn. (I hope this buzzword still exists by the time you read this. Chances are that by then it has already changed to Super-Info-TGV, Info-Concorde or something similar.) As for e-mail, we will mainly be looking at Internet Mail (RFC 822, MIME). So what is wrong then with X.400 for connecting to the outside world? Basically, nothing. To get back to the telephone metaphor: you can lead a satisfying life with a rotary pulse dialer, you can even connect an acoustic modem to it and send pictures and speech. The advantages of a cellular ISDN connection are rather obvious though. As for X.400, it will allow you to use mail, possibly Multi-Media, to other X.400 users and even to other mail communities, such as the Internet, using gateways. Full stop. Similar observations can be made about other mail systems, such as PC-LAN mail systems and even MIME. For the modern Infobahn user however, mail is only one of many purposes of the net (though still the most popular...) To see that connecting to the Internet will connect you to more than just universities, please look at figure 1, which shows the distribution of different sorts of organisations already connected to the Internet [source: 11]. Figure 1. Organisations on the Internet If we consider all registered networks in the Internet (i.e. networks that may or may not be actually connected yet), the percentage of commercial networks is even more impressive: 75 % [source: 12]. 3. Getting connected There are different ways of getting connected to the Infobahn. The most trivial way is if you are already using a complete TCP/IP protocol suite internally (or are planning to do so). You connect your internal network through a router to the Internet and you're ready for take-off. Although in my humble opinion this is the preferable solution (remember, this way you not only get e-mail connectivity, but the full range of Internet services), it is not within the scope of this paper.... A more restricted variation to the full connectivity solution is to dedicate one machine to become part of both your local network and the Internet. This firewall solution has received much attention lately, mainly because the accounting and security procedures are easier than in the full connectivity solution. The scope of this paper only allows us to look at a solution where you have an isolated e-mail system, or maybe even an internal X.400 connected to the PTT routing infrastructure (the ADMDs: Administrative Management Domains) and you want to get pure e-mail connectivity to the Internet. Independent of whether you are using X.400 software, MS-Mail, or any other popular mail solution, you will have to make a number of choices: - Which mail gatewaying software to use? - What do the addresses look like across the gateway? - Who will operate the gateway? - At what service level? - At what cost? Some of those questions will be discussed in more detail in the rest of this paper. 4. Gateways The basic function of a mail gateway can be described as follows: receive a mail from one mail world, translate it into the formats of the other mail world and send it out again using the routing rules and protocols of that other world. Some of the simplest examples of what has to be translated are: - Service elements E.g. Internet: From: <- > X.400: P2.Orginator - Types E.g. Internet: ASCII <-> X.400: IA5Text - Values E.g. Internet: user@domain <-> X.400: user(a)domain Especially if a message crosses more than one gateway, it is important that all gateways have the same understanding of how things should be translated. A simple example of what could go wrong otherwise is the following: A sends a message to B through a gateway and B's reply to A is being routed through another gateway. If the two gateways don't use the same translations, the 'From' and 'To' addresses in the original message and in the answer will not match. This is, to say the least, very confusing for the end-users (consider what happens if the end-users of the service are automated processes that communicate via mail). More serious things can happen to addresses if a message crosses more than one gateway on its way from the originator to the recipient. As a real-life example, consider receiving a message from: Mary Plork This is not what you would call a user-friendly service. If you are lucky, a reply to this address will take a very inefficient route. As said, this was a real-life example and I replied using the so-called 'send-and-pray' method. My prayers were not heard though... The example shows the importance of transparency towards the end- users, which can only be guaranteed if all gateway translations are reversible. All gateways thus must know which translations were made by other gateways. The easiest solution is of course for all gateways to use the same standard translations. For gateways between X.400 and the Internet, such standard translations are described in RFC 1327: "Mapping between X.400(1988) / ISO 10021 and RFC 822" [3], written by Steve Kille in 1992. Most operational X.400 to Internet gateways are indeed RFC 1327 conformant nowadays. As expected, the few that aren't regularly cause problems for the others (as well as for themselves and their customers). For those readers with USEnet or FTP connectivity, there exists a list of RFC 1327 conforming gateway services and products [10], which also contains a list of 'sinners'. My personal advice is to read this list and avoid e-mail management overhead by never using a non-conformant gateway. For non-X.400 e-mail services the situation is even more complicated. A large number of products using proprietary mail protocols (MS-mail, MHS, etc.) come with as many proprietary solutions for gatewaying. The more protocols the gatewaying software can translate, the better the vendor claims it to be. Gateways to other protocols, both proprietary and 'standardised', such as Internet mail, X.400, and UUCP are offered. The software suppliers are however not in any way co-ordinating the global behaviour of such gateways, so that there is in general only transparency between end-users with the same proprietary mail systems. That is, _if_ they are also using the same gatewaying software and the mail is not routed through any other gateways, which can normally not be controlled by the end-users themselves. And even when a message crosses only one type of gateway twice, transparency is not a natural thing to expect. In reality there are no 100% transparent gateways. Almost every gatewaying step will cause some loss of functionality. If you cannot use Internet mail or X.400 internally, the rule of thumb has to be: "The less gatewaying steps, the better". For example, if you want to connect your MS-mail system to the Internet, avoid using software that can only gateway MS-mail to X.400 and thus forces you to use another gatewaying step: X.400 to Internet mail. Please keep the above in mind when your PC-LAN mail system is already connected through a gateway to an X.400 service provider and that provider offers you to gateway your mail to the Internet as well. You can probably save quite some problems if you can find an Internet service provider that can gateway you PC-LAN mail to the Internet directly. Within RARE (the Association of European Research Networks) and the IETF (Internet Engineering Task Force), efforts are being made to harmonise all these gateways' behaviour by creating a list of meta-requirements for mail gatewaying requirements. Whether the vendors will follow these meta-requirements in the very near future is very questionable though. They can do more after-sales if there are gatewaying problems and for most customers it is difficult to realise; let alone to stress to the vendors, that what they really want is a globally transparent gatewaying service. 5. Servers Especially in the Internet, e-mail systems are increasingly used as an underlying service for so-called Mail Based Servers (MBSs). Examples for MBSs are echo servers, distribution lists (a.k.a. Mailing Lists), Mail Based Directory Enquiry Servers and Mail Based File Servers. [5] gives a complete classification of all types of MBSs. The most typical sorts of MBSs you will encompass when you get connected to the Internet are Mailing Lists and Mail Based File Servers. A mailing list is a process with a certain mail address that will, upon receiving a message, distribute that message to a well- defined list of recipients, the list members. Mailing lists are mostly used for the same purposes as the USEnet services described in chapter 3, i.e. to share knowledge amongst a distributed group of people. Likewise, a Mail Based File Server is a process with a certain mail address that will, upon receiving a message, interpret commands in the body of that message (typically: SEND filename, HELP, LIST directory). It will then execute those commands and send the results back to the originator of the original message. Since MBSs deal with automatically receiving, forwarding and replying to messages, which may themselves have been generated by automated processes, strong requirements are needed on the one hand to minimise human effort to manage such servers, and on the other hand to make the behaviour of MBSs deterministic enough to build reliable tools upon them. A classic example of what can go wrong is when a mailing list contains an invalid address. The remote mailer generates a non- delivery message and sends it to the originator of the original message, which, under some (wrong) circumstances, could be the list itself, which then distributes the non-delivery message to the list, and thus also to the erroneous address, etc. Following strict recommendations on how mailing lists should deal with message header fields can avoid such looping-endangered situations. [6] is the first in a series of such requirements. It defines such requirements for a sub-class of MBSs, the answering servers. This class includes echo servers (a.k.a. auto-answer) and vacation servers. 6. Traffic implications As stated in chapter 1, getting connected to the Internet can result in an exponential increase in traffic (mainly incoming). Note that this increase can mainly be explained by the huge amount of remote parties reachable on or through the Internet. A study of the e-mail traffic over the Norwegian X.400 to Internet gateway in 1992 showed that 30% of the traffic coming in from the Internet could be proven to have been originated by distribution lists and file servers. As MBSs often have addresses indistinguishable from normal users' addresses, it is difficult to work out the real figure. To me, 50% seems a reasonable estimation. 7. Conclusion If you are using X.400 internally, make sure your gateway to the Internet is conforming to RFC 1327. You might run such a gateway yourself, which will of course give you more control over the system. Consider extensibility, accounting, etc. However, since the fixed costs are rather high (typically one dedicated machine, one router, one leased line, plus administrative overhead of approximately 1 man-power), smaller companies should rather try to get an X.400- or Internet service provider to run the RFC 1327 gatewaying service for them. If you are not running Internet mail or X.400 internally and you want to connect to the Internet, contact your nearest Internet service provider for further help. Again, you will have the options of outsourcing the gatewaying service or running it yourself. The pros and cons are similar as with the X.400 to Internet gateways described above. Finally, consider the following situation: You want to get connected to the Internet and at least one of the following bullets holds (mind that they can overlap): - You are using X.400 internally - You are already connected to an X.400 service provider - You would also like to get connected to an X.400 service provider In this situation I would like to advise you to always compare offers from more than one service provider. In many cases, using an Internet service provider will be preferable. Typical reasons might be: - Easier to upgrade to full Internet connectivity (sse chapter 3) - Only one gatewaying step (see chapter 4) - No tariff per incoming mail (see chapter 6) Finally there is the possibility of not outsourcing the gatewaying service at all, but run one locally. The viability of this option will mainly depend on the size of you internal e-mail network and your (future) requirements for Internet connectivity beyond mail services. After my own final proof-read of this paper, I added this last paragraph in which I would like to stress that my goal has not been to promote the Internet service providers. In the context of this paper, 'Internet gatewaying', they often compare favourably to X.400 service providers though. A. References and further reading [1] Crocker, D., "Standard for the Format of ARPA Internet Te