TUNING.FAQ
==========

Changes since 23rd June 1995:


          *    Update information on V34 access
          *    Update information on vPoPs
          *    Update information on news server
          

     If you have any comments on the contents of this FAQ, please let me
     know

Changed text is marked by #s



CONTENTS
========

 0.  Introduction
 1.  Do I have a performance problem ("Am I normal ?")
 2.  MODEM.TXT
 3.  MNP5 and other compression
 4.  Hardware overruns
 5.  How fast should I run the link to my modem ?
 6.  Does Windows affect link speeds ?
 7.  Eliminate SMARTDRV, TSRs etc.
 8.  Serial chips
 9.  The KA9Q asystat command
10.  Internal modems
11.  IP fragmentation
12.  NNTP request queues
13.  HISTORY
14.  PPP versus SLIP
15.  Telnet speed
16.  Other improvements
17.  OS/2
18.  The Authors
19.  Disclaimer

0.   Introduction
==   ============

     This document mainly concerns itself with tuning your KA9Q system to
     access the Internet as efficiently as possible. Time is money (and the
     phone company will send you their bill eventually).

     Because people don't always like blindly following instructions some
     explanations are given as to why various things are useful. You can
     skip these sections if you wish and just push the buttons :) Only
     limited initial knowledge is assumed, but by the time you read to the
     end you too could be an expert.

     Although this document is specifically aimed at DOS and OS/2 KA9Q
     users, the general discussion is much more widely applicable.

1.   Do I have a performance problem ("Am I normal ?")
==   =================================================

     If you have a USR V 34 or VTerbo modem you should connect to demon via
     the new vPoPs. Most of the VPoPs have VTerbo modems, although these
     are now slowly being updated to V34 standard. Demon have also said
     that they hope to increase the serial port speed to 58,700 or 115,000
     as this occurs. I am currently in the process of gathering figures for
     V34 connections.

     If you own a 14.4K modem which can do V42bis (such as a Sportster) and
     you connect to it at 38400 bps (or more) you should be getting a cps
     (characters per second) throughput of at least:

               2700 cps for text
               1650 cps for UUENCODED text
               1600 cps for ZIP files

     By text is meant general Usenet newsgroups and email. UUENCODE is the
     format used for sending programs, pictures and sounds around within
     text-based newsgroups. ZIP is a popular pre-compressed format for
     binary files transferred by the FTP file transfer protocol.

     A 9600 bps modem with V42bis should achieve:

               1850 cps for text
               1050 cps for UUENCODED text
               1000 cps for ZIP files
          
     A 9600 bps modem which does no compression at all should achieve a
     touch over 900 cps for all types of data.

     Obviously, if you take a mixture of newsgroups some of which are
     binary and some of which are not then you will get intermediate
     performance figures. Also if your newsfeed is mixed in with a lot of
     incoming email then you will get a slower news transfer while the link
     is being shared.

     You will also be lucky if you consistently achieve these speeds at
     very busy times (such as early weekday evenings). When there can be
     over six hundred people all transferring data, there is
     unsurprisingly, some degradation in performance. The servers are
     busier, and there is more contention for bandwidth on the Demon LAN.

     A new news machine was installed at Demon just before Christmas. It
     has the new Solaris 2.4 operating system and the latest NNTP software.
     This initailay lead to an increase in speed from the server.
     Thoughput is not as high as it could be though due to a bug in the
     Solaris operating system causing retransmittions.

#    Over the last few weeks the demon news server has been improved
     considerably so that the 400 server too busy message is rare. The
     problem with retransmizssions is still there though. A number of
     seperate syncronised news servers have also been added. The NNTP
     software has also been optimised by aDe.

     There is also some doubt that a very slow machine such as an Amstrad
     1512 can ever reach these speeds, although 12MHz 286s are reported to
     manage just fine. Also, running OS/2 on slow 386s seems to reduce
     maximum news throughput. If you have a slow machine you are advised to
     work through the rest of this document to check that your system is
     optimally tuned; but you may well not achieve the suggested "normal"
     throughput figures. If you are using OS/2 there is some specific
     advice in section 17.

     KA9Q will show you the news transfer rate you are achieving if you
     have "nntp verbose on" (type this at the keyboard or add it to
     autoexec.net, perhaps with the DIS front end [D A f6]). The figures
     are also recorded in _\spool\nos.log for posterity. Even with a
     "perfect" set-up you will find the speed achievable for Usenet news
     varies from day to day, but that ZIP and UUENCODE transfer speeds are
     pretty constant.

     If you want to try some specific tests then Demon keep some special
     files for FTP access on ftp.demon.co.uk. You will find them in
     /pub/test. The most useful ones for general testing are the various
     "fullfile"s, since "emptyfile" and "regularfile" are easily compressed
     out of existence by almost everything! In fact "emptyfile" is usually
     so compressed across the phone link that it serves merely to measure
     the maximum throughput of the rest of the system.

     If you regularly fail to get the figures we mention then there are a
     number of possible reasons. You will need to check your modem is
     properly configured. You will need to eliminate any hardware or
     software overruns. Since the default KA9Q settings are wrong (!) you
     will probably need to tweak some parameters to eliminate IP level
     fragmentation. All of these changes are discussed below.

     If you suddenly get a slower transfer then it may be temporary. Maybe
     you picked a busy time, you got a noisy phone line or a system problem
     at Demon caused difficulty. If not a peak time, check the
     demon.announce newsgroup for apologies from Demon, or the
     demon.service newsgroup for other sufferers. It is also a good idea to
     finger status@gate.demon.co.uk as many minor problems are often only
     reported their. If it persists then it was probably the change you've
     just made to your machine. If it just affects news then you must check
     if your history file is too big (see section 13).

     Finally, before moving on to practical advice, please note that the
     figures stated are pretty much the best you can hope for (within a few
     percent), and assume that the only limiting factor is the link between
     you and Demon's Finchley network. If you are transferring data from
     the other side of the planet you will be competing for bandwidth
     across the Atlantic, through the US routers and then for some
     attention from a busy server in California (or wherever).

     Similarly, though less dramatically, when you use one of the local
     access tPoPs (Edinburgh, Cambridge etc.) then besides all the other
     factors, you will competing with the other local users for a share of
     the 64K line to London. Thus unless you are logging on to a modem in
     Finchley you may never be able to achieve quite the suggested
     throughput. All the vPoPs are connected directly to the Finchley, only
     the tPoPs compete for the bandwidth. If you are using a tPoP you will
     have to see if the saving of the local call outweighs the degradation
     of the service.

     As I've stated in the case of the Energis vPoPs, this situation is
     eased as all the modems are actually connected directly to the
     Finchley network. Therefore if you have the choice between a local
     vPoP or tPoP I would suggest that you use the vPoP which should
     increase your throughput. The numbers for these lines are available in
     Welcome.txt (on ftp.demon.co.uk)

2.   MODEM.TXT
==   =========

Advice:
     Read MODEM.TXT (this advice is traditionally chanted in unison)!

How:
     Demon maintain and publish a related document to this one called
     "MODEM.TXT" which you should find in the KA9Q distribution ZIPfile, or
     on ftp.demon.co.uk as /pub/doc/Modem.txt (the M is a capital).

Why:
     MODEM.TXT explains about using proper cables to your modem and the
     basics of setting up your modem for use with KA9Q. Suggestions that it
     fully explains the meaning of life are unfounded, but it (like the
     other Demon produced documentation) does cover a lot of ground and
     reading it is to be highly recommended.

     The purpose of MODEM.TXT is to get you connected and data flowing.
     This document on the other hand is intended to get that data flowing
     at an optimum rate.

     If you post to the demon.ip.support.* newsgroups and cannot say that
     you have read MODEM.TXT (and of course this document TUNING.FAQ) then
     you are wasting everyone's time, including your own. People may well
     point this out to you by email; more or less politely.

3.   MNP5 and other compression
==   ==========================

Advice:
     Configure your modem to use V42bis not MNP5.
     If you only have MNP5 then use it only for text transfers

How:
     RTFM! Sorry, but the AT commands for configuring your modem are not
     standardised across different models. MODEM.TXT contains some set-up
     strings which may help you.

Why:
     MNP5 is a data compression scheme which modems use whilst the data is
     transferred across the phone link. Unfortunately, although it will
     compress text fairly well, and is said to be especially good at
     UUENCODED text, it can (and does!) make ZIP files bigger! V42bis is a
     compression scheme which is smart enough to quit if it is making
     things worse.

     If your modem does V42bis then use this and turn off MNP5. Otherwise
     you should turn off MNP5 unless you know that you will only be
     transferring text. Some people go as far as to arrange two separate
     set-ups and they log off and redial with the other set-up if they are
     going to FTP some ZIPs rather than collecting their newsfeed.

4.   Hardware overruns
==   =================

Advice:
     You MUST try to eliminate hardware overruns.

How:
     You can tell if you have hardware overruns by using the KA9Q asystat
     command (just type it at the NET prompt). If 'hw over' is non-zero
     then you have a problem. If 'sw over' is non-zero then you also have a
     problem. For more about asystat in general, and software overruns in
     particular, see section 9.

     To fix a hardware overrun problem:

     Slow down the link to the modem         see section 5
     Try using a Windows DOS box             see section 6
     Try not using a Windows DOS box (!)     see section 6
     Get rid of TSRs, SMARTDRV &c            see section 7
     Buy a 16550A                       see section 8

Why:
     If you are getting hardware overruns then your throughput will suffer!
     Because of the nature of the TCP/IP link you will suffer a lot more
     than you would on normal comms links to Bulletin boards. Thus a set-up
     which works marginally to, say, CIX can work very badly indeed to
     Demon.

     Your modem is generating characters in an unstoppable sort of way,
     though at a maximum fixed rate (9600, 19200, 38400 etc.). When the
     characters turn up at the PC end of the cable they must be recorded
     into the PC memory before the next character turns up. If they are not
     "read" and transferred to memory then they will be lost. When the
     standard 8250 serial chip "interrupts" the PC to tell it that a
     character is ready then the PC has just one character time to read it
     before the next one appears and replaces it. (Of course if the next
     character is late arriving because the link is not fully utilised then
     the PC has correspondingly longer to react.)

     If the PC loses a character then the entire packet it formed part of
     will have to be sent again. Packets may well be 1500 bytes long so
     this can take some time. Further, because of the way the TCP/IP
     protocols work the re-sending may not start until after a non-trivial
     timeout has elapsed.

     Worse, if you are receiving data from a long way away then at any
     given time there is a lot of data "in the pipeline" coming to you.
     When the other end realises that you've lost some data it will resend
     that data and continue on from there. Naturally KA9Q will hang onto
     the good data which turns up after the damaged packet and thus once
     the retransmission starts and the missing packet arrives it can (and
     does) acknowledge everything it has. However, by the time the other
     end receives this acknowledgement the pipeline may be refilled and
     thus, despite KA9Qs best efforts, you will still receive a lot of
     duplicate data, which takes time to be transferred to your PC and then
     discarded.

     All of this means that hardware overruns have a significant effect on
     overall throughput, and their elimination is extremely important.

5.   How fast should I run the link to my modem ?
==   ============================================

Advice:
     Modern modems compress the data they send across the phone line, so to
     keep up you need to run the serial link to them quite a bit faster
     than the speed they have apparently connected at to the other end.

     Use the fastest possible serial link speed to connect to your modem
     PROVIDED that you do not get hardware overruns. 19200 bps is too slow
     for a 14.4K connection.

How:
     The serial link speed is configured by the DIS front-end program, [D
     A] (your chosen setting eventually ends up in the ppp attach command
     in autoexec.net in the main KA9Q directory).

     e.g. attach asy 0x3e8 4 ppp sl0 4096 1500 38400
                                               -----
Note:
     The letters "AT" at the start of each modem command form a (carefully
     chosen) bit pattern which allows your modem to automatically detect
     the serial link speed. Thus modems do not usually need any
     reconfiguring when you try different speeds.

#    It should also be noted that the current version of KA9Q cannot handle
     this throughput being set to 115000 due to a bug which should be fixed
     in the next release.

Why:
     There are two speeds involved in communications. First there is the
     speed at which bytes of data are transferred to and from the modem.
     This will be either down a cable to an external modem, or in the case
     of an internal modem, the speed at which the interface chips pass data
     to the rest of the card. This document calls this the "serial link
     speed", but elsewhere you may see it called the DCE/DTE speed. The
     second speed is the connection speed across the telephone wires.

     Not all that many years ago modems ran at 1200 baud, and you used 1200
     bps serial links to them. Modern modems use complicated encoding
     methods so that transitions on the wire no longer directly correspond
     to bits of data (this is the distinction which professionals and
     pedants like to make between bps and baud). Furthermore, if requested,
     the data itself is compressed (using MNP5, V42bis etc.) before it is
     transmitted and it is then re-expanded at the other end. As a result
     you can regularly get text transferred at 3000 or more bytes per
     second down a 14.4K link. (In fact, sequences of identical bytes can
     be compressed almost out of existence. The protocol headers on TCP/IP
     packets prevent this occurring, so it is very unusual to see speeds
     much above 3200 bytes per second).

     Since there are 10 bits per byte on a serial link, using 19200 bps
     will only allow you to transfer 1920 bytes per second. Obviously if
     you have a 14.4K modem which can move over 3000 bytes per second of
     textual data this can create a bottleneck. Using 38400 bps will almost
     invariably make the phone link the limiting factor. At the other end
     Demon connect their computers to their modems at 38400, thus overall
     (in a steady state) 38400 bps is sufficient.

     But a steady-state analysis is not the end of the story, since your PC
     might get a little behind in servicing the serial port (because of
     writing to disk for example). Therefore it is theoretically helpful to
     run at 57600 so that your PC can catch up, should it ever get behind
     in this way. However, since 38400 already exceeds the transfer rates
     that you will get on real data (which never contains indefinite runs
     of identical characters), then if your modem or serial card will only
     do 38400 this is not a matter to cause you much concern.

     At faster speeds your PC has less time to deal with incoming
     characters before the next one appears. For example, 38400 produces
     characters four times as fast as 9600. This can produce hardware
     overruns - which as discussed above can severely degrade throughput.
     If you get hardware overruns at 57600 then try 38400 since this will
     be just as good in practice.

     Similarly, if you get overruns at 38400 then try 19200. Some people
     state that a few hardware overruns at 38400 gives them better overall
     performance than no hardware overruns at 19200. The trade-off will
     depend upon how soon the TCP/IP link resends data and how quickly your
     acknowledgements reach the other end and this will in turn depend upon
     how much Internet there is between you and the data sender. It would
     naturally be better to try the other advice in this document and
     achieve 38400 and NO overruns!

     Of course not everyone has a 14.4K or 28.8k modem. Demon do not let
     you connect at 1200 because you cannot get packets back and forth
     quickly enough and some protocols time out. Many people use 2400 or
     9600 bps modems ... they may well find that there is no advantage in
     increasing serial link speeds past 4800 or 9600. However, provided
     that there are no hardware overruns there should be no disadvantage to
     higher speeds, so use them if you can.

#    Demon are now in the process of updating all their modems to v34. All
     the vPoPs have now been upgraded and work on the the older PoPs is
     being planned. Demon have also increased the speed of their terminal
     servers from 38400 to 115,000. This should give a considerable
     increase in performance if you have a V34 modem.

6.   Does Windows affect link speeds ?
==   =================================

Advice:
     Try running KA9Q in a full screen Windows DOS session !
     (or possibly, avoid running KA9Q in a Windows DOS session !!)

How:
     Provided your SYSTEM.INI file correctly lists your hardware you should
     have no problems using serial ports from a DOS session. Beware of
     serial mice ... in the usual configuration of a PC COM1 and 3 share
     the same IRQ (as do 2 and 4). This might not show up if you don't load
     a mouse driver except within Windows.

Why:
     Under Windows real hardware devices are handled by device drivers, and
     programs like KA9Q do not actually access the hardware directly. In
     principle this "virtualisation" slows things down so DOS sessions
     should be worse than not using Windows. However, because Windows
     device drivers are well integrated into the system, you may find you
     can use faster speeds without getting hardware overruns.

     Even the slowest PCs are quite fast enough to handle 3000 interrupts a
     second from a serial port. However, besides processing the incoming
     data the information is also stored on disk, and text is written to
     the screen to keep the user appraised of progress. Unhappily, whilst
     doing this DOS and the PC BIOS can lock out interrupts for relatively
     long periods, and so the PC is not able to break away from another
     task and service the serial port. Thus you can get hardware overruns.
     Avoiding any quasi-religious discussion of the merits of alternative
     operating systems, we can note that Windows has avoided these problems
     with DOS and the BIOS (in the jargon, it can handle short crisis time
     devices), and so you probably get fewer overruns than from raw DOS.

     The word "probably" in the last sentence was chosen carefully.
     Certainly many machines do run better, but some people do worse in
     Windows DOS sessions than otherwise, though seldom so much worse as to
     make it worth changing. Fast machines and SCSI drives seem to help you
     do better under Windows than DOS. Slow machines seem to work better
     under DOS than Windows. The only plausible advice seems to be "try it
     and see".

     Running in a Windows DOS session is much less likely to be beneficial
     if you are not in full screen mode. When the session is made a window
     on the desktop there is a significant overhead involved in rolling
     text up. This can cause data loss. If you change away to another
     window then how much attention KA9Q gets will depend upon the other
     program's ability to share machine time, and on the various priorities
     of the tasks you are running.

7.   Eliminate SMARTDRV, TSRs etc.
==   =============================

Advice:
     If you are getting hardware overruns then experiment in running
     without DBLSPACE, SMARTDRV, MIRROR or inessential TSRs.

     Even if you don't remove these utilities, some configurations are
     worse than others, so parameter twiddling may help.

How:
     RTFM (for DOS or the TSR)

Why:
     As indicated in the previous section, you get overruns when your PC is
     not informed (by an interrupt) of an incoming character, until it is
     too late. Disk compression and disk caching software can disable
     interrupts for long periods and thus cause problems. In general, the
     faster your machine the faster it will be executing these critical
     sections, and so the less likely you are to have problems.

     DBLSPACE seems to have caused a lot of problems to people who have
     tried to use it. A common workaround is to arrange for the incoming
     newsfeed (in _\spool\articles) to be on an uncompressed disk, whereas
     the newsbase (_\spool\news\newsbase) is kept on a compressed area
     since it is only used off-line when disabling interrupts is of little
     consequence. If you want to change the incoming temporary directory
     then either edit the configuration files directly, or use the DIS
     front-end [E J A] to change the line starting "newsdir"; then [D F A]
     to change the line starting "nntp dir" to correspond. You then need to
     edit DEMON.BAT by hand; in the section starting :net change the
     directory tested for BATCH.TXT.

     SMARTDRV has also caused problems, though many people seem to feel
     that it can improve throughput if you can make a big cache with
     delayed writes. Small caches with delayed writes seem to cause
     problems. Big caches without delayed writes generally seem to be OK.
     These results may just be caused by the bigger caches making it less
     likely that *any* disk transfers are needed. Similarly, since most
     data is incoming, turning off write caching will mean that a disk
     cache has little enough to do whilst you are on-line.

     This is obviously an area which requires experimentation to determine
     the best solution for your configuration. You could try editing the
     DEMON.BAT file to add extra enable/disable commands before and after
     running KA9Q.

8.   Serial chips
==   ============

Advice:
     Use a 16550A serial chip

     If you fit a 16550A you can probably ignore all the other advice in
     this document about eliminating hardware overruns, because you will
     find that no matter what, none will occur. Or to put it another way,
     if 16550As were free, sections 4..8 of this document could be
     discarded altogether.

How:
     Most PCs are still shipped with 8250 serial chips (UARTs). If socketed
     they can be directly replaced by a 16550A. If soldered in, or if your
     serial card is a modern highly integrated device (doing parallel,
     disks, washing machine etc.) then you will need to add another serial
     card. If you don't have a spare slot then you'll need to buy a
     multifunction card with a 16550A on it.

     A quick glance at MicroMart (Thursdays, 60p) will yield literally
     dozens of suppliers of 16550A boards and chips. They change so fast
     that specific recommendations are impossible, but as a guide you
     should pay no more than 20 pounds (+VAT & postage) for a serial card
     and a fiver or more less than this for just the chip on its own. Fans,
     and one-stop shoppers, can also purchase these items directly from
     Demon, but although service and support may be better, their prices
     are less keen.

     If you aren't sure what sort of chip you have then the Microsoft
     supplied program MSD.EXE (in your DOS or Windows directory) can
     probably tell you. Run the program outside of Windows for the most
     reliable answer. However MSD can become confused by some multi-
     function cards, and if you have an unusual configuration it can fail
     to identify which port is which! Numerous other utilities exist which
     will check out your serial cards. Try ftp.demon.co.uk for
     /pub/simtel20/msdos/modem/modemd52.zip.

     Also, of course, KA9Q will tell you the details of the port which it
     is using. See the description of asystat in the next section.

     In passing, it should be noted that there are a fair number of
     alternative serial chips, as well as dual port versions of the 16550A.
     The 16550 is not suitable (it contains a FIFO, but it does not work
     properly). KA9Q correctly detects all the chip variants whose FIFO can
     be used. Christian Blum has collated an excellent FAQ called
     The_Serial_Port which covers the various alternative chips in detail.
     You can find this at pfsparc02.phil15.uni-sb.de in the directory
     /pub/E-Technik/afd. We will not repeat all of the information here
     since:

          *    the 16550 has not been manufactured for years so now you are
               unlikely to be offered anything other than a proper 16550A,

          *    the various other letters at the start and end of the chip
               part number are manufacturer specific, or just tell you
               whether it is NMOS or CMOS.

     so just order a "16550A" card or chip, and of course if it is a
     replacement then tell the supplier what the current chip designation
     is. In the unlikely event you get the wrong device then the Sale of
     Goods Act will protect your investment :-)

Explanation:
     The important difference between an 8250 and a 16550A is that the
     latter contains a 16 byte first-in first-out ("FIFO") buffer. This
     means that when bytes turn up on the serial link the "crisis time"
     before they are overwritten by further characters is significantly
     extended. As previously mentioned, when a PC is doing nothing but
     processing serial data there is no problem in it keeping up, and it
     turns out that the modest increase in buffering provided by a 16550A
     is quite sufficient for current applications and modem speeds.

     In fact the buffering is more than is needed in practice, so that a
     secondary benefit is possible. Instead of interrupting as soon as a
     character turns up (and giving the PC 15 or so more character times to
     respond) the chip can be configured by software to buffer several
     characters before interrupting at all (whilst still leaving several
     spare slots in the FIFO for further characters). This means that the
     PC is interrupted less often, and this can improve performance
     slightly. Naturally, there is a scheme whereby if no further
     characters seem to be appearing, then an interrupt is generated for
     the partially filled buffer - you can detect this happening with the
     asystat command.

     The reduction of interrupt load of a 16550A is of particular benefit
     to Windows which you will recall must "virtualise" the hardware
     device. This is all overhead, and the less the better. Sadly the
     standard drivers shipped by Microsoft take limited advantage of the
     16550A. There are some third-party alternatives such as the Cybercom
     driver on ftp.demon.co.uk in /pub/ibmpc/windows/utilities/cyberc.zip.
     It uses different settings to make better use of the FIFO buffers. The
     difference is said to be greatest on Windows 3.0, and hard to measure
     on 3.11.

     The bottom line is that a 16550A is a Good Thing and as many people
     know, not only in theory! As modem speeds increase it will become ever
     more necessary. Further, it can help a bit even if you weren't getting
     hardware overruns - and it might let you put back some of your TSRs
     (DBLSPACE, SMARTDRV or whatever).

     Of course, if you want to be really fancy you can buy proprietary
     enhanced serial interfaces or even serial boards with 16K buffers on
     them... but not for the same price as 16550As!

9.   The KA9Q asystat command
==   ========================

Advice:
     Use the asystat command to check for hardware and software overruns.

Discussion:
     When you type asystat at the net> prompt KA9Q will tell you a lot of
     useful low level information about how the serial link has been
     performing so far this session.

     e.g.

     sl0: [NS16550A] [trigger 0x7e] 38400 bps
     RX: x int, x chr, x hw over, x hw hi, x fifo TO, x sw over, x sw hi
     TX: x int, x chr, x q, x MS int, x THRE TO

     The first line tells you about the hardware configuration:

     'sl0' is the mnemonic name for serial link interface zero.

     '[NS16550A]' shows that KA9Q has detected that a 16550A is fitted and
     it is using the FIFOs. It will be absent if you have some lesser chip.

     '[trigger 0x7e]' is to do with the protocol used, and is not of
     general interest.

     '38400bps' is the serial link speed between the modem and PC.

     The first line will also tell you if CTS flow control and/or RLSD
     (carrier detect) line control is enabled.

     The second line gives statistics for received (RX) information.

     'int' is the total number of interrupts which have occurred.

     'chr' is the total number of characters received.

     These numbers show you how busy your link has been. On a 16550A
     (though not necessarily on the Hayes ESP interface) you will see that
     'chr' is much more than 'int' because characters are transferred in
     batches. These are raw numbers, including all protocol headers, escape
     characters and any duplicate data.

     'hw over' is the total number of hardware overruns which have
     occurred. ie this is the number of individual characters which have
     been lost. FOR BEST RESULTS THIS VALUE SHOULD BE ZERO!

     'hw hi' is the highest number of characters read during a single
     interrupt. It is reset to zero every time you do an asystat command.
     As this number approaches the buffer size in your chip it indicates
     how much of a risk you are running of getting a 'hw over'. If you run
     under Windows you may well see values of 30 or more here. This is
     showing you how many characters the device driver is buffering, rather
     than the size of the buffer in the chip itself.

     'fifo TO' is only reported for 16550As. It is the number of times
     interrupts were generated because characters were in the buffer but no
     more were arriving. TO stands for time out.

     'sw over' is the number of times that the KA9Q buffers have
     overflowed. Just as with hardware overflows this is bad news. FOR BEST
     RESULTS THIS VALUE SHOULD BE ZERO!

     If you are getting 'sw over's then you need to increase the buffer
     sizes by altering the attach command to allocate a larger input "ring
     buffer":

     e.g. attach asy 0x3e8 4 ppp sl0 4096 1500 38400
                                     ----
     Try 8192 or even 16384. Actually, any value you use will be taken to
     the nearest multiple of 8, so you don't need these very "techie"
     powers of 2 for the size, so feel free to try 10000, 11000 etc.

     'sw hi' is the "high water mark" showing the maximum space ever used
     within the KA9Q buffer. If this value is always much less than the
     buffer size then you could safely reduce the buffer size, and free up
     memory for other purposes. This value is reset to zero by the asystat
     command.

     The third line gives statistics for transmitted (TX) information.

     'int' is the total number of interrupts which have occurred.

               'chr' is the total number of characters transmitted.

               'q' is the total characters currently queued for sending.

               'MS int' counts modem status interrupts. You'll usually see
               a handful of these, corresponding to your modem managing to
               CONNECT. If you are using CTS or RLSD line control then this
               number will tell you how often this is occurring.

               'THRE TO' relates to transmit interrupt timeouts. It is not
               of general interest.

10.  Internal modems
==   ===============

     All of the advice given in sections 4..7 and 9 applies equally well to
     internal modems.

     An external modem lives at the other end of a cable from the computer.
     As characters arrive over the phone line it will send them down the
     cable to the computer whether or not the last one has been dealt with
     yet. An internal modem is directly connected to the serial port
     hardware within the computer, in fact they will be on the same board
     if not the same chip. Thus an internal modem has access to the serial
     chip and can therefore, in principle, "know" whether or not the
     computer has processed the last character yet. This allows internal
     modems to use their own buffering system to avoid "overruns", should
     the computer occasionally be slow at processing incoming characters.

     For their serial port hardware internal modems may actually use an
     8250 or a 16450 chip (much the same as far as this document is
     concerned), or more likely they will contain some custom chips which
     emulate one or the other. Either way, for the reasons just mentioned,
     they will not generally exhibit the same sort of overrun problems that
     an external modem with a real 8250 would suffer from. Thus you should
     not be concerned if your modem does not contain a 16550A.

     To be perfectionist, everything else being equal, then you would
     prefer an internal modem to emulate a 16550A rather an 8250 because of
     the lower interrupt load. However, provided the buffering is adequate,
     the difference in KA9Q performance caused by fewer interrupts may be
     hard to measure.

11.  IP fragmentation
===  =================

Advice:
     Make sure that incoming packets are not being fragmented.

How to detect fragmentation:
     Use the KA9Q "ip status" command. Look at the variable (14) called
     ipReasmReqds. If it is not zero then you are getting fragmented
     packets and this must be corrected.

     The other statistics count IP packets and most of the other
     fragmentation values (the ones towards the end of the list) should
     also be zero except ipReasmTimeout which will be 30. This is the time
     KA9Q waits for the rest of a fragmented packet to appear and hence it
     is correct that it is not zero!

Background - what is fragmentation:
     When data is sent from machine to machine over a TCP/IP link it is
     parcelled up into packets. The end machines agree the size to be used
     which is called the maximum segment size (MSS). The data has 40 bytes
     of TCP/IP header added and is then placed inside of a hardware
     datagram, on a Ethernet, X25 network or whatever is used to move the
     data across the Internet, possibly through 30 or 40 hops from the
     other side of the planet to your machine.

     If at any stage the data will not fit into the datagram for the next
     hop it will be split up (fragmented) and the fragments travel onward
     to be reassembled within KA9Q. Since the fragments each have their own
     header there is an extra overhead associated with fragmentation. You
     might think that this overhead was the difference between (header(40)
     + data(n)) and (40 + n/2 + 40 + n/2 = 80+n) but it is in fact a great
     deal worse than this...

     ... On PPP connections (and indeed on SLIP as well) the packet headers
     are usually sent using "Van Jacobson" (VJ) compression. This very
     clever scheme means that headers are typically transferred in only 5
     bytes or so instead of the usual 40. However, fragmented packets are
     never compressed (since in a well-ordered network they should never
     occur). Thus, in the example, the difference fragmentation makes is
     between (5 + n) and (80 + n) [approximately]. On a typical connection
     to Demon using the standard KA9Q configuration fragmentation will
     degrade your performance about EIGHT PERCENT. This is very
     significant, and is well worth avoiding.

     There is a scheme which is being introduced across the Internet to
     allow machines to dynamically determine the MTU (maximum transfer
     unit) between them. This "path MTU discovery" is used to ensure that
     datagrams are never bigger than the smallest size on any link they
     must traverse. Sadly, many machines do not use this scheme, so human
     intervention is required to set the values correctly.

More advice:
     Use an MSS of 536. Then set the KA9Q ppp link parameters to use MSS+40
     (ie 576) as the datagram size (the MTU). This in most cases is the
     default setting.

     You can do about 0.6% better than this in one special case...

     If you are connecting ONLY to Finchley (not to any other PoPs) AND you
     will not be using the trans-Atlantic link (ie you are only going to
     receive email and Usenet news, or use the local ftp.demon.co.uk) THEN
     (and only then) you can use an MSS of 1460. Set the ppp link
     parameters to MSS+40 (ie 1500). You MUST also change your login script
     to specify the protocol as "PPP,mru=1500" rather than just PPP. (mru
     is an equivalent acronym for MTU!). Remove the mru=1500 if you change
     away from 1500::1460 otherwise KA9Q will fail to connect.

Note:
     In the past Demon used different datagram sizes on their links. In
     that different world, different settings were optimal. Any previous,
     now out of date, advice should be ignored!

     Always using the 576::536 (MTU::MSS) setting will cost you less than a
     1% overhead compared with "optimum" 1500::1460 value for Finchley only
     usage. Using 1500::1460 to any other PoP, or across the Atlantic will
     immediately risk an 8% degradation. So unless you are very sure that
     your usage fits the special case you should pick 576::536.

     If you regularly transfer files from non-Demon server machines then
     you should check for IP fragmentation. Although 576::536 is a good
     setting for most places on the Internet there is no guarantee that
     smaller values are not being used on some of the links the packets
     traverse.

     If you use telnet a lot then more advice is given below.

How:
     Decide if you are going to use an MSS of 1460 or 536 (or perhaps less,
     see later advice).

     Now edit the autoexec.net file in the main KA9Q directory. You change
     the lines:

          attach asy 0x3e8 4 ppp sl0 4096 1500 38400
                                          ----
          tcp mss 1460
                  ----
          ppp sl0 lcp local  mru 1500
                                 ----
          ppp sl0 lcp remote mru 1500
                                 ----

     Note that sl0 is "s" "el" "zero" (serial line 0); an error-prone
     choice of identifier! You change the 1460 value to your chosen MSS
     (usually 536) and the 1500 value to MSS + 40 (usually 576).

     Also, if you are not using 576::536 (for example if you choose to use
     1500::1460) then you need to edit the dialler sequences to respond
     "ppp,mru=1500" to the "Protocol" prompt. You can do this from the DIS
     front end (option D, option A, f5) or by editing the DIALER file
     directly. The mru= value should correspond to your MSS + 40 value.
     There is no harm in using mru=576, but it is not necessary.

Why:
     The situation with remote tPoPs (Warrington, Reading etc.) is simplest
     to explain. Demon have configured the link to and from Finchley to use
     576 byte datagrams, ie there is room for 40 bytes of TCP/IP header and
     536 bytes of data. They have done this because they want short packets
     travelling the link, so that when you call the PoP the login sequence
     can transfer information back and forth without having to wait for
     very long packets to finish. This 576 value is fixed, so you must set
     the MSS to 536 or less (and therefore set the total packet size in the
     ppp parameters to 576 or less).

     As the vPoPs are based at the Finchley Network Centre they should have
     the same performance as the original Finchely tPoP.


     Similarly, the transatlantic link is configured to use 576 byte
     datagrams (again to improve general responsiveness). Any packets
     larger than 576 are fragmented before they can be carried across it.

     The routers at Finchley have a default configuration of 576, so there
     is no problem in connecting to them at this size.

     However, Demon's Ethernet LAN uses 1500 byte datagrams, and it is
     possible to configure the Finchley routers to use 1500 byte datagrams
     on the phone link by adding "mru=1500" to the login sequence. You then
     set the MSS to 1460 and configure the KA9Q ppp link to 1500. Provided
     your traffic just flows across the Demon LAN (ie is just incoming
     email, Usenet news and local FTP) then it will not be fragmented.

     In theory, you would only need to alter the KA9Q PPP settings, and the
     magic of the PPP protocol will correctly reconfigure the router at the
     other end of the link. However, the routers at Finchley have been
     configured to 576 in a "broken" way (ie not conforming to the RFCs)
     and so if you set 1500::1460 (the default KA9Q configuration!) then
     the link will actually be configured to run at 576::1460 ie with
     fragmentation occurring and 8% less throughput than you might expect.
     Setting mru=1500 stops this problem occurring. Configuring to 576::536
     also prevents the problem.

     When the ramifications of the "broken" routers on IP fragmentation
     were first beginning to be understood it was found that you could get
     1500::1460 by asking for 1501::1460 (!) and for a while setting 1501
     was the best available advice. This still works, but the advice above
     is simpler to understand and apply.

     Remember that all of the numbers given above (except for 1501::1460)
     correctly use the relationship "n"::"n-40". The values explicitly
     EXCLUDE the PPP header byte and various other red tape associated with
     using PPP, which can be ignored when determining the numbers in this
     section.

     As it happens, there is a bug in the PPP code of KA9Q (reported, but
     not yet corrected). This means that negotiating a link value which is
     smaller than the link value at the Demon end will fail, and you will
     get to the HELLO prompt but no further. Negotiating to higher values
     does not seem to suffer from this problem ... except for the problem
     of negotiating up to 1500 discussed above.

     The simplest way of avoiding any problems is to always set the mru=
     value on the login line to the same as the value you want KA9Q to
     negotiate. Thus if you wanted to use, say, a small mss value like 266
     then you must set the ppp negotiation to 306 (ie 266+40) and you
     should set the login line sent by the dialler to be "PPP,mru=306".

Even more advice:
     If you regularly use telnet at the same time as an NNTP news transfer
     (or FTP file transfers) then consider a smaller MSS.

How:
     As explained above, set the tcp mss command to "n" and the attach and
     two ppp setup commands to n+40. You must also set the dialler login
     line to specify mru= the n+40 value.

Why:
     When you are using the telnet protocol you are sending very small
     packets (with just a character or two in each). In the absence of
     fancy priority queuing these packets must wait their turn behind the
     big 1500 byte packets. Since 1500 bytes of FTP will take about a
     second to come over the link you can see that this can make telnet
     response rather jerky. Smaller packets mean that the telnet
     information can flow more smoothly. This is in fact much the same
     scenario as the speeding up of remote PoP logins discussed above.

     There is obviously some overhead in using smaller packets. However
     because the headers are compressed so much, it is not very high. Van
     Jacobson's RFC on compression recommends choosing 240 at 9600 bps. At
     this size, you can send 1460 bytes in just over 6 packets. Thus the
     extra overhead is about 25 bytes per 1500. ie even with quite small
     packets the "cost" is only about 1.6%.

     Note that if you are *only* doing telnet then the only packets for
     transfer will be telnet packets and so you will see no difference in
     response no matter what transfer sizes you use. See also section 15
     for other reasons for slow telnet response.

12.  NNTP request queues
===  ===================

     Some of the throughput problems with Usenet news are caused by delays
     at the Demon NNTP server. It is possible to improve the flow by an
     "nntp batch on 12" command in autoexec.net. This issues requests for
     articles 12 ahead so that the overworked news machine is able to start
     on fetching further articles whilst KA9Q is still receiving a previous
     one. You can change this setting from the DIS front end program [D A
     f6].

     [The value 12 is chosen random(ish)ly. It has been suggested that even
     with higher values the flow is still more "jerky" than might be
     desirable.]

13.  HISTORY
===  =======

Advice:
     If you are getting slow transfer times then slim down your HISTORY
     file (by hand or by expiring some news).

     Slow transfers are a problem that comes suddenly upon people after
     days or weeks of acceptable performance. It only affects NEWS; email
     and FTP will still work at full speed.

How:
     Besides discarding or archiving old news the EXPIRE program will also
     trim your HISTORY file (kept in _\spool\news) according to the
     criteria set in EXPIRE.DAT (same place).

     The relevant command is: [tail] 1000

     where 1000 is the number of news article identifiers to be retained in
     the HISTORY file. A sensible value is just over one day's worth of
     articles, but anything under 2000 is likely to be OK.

Note:
     If you run expire directly (as in "expire 10") then it will not use
     EXPIRE.DAT and it will not reduce the size of your HISTORY file. You
     will need to take other steps to do this.

Why:
     The purpose of the HISTORY file is to record article identifiers which
     have already been downloaded to your system, so that even if you are
     offered them again you will not refetch them, but will instead see
     them reported as duplicates. KA9Q keeps this list of identifiers in
     memory if possible, but failing that it has to read the list off disk
     in order to check. It is these disk transfers which ruin the
     performance.

     You can of course use different values than 1000 for tail, indeed some
     people use 0, and can thus avoid using expire but just delete the
     HISTORY file altogether. If you use 0 then any duplicate news will be
     downloaded, but the SNews unbatch program will still discard the
     duplicates using the HISTORY.SNW file (whose contents are controlled
     by the [life] parameter).

     It is usual to be offered a few duplicates at the start of each NNTP
     session (because of the way the "last fetched news" time is handled).
     Therefore, if you use [tail] 0 you must be prepared to waste time
     downloading a few articles which will be discarded by unbatch. A
     higher setting is therefore to be recommended.

Note:
     Some people have managed to damage their EXPIRE.DAT so that the [tail]
     command is missing altogether. Of course, running expire in this state
     will not reduce the size of the HISTORY file.

14.  PPP versus SLIP
===  ===============

Advice:
     Use PPP.

How:
     Send "PPP" to the protocol prompt. This is the default set into the
     DIS front end program. (Better, usually is "PPP,mru=1500" or whatever
     size you require; see section 11 above.)

Discussion:
     Demon recommend using PPP for connections with KA9Q, and the latest
     versions will have had SLIP removed. There are better diagnostic tools
     at the Demon end for PPP problems, and this doubtless contributes to
     their preference for it. However, Windows applications working through
     Winsock usually seem to use SLIP instead. Sometimes people ask why...

     SLIP is a fairly simple scheme for sending TCP/IP packets down a
     serial wire. The code is trivial (assuming you are happy about
     interrupts and serial chips and such). Compressing SLIP headers
     (RFC1144) is not entirely trivial, but is not a vast amount of code
     and improves telnet responsiveness especially (and reduces link
     traffic generally). Some example code is in the RFC so it is pretty
     easy to add to an existing program. This means that people can easily
     support SLIP - so they do. Some people use the name CSLIP for SLIP-
     plus-header-compression. This document does not bother to distinguish,
     because there are very few implementations of SLIP which are not CSLIP
     as well.

     PPP is a rather fancier animal altogether. It will handle multiple
     protocols down the same wire (which is not very relevant in this
     context). It can also deal with links which use software flow control
     or which cannot transmit some characters (which is also not very
     relevant). With suitable compression negotiated you get much the same
     overhead per packet as with SLIP (it rather depends quite what you
     send!). Since it is a bigger (and newer (the latest RFC came out just
     before Xmas)) protocol there are rather fewer implementations around,
     however KA9Q does have PPP built in.

     Winsock protocol stacks often use packet drivers for their network
     connections. The Crynwr set of freely available packet drivers does
     not at present contain a PPP driver. There are in fact very few PPP
     packet drivers around (which you don't have to pay for) and their
     reputation for using 8250s (as opposed to 16550As) is not very good.
     The Trumpet shareware Winsock stack can either use a packet driver or
     its own internal SLIP. The very latest beta versions of the Trumpet
     stack now support PPP and some people have used this successfully (and
     others have not). All of this means that at present most Winsock users
     are connecting with SLIP.

     The received wisdom is that PPP is a teeny bit faster in practice than
     SLIP, but only by a little bit. Since they both tend to add the same
     protocol overheads of single bytes each end of a packet, and can both
     use header compression the transfer rates will depend upon horrible
     practical issues of error recovery which will depend on the sort of
     corruption you get on your link; whether your error correcting modem
     actually does correct your errors; and how many data octets [ that's
     bytes :-) ] need escaping.

     This similarity in performance means that other issues elsewhere in
     the software can easily swamp the differences between competent
     implementations of either protocol. So the bottom line is that you
     should use PPP when you can, and SLIP if you can't, and it doesn't
     make a lot of difference either way.

15.  Telnet speed
===  ============

     Telnet responsiveness was discussed above in the section on IP
     fragmentation. However, you should also be aware that you can get very
     "soggy" links when the remote end is echoing what you type.

     The reason is simply that the other machine is a long way away. It's
     typically 300ms to Finchley, 800ms to the US and 1.6 seconds or more
     to Europe (because European traffic crosses the Atlantic twice for
     reasons more to do with politics than technical issues). You can't
     change these routes - you're stuck with them - and so echoes from the
     remote machine can take a long time. Also, when dealing with European
     machines, you will find that the infrastructure is slower and many
     more machines are connected by fairly slow and congested links.

     You can use "ping" to find out how far away the remote machine is. Try
     "ping xx.xx.xx" when you've gone "telnet xx.xx.xx" to see how far away
     the machine is. If you're interested in the path taken there then try
     "hop check xx.xx.xx" and you'll see the little tour of the Eastern US
     states that most traffic takes.

     When using "ping" you must remember that the packets it is sending are
     queued for transmission in the normal way. Thus if you "ping" during
     news transfer you will get higher values than otherwise. Since this is
     pretty much what happens to telnet traffic, this may contribute to the
     information you need to tune your packet sizes.

16.  Other improvements
===  ==================

     Whilst receiving news and email the incoming data must eventually be
     written to disk. The usual suggestions for increasing disk performance
     apply : defragment the drive, set BUFFERS= to a sensible value, use
     SMARTDRV, use a RAMDISK etc. All of these are Good Things provided, as
     discussed extensively above, you check that they don't introduce
     hardware overruns.

     Several people with more than one drive have reported improvements
     from ensuring that the incoming _\spool\articles\BATCH.TXT file is
     placed on the faster drive.

     Finally, it has been suggested that provided your modem has software
     flow control disabled (ie you can send _Q and _S (XON, XOFF) as data)
     then there is little point in having the ppp protocol escape these
     values. This change would make a marginal difference to ZIP transfer
     speeds, but none at all to news and other text transfers. Has anyone
     tried this ?

17.  OS/2
==   ====

     All multi-tasking operating systems use some of the available machine
     performance to do their magic, so if you are short of machine power
     then OS/2 (or indeed NT or plain Windows) will use machine cycles
     which would otherwise be used to execute KA9Q.

     However, most modern machines are fast enough that you are in practice
     unaffected by the overhead, and you are actually limited by the serial
     link speed. On such a machine (like a 25 MHz 486) you will see no
     difference between, for example, KA9Q for OS/2, KA9Q in an OS/2 DOS
     box or KA9Q running under real DOS. On a more modest PC (like a 20MHz
     386SX) you will find that using OS/2 will slow down News collection
     considerably (1400 cps rather than 2700, say), but that FTP speeds
     will be almost unaffected.

     You have the option of running two different flavours of KA9Q under
     OS/2:

          *    DOS KA9Q v2.16 in an OS/2 DOS session earlier versions do
               not have important buffering fixes.

          *    OS/2 KA9Q v2.17 (OS/2 version 2.40) running as a PM
               application and including Archie and Gopher clients.

     again, avoid earlier versions because OS/2 version 2.40 fixed a long-
     time bug that caused lockups. The latest version is on ftp.demon.co.uk
     in /pub/os2/netos2.
     This version is exempt from advice in earlier versions of this
     document to specially boost the process priorities.

     You should install Ray Gwinn's shareware SIO/VSIO serial port drivers
     to get a virtual buffered serial port with a 4096 character buffer.
     These drivers provide most benefit to DOS comms applications running
     under OS/2. Find these on ftp.demon.co.uk in /pub/os2/netos2 as the
     file sioXXXX.zip, where XXXX is the version number. Don't forget to
     register them.

     A 16550A is strongly recommended for use with OS/2, although the
     SIO/VSIO drivers can provide near 16550 performance on machines with
     unbuffered UARTs.

     To set the "performance baseline" for your machine you could try
     booting "real DOS" and running DOS KA9Q v2.16. (This will only be
     possible if you have dual boot or boot manager installed and KA9Q is
     running from a FAT partition.) You can use this performance level to
     assess how well your OS/2 setup is doing.

     Note that the OS/2 and DOS versions of KA9Q will happily share the
     same configuration files, so swapping around to experiment is
     relatively easy.

18.  The Authors
===  ===========

This FAQ is maintained by Richard Palmer (Tuning@blackbrd.demon.co.uk It is
based on the Tuning FAQ by Richard Clayton and Phineas R John. Many of the
ideas and advice that have been culled from the demon.ip.support.*
newsgroups. I hope that the original authors will be pleased to see their
ideas here, even if their names are absent.


19.  Disclaimer
===  ==========

Although whilst writing this document I have tried to be accurate, and to
give good advice, I have not spent the time or effort on it that would be
necessary for it to be in any sense authoritative. In particular the
efforts I have made fall short of the sort of standard which would be
expected from us in our professional capacities. You follow any advice
within this document entirely at your own risk. Take a note of settings
before you change them, and always take a back up copy of data which is of
value to you.

Naturally I would be pleased to receive email with corrections and
suggestions for improvements and alterations. Please write to:

     Tuning@blackbrd.demon.co.uk


---------------------------------------------------------------------------
Copyright 1995 Richard J Palmer (Tuning@blackbrd.demon.co.uk)
Original FAQ Copyright Richard Clayton & Phineas R John.

This file may be freely distributed provided that it remains unedited from
its current form. The latest version is posted regularly to the newsgroup
demon.ip.support.pc. It is available by FTP from ftp.demon.co.uk as
pub/doc/ka9q/Tuning.faq or can be obtained upon email request to
Tuning@blackbrd.demon.co.uk

end of TUNING.FAQ Issue 29                             10th July 1995

---------------------------------------------------------------------------