Text preview for : IFS_Status_and_Plans_Nov77.pdf part of xerox IFS Status and Plans Nov77 xerox alto memos_1977 IFS_Status_and_Plans_Nov77.pdf
Back to : IFS_Status_and_Plans_Nov7 | Home
Inter-Office Memorandum
To IFS Project Date November 10, 1977
From Ed Taft and David Boggs Location Palo Alto
Subject IFS Status and Plans Organization PARC/CSL
)(EROX
Filed on: StatusAndPlans.bravo
IFS is now at a stage in which it is available as a limited service, with limited capabilities
and reasonably good reliability. It is now time to pause and consider how much additional
effort should be invested in the project, and in what areas.
We recently conducted a review of the project, and our tentative decisions are presented here
also. Members of the review committee were Chuck Geschke, Butler Lampson, Severo
Ornstein, and Mike Schroeder.
Present Status
As a file server, IFS now supports most of the facilities available on Maxc. Exceptions
include file protections, direct shipment of files for printing, and various odds and ends.
With completion of the Backup system and Scavenger, file storage is now quite reliable.
The most serious shortcoming of the present system is its performance. We presently permit
only five simultaneous users, and even with that load the system's performance is often very
poor. Until recently the system crashed occasionally due to deadlocks arising from running
out of memory.
We have not yet made a detailed analysis of the performance bottlenecks, but we have a good
idea of where the problems lie. Unfortunately, they are not amenable to straightforward
sol ution.
The Alto's main memory is divided into three regions: resident code, resident storage, and
paging buffers for the software virtual memory. At present there are 32 1024-word paging
buffers, which are used for a variety of purposes: code overlays (of which there were 61 at .
last count), server stacks (one per FTP or Chat user), stream buffers (one per file actively
being transferred), B-Tree pages, large blocks allocated temporarily for scratch use, and
several others.
The manner in which the Bcpl stack and overlay mechanisms work exacerbates the main
memory shortage. Server stacks are allocated contiguously and can grow quite large (nearly
1024 words each). Once created, they cannot be moved or swapped out until the server is
destroyed. Similarly, a code overlay cannot be moved or swapped out if any of its
procedures is pending on any process's stack. Obviously, if there are several processes doing
. different things, many overlays will be locked in memory and few paging buffers will be left
. for other purposes.
2
We have recently expended about two man-weeks of effort in an attempt to increase the
number of paging buffers. The result has been an increase from 27 to 32, which we hope
will reduce the likelihood of deadlocks but is otherwise unlikely to improve performance in
a significant way. We have reached the point of diminishing returns in this effort. There is
little prospect of gaining any more Alto memory for paging.
We have given some thought to the possibility of using an Alto-II with extended memory.
(Indeed, it was this possibility that led to the redesign and generalization of the extended
memory option.) There is no straightforward way of using the extended memory in a
general manner from within the Bcpl environment. However, there are a few instances of
objects (such as stream buffers) that are referenced in sufficiently limited and stylized ways
that it may be possible to locate them in the alternate bank.
Another possible use of the extended memory is as a cache for the disk, particularly for code
overlays. We believe that swapping of code overlays accounts for a substantial fraction
(perhaps more than half) of the disk transfers during heavy load. While use of a cache will
not directly alleviate the situations that lead to deadlocks, it may improve throughput
sufficiently that the number of simultaneous users will become fewer. (On the other hand,
this will encourage more usage, so it won't help in the long term.)
Severo Ornstein is arranging that CSL's one extended-memory Alto will go to the Juniper
project (in exchange for one of theirs), and Howard Sturgis has agreed that we can share
access to that machine to facilitate experiments. If the results are favorable, that machine
might replace the Alto-( that now serves as the IFS machine.
In any event, the disappointing performance of IFS is the principal reason we have not yet
taken the plunge to move common files (such as the Alto dire,ctory) from Maxcl onto IFS.
At this point, we believe it would be unwise to do so until performance has been improved
sign ifican tty.
Our List of Things To Do
This section contains a fairly complete list of all the IFS-related work we have contemplated
for the immediate future. As can be seen, it is rather formidable, so it is unlikely we will
reach the end of it before either abandoning the project or turning it over to someone else.
The ordering reflects our assessment of the relative importance of the various improvements.
I. A Low-Level File Access Protocol
The fundamental reason for the present performance problems is simply that we are trying
to make the server machine do more than it has capacity for. Of course, we have felt all
along that a lot of the present machinery (and in particular, everything having to do with the
user interface) ought really to run in the user machine; ollr attitude has been that the
present Chat interface to many of the file system functions is merely a stopgap measure
while we design a better file access protocol. However, we failed to anticipate the
performance problems that the interim arrangements have encountered.
For some time we have considered the design of a Page-Level Access Protocol that would
provide remote access to files at a very low level. Implementation of this protocol by a
server would be relatively inexpensive, so the server could support simultaneous access by
large numbers of users. An existing example of such a protocol is the one used by the
Woodstock Fi Ie System.
A Page-Level Access Protocol called "Pine" has been developed as part of the Juniper
project. An attempt has been made to design the protocol in such a way that it may be used
to access conventional file systems, such as IFS, as well as Juniper.
3
Given the ability to access IFS using such a protocol, we would then be in a position to
construct a subsystem (and a set of packages) that would run in the user's Alto in place of
FTP and Chat. All of the user interface and a substantial portion of the remaining code that
now runs in the IFS server would then be contained in the user machine, leaving the server
with more capacity to perform its basic functions.
Besides reducing the amount of code contained in the server, we expect this style of access to
relieve the server's burden in a second way. A Pine request may be thought of as a remote
procedure call: the user machine issues a single request, the server processes the request and
generates a response. and the interaction is finished. The amount of state information that
the server must maintain between requests is relatively small. It consists of connection state
(for reliable transmission and perhaps authentication) and open file information.
The important point is that given the remote procedure cal1 organization, a single server
process can service requests for all users, rather than requiring one serv~r process per user as
is the case for the present ITP and Chat servers. This will result in a significant savings in
main memory. (Actually, it is likely that we will distribute the requests among several
servers so that long-running operations, such as deleting files, will not needlessly delay
service of simpler requests.)
We therefore consider the implementation of a Pine protocol in IFS to be our highest-
priority task. Reinforcing this, we have already committed to providing a page-level
interface for use by Officetalk; the deadline for this is sometime during the first quarter of
next year.
This effort may be divided into several, relatively independent projects. First, we must
complete the design of the protocol itself (or mOdify the Juniper Pine protocol to serve our
needs). Second, we must implement the server process that maps incoming requests into calls
on existing internal IFS primitives. This should be relatively straightforward because we
have already organized the system primitives with precisely this application in mind.
Finally, we must implement the program that runs in the user machine. This entails a
considerable amount of work, depending on how ambitious we are to produce a pleasing user
interface. (It doesn't real1y matter whether it is written in Bcpl or Mesa.)
Recommendation: We will proceed with this effort, in cooperation with the Juniper project.
Will Crowther, who recently joined CSL, will participate in the protocol design and probably
a Mesa implementation. This work will be done in such a way as to benefit both IFS and
Juniper. Our hope is that the outcome will be a set of user programs compatible with IFS
and Juniper, at least for the most common functions.
2. Inter-Machine Access
The existence of multiple IFSs operated by different organizations has led to the obviolls
problem of sharing files among users of different systems. Since this problem is among the
ones considered by the Juniper project, it is inappropriate for us to expend too much effort
on it in IFS. However, some workable short-term solution is required.
This problem appears to break down into two cases. First, there are a number of large
collections of files that are so heavily used that they must be duplicated on each IFS. These
include such things as Alto subsystems, the Mesa system and library, fonts, documents, and
so forth. Some sort of automatic distribution and update mechanism is clearly called for.
(This problem has already been considered at some length by people in SOD and XEOS, and
an FTP-based update system is presently being constructed. It is somewhat cumbersome
because the primitives provided through FTP are inadequate. However, we might be a-ble to
solicit help from these quarters.) .
Second, there are files that individual llsers wish to share with others, some of whom might