June 2001 - Tony Stoneley
Preface
This tutorial is aimed at institutional computer officers in Cambridge University who need to get up to speed in simple use of the central IP address database. It assumes little, and skims over much. Experts kindly stand aside and keep quiet! The prime message of this tutorial is that there is no need to be frightened of the database.
Addresses, names, the DNS and the Database
A networked computer needs an address and a name. Both identify the machine uniquely. Given one you can find the other, once set up. The address, that thing made of four numbers joined with dots, is a low level thing, those numbers telling the network how to route the traffic. The name, that thing ending in cam.ac.uk, is a higher level invention that avoids mere humans having to worry about things designed for machines, and reflects administrative affairs such as responsibility rather than the nitty gritty detail of traffic delivery.
The apparatus that translates names into addresses and vice versa is the DNS, which is a specialised and distributed database. On its own, however, it's not really adequate for keeping administrative records as well.
Administrative records? Huh? What? Why? Well, this isn't the place for a lengthy justification of the need. Just take it as read that central records of what each and every address is being used for are both necessary and required. We now keep all that info in an Oracle database, hereinafter referred to as "the database" which can be updated via the web by authorised institutional computer officers. The DNS is automatically brought into line (usually at end of working day) to reflect the content of the database.
Access to the database
The database is a collection of tables held as part of a much larger database, the Jackdaw database that underpins all Computing Service user administration. To get access to the IP address database, you need a Raven account (which we assume you already have), then you can mail ip-register@uis.cam.ac.uk to ask for access.
Note that transfer of records from the old files used (only) by hostmaster into this new database is far from complete. Nearly all colleges are transferred, but rather few departments, as of June 2001. If your application seems to have got stuck at stage two, the reason is probably that your institution is not yet moved over.
Currently the only method of interacting with the database (other than asking hostmaster to do it for you) is via a web browser, and one of the overwhelmingly commonplace ones at that. Netscape is preferred, being what the developer uses. Internet Explorer has been reported to work, and the developer will make every effort to make it so. With any other browser you're on your own, at any rate for the time being.
If you are paranoid about security, and full marks if so, you may previously have restricted your browser and be wondering about what if anything needs opening up. Very little needs opening up. You do need cookies in respect of the host you are interacting with, but not more. You don't need Javascript, though a small number of easements such as automatic cursor positioning may eventually appear if Javascript is enabled. You certainly don't need Java, and there is no intention of using it at all in the near or medium future.
Ready to go? Point your browser at http://jackdaw.cam.ac.uk/, follow the link to the "Web based interface to the Jackdaw database", follow the prompts through the login, follow the link to "Administer IP addresses" and thence, if this is your first visit, to "single_ops".
Dealing with a single address/name/workstation - single_ops
single_ops lets you deal with simple workstations, their
names and their addresses, one at a time and fully interactively. It's
certainly the place to start. Let's begin by displaying the data for a
workstation that's already up and running, perhaps the one right in
front of you. At any event, pick one that is within your sphere of
authority, else you won't be able to see it, let alone modify it!
Either type its address (e.g. 131.111.256.1) into the "IP
address" field and click info_for_address, or type
its full name (e.g. possum.botolph.cam.ac.uk) into the
"hostname" field and click info_for_name. Either
way, the info recorded for the workstation should appear. With that
example in front of us, let's discuss the individual fields and what
they mean.
Right at the top of the page is a short section common to all these pages, containing things generically required, such as a link (help) to some documentation, links to other forms, a button for clearing the form, and most importantly a button for logging off. Your access to this database is privileged and becomes dangerous if it falls into the wrong hands. Don't walk away from a logged-in unprotected browser!
Below the line ruling off the generic stuff we come to this form in particular, where the first field we see is "Management zone", more often abbreviated as "mzone". Management zones are notional spheres of managerial interest and authority, and the field displays the name of the one in which this workstation lies, perhaps BOTOLPH. Typically a college or a department is a single mzone. Any single IP address lies in one and only one mzone. There is no overlap of mzones. All the addresses available to St Botolph's will lie in BOTOLPH. Often there will also be only one DNS domain, but possibly there may be more. A domain also belongs to one and only one mzone. So what does all that do for you? Well the nub of the matter is that you yourself are associated with one or more mzones, and what you can see (and modify) in the database is limited to those mzones. Being CO for St Botolph's you are associated with the BOTOLPH mzone and can see into it, but since you have only one job rather than going part time with St Bene't's you cannot see into the BENET mzone.
Alongside the "Management zone" field there's a button
mzone_info. Go on! Click it! Down at the bottom of the
page there should appear some summary info about the mzone, including
a list of subnets in which the mzone has a stake. Be not deceived,
though, because that doesn't mean the mzone owns those subnets in
their entirety. Ownership of just one address in a subnet is
sufficient to flag an interest. Often though, you do in fact own one
or more complete subnets. Just remember that strictly it's individual
addresses that belong, not subnets.
The next field we encounter is labelled "LAN", and quite possibly it contains "MAIN". A LAN in this context is not (or at any rate not necessarily) a physical LAN, rather an abstraction under which addresses can be grouped. Just as with mzones, each address belongs to one and only one LAN. The difference is that this time it's under your control, whereas mzones and their contents are fixed by the CS. You can invent LANs at will, and move addresses about between them. LANs (this type) did not exist in the records before the new database, and when the records were moved over someone made an initial guess at what LANs it might be useful to invent. At the very least one is required in each mzone, and there are plenty of mzones with just the one, conventionally called MAIN. Sometimes a college has a range of addresses reserved for some purpose such as administrative machines behind some firewall, and a LAN such as ADMIN has been invented to identify them. What's the point? This comes when we ask the system to pick an address for a new machine. If your address space is carefully carved up it's important that the system honours this. You force this by specifying which LAN to choose from. This is used in the example lower down.
The next two fields, subnet base and subnet width mean pretty much what you would think, the width being the mask width, e.g. the 24 in 131.111.256.0/24. You can use the subnet to constrain an automatic choice of address, but the LAN mechanism is usually better. Beware that to a degree a subnet is also an abstraction, and there are instances in the database of overlapping subnets.
Finally, in this section, we see "Address selection range", which can be used to constrain automatic choices of address during "register", described below.
Below a horizontal line ruling off the generic stuff we find the "IP address" and "hostname" fields, which we've already used (or used one of).
Following this is a "rename as" button plus field. Yes, it does the obvious. There was a choice between putting this close to the (current) hostname field, or with the other action buttons. It is described with them below.
After another rule-off we have a collection of descriptive fields said to be "Modify"able. For the most part their intended purposes are clear (I hope!), and I will be brief with them. "Equipment" is a rough description of what kind of machine this is and "location" of where it is.
"owner", "end user" and "sysadmin" indicate who's responsible for it. The owner might be and individual or the college ("College") or deparment ("Dept"). The sysadmin is the person who looks after it, perhaps an individual, maybe the same as the owner, perhaps the CO (you!), in which case "CO" is usually written. The end user is primarily an identification of whose desk it sits on, often its owner, but largely to help identify it in, say, a shared office. "n.a." or some such is used when no single individual can thus be singled out.
"remarks" is for, er, remarks.
"review date" can be useful, especially in respect of equipment or people who are here for probably limited periods, such as students. You can set the value to a date in the format illustrated where it says "last updated on" immediately below. It is then possible to search for all entries for which review is due or overdue, as described below.
The occasion on which the record was last updated is then given, together with the crsid of who did it.
We now come to a row of action buttons. Care! These are live, and there is no "undo"!
Finally, if we've just caused a display, the details are re-iterated as plain text.
So much for the fields and what they mean. Now to using them. We'll consider the action buttons in order of increasing awfulness.
Searching on incomplete data
The search button provides a limited but still useful means of
searching when you only know a little about the object(s) of your
search. It must be said that it's very inefficient and slow, and needs
to be used with discretion. Please don't cane the system with it,
thereby forcing it's early withdrawal! Anyway, it does a match on all
non-empty fields, with % being available as a wildcard
matching any string. Thus you could clear the form (so that relics in
other fields don't limit the search), put
pc%.botolph.cam.ac.uk in the hostname field and click
search, getting anything that has a name like pc01.botolph.cam.ac.uk,
pc02.botolph.cam.ac.uk, pcplod.botolph.cam.ac.uk, etc. The first or
only match, if any, is displayed in full detail. If more than one
thing matches you will also be told and offerred a selector+button
for getting any one of the others. Having done that, it's often
useful to use the browser's "Back" button to re-acquire
the selector, which is not carried forward to the new display.
The review date field is different. Rather than accepting a wildcard match as described above, which if you think about it would be pretty useless, a date written there for a search selects only those entries whose review date is less than or equal to it. In addition, you can write TODAY in the review date field for a search (and only for a search), which causes the search to pick out only those entries that are due or overdue for review now. (Other non-empty fields do still contribute to the search constraints - you may want to ensure that they are all cleared first.)
Modifying records
When a record is displayed, any of the "Modify"able data can be changed, even several items at once. Just change the fields as desired and click "modify". Et voila! Done!
Renaming
When a record is displayed it, can be renamed simply by typing the new name into the "rename as" field and clicking "rename".
(Renaming is conceptually different from modifying, for reasons which will become more apparent the more you know. Accept it for now. And no, you can't change an address in like manner.)
Registering a new machine
You have a new machine (or more likely your lucky user has) and you need a name and address assigned. First I suggest clearing the whole form with the "clear" button near the top. You could just clear fields you know you want cleared, leaving ones you want left, but I want to keep the example simple. First fill in everything you are sure about, like the equipment type and location and its owner/user/minder. You will have to choose a name for it: do so, and put it (fully qualified, e.g. drool.botolph.cam.ac.uk) in the hostname field. If you are highly organised and know what address you want assigned you can simply write it into the address field, but more likely you simply want the system to choose one, in which case leave the field empty. Given all that and no more, the system would pick any of the unused addresses available to you that is consistent with the domain name you have specified, but you may well want to constrain the choice. If you are well organised, the best way to do this will probably be by specifying the mzone and LAN. You can alternatively specify the subnet base (and optionally the width), or you can restrict the range within which the address must lie. Empty fields imply no such constraint. Having got everything ready take courage and click "register". It shouldn't need too much courage, since you can always "rescind" anything you can "register".
If the registration request fails, it will tell you why. With a bit of luck the error message will be plain English. If it's a frightening ORAgobbledegook message then read it slowly and try to use intuition about what it means. This usually works, honest! A probable one is "constraint mumble violated", and the particular mumble quoted will give you a clue about what it means. What you are seeing here is the underlying database structure. That low level is where the real security and integrity is built in, but down in the engine room there is little knowledge of what those on the bridge are trying to do, so the only possible error messages from such depths are necessarily technical. The higher level code, the code that implements these web forms, itself tries to stop you doing silly things and reports in your terms, but it doesn't always succeed.
If the registration succeeds as we hope, the consequentially established values such as IP address will be plonked into the appropriate fields of the form and a text message summarising the registration appears at the bottom of the page. Seen something like it before? Yes, this is the text that hostmaster would have cut/pasted into the reply if you had asked hir to do the registration for you.
Rescinding a registration
Rescinding a registration is easy. Just get it displayed, using
info_for_addr or info_for_host, and click "rescind". It's
arguably too easy, since there is no undo operation. Make darn sure
you mean it before clicking rescind!
Actually, you often can in a sense undo an accidental rescind, if you notice it soon enough. Use your browser's "Back" button to redisplay the previous screen, where you had the registration detail displayed just prior to the disaster, and then click "register". With a bit of luck a brand new registration identical to the old one should occur. Phew!
Dealing in bulk
Dealing with registrations interactively one by one is all very well,
but it's not so hot if you have a large batch or if you want to avoid
re-keying data to or from some other local database or whatever.
list_ops provides bulk data retrievals to local files and
bulk updates from prepared local files.
Go to the list_ops form and have a look at the display. It's divided
into two parts, one concerned with download of data from the database
to your workstation, the other with upload of data from you to the
database.
Let's start with a download. You can download all the records of all
registrations in (any of) your mzone(s), all records in a particular
name domain, all records in a particular LAN or all records in a
particular subnet. It is obvious, isn't it? For example, just
write BOTOLPH into the management zone field and click list_mzone, or
put botolph.cam.ac.uk into the domain field and click list_domain. The
result is a stream of data sent down to your browser, which should
prompt you for where to store it.
Note the option for specifying the record separator to be used. The default was chosen with some thought, but may not be what you want, and I have no reliable means of knowing what your workstation really wants.
You now have a file of data, so have a look at it. It's an ordinary plain text file so you can look at it "in the raw" with any text tool, but it may not look good that way and it's really intended for program consumption. In particular it should be easy to load as a spreadsheet. The file actually contains a set of tab separated columns. Each row describes one database entry, except that the first row consists of column names. The order of the columns is not defined and is subject to change: you must parse the column headings anew for each file.
Before we go on to consider uploading a word of warning is in order,
several people having rushed off with the wrong idea at this point. It
is convenient to speak of downloading, and not inappropriate, but when
we come to think about the converse a number people have seriously
mis-understood what is going on. The model is decidedly not that you
take a copy of (part of) the database, tweak it, and then spit it back
as bulk replacement. Rather, what you can upload, in fact the only
thing you can upload, is a batch of requests for registration of new
machines or modification or rescinding of existing registrations, and
you can't mix the operations within a batch. What you have neither
more nor less than a means of bulk submission of the same requests
that you could have made one at a time with single_ops.
The format for files to be uploaded is the same sort of thing, tab
separated columns, in any order, the first row identifying the columns
and subsequent rows representing one database request each. A single
file represents a number of requests all of the same type, registrations,
modifications, renamings or rescindings, the same operations that are
available with single_ops but here applied to a whole batch. You don't
need to supply all the same columns that the download gave, just the
ones necessary for the operation requested.
For example, a batch for registration might look like
name lan equipment location owner end_user sysadmin bursar.botolph.cam.ac.uk MAIN Pentium PC Bursar's Office College bursar CO master.botolph.cam.ac.uk MAIN Hexium PC Master's Lodge College Master CO
(I have lined up the columns for visibility here, rather than separating them with the tabs that would be required for real.)
Note the absence of an address column in this example: we are asking the system to choose addresses for us. To execute this batch of registration requests we simply insert the name of the file and click register. Each request item is attempted, and the results are spat back into the text display area at the bottom of the viewed page. You can of course save this using the browser's own save facility if you wish.
Do make sure you inspect all the returned text. The requests in the batch are treated entirely independently, and there may be an isolated refusal in the midst of a batch of successes. (Or an isolated success in the midst of a batch of refusals!)
For both modify and rescind, you must specify either the hostname or
the IP address to identify the item. In the case of rescind you
mustn't specify anything else. In the case of modify you can specify
any of the fields that would be modifiable with single_ops, but note
that you have to specify the same fields for all requests in any one
file, and remember that empty means nullify, not ignore!
For rename, you must include columns old_name and
new_name (and nothing else).
The limits of the simple
That completes the substance of this tutorial, but rather than leaving you alone on the edge of uncharted territory we conclude with a vista point, as it were.
We've already hinted that these simple forms are hiding a more complex underlying reality. There has to be that complexity at heart, because the world is potentially complex, but the overwhelmingly common cases are relatively simple, and these forms (attempt to) hide the complexity for those cases. What then are the potential complexities?
Perhaps the most obvious complexity is that a single machine can have more than one network connection, either real or virtual, whereas the simple forms all assume there can only be one. The possibility of more forces a completely different underlying structure, and a simple registration in fact involves simultaneous and coordinated updating and linking two quite separate tables, one of pieces of equipment (boxes), and one of addresses. The underlying database is perfectly capable of handling hosts with several addresses, but the simple web forms are not, or at any rate not yet.
What we do have to rescue us in the face of complex requirements
is a set of forms (table_ops) that lets you interact directly with the
underlying tables. If you know what you are doing, the limitations of
the simple forms thus melt away, but you may well prefer to ask
hostmaster to cope with such cases on your behalf. That said, by
all means have a look at table_ops.
Finally, a word about the history trace tables. For every main table
in this database there is an automatically maintained history table,
entries being retained for a very long time. There is no easy
roll-back after mistakes, but there is this audit trail that does
allow investigation of what was done and what is no longer visible.
You can see these history tables with table_ops. Treat them as an
audit trail, as was intended, not as a back-up, which they certainly
aren't, but if you do have a disaster remember their existence. If you
are going to do bulk updates with list_ops then you are strongly
advised to obtain checkpoint downloads first.