Module loading in dojo (1.9)

How Dojo locates a module

The official Dojo tutorials unfortunately contain many errors or omissions. So, I had to write down my findings from my experiment. When we request the Dojo loader to load a module with id, say, foo/bar, it will basically start the search from the "base URL". The base URL by default is the folder containing dojo.js. Then it will drill down the path using the module id. For example, if dojo.js is loaded like:

then the base URL will be http://www.abc.com/js/dojo-dist/dojo and therefore, in principle, the Dojo loader will drill down and try to load http://www.abc.com/js/dojo-dist/foo/bar.js file. However, because Dojo is distributed with the following folder structure, it will mean that you need to put your foo folder into the dojo folder (which contains the core files of Dojo):

 js
  |
  +-- dojo-dist
       |
       +-- dojo  <--- 1: the base URL by default
       |     |
       |     +-- dojo.js
       |     |
       |     +-- foo  <--- 2: drill into here
       |          |
       |          +-- bar.js
       |
       +-- dijit
       |
       +-- dojox
       |
       +-- util

This doesn't make sense. Therefore, Dojo by default assumes that your files are put into a folder that is a sibling of the folder pointed to by the base URL. That is, after locating the base URL, it will go back one level before drilling down using the module id:

 js
  |
  +-- dojo-dist  <--- 2: go back to here
       |
       +-- dojo  <--- 1: the base URL by default
       |      |
       |      +-- dojo.js
       +-- dijit
       |
       +-- dojox
       |
       +-- util
       |
       +-- foo  <--- 3: drill into here
            |
            +-- bar.js

To observe the effect of this "going back one level", you can set the tlmSiblingOfDojo ("top level module sibling of Dojo") option between true (default, go back) and false (don't go back):

Putting the foo folder as a sibling of the dojo folder is better (at least the foo folder is not mixed with the core Dojo files). You might interpret the dojo-dist folder containing all Dojo related files, i.e., files from the Dojo Foundation and files created by you that use Dojo.

Putting your own files outside the Dojo folder structure

However, you may not like the above folder structure and decide to put your own files outside the dojo-dist folder, like:

 js
  |
  +-- dojo-dist
  |    |
  |    +-- dojo
  |    |     |
  |    |     +-- dojo.js
  |    +-- dijit
  |    |
  |    +-- dojox
  |    |
  |    +-- util
  |
  +-- foo
       |
       +-- bar.js

Then you must set the base URL yourself. You may set it to the dojo-dist folder and continue to rely on the default tlmSiblingOfDojo to go back to the js folder, before it drills down to the foo folder:

 js  <--- 2: let it go back to here
  |
  +-- dojo-dist  <--- 1: set the base URL to here
  |    |
  |    +-- dojo
  |    |     |
  |    |     +-- dojo.js
  |    +-- dijit
  |    |
  |    +-- dojox
  |    |
  |    +-- util
  |
  +-- foo  <--- 3: drill into here
       |
       +-- bar.js

Obviously, when setting the base URL yourself, it no longer has the meaning of "the folder containing dojo.js" and thus the tlmSiblingOfDojo option is really meaningless; it only means whether it should go back one level. The code is like:

As going back one level in this case is really confusing, it is better to just turn that off and set the base URL to the js folder:

 js  <--- 1: set the base URL to here and do NOT go back one level
  |
  +-- dojo-dist
  |    |
  |    +-- dojo
  |    |     |
  |    |     +-- dojo.js
  |    +-- dijit
  |    |
  |    +-- dojox
  |    |
  |    +-- util
  |
  +-- foo  <--- 2: drill into here
       |
       +-- bar.js

The code is like:

Instead of a full URL, the base URL can also be set as an absolute path (on the same host with the same protocol used to access the enclosing HTML file) or a relative path (to the enclosing HTML file). For example, If the enclosing HTML file is a sibling of the js folder, then it is better to use a relative path:

A more flexible location mechanism

Putting the modules as direct children of the base URL is fine, but may be too limiting. If you would like to have complete control over where to put each module, you can make use of the concept of a "package". A package can be considered a location with a name. For example, you can define a package named "foo" whose location is http://www.abc.com/js/myapp, then when you're trying to load the foo/bar/baz module, Dojo will note that the module id (foo/bar/baz) starts with foo, so it is in the foo package and thus it will go to http://www.abc.com/js/myapp first and then drill down to the bar folder and load baz.js:

Instead of using a full URL as the package location, it is possible to use an absolute path (use the same protocol and host as the base URL) or a relative path (to the base URL). Note that when locating a module through through the package method, which is much more flexible than the "assuming everything is a sibling of dojo", the tlmSiblingOfDojo is NOT used at all. That is, Dojo will locate the base URL, never go back one level, combine it with the package location (absolute or relative path) to determine the full URL for the package. Here is an example of a relative path:

Here is an example of an absolute path:

Of course, it is possible to define multiple packages:

Beauty of Scala: construction of hierarchical objects

In Java, one would construct a GUI (e.g., Swing) like:

JMenuBar menuBar = new JMenuBar();
JMenu file = new JMenu("File");
menuBar.add(file);
JMenuItem open = new JMenuItem(new AbstractAction("Open Word Bank") {
    public void actionPerformed(ActionEvent e) {
        openWordBank();
    }
};
file.add(open);

But in Scala, it is done like:

menuBar = new MenuBar {
  contents += new Menu("File") {
    contents += new MenuItem(new Action("Open Word Bank") {
      def apply {
        openWordBank
      }
    })
  }
}

That's the beauty of Scala! The code can be written like that because we can write initialization code (usually inside a constructor in Java) directly inside a Scala class.

Network access related protocols: PPP, EAP, RADIUS, 802.1x, L2TP, GRE

PPP

• PPP is protocol to negotiate network layer protocol parameters (e.g., IP) and a encapsulation to carry IP packets for transmission over a point-to-point link (a layer 2 functionality). The former allows the server to assign IP, netmask and DNS server etc. to the client. The latter allows the actual transmission of IP packets.
• When a packet is sent to the PPP's IP interface (ppp0 on Linux), PPP will put the IP packet into a PPP packet and deliver it to the point-to-point link. The other side will extract the IP packet from the PPP packet. This is very much link IP over Ethernet.
• Before allowing the client to connect, the client needs to authenticate with the server. PPP has built-in authentication protocols like PAP and CHAP.

EAP

• EAP was devised to run on top of PPP (without IP). Instead of requiring a particular authentication mechanism like PAP and CHAP, it allows different authentication mechanisms to be plugged into it.
• As EAP is not run on top of IP or TCP, it handles re-transmission (for packet loss) by itself. However, it does assume that packet ordering is preserved by the transport (typically ensured by point-to-point link).
• EAP is a peer to peer protocol. Either side can authenticate the other side. The side being authenticated (e.g., a remote access client) is called the supplicant or the peer, the other side is called the authenticator (e.g., a network access server).
• The authenticator can authenticate the peer itself or can pass through to another server (authentication server). The latter approach is useful to centralized the user accounts. Typically, it is a RADIUS server.
• EAP-TLS is an authentication mechanism of EAP which uses TLS for authentication. That is, using the TLS handshake protocol (using the certificate of the peer) for authentication. The last step of the TLS handshake protocol is to verify that both sides have the master secret which was established by random numbers generated by each side and sent to the peer encrypted by the public key of the peer.
• PEAP (protected EAP). There are some weaknesses in EAP with some mechanisms such as sending the identity of the peer in clear. To avoid the problem, PEAP works by establishing a TLS session with the authenticator (or authentication server in pass-through mode) first and then perform EAP (so, the actual mechanism is needed such as EAP-CHAP). This way, everything is encrypted. PEAP is not an open standard but created by Cisco and MS.

RADIUS

• RADIUS provides its own authentication (quite weak) using a central user database. It can also return additional information through the attributes to the authenticator (e.g., the privilege level of the user).
• Even though RADIUS has its own authentication protocol, it can support CHAP and EAP using its attributes.
• A RADIUS attribute has a type (an integer) and a value. It is called attribute-value pair (AVP). There are some standard RADIUS types and vendor proprietary attribute types.

802.1x

• 802.1x (EAP over LAN) is basically just EAP over 802 (Ethernet or 802.11 wireless LAN) so that a switch or an AP can authenticate a device being connected to it. This is like the case in PPP for allowing the client to connect or not, except that PPP deals with a remote client while 802.1x deals with a LAN client. As on a LAN there is no need for another layer 2 protocol, only the authentication part (EAP) of PPP is needed.
• As the authentication is done in layer 2, there is no IP yet. So, just like normal EAP, EAP packets are encapsulated in layer 2 frames (here, in 802 frames). 
• The switch or AP is called the authenticator as it needs to authenticate the peer.
• In pass through mode, the authenticator will use RADIUS protocol with EAP support to talk to the authentication server (for central management).
• A RADIUS server can return RADIUS attributes telling the switch which VLAN to put the client/port into.

L2TP

• L2TP is used to simulate a point-to-point link with an IP network such as the Internet. Therefore, PPP can be run over L2TP. This way, a VPN tunnel can be built across the Internet. Essentially L2TP serves as a layer 2 protocol that runs on top of a layer 3 protocol (IP). When people say it is an L2TP VPN, actually it is L2TP and PPP VPN as the address assignment and etc. are done with PPP. The authentication is also the same (e.g., EAP).
• L2TP itself provides no encryption, so typically it is run over IPSec to protect the traffic. So, it is PPP over L2TP over IPSec.

GRE

• GRE is very much like L2TP except that it can tunnel any protocol over any protocol (i.e., the transport is not necessarily IP).
• When using for VPN, PPP is also run over GRE. In addition, for security, the PPP payload is typically encrypted and this modified version of PPP is PPTP (proprietary by MS).

Concepts of Nagios

• Purpose. Basically Nagios is used to monitor if your servers and network devices are up and working properly. If something is not working, it can notify you (the administrator) so that you can fix the problems ASAP.
• How it works (basically). Nagios runs on a central server so that you can view the status of everything in one web interface. It will check each host (server or network device) by pinging its hostname or IP, say, every 5 minutes ("check interval"). If a host is down, it will notify the administrator ("contact") or the administrators (the "contact group").
• Notify on state change. Once a host is detected as down, the checking will continue as usual to detect if it is up again.
• If it is still down, usually no further notification is sent (to avoid bombarding the admins). However, to avoid the problem from being forgotten, you can configure it to send a notification at a certain interval ("notification interval").
• If it becomes up, another notification will be sent so that the admins will know that it has recovered.
• Therefore, notification is sent when the state is changed, not when a host is down.
• Soft and hard state. Sometimes a host is not down, but just very busy or the network is very slow so some pings may timeout. Therefore, if a host is considered down by ping, the state is changed to a "soft" down, meaning that it is possibly down. No notification is sent yet. Nagios will check it ("retry") a few times more (the "max check attempts"). If it is still down for these checks, the state will be changed to a "hard" down and then a notification is sent. Similarly, when a host recovers from a hard down to up, it is just a soft up. Only when it is still up in the subsequent checks, it will change to a hard up and then a notification will be sent. In summary, notification is sent only when the state is changed into a different hard state.
• Unreachable state. In addition to up or down, there is a third possible state: unreachable. It can happen if the host is running fine, but a router between them is down. But how can Nagios distinguish which case it is when ping fails? You can create a host record for the router in Nagios  and tell it that the router is the "parent" of that server host (on the path from the Nagios server to that server host). This way, when ping to that server host fails, Nagios will check if the router is up or down. If it is down, it will mark the server host as unreachable instead of down. Again, a one time result only leads to a change to a soft state.
• Will the admins still get a notification if it is changed to a hard unreachable state? By default yes but you can configure it (below).
• Command. In Nagios you can configure the actual command used for ping or for sending notifications. To allow you to refer to the command by a simple name without worrying about the actual path, each command is assigned a name for later use. Many such commands will also retrieve information from environment variables such as HOSTADDRESS ("macro" and set up by Nagios). Below is the command typically used for ping:  

define command {
  command_name check-host-alive
  command_line /var/lib/nagios/plugin/check_ping -H $HOSTADDRESS$
}

Below is a simplified example of a command used for sending email notification (again, Nagios will provide the information such as email, host name, host state through the macros):


define command {
  command_name notify-by-email
  command_line /bin/echo $HOSTNAME$ in $HOSTSTATE$ | /usr/bin/mail $CONTACTEMAIL$
}
• Contact. A contact is simply a configuration item for sending a text message (a command line) to a person. It specifies the email address, pager/mobile number and etc. so that the command know where to send the message to.
• Notification options. It can also specify what kinds of notifications it will accept: it can choose to accept down (from up to down), recovery (from down to up), unreachable (from up or down to unreachable), etc.
• Notification period. It can also specify what time of the day on which days notifications can be sent. Usually it should be 24x7 for critical hosts, but if the host is unimportant and this is contact sending SMS, you may configure it to only send in business hours. The notification outside of the period is not lost though: Nagios will schedule it to be sent at the start of the next time slot (e.g., next morning).
• Notification settings for a host. The notification options and notification period can also be specified for a host. It means some hosts may be more important and some admins may be available only for certain issues in certain time periods.
• Below is an example of a contact (using the notify-by-email command above):

define contact {
  contact_name kent
  email kent@foo.com
  host_notification_options d,r,u
  host_notification_period 24x7
  host_notification_command notify-by-email
}

• Contact group. A contact group is just a set of contacts (e.g., for all the administrators). You define the individual contacts first and then add them to a group. This way, you can configure the host checking to notify a contact group instead of individuals. Below is an example:

define contactgroup {
  contactgroup_name admins
  members kent, paul
}

• Check period. Instead of performing checking at all time, Nagios requires that for each host you specify a period in which checking is performed. You can set it to 24x7 or your business hours depending on the supposed up-time window of the host.

• Host configuration example. The configuration of host in Nagios includes the address (hostname or IP) and the various checking and notification settings mentioned above. Below is an example (using the check-host-alive command above):

define host {
  host_name host1
  address 192.168.1.10
  parents router1
  check_interval 5
  check_period 24x7
  check_command check-host-alive
  max_check_attempts 4
  contact_groups admins
}

define host {
  host_name router1
  ...
}

• Time period. The time period used above such as 24x7 is defined like below:

define timeperiod {
  timeperiod_name 24x7
  sunday 00:00-24:00
  monday 00:00-24:00
  tuesday 00:00-24:00
  ...
}

• Object and object inheritance. Each item above such as host, contact, command is called an "object" in Nagios. To save on specifying all those options, Nagios allows you to define an object template that can be inherited by objects. For example, there is a template coming with Nagios called generic-host defined as below to set sensible values for many settings. The name is the name of the template so that you can refer to it by name and "register 0" tells Nagios not to treat it as a real host:

define host {
  name generic-host
  register 0
  max_check_attempts 10
  check_command check-host-alive
  contact_groups admins
  ...
}

• To use ("inherit") the template, just do it like this:

define host {
  host_name host1
  address 192.168.1.10
  parents router1
  use generic-host
  check_period 24x7
}

• Service monitoring. Just monitoring if a host is up or down isn't enough. It is needed to monitor the services such as website or email services are running properly. This is configured with a service object. A service object specifies the command used for checking (so the actual meaning of "up" is up to you to decide) and one or more hosts to provide the IP addresses. It means Nagios will perform the checking on each such host independently and pass the IP to the command via the $HOSTADDRESS$ macro. This is useful if, say, you have Apache running dozens of hosts, as you only need one service object. So, the service object is not really a single service, but just a way to apply a type of service checking across multiple hosts. Other than this, a service checking is very much similar to host checking and support similar settings such as max check attempts, check interval, contact, notification options, etc. Below is an sample service object (the check-http command will simply check if it can successfully retrieve a web page from the IP):

define service {
  service_description website
  host_name host1, host2, host3
  max_check_attempts 4
  check_interval 3
  check_period 24x7
  check_command check-http
  notification_options c,w,r
  notification_period 24x7
  contact_groups admins
  ...
}

• Service state. Instead of up, down, unreachable, a service (running on a particular host) can be up, critical (obvious error such as no response or connection refused or 404 error), warning (e.g., slow response. The actual meaning is defined by the command )  or unknown (usually meaning that the check command itself has failed). The corresponding notification options are r (recover), c (critical), w (warning).
• Service notification settings in a contact. Probably because the notification options for service checking are different from those for host checking, Nagios requires that in a contact object you  specify the notification settings separately for host notifications and for service notifications:

define contact {
  contact_name kent
  email kent@foo.com
  host_notification_options d,r,u
  host_notification_period 24x7
  host_notification_command notify-by-email
  service_notification_options c,w,r
  service_notification_period 24x7
  service_notification_command notify-by-email
}

• Passing explicit arguments to a command. Just checking if a web page can be retrieved may not be good enough. A better check is to check if the web page contains the right content (e.g., containing a particular string). To do that, you can define your own command and pass the expected string (e.g., abc) to it. This is done using an exclamation mark and it is retrieved by $ARG1$ in the command definition:

define service {
  service_description website
  host_name host1, host2, host3
  max_check_attempts 4
  check_interval 3
  check_period 24x7
  check_command check-http-string!abc
  ...
}

define command {
  command_name check-http-string
  command_line /usr/lib/nagios/plugins/check_http -H $HOSTNAME$ -I $HOSTADDRESS$ -s $ARG1$
}

• Finding out the plugins. There are many commands (both the defined command objects and the actual programs) coming with Nagios. They are called plugins and many 3rd party ones can be installed. To find out what is available and how to use each one, just do it like:

# find the command object definitions by the plugins
$ ls /etc/nagios-plugins/config
  ...
# learn about the options supported by each program
$ /usr/lib/nagios/plugins/check_http --help
  ...

• Inheritance for service. Just like a host object, you can use object inheritance for a service object too. Nagios comes with a service template called "generic-service" defining many defaults for you to use:

define service {
  service_description website
  use generic-service
  check_command check-http-string!abc
  ...
}

• Host group. Instead of listing individual hosts in a service object, it may be a better way to define a "host group" named web-servers containing such hosts and just let the service object refer to the host group. This is particularly useful if you need to perform two kinds of service checking on the same group (e.g., checking web and SSH).

define service {
  service_description website
  hostgroup_name web-servers
  ...
}

define hostgroup {
  hostgroup_name web-servers
  members host1, host2, host3
}

• Service dependency. If the DNS or LDAP is down, then many other services will also not function. So, you will receive lots of notifications, burying the real issue. If you don't want to receive such "bogus" notifications, you declare that such services are dependent on the DNS or LDAP services. This way, when an upper level service fails, Nagios will check if the lower level service also fails. Given a specific option, you can tell Nagios to not send notification for the upper level service when the lower level service is in a certain "failure" states (e.g., critical or warning). As this dependency is about a service on a host depending on another service on another host, you must specify the hosts in addition to the services:

define servicedependency {
  dependent_service_description website
  dependent_host_name host1
  service_description dns
  host_name host2
  notification_failure_criteria c,w
}

• Service-host dependency. In principle, the same dependency should be specified for service and host as if a host is down, it is impossible for the service to continue to run properly. However, at present Nagios doesn't support this concept. So, if a host is down, you will still receive notifications for the services running on it.

Concepts of SNMP (including v3)

SNMP

• Purpose. SNMP is a protocol for getting the status (e.g., CPU load, free memory, network load) of computing devices such as routers, switches and even servers.
• Object descriptor, managed object. The client can provide a globally unique names such as cpmCPUTotal5secRev (the average CPU load of a Cisco device for the past 5 seconds) to indicate the information that it wants, then the server should return such information. Such a textual name is called the "object descriptor". The word "object" or "managed object" refers to the concept of CPU load. The actual CPU load in the device is called the "object instance".
• Object identifier (OID). To make sure that each object descriptor is unique, actually it is defined using a list of integers such as 1.3.6.1.4.1.9.9.109.1.1.1.1.6. Each integer is like a package in Java. For example, the integers in 1.3.6.1.4.1.9 represents iso (1), org (3), dod, i.e., department of defense (6), internet (1), private (4), enterprises (1), cisco (9) respectively. This allows the Internet authority to delegate the management of the namespace hierarchically: to private enterprises and then to Cisco, which can further delegate to its various divisions or product categories. Such a list of integers is called an "object identifier". This is the ultimate identification for the managed object.
• Even though the object descriptor should be unique, it is useful to see the hierarchy. Therefore, usually the full list of object descriptors is displayed such as iso.org.dod.internet.private.enterprises.cisco...cpmCPUTotal5secRev.
• Why use integers instead of symbolic names? Probably to allow the network devices (with little RAM or CPU power) implementing SNMP to save space in processing. Symbolic names such as object descriptor can be used by human in commands, but in the protocol's operation it is done using object identifier.
• In principle, the object a.b.c.d and the object a.b.c.d.e on a device have NO containment relationship. That is, they are NOT like a Java object containing a child object. In fact, the value of each object in SNMP is basically a simple value (scalar) such as an integer or a string. The only relationship between them is their names.
• Identifying an instance. Now comes the most complicated concept in SNMP. Consider the concept of the number of bytes that have been received by a network interface on a router. This concept is an object. As a router should have multiple interfaces, there must be multiple instances of that object. Then, how can an SNMP client indicate to the SNMP server which instance it is interested in? The solution is more or less a kludge: to allow the instance of, say, a.b.c.d, to represent a table (a compound, structural value), which contains rows (also compound, structural value) represented by a.b.c.d.e. Each row contains child object instances (with scalar values only). Each child object is called a "columnar object". For example, each row may contain three object instances: a.b.c.d.e.f, a.b.c.d.e.g, and a.b.c.d.e.idx. If you'd like to refer to the a.b.c.d.e.f instance in a particular row, you will write a.b.c.d.e.f.<index>. The meaning of the index is defined by a.b.c.d.e (the row). For example, it may be defined as finding the row in the table which contains a columnar object a.b.c.d.e.idx whose value equals to <index>, then return the columnar object a.b.c.d.e.f as the result.
• Note that this is the only situation where the value of an object can be a structure and that there is object containment relationship in SNMP.
• What is confusing is that a.b.c.d.e.f is used both as an object identifier and the lookup key to find the child instance in the row. Unlike other object identifiers, the identifier now represents an object containment relationship so it must have a.b.c.d.e as the prefix, otherwise the server won't know which table to look into and what is the definition for the index.
• The complete identifier a.b.c.d.e.f.<index> is called an instance identifier.
• Here is a concrete example: Consider iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifInOctets.1.  The definition of iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry says that to find the row in the table, it should search for a row which contains a child object iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifIndex with the value of 1 (the index specified), then it will return the value of child object iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifInOctets in the row. Of course, for this to work, the  iso.org.dod.internet.mgmt.mib-2.interfaces.ifTable.ifEntry.ifIndex child object in each row must have been assigned with sequential values 1, 2, ..., etc. (which is indeed the case). 
• Finally, a simple case is that there is no table at all. For example, to find the up time of the device, use the object identifier iso.org.dod.internet.mgmt.mib-2.host.hrSystem.hrSystemUptime and append a .0 as the index, so the instance identifier is iso.org.dod.internet.mgmt.mib-2.host.hrSystem.hrSystemUptime.0. BTW, "hr" stands for "host resources".
• MIB (management information base). A MIB is just a a collection of managed objects (not instances). There are standard MIBs so that device manufacturers can implement and users can find the right object identifiers to use. There are also proprietary MIBs such as those designed by Cisco to provide information only available on its devices.
• Finding the supported OIDs. How can you find out the MIBs or the object identifiers supported by a device? It is easier to just "walk the MIB tree": to show all the instances in the tree or in a subtree. On Linux, this is done as below. You specify the IP or hostname of the server and optionally specify a node so that only that subtree is displayed (the object identifier starts with a dot, otherwise it will be assumed it is relative to iso.org.dod.internet.mgmt.mib-2):

# snmpwalk <some args> localhost
# snmpwalk <some args> localhost .iso.org.dod.internet.mgmt.mib-2.system
# snmpwalk <some args> localhost system

• Getting an instance. Just specify the instance identifier:

# snmpget <some args> localhost .iso.org.dod.internet.mgmt.mib-2.host.hrSystem.hrSystemUptime.0
# snmpget <some args> localhost host.hrSystem.hrSystemUptime.0

• SNMP enttiy, engine and applications. SNMP is a peer to peer protocol. There is no concept of a server and a client (these terms are used here for simplicity). Instead, both peers have the same core capabilities such as sending or receiving SNMP messages, performing security processing (see below), integrating v1, v2c and v3 processing (dispatching) and etc. This part is called the SNMP engine. On top of the engine, there are different "applications": one application may only respond to requests for object instances (called SNMP agent in v1 and v2), another application may probe others (called SNMP manager in v1 and v2), yet another may forward SNMP messages (SNMP proxy). The whole server or client is called the "SNMP entity".
• Context. On some devices there are multiple copies of a complete MIB subtree. For example, a physical router may support the concept of virtual routers. Each such virtual router will have a complete MIB subtree of instances. In that case, each virtual router may be indicated as a "context" in the SNMP server. A context is identified by name. For example, a virtual router could be identified as the "vr1" context. There is a default context with empty string ("") as its name. When a client sends a query, it can specify a context name. If not, it will query the default context.
• Notification (trap). An SNMP server may actively send a notification to the client when some condition occurs. This is very much like a response without a request. Otherwise, everything is similar. The condition (e..g., the changes of the value of an instance or its going out of a range), the destination, the credentials used (see the security section below) and etc. are configured on the server.
• Transport binding. Typically SNMP runs on UDP port 161.

SNMP security

• SNMP v1 and v2c security. In SNMP v1 and v2c (v2 was not widely adopted), there is little security. The only security is the "community string". That is, the server is configured to be in a community identify by a string such as "foo", "public" (commonly used and the default for many devices to mean no protection) or "private". If the client can quote the community strnig, then it is allowed access. As the community string is included as plain text in SNMP packets, it practically provides no security. Therefore, in v1 and v2c, to access an SNMP server, you will do something like:

# snmpwalk -v 2c -c public localhost
# snmpget -v 2c -c public localhost <INSTANCE ID>

• SNMP v3 security. In SNMP v3, there is user-based security. That is, the client may be required to authenticate the messages to the server as originating from a user using a password (authentication password). In addition, the client may be furthered required to encrypt the messages using another password (privacy password). This security requirement is called the "security level" (no authentication needed, authentication but no privacy, authentication with privacy). Therefore, in v3, you will access the server like:

# snmpwalk -v 3 -l noAuthNoPriv localhost
# snmpwalk -v 3 -l authNoPriv -u kent -A "my auth passwd" localhost
# snmpwalk -v 3 -l authPriv -u kent -A "my auth passwd" -X "my priv passwd" localhost 

• Client configuration file. To save typing all those every time, you can store these parameters into the snmp.conf file as defaults.
• Security limitation. It is a bad idea to specify the password on the command line as it can be revealed by local users using "ps". Storing it into the configuration file is better. However, the file only allows a single authentication password and a single privacy password, not enough to handle the case of using different passwords for different servers.
• Security name. A security name is just a user name. No more, no less. That's the term used in the RFC (maybe in the future it could be something else?)
• Algorithm. Further, there are different algorithms for authentication (HMAC using MD5 or SHA) and for privacy (encryption using DES or AES). So, you need to specify the algorithms to use:

# snmpwalk -v 3 -l noAuthNoPriv localhost
# snmpwalk -v 3 -l authNoPriv -u kent -a MD5 -A "my auth passwd" localhost
# snmpwalk -v 3 -l authPriv -u kent -a MD5 -A "my auth passwd" -x DES -X "my priv passwd" localhost 

• Ensure the algorithms match. As SNMP uses UDP and each query and response may use just a single UDP packet, there is no negotiation of algorithm at "connection phase" at all. In fact, presumably for simplicity in implementation, the algorithms used are not even indicated in the message, so the client must use the agreed-on algorithms as configured in the user account on the server, otherwise the server will simply fail to authenticate or decrypt the message.

• Localized keys. The authentication password and privacy password of a user account are not used directly. The idea is, most likely you will use the same password for all the user account on all devices on site. If it is directly used, then a hacker controlling one device will be able to find the password and use it to access all the other devices. Therefore, when creating a user account, you specify the password, but the Linux SNMP server will combine it with a unique ID (called the "engine ID") generated for the device (such as the MAC or IP and/or a random number generated and stored on installation), hash it and use the result as the password (the "localized key"). This way, even if the hacker can find this localized key, he will still be unable to find the original password.
• But how can a client generate the same key? It has to retrieve the engine ID first and then perform the same hashing. This is supported by the SNMP protocol.
• User accounts creation. Due to the need to generate localized keys, the way to create user accounts on Linux is quite weird. You stop the server, specify the user account's name and password in a file, then start the server. It will read the password, convert it to a localized key and overwrite the file. This file is /var/lib/snmp/snmpd.conf on Linux:

createUser kent MD5 "my auth password" DES "my privacy password"
createUser paul SHA "my auth password, no encryption needed"

• Access control. The access control can specify the user account, the lowest security level required, which part of the MIB tree is accessed (may use an OID to identify a subtree), the type of access (read or write), in order to grant the access. Here are some example settings on Linux (although human user names are used, but in practice they should be representing devices):

rouser john noauth .iso.org.dod.internet.mgmt.mib-2.system
rouser kent priv .iso.org.dod.internet.mgmt.mib-2.system
rouser kent auth .iso.org.dod.internet.mgmt.mib-2
rwuser paul priv 

• View. How to specify several subtrees in an access control rule? You can define a view. A view has a name and is defined as including some subtrees and excluding some subtree. Then you can refer to it by name in access control:

view myview included .iso.org.dod.internet.mgmt.mib-2.system
view myview included .iso.org.dod.internet.mgmt.mib-2.host
view myview excluded .iso.org.dod.internet.mgmt.mib-2.host.hrStorage

rwuser paul priv -V myview 

• Access control for v1 and v2c. For v1 and v2c, access control can specify the community string, the IP range of the client (the "source"), the subtree (OID) or the view:

rocommunity public 192.168.1.0/24 .iso.org.dod.internet.mgmt.mib-2.system
rwcommunity private localhost -V myview

• Most flexible access control model. The above access control model is called the "traditional model". The new, most flexible access control model is called the "view-based access control model (VACM)", even though the former can also use views. It may be more suitable to called it group-based access control as it uses user groups in the rules (NOT the precise syntax yet!):

group g1 kent
group g1 paul
#access <group> <context> <min sec level> <exact context?> <view for read> <view for write> <view for notify>
access g1 "" auth exact myview1 myview2 myview3

• Mapping community string to user name. When using the VACM, instead of granting access to community strings, you need to merge v1 and v2c into the user-based access control processing. To do that, a community string along with the source can be mapped to a user name (the user name mapped to do NOT have to be existing):

com2sec user1 192.168.1.0/24 public
# "default" source means any
com2sec user2 default private

• Security model. Even though the different types of identity in the different SNMP versions are represented uniformly as a user name, their trustworthiness is still significantly different. So, in specifying group memberships and access control rules, you are required to specify the "security model" (v1, v2c or the user security model as in v3) and that's the correct syntax:

group g1 usm kent
group g1 usm paul
group g2 v2c user1
group g2 v1 user1
group g2 v1 user2
access g1 "" usm auth exact myview1 myview2 myview3
access g2 "" any noauth exact myview4 myview5 myview6

Basic concepts of OpenLDAP

LDAP

• LDAP is just a type of database specialized for look ups. Reads are fast while writes are supported but relatively slow. That's why LDAP is commonly used to store centralized user accounts.
• In LDAP, data is stored nodes (much like a record in a relational DB). Each node has some named attributes. For example, a node representing a user may have the following attributes (cn stands for "common name", while sn stands for "surname"):

cn: Kent Tong
sn: Tong
givenName: Kent
email: kent@ttdev.com

• Just like a record in a DB or an object in Java, the attributes that a node can have is fixed. This is specified by an attribute named objectClass. For example, there is an objectClass named "person" that says that it can have cn, sn and email, then a node belonging that this "person" class can have such attributes:

cn: Kent Tong
sn: Tong
email: kent@ttdev.com
objectClass: person

• It is possible for a node to belong to multiple object classes. For example, there may be an objectClass named "securityObject" that says that it can have a password attribute, then if you need to put a password into a node, you can do it like:

cn: Kent Tong
sn: Tong
email: kent@ttdev.com
objectClass: person
objectClass: securityObject
password: abc123

• How to use a node to present a department or a company? This is done with the "organizationalUnit" objectClass ("ou" stands for "organizational unit"):

ou: Foo Ltd.
objectClass: organizationalUnit

ou: Finance dept
objectClass: organizationalUnit

• If the person Kent Tong is in that Financial department, how to represent that relationship? Unlike a relative DB where such relationship is represented a foreign keys, in LDAP one node can be put under the other as a child node. For example, let's put the person node as a child node of the financial dept node which is put as a child node of the company node:

dn: ou=Foo Ltd.
ou: Foo Ltd.
objectClass: organizationalUnit

dn: ou=Finance dept,ou=Foo Ltd.
ou: Finance dept
objectClass: organizationalUnit

dn: cn=Kent Tong,ou=Finance dept,ou=Foo Ltd.
cn: Kent Tong
sn: Tong
email: kent@ttdev.com
objectClass: person

• This parent-child relation is represented by an attribute called "dn" (distinguished name, just like an full path to a file). That is, you can determine the dn of the parent node from the dn of the node.
• Usually you will add the cn to the dn of the parent to get the dn of the node itself. However, you don't have to use cn. You are free to any attribute to serve that purpose (called "relative dn") as long as it is unique among the siblings. For example, cn is good if there are no two people with the same cn values  the dept. This is so that given a dn, it is very fast to locate the node.
• When you create the top node of the directory, you need to specify its dn. Usually, you will use the DNS domain name of your company (let's say it is foo.com):

dn: dc=foo,dc=com
ou: Foo Ltd.
objectClass: organizationalUnit

dn: ou=Finance dept,dc=foo,dc=com
ou: Finance dept
objectClass: organizationalUnit

dn: cn=Kent Tong,ou=Finance dept,dc=foo,dc=com
cn: Kent Tong
sn: Tong
email: kent@ttdev.com
objectClass: person

• in which dc means "domain component", just a label in a domain name.
• Some object classes such as person or organizationalUnit are designed to be used as the "main" object class for the nodes, while some other object classes such as securityObject (officially, it is called "simpleSecurityObject") are designed to be "auxiliary" and to be added to the nodes. The former is called a "structural object class" while the latter an "auxiliary object class".
• For each node, it must have exactly one structural object class. It can have zero or multiple auxiliary object classes.

Schema

• The attributes and object classes are defined instead of hard-coded. For OpenLDAP on Debian, you can find the schemas in /etc/ldap/schema. For example, you can find the following object class in core.schema:

objectclass ( 2.5.6.6 NAME 'person'
        DESC 'RFC2256: a person'
        SUP top STRUCTURAL
        MUST ( sn $ cn )
        MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) )

• It says that it must have the "sn" and "cn" attributes and may have a "userPassword" attribute and etc. It says that its super class is "top" which is a built-in object class representing the topmost level object class. It also says that the "person" object class is a structural class.
• You can find the most commonly used object classes and attributes here.

Authenticating with LDAP

• A straightforward way is to retrieve the node according to the username, get the password and compare it to the password the user entered.
• However, this allows retrieving sensitive information (password) of any user over the network by services. Therefore, there is another way: you send dn and the password to the LDAP service and let it perform the checking. This is called "binding" to that dn. Essentially you are logging in as a particular node.
• Both methods above will transfer the password in the clear. So, you should probably use TLS when accessing the LDAP service (using ldaps://)
• The above methods are most commonly used by other services such as mail, Samba or Apache for checking the passwords of the users.
• When performing administration with OpenLDAP, you may use the "bind" approach above to log in as the node representing the administrator. This is called "simple authentication":

# ldapsearch -x -W -D cn=admin,dc=foo,dc=com ...

• However, sometimes it is more convenient to authenticate not as a node in the directory but just as an administrator. For example, if your site is using Kerberos, you can perform single sign-on with OpenLDAP. This is done with its SASL support. SASL in turn supports many mechanisms such as PLAIN (plain text), GSSAPI (Kerberos), EXTERNAL (using the transport such as the client cert with TLS). When SASL to authenticate, OpenLDAP will make up a dn to represent your identity. For example, for user Kent in the Kerberos domain foo.com, the made-up dn will be uid=kent,cn=foo.com,cn=gssapi,cn=auth. This is needed for granting permissions.

Setting up and running OpenLDAP on Debian

• When setting up OpenLDAP on Debian, you need to reconfigure the package so that it creates the initial directory for you as well as the node representing the administrator:

# dpkg-reconfigure -p low openldap

• Then, to check if it is working, you can use SASL EXTERNAL mechanism and use Unix domain socket (ldapi:///) as root to connect to it:

# ldapsearch -b cn=config -H ldapi:/// -Y external

Query

• Just like a relation DB, you can perform queries to retrieve the nodes and their attributes. For example, you can try to find all nodes with cn=Kent Tong (the "filter") under the node whose dn is dc=foo,dc=com (the "base" of the search):

# ldapsearch -b dc=foo,dc=com -H ldapi:/// -Y external "cn=Kent Tong"

• You may try to find all nodes whose objectClass is "person" and whose cn contains the word "Kent":

# ldapsearch -b dc=foo,dc=com ... "(&(objectClass=person)(cn=*Kent*))"

• You may tell it to retrieve only a certain attributes:

# ldapsearch -b dc=foo,dc=com ... "(&(objectClass=person)(cn=*Kent*))" dn sn

Updating the data

• To add a new node, make sure the parent already exists, then specify the dn and other attributes in a text file (called an LDIF file) and mark it as an "add" request:

dn: cn=Paul Chan,ou=Finance dept,dc=foo,dc=com
changetype: add
cn: Paul Chan
sn: Chan
email: paul@ttdev.com
objectClass: person

• Then feed the LDIF file to ldapmodify:

# ldapmodify < myfile.ldif

• To modify (add, change or delete) the attributes of an existing node, specify the node first and then list each attribute change and separate them by a line containing a single hyphen. Finally, end the node with an empty line:

dn: cn=Kent Tong,ou=Finance dept,dc=foo,dc=com
changetype: modify
add: givenName
givenName: Kent
-
replace: email
email: kent@foo.com
-
delete: phoneNumber

• To delete a node:

dn: cn=Paul Chan,ou=Finance dept,dc=foo,dc=com
changetype: delete

Basic IPv6 concepts

• Basic concept. IPv6 is a completely separate protocol from IPv4! It means that the configuration and operation of IPv6 have nothing to do with those of IPv4 at all. They will run independently just like IPv4 and IPX or AppleTalk.
• Format of IPv6 address
• An IPv6 address contains 128 bits and is written in 8 groups of 16 bits like abcd:0000:1111:2222:1357:2468:5555:6789. Each group contains 4 hexadecimal digits (so, 16 bits).
• Many IPv6 addresses contain many consecutive groups of 0000 such as abcd:0000:0000:0000:0000:0000:0000:1234. To save typing, such consecutive groups of zero can be written as ::, so the address above can be written as abcd::1234. Or working from the reverse direction, because there are eight groups in an IPv6 address but only two groups were given, so :: is representing the six groups that are all zero.
• Different types of IPv6 addresses
• Just like IPv4, there are global IPv6 addresses that can be obtained from ISP. They begin with 2???: or 3???:.
• Just like private IPv4 addresses like 192.168.?.?, in IPv6 we use fec?:, fed?:, fee?:, fef?:. These are called site local addresses as they can only be used privately on a site. However, IPv6 provides a better alternative called "unique local address" in the range of fd??. To use it you will first generate a random number and append it to fd to form a random prefix, so that it will not collide (in practice) with any other range used by other private networks.
• Just like 127.0.0.1 in IPv4, there is ::1 (i.e., 0000:0000:0000:0000:0000:0000:0000:0001) in IPv6.
• Just like 169.254.?.? in IPv4, there are fe8?:, fe9?:, fea?:, feb?: address ranges in IPv6. They are called link local addresses as they can only be used privately on a link (a broadcast domain such as a switch or a set of interconnected switches). They are mainly used for network discovery (see below).
• Netmask and subnetting. The host bits in IPv6 take up at least the lower 64 bits (and maybe more), so the netmask is always <= 64 bits. The only exception is the loopback address ::1 which has a netmask of /128. For a LAN, usually just use /64 as the netmask. The ISP may assign a /64, /56 or /48 address block to a site.
• Assignment.
• Stateless auto configuration. In IPv6 it is fine to let a host configure its own address by getting the prefix (the network ID) from the router and then using its own MAC address as the host ID (with some encoding). Therefore, you only need to configure the router with static IPv6 address (so it knows the prefix). How can the host find the router? The host can use multicast to discover the router (part of the IPv6 Network Discovery Protocol) so you don't need to manually configure the default gateway of the host. The idea is that it can detect the running router automatically and perform fail-over as needed.
• DHCPv6. It is very much like DHCP for IPv4. Use it if you need centralized control.
• DNS.
• Domain name to address. You use an AAAA record to map a domain name to an IPv6 address. You can use either IPv4 or IPv6 to query the server and it can return A or AAAA records in either case (depending what record type you are searching for).
• NS, MX, etc. You can just put a domain name there and the name server will try to find its A and AAAA records and return them in the additional section.
• PTR. The reverse domain name is in the form of a.b.c.d.?.?.?.?.?.?.?.IPv6.arpa where a, b, c and d represent a hexadecimal digit (the address is like ??:??:??:...:dc:ba).
• Binding with application protocols. Most higher level protocols already support IPv4 and IPv6 simultaneously (e.g., ping/ping6, OpenSSH, Chrome, Firefox, Apache). To use an IPv6 address directly, you may need to put it in square brackets, e.g., http://[2001::1] in a browser otherwise the colon may look like starting a port. If you use a domain name and already has an AAAA record, the OS will return both the IPv4 and IPv6 to the application. According to the standard, actually global IPv6 are listed before  (preferred) public IPv4.
• In Linux, you can configure a static IPv6 address just like IPv4 in the interfaces file:

iface eth0 inet6 static
address 2001::1
netmask 64

• Checking the IPv6 addresses can be done with:

# ip addr show