GSoC 2012: Modular nodewatcher SL

Among projects accepted for Google Summer of Code 2012 was also development of core components for the next version of nodewatcher platform, version 3.0. While version 2.0 was developed for needs of wlan slovenija network, the goal of version 3.0 is to establish a customizable platform for planning, deployment, monitoring, maintenance, and management of open but diverse wireless networks. The main idea is modularity of the platform on all levels so that each network can completely adapt its installation to needs, use cases, goals, and values of the network. Practice has shown that all this varies between networks and communities and that it is impossible to develop an "one size fits all" solution by itself. Based on this, nodewatcher 3.0 will not be only one solution, but customizable platform with ecosystem of modules and communities around it.

The rest of this blog entry presents description of the architecture of all components from which the new platform is made.

Node configuration and modularity of data models

The concept of modularity brings new challenges to the nodewatcher platform. Foremost, a question poses itself: how to enable modules to add their own configuration options to registered nodes. The platform has to provide interfaces for:

  • Persistent storage of configuration data, where schema is defined by module developer.
  • Option that user through web interface (which should be automatically generated from schema so that developer does not need to define it twice) enters configuration values.
  • Validation of all data at entry time in broader context of node configuration: one configuration value could have dependencies on other modules which require suitable validation at value change. Furthermore, to satisfy needs of the firmware generator, we wanted to assure that user gets feedback about possible invalid configuration for particular hardware immediately, so that she can correct the error directly, and not just after firmware generator fails at some later stage.
  • Specification of context-sensitive default configuration values. For example: node in project X has in the case that hardware supports multiple SSID on interface of ad-hoc type by default SSID "mesh.wlan-si.net", otherwise, in the case hardware supports only one SSID, the default should be "open.wlan-si.net". To satisfy requirement for such complex rules, we developed a simple specialized language to define such rules.

To cover all requirements listed above we developed component named registry. It is a thin layer above existing Django ORM which allows defining schemata (example of schema for node configuration can be found here), automatic generation of corresponding configuration forms, setup of default configuration and validation of provided schema. Furthermore, hierarchical relations between elements are supported, too (eg. one radio can have multiple interfaces which can in turn have multiple addresses) which is a challenge especially when generating suitable user interface and rules evaluation.

Extending and adding functionalities is possible at multiple points so that, for example, developer can change how forms behave (which can dynamically adapt to values of other fields). It is possible at successful validation of configuration to execute various actions (eg. automatic IP addresses allocation based on selected requests, or validation configuration for particular hardware in the case of the firmware generation).

nodewatcher 3.0 uses this component for storing logs about nodes and for extracting configuration for all other modules (firmware generator, network monitoring, etc.).

Network monitoring

The new version of the platform provides a component for network monitoring which also supports modular extending of its functionalities. Each module can add its "processors" which are invoked either on the whole network (all nodes), or for each connected node separately. The latter allows automatic parallelization of parts of monitoring which works on multiple nodes. It is possible to define a completely arbitrary workflow of these processors. Furthermore, modular approach supports various data sources and various ways of data processing and analysis (eg. notification generation at detected deviations from configuration).

It is a framework which allows the development of custom solutions and workflows for network monitoring. Currently implemented is support for fetching topology data from OLSR routing protocol and from existing nodewatcher version 2.0 format which is being used by nodes with our firmware.

A module named datastream has been developed, too. It supports storing all data as time-series and further post-processing for better visualizations of data changes through time.

Firmware image generator

The third key component is firmware image generator named Configuration Generating Modules (CGMs). They are again modules which know how to from configuration in registry generate configuration for operating system which runs on the target router (currently only OpenWrt is supported through UCI configuration). This allows system to prepare firmware for each particular device from existing configuration. This component is not yet finalized, it is still necessary to add support for package management and preparation of final firmware image.

Plans for the future

It is necessary to finalize the firmware image generator and start developing basic elements of graphical interface. Of course, all other modules have to be finalized in the way to have all functionalities at approximate the same level as they are in current 2.0 version.

All development is happening in development branch on GitHub.