Archives

Galera – Information and Maintenance

Ringfree’s PBX databases exist within a MySQL/MariaDB clustering scheme known as Galera. Essentially it’s a high-availability, multi-master, self-healing solution for data redundancy. Ringfree’s Galera cluster consists of five (5) nodes in five (5) locations distributed amongst three (3) infrastructure providers. Those nodes are as follows:

  • galera-atl-02.ringfree.biz – Resides within Ringfree operated infrastructure in the Atlanta DC.
  • galera-dfw-03.ringfree.biz – Running on a vultr.com VPS operating in Dallas, TX, USA.
  • galera-nyc-02.ringfree.biz – Running on a digitalocean.com VPS operating in New York, NY, USA.
  • galera-sfo-02.ringfree.biz – Running on a digitalocean.com VPS operating in San Francisco, CA, USA.
  • galera-fra-02.ringfree.biz – Running on a digitalocean.com VPS operating in Frankfurt, Germany.

Accessing the Nodes

To gain access to the Atlanta node, first access papalmainframe.ringfree.biz via SSH, run the rfc command to access the list of available servers, enter node003.ringfree.biz and run rfctl enter 4026 to gain root access.

To gain access to any of the other nodes, you can SSH into them directly.

Once in a node, the only service of any relevance that is running is MariaDB. Apache may be running but anything currently being served is obsolete and has been out of service for quite some time. You can gain root access to the MariaDB instance using the following:

mysql -u root -p

And entering the appropriate password.

General Galera Cluster Information

There are two primary markers to observe when checking the health and status of the Galera cluster: the cluster size, and the state UUID. If the cluster size is ever anything other than five (5), then there’s a problem somewhere. If the state UUID is different between nodes, then those nodes are out of sync, thus indicating a problem somewhere. To check the cluster size, run the following as the MariaDB root user:

SHOW GLOBAL STATUS LIKE 'wsrep_cluster_size';

To check the state UUID for any given node, run the following as the MariaDB root user:

SHOW GLOBAL STATUS LIKE 'wsrep_cluster_state_uuid';

If you wish to check all available Galera status indicators, use a wildcard in the query:

SHOW GLOBAL STATUS LIKE 'wsrep_%';

For additional information please check the official documentation.

Node Troubleshooting and Repair

By and large, Galera is extremely reliable and is generally both trouble and maintenance free. In the event that a problem is identified, it’s usually following a large scale network outage or a longer-than-intended maintenance period with one of the third party infrastructure providers.

If a node is not operating/syncing normally (as indicated by the cluster size and state UUID), often the fix is to simply restart the service:

sudo systemctl restart mysqld

Depending on how far out of sync the node is, the restart may time out several times before the service successfully starts up again. What has worked in the past has been to try to restart it twice, then wait a couple of hours and try again a couple of times. If this doesn’t work, then the node has likely failed and you’ll need to refer to the official documentation on node failure and recovery.

superEngine – FreePBX/NetSapiens Migration Utility

superEngine is an extensible command line utility that performs actions against the NetSapiens API and migrations from FreePBX to NetSapiens. Among its various uses are the ability to perform CRUD operations on any currently implemented NetSapiens object.

Installation & Configuration

To install superEngine, run the following commands on a Linux or Mac operating system with Git, PHP 7.3+, and Composer installed. Installation for Windows machines varies according to the deployment specifics of your PHP and Composer installations and is not recommended.

git clone https://github.com/Ringfree/super-engine.git
cd super-engine
composer install

Upon cloning the repository you will be prompted for your Github credentials as the repository is private. Upon running composer install you’ll be prompted to create/copy an access token as one of the dependencies, the NetSapiens PHP SDK, is also in a private repository.

At this point you’ll need to add configuration parameters so that superEngine knows where you’d prefer to store files and how to communicate with the various services. A default (and almost empty) configuration file called config.ini is available within the super-engine directory. It’s recommended to copy the file to /etc/superEngine.ini as superEngine looks for the file there by default.

Within the configuration file there are four sections. Here’s a brief rundown of everything you need to know:

storage

  • dir – Where superEngine should write/download files used during migration actions.

netsapiens

  • apiUrl – The domain of the NetSapiens instance you’ll be communicating with. This should be limited to the subdomain, second-level domain, and top-level domain.
  • clientId – The OAuth client ID you’ll be using to access the NetSapiens API. OAuth clients are configured within the SiPbx interface.
  • clientSecret – The OAuth client secret/password. This is also configurable within the SiPbx interface.
  • username – The username of the user you’ll use to access the NetSapiens API. For superEngine it is recommended that you create a dedicated API user in the “Super User” scope.
  • password – The password of the user you’ll use to access the NetSapiens API.

portal

  • apiUrl – The domain of the Ringfree Portal instance you’ll be communicating with. This should be limited to the subdomain, second-level domain, and top-level domain.
  • username – The username of the user you’ll use to access the Ringfree Portal. Any user with the api-basic-auth:read permission will do.
  • password – The password of the user you’ll use to access the Ringfree Portal.

freepbx

  • username – The username of the user you’ll use to access FreePBX instances. Note that Ringfree has a standard common username for this that should allow access to all instances.
  • password – The password of the user you’ll use to access FreePBX instances. Note that Ringfree has a standard common password for this that should allow access to all instances.

Basic Application Structure

superEngine is a simply structured application with the only hard dependencies being the NetSapiens PHP SDK, prim/httpfactory, and league/container. When run, the application first initializes the service container, reads the action from the command line arguments, and attempts to perform that action using any other command line arguments specified.

In superEngine terminology, an action is a class located within the src/Action directory. Any action located within this directory can be called via the command line so please don’t put any internal classes here. An action can be something like CreateDomain or ImportSubscribers and naming is generally consistent. You can add a Test action here that will be ignored by Git.

Services provided by the service container implement interfaces found within the src/Policy directory. If you wish to swap a service implementation with a new one, you just need to modify what’s being added to the container in the appropriate class within the src/Service directory. If you’re writing a new implementation, please put your business logic either within the src/Internal directory or within a new component. With the exception of the two aforementioned hard dependencies, it’s expected that appropriate interfaces will be added and the service container will be used for all dependencies.

Using superEngine

To use superEngine, run the application from within the super-engine directory as follows:

php superEngine --action="Test"

You may add additional arguments in any order following php superEngine command but an action argument is always required. Various actions require various different arguments. The specific arguments necessary for any action are available in the file level docblock for that action and are referenced with the @uses tag.

As an example, the CreateReseller action contains the following file level docblock:

/**
 * Creates a new Reseller in NetSapiens.
 * 
 * @uses territory      The name of the reseller.
 * @uses description    A short description of the reseller.
 */

Therefore in order to create a new Reseller within NetSapiens, superEngine will need to be run as follows:

php superEngine --action="CreateReseller" --territory="Demo" --description="Demo"

Any time you run superEngine you have the opportunity to specify a configuration file by passing the config argument with the path of the config file you wish to use. If you do so, the specified file will completely replace the default configuration file for that execution of the application. Basically you can’t override just a couple of parameters, rather you need to specify everything in your file. For example:

php superEngine --action="Test" --config="/path/to/config.ini"

Words of Caution

Because of the nature of superEngine, it’s incredibly easy to damage or delete things critical to the operation of a NetSapiens phone system. Please exercise caution when running any commands and double check your inputs.

Additionally, the NetSapiens PHP SDK follows the API specification very closely. Because the API is poorly and often incorrectly documented, sometimes things don’t always work as expected. This typically presents as Bad Request errors, however sometimes API requests will “succeed” in unexpected ways. Fixes and workarounds should be implemented within the NetSapiens PHP SDK, not within superEngine.

Extending the Application

There are three ways in which superEngine may normally need to be extended:

  1. Adding new action classes.
  2. Supporting new command line arguments.
  3. Putting new dependencies in the service container.

Also of note is that superEngine is written against PHP 7.3 so you should be using either 7.3.x or 7.4.x when running it. It’s possible that it’ll work with 7.2 but hasn’t been tested. superEngine also uses strict types everywhere and this convention should be followed moving forward.

Adding a New Action

Simply create a new class within the src/Action directory and adhere to the following:

  • Include a file level docblock containing @uses tags for all command line arguments that will be used by the action.
  • Declare a public constructor that takes a League\Container\Container object as the only argument.
  • Keep all business logic exclusive to the action within the constructor or private methods. Reusable business logic should be factored out into classes within the src/Internal directory.

Adding a New Command Line Argument

Open the src/Internal/Args.php file and do the following:

  • Add a private class property for the new argument. Also be sure to add that property to the docblock.
  • Within the constructor, add the argument suffixed by two colons to the $longOpts array.
  • Within the constructor before $this->config is set, set the property from the $args array and provide a default value. null is an acceptable default value.

Service Container Dependencies

superEngine uses league/container, which is a PSR-11 compliant service container implementation. Service definitions are located within the src/Service directory. To add a new service, please do the following:

  • Create a new class within the src/Service directory that extends the League\Container\ServiceProvider\AbstractServiceProvider class.
  • Add a protected property called $provides that contains an array of the interfaces the new service will provide.
  • Add a public method called register that accepts no arguments and returns void. Within this method, register the classes implementing the aforementioned interfaces to the container. Consult the official documentation for specifics on how to do this.
  • Open the src/ServiceProviders.php file and, within the init function, add the service to the container using the service’s fully qualified namespace.

4. Assembling views

All the functionality in the world is useless if you don’t have some method of interfacing with it. For this reason, the portal has a powerful and extensible way of implementing views. Views an be implemented very simply, however the portal has a built in framework for handling most of the tedious bits for you. Let’s start with a basic view example and go from there.

Implementing a “Hello World” View

Fundamentally a view can be implemented with only two things: a registration to the VIEW hook, and a function that outputs some content:

namespace Papal\View;
__register('VIEW', '/hello', ['callback' => '\Papal\View\Hello::display']);

class Hello {
  public static function display() {
    echo "Hello World";
  }
}

In this case, visiting the /hello route within your application will result in the application displaying the output of the specified callback function. If you navigate to the route in your browser you’ll see just the following:

Hello World

Pretty simple stuff, right? That said, there’s a bit more going on under the hood. To demonstrate, let’s update our function definition a bit:

namespace Papal\View;
__register('VIEW', '/hello', ['callback' => '\Papal\View\Hello::display']);

class Hello {
  public static function display($args) {
    var_dump($args);
  }
}

Now if you navigate to the view, you’ll be presented with the following output:

array(1) {
  ["urlpath"]=>
  array(1) {
    [0]=>
    string(5) "hello"
  }
}

As you can see, the components of the URL get passed to the function so that the function can act upon them.

Regular Expressions in URLs

One feature of the VIEW registration hook is that regular expressions can be used as URL components. Consider the following example:

namespace Papal\View;
__register('VIEW', '/hello/([A-Za-z]+)', ['callback' => '\Papal\View\Hello::display']);

class Hello {
  public static function display($args) {
    echo "Hello, {$args['urlpath'][1]}";
  }
}

Now if we navigate to /hello/George we’ll be presented with the following output:

Hello, George

The ViewHandler Class

The above is all good and well if you have extremely basic needs from your view. But what if you want your view to actually integrate with the rest of the application? Or what if you have certain permissions requirements for the view? For this we have the \Papal\ViewHandler class.

The Viewhandler class is intended to serve as a parent class for your views. By extending it, you give your class support for authentication and permissions requirements, and a nifty way of mapping arrays to your view object’s properties. The ViewHandler class is also the “correct” way of implementing views so that potential future updates to it will affect all of your views without you having to modify each of your views. Let’s take a look at a basic implementation that uses the portal’s Body template:

namespace Papal\View;
__register('VIEW', '/hello/([A-Za-z]+)', ['callback' => '\Papal\View\Hello::display']); 

class Hello extends \Papal\ViewHandler {
    public function __construct($args = null) {
        $setup = $this->setup($args);
        if ($setup === true) {
            \Papal\Template\Body::open("Hello World", "Hello World");
            $this->markup();
            \Papal\Template\Body::close();
        }
    }

    private function markup() {
        echo "Hello, {$this->urlpath[1]}";
    }
}

As you can see, we’re extending the ViewHandler class and declaring a constructor overload that accepts a single argument (which defaults to null). The constructor then calls the setup() object method from the ViewHandler class which determines whether to display the the view, a login page, or a 404 page. If there are no issues with the setup, the constructor then displays the output of the markup() object method in between the opening and closing of the Body template.

The private markup() object method in this case simply outputs the same string from our more basic example above, but note that this time it’s accessing urlpath as an object property rather than as part of an array passed directly to the display() method (which is now defined within the ViewHandler class and is simply a shortcut to call the object constructor). The setup() method mentioned above, takes the entries in that array and creates object properties out of them.

So what about authentication and permissions? By default all classes extending the ViewHandler class require the user to be logged in and don’t have specific permissions requirements, but this is very easy to set. Let’s take our class definition from above and make the requirements explicit and add the requirement that the user have the search capability:

class Hello extends \Papal\ViewHandler {
  public function __construct($args = null) {
    $this->requiresAuth = true;
    $this->requiresCaps = 'search';
    $this->requiresAllCaps = false;

    $setup = $this->setup($args);
    if ($setup === true) {
      \Papal\Template\Body::open("Hello World", "Hello World");
      $this->markup();
      \Papal\Template\Body::close();
    }
  }

  private function markup() {
    echo "Hello, {$this->urlpath[1]}";
  }
}

Let’s break it down:

  • requiresAuthboolean – Whether or not the view requires the user to be authenticated. For most views, you’ll want to leave this alone, however you may find it useful to override this if you have pages you wish to display publicly or if you’re, for instance, building an API route that authenticates differently.
  • requiresCapsstring | array – If this is a string, the user accessing the page must have the listed capability in the permissions list for his/her user role. If this is an array, the user accessing the page must have one of the capabilities in the permissions list for his/her user role.
  • requiresAllCapsboolean – Makes it so that the user must have all the capabilities in the requiresCaps array, rather than any single one, in order to access the view.

Structuring your HTML

While there are no specific requirements for HTML structuring, there are definitely a few best practices and a few things to keep in mind. At the moment and for the foreseeable future, the portal’s default templates make use of Bootstrap 4 and it’s generally a good idea to use it in your views as well. In fact the Bootstrap CSS and Javascript are both included in the default Header template (which is called from within the Body template). If your view requires custom CSS you can add it by passing the necessary argument to the \Papal\Template\Body::open() function or by including style tags in the view output.

Some modules provide custom registration hooks which can be used to assist with and/or automate the construction of various HTML components. For instance the PapalAdmin module provides the ability to quickly and easily assemble forms, tables, modals, and jQuery form submissions by using arbitrarily defined registration hooks.

In some rare cases, you may find that the default templates included in the portal’s core code are not sufficient for what you’re trying to accomplish. In this case you’re welcome to implement your own templates or assemble your own views from scratch. A good use case for this would be implementing publicly viewable pages with a different “theme” than the administrative back-end. You could also use some logic within the view to display different templates based on other factors such as the user ID, the time of day, or the value of a setting in the database.

As an example let’s say a developer has a user base that is still largely dependent upon an old version of Internet Explorer for reasons beyond their control. The developer could write new templates that work with the old IE version and then test the value of $_SERVER['HTTP_USER_AGENT'] to determine which templates to load.

Portal Registration Hook Reference

The following is a reference for all registration hooks offered within the portal’s core code. This reference presumes you are familiar with how registration hooks work and that you understand the terminology in use. For more information on these topics, please review section 3 of the portal developer’s guide: Understanding and using registration hooks.


INIT

Description: Arbitrary functions to be executed each time the application is accessed. The functions will be executed after the session has been loaded but before any other other actions are taken (such as form handling or routing).

Trigger: Unused except for logging.

Args:

  • callbackstring – (required) A fully qualified and namespaced function name without arguments or parentheses.
  • argsarray – Arguments to be passed to the function specified in the callback.

Example:

__register('INIT', 'demo', ['callback' => '\Papal\Log::write', 'args' => ['Demo']]);

PERM

Description: Defines a slug to appear as a permission which can be assigned to various user roles. The slug itself has no value in and of itself; rather it’s up to the developer to make use of the slug when implementing permissions-related functionality in their modules.

Trigger: The slug to be defined as the permission. Please use a proper slug beginning with a lowercase letter and consisting of lowercase alphanumeric characters and hyphens.

Args: Unused.

Examples:

__register('PERM', 'search');
__register('PERM', 'system-alerts');

FORM

Description: Defines a handler for form submissions. The portal will look for a POST data key that matches the trigger and will execute the specified function. Arguments may optionally be passed to the function by specifying an array of addtional POST data keys from which the values will be taken.

Trigger: The POST data key that the portal is to look for.

Args:

  • callback  – string – (required) A fully qualified and namespaced function name without arguments or parentheses.
  • argsarray – POST data keys from which to use the values for as arguments to be passed to the function specified in the callback.
  • requiresstring | array – Permission triggers required by the user submitting the form.
  • requiresComparestring – Accepts the literal values ‘AND’ and ‘OR’. If the ‘requires’ parameter is an array, pass AND to require all permissions or pass OR to require any one of the specified permissions.

Example:

__register('FORM', 'testForm', ['callback' => '\Papal\Log::write', 'args' => ['content']]);

The above presumes a $_POST variable with structure similar to the following:

array(2) {
  ["testForm"]=>
  int(1)
  ["content"]=>
  string(4) "Test form content."
}

Additionally you could specify that the user submitting the form be required to have both the search and system-alerts permissions:

__register('FORM', 'testForm', ['callback' => '\Papal\Log::write', 'args' => ['content'], 'requires' => ['search', 'system-alerts'], 'requiresCompare' => 'AND']);

VIEW

Description: Associates a URL with a function that outputs content, such as HTML or JSON. This can also assign a context to the session which can be used by other parts of the application to determine things such as conditional menu display.

Trigger: A string containing the URL to route to the view with optionally embedded regular expressions for pattern matching. An array of each URL component is passed as an argument to the specified callback.

Args:

  • callbackstring – (required) A fully qualified and namespaced function name without arguments or parentheses. Preferably one that outputs content usable by something like a browser or cURL.
  • contextstring – The context of the view.

Examples:

__register('VIEW', "/admin", ['callback' => '\Papal\View\Admin::display', 'context' => 'admin']);
__register('VIEW', "/admin/users/[0-9]+", ['callback' => '\Papal\View\UserEdit::display', 'context' => 'admin']);

CRON

Description: Functions to be executed whenever the cron.php file is executed. This is similar to the INIT registration hook, however a cron job should be set up on a reasonable schedule to ensure the regular execution of these functions. In the future, scheduling will occur via arguments passed to the registration hook. For now it executes the function with every execution of the cron script.

Trigger: Unused except for logging.

Args:

  • callbackstring – (required) A fully qualified and namespaced function name without arguments or parentheses.
  • args  – array – Arguments to be passed to the function specified in the callback.

Example:

__register('CRON', 'sessionClean', ['callback' => '\Papal\Session::clean']);

MENU

Description: Adds an entry to the menu.

Trigger: The menu link destination. Relative URL paths can be used, but should be used with caution. It’s preferable to link URLs within the application by using their path relative to the base URL. If you’re using a URL for a page outside of the application, perhaps ask yourself why that item needs to be in the menu in the first place.

Args:

  • menustring – (required) Which menu to include the item in. The portal includes three default options (‘main’, ‘sub’, and ‘user’) however developers are free to implement their own menus either in the sidebar or in other parts of the application.
  • labelstring – The text to appear as/with the menu item (if applicable).
  • altstring – The HTML alt text for the menu item.
  • iconstring – The icon to appear as/with the menu item (if applicable). By default the portal uses the free, solid icon set from FontAwesome.
  • priorityint – Where in the menu order the item should appear. Low numbers are placed first whereas high numbers are placed last.
  • permsarray – If the user isn’t capable of any of the permissions listed here, the menu item will not display.
  • contextarray – Only display the menu item if an item in this array matches the context of the current view.

Example:

__register('MENU', "/infrastructure/datacenters", ['menu' => 'sub', 'label' => "Datacenters", 'alt' => "Datacenters", 'icon' => "hotel", 'priority' => 10, 'perms' => ['manage-infrastructure'], 'context' => ['infrastructure']]);

 

2. References, metadata, and the database

The data and object implementation within the portal generally follows one simple rule: make everything as abstract as possible. The more abstract it is, the easier it should be to extend and because none of the data in the portal is speed critical, we can afford to trade some amount of optimization for flexibility.

Data in the portal is almost always abstracted into two categories: references and metadata. The former containing no more than an ID and a type, the latter containing all of the information associated with that reference. For example, let’s say we need some way to represent a vehicle. The reference would simply contain an automatically assigned ID, and the type vehicle while the metadata would include things such as the make, model, color, year, mileage, and other pertinent bits of data. To better understand this, let’s take a look at the database schemas for each type. Here is the refs table schema:

+-------+------------------+------+-----+---------+----------------+
| Field | Type             | Null | Key | Default | Extra          |
+-------+------------------+------+-----+---------+----------------+
| id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| type  | varchar(128)     | NO   | MUL | NULL    |                |
+-------+------------------+------+-----+---------+----------------+

And here is the meta table schema:

+-------+------------------+------+-----+---------+----------------+
| Field | Type             | Null | Key | Default | Extra          |
+-------+------------------+------+-----+---------+----------------+
| id    | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| type  | varchar(128)     | NO   | MUL | NULL    |                |
| slug  | varchar(128)     | NO   | MUL | NULL    |                |
| refid | int(10) unsigned | NO   | MUL | NULL    |                |
| value | longtext         | YES  |     | NULL    |                |
+-------+------------------+------+-----+---------+----------------+

Using our above example, let’s presume we need to store data that represents a 1994 Honda Civic. Once created let’s say we have an id of 42. Here would be the example reference and metadata as stored in the database:

+----+---------+
| id | type    |
+----+---------+
| 42 | vehicle |
+----+---------+

+------+---------+-----------+-------+---------+
| id   | type    | slug      | refid | value   |
+------+---------+-----------+-------+---------+
| 3121 | vehicle | type      |    42 | vehicle |
| 3121 | vehicle | make      |    42 | Honda   |
| 3122 | vehicle | model     |    42 | Civic   |
| 3123 | vehicle | year      |    42 | 1994    |
| 3124 | vehicle | color     |    42 | green   |
| 3125 | vehicle | mileage   |    42 | 127235  |
| 3126 | vehicle | trimLevel |    42 | CX      |
+------+---------+-----------+-------+---------+

So why not create a vehicles table with the relevant columns? The answer is that in this paradigm, anyone or anything can then add more metadata without there having to be any changes to the original code or to the database structure, and all the base level data access and manipulation functions can be contained within a couple of classes. Also note that the indexes in the meta table schema should provide sufficient performance for most use cases.

Of course there’s nothing stopping a developer from creating their own database tables if they require optimization beyond the reference/metadata paradigm. Creating and dropping those tables would be a good use case for the onActivate and onDeactivate parameters specified in the module’s meta.txt file.

There are classes specifically for interacting with the database and for handling references and metadata. Unless you have very particular needs, it’s extremely unlikely that you’ll ever need to interact with the \Papal\Database class directly. Rather most everything written to or read from the database will happen via the \Papal\Reference class, a child of it, or the \Papal\Meta class.

The Reference Class

Almost every “thing” that you’ll represent within the portal is done so via the \Papal\Reference class or a child class which extends it. If you look in the code of most existing modules, you’ll notice that there are many instances of classes extending the Reference class to provide implementation specific functionality or to override the default functions. The Reference class or an extension of it provides a clean and simple way of creating just about any sort of object representation within the portal.

The fundamental anatomy of a Reference object and how it maps to the database is incredibly simple: the reference object represents an item in the refs table and all of the object properties represent the associated items in the meta table. Consider the following example:

$ref = \Papal\Reference::new('vehicle');
$ref->make = 'Honda';
$ref->model = 'Civic';
$ref->year = '1994';
$ref->save();
echo $ref->id;

This creates a new reference of type vehicle. Note that the new() static method writes the reference to the database and returns the object with a couple of default properties: id and type. Next we set a few of the values used in our previous example and call the save() object method to write the object properties to the database as metadata.

It’s extremely unlikely that you’ll ever need to call the Reference class constructor directly. Doing so without the reference ID will simply return an empty object. Creating new Reference objects is almost always done via the aforementioned new() static method and retrieving existing ones from the database is almost always done with the constructFromId() static method as is done in the following example:

$car = \Papal\Reference::new('vehicle');
$car->make = 'Chevrolet';
$car->save();

$chevy = \Papal\Reference::constructFromId($car->id);
var_dump($car);
var_dump($chevy);

The above will result in the following output:

object(Papal\Reference)#8 (3) {
  ["id"]=>
  string(3) "625"
  ["type"]=>
  string(7) "vehicle"
  ["make"]=>
  string(5) "Chevy"
}
object(Papal\Reference)#7 (3) {
  ["id"]=>
  string(3) "625"
  ["type"]=>
  string(7) "vehicle"
  ["make"]=>
  string(5) "Chevy"
}

Note that you now have two objects that represent the same reference. While you’d probably not want to do this in the module you’re planning to write, it serves as an appropriate example of how to create and access Reference objects.

There are additional static methods and object methods for accomplishing various tasks with and within the Reference class. There are methods available to retrieve multiple objects, delete/purge objects, set timestamps, set object properties from JSON data, etc. For a complete reference please look at the code on Github.

Extending the Reference Class and Using Utility Classes

So what if your “thing” needs some additional functionality beyond what is found in the Reference class? Or what if all of your references have certain metadata values that need to be associated with them? There are two ways to accomplish these sorts of things. Consider the following example of a class that extends the Reference class:

namespace Papal;
class HondaCivic extends Reference {

  public static function new($placeholder = null) {
    $ref = parent::new('vehicle');
    $ref->make = 'Honda';
    $ref->model = 'Civic';
    return $ref;
  }

  public function setMileage(int $miles) {
    $this->mileage = $miles;
  }
}

And this example of a separate utility class:

namespace Papal;
class HondaUtility {

  public static function new() {
    $ref = Reference::new('vehicle');
    $ref->make = 'Honda';
    return $ref;
  }

  public static function newCivic() {
    $ref = self::new();
    $ref->model = 'Civic';
    return $ref;
  }

  public static function setMileage(Reference $ref, int $miles) {
    $ref->mileage = $miles;
  }
}

Both classes effectively accomplish the same things and one implementation is not necessarily better than the other. If you prefer a more object-oriented coding style, extending the Reference class might make more sense to you. If you prefer a more procedural approach, a utility class might suit your workflow better.

The Meta Class

While most of your interactions with metadata will be via Reference object properties, sometimes you may find that it makes sense to interact with the metadata directly. The \Papal\Meta class exists for this reason. There are also various static methods for retrieving reference IDs or multiple pieces of metadata. The Meta class isn’t intended to be an abstracted away, “lower level” class in the same vein that \Papal\Database is, rather it’s intended to be used anytime you need to manipulate or filter for metadata but when you don’t necessarily want or need the extra overhead of Reference objects. In fact the Reference class itself makes use of the Meta class for a lot of underlying functionality.

Say for instance we want an array of all references that represent green vehicles. We can leverage the Meta class to assist with this:

$refIds = \Papal\Meta::getRefIdArray('vehicle', 'color', 'green');
foreach ($refIds as $id) {
  $refs[] = \Papal\Reference::constructFromId($id);
}

Or let’s say we need an array of all vehicle colors currently in the database perhaps to display in something like a select input or a report:

$colorData = \Papal\Meta::getRefIdsAndValues('vehicle', 'color');
$colors = array_unique(array_values($colorData));

It should be noted that the Meta class is a final class and that it only contains static methods. Basically there’s no reason to ever try and instantiate a Meta object as metadata should only ever exist as primitives and Reference object properties. For a complete Meta class reference, check out the code on Github.

Next we’ll take a look at the portal’s powerful registration hooks, how they work, and how to use them:

Understanding and using registration hooks.

3. Understanding and using registration hooks

Portal development makes heavy use of a concept called registration hooks. Say you need to create a new view, or add an item to the menu, or insert some javascript code into the header? The answer is to do so via registration hooks. There are a handful of default hooks you can make use of, but it’s really simple to create your own custom hooks to do with whatever you like.

Portal Registration Hook Reference

Let’s look at an example from the Homepage module:

__register('VIEW', "/", ['callback' => '\Papal\View\Homepage::display']);

Note that we’re calling the __register() function with three arguments. The first is the registration hook, in this case we’re registering a new view. The second is the trigger to be registered, in this case we want to the view to be displayed at / or the base URL for the application.

The third argument is where the real magic happens. Different registration hooks will have different usage requirements, but this is basically an array of data that tells the hook how to operate. In the case of the VIEW hook, the only thing necessary is a callback which is the function that will output the HTML for the view.

Let’s take a look at another example, this time from /templates/Sidebar.php in the portal core code:

__register('FORM', 'ajaxDisplayMenu', ['callback' => '\Papal\Menu::setDisplayMenuCurrent', 'args' => ['value']]);

In this case we’re registering a form handler with the ajaxDisplayMenu trigger. How this works is actually quite simple: Whenever a POST request is submitted to the application that contains a value for the ajaxDisplayMenu key, it executes the specified callback with the specified arguments using the appropriate POST values. The above would functionally do the same as the following:

if (isset($_POST['ajaxDisplayMenu'])) {
    \Papal\Menu::setDisplayMenuCurrent($_POST['value']);
}

So why use this paradigm instead of something more conventional? The answer is simple: registration hooks make it very easy to build extensible features. Because of how they’re implemented any developer can, for example, register a new menu item from any PHP file that gets evaluated in their module.

For another example, say a dev named Jim uses registration hooks to build a table for displaying CDR data. Later on another dev named George could then add columns to Jim’s table to display additional data that wasn’t part of the original scope. And George can do it barely having to look at Jim’s code, much less modify it in any capacity.

How it Works

So let’s get under the hood and take a look at how the __register() function works so you can learn how to better make use of registration hooks and even implement your own. In the portal’s index.php the function is declared as follows:

function __register($hook, $trigger, $args = true) {
    $GLOBALS["_PAPAL_{$hook}"][$trigger] = $args;
}

As you can see, the __register() function is simply a clean way of leveraging the PHP superglobal variable called $GLOBALS. All of the code that actually works with registration hooks utilizes $GLOBALS directly. As an example, here is part of the \Papal\Form::handle() function:

foreach ($GLOBALS["_PAPAL_FORM"] as $trigger => $form) {
    if (isset($_POST[$trigger]) || isset($_FILES[$trigger])) {
        ...
    }
}

Custom Registration Hooks

So how would you go about implementing a custom trigger? Simple, just write a function that uses the registration hook paradigm and call it from somewhere. Here is an example:

namespace Papal;
class LogRegister {
  public static function logIt() {
    foreach ($GLOBALS["_PAPAL_LOG"] as $trigger => $data) {
      Log::write("{$trigger} -- {$data['content']}");
    }
  }
}

Here would be the registration hook:

__register('LOG', 'test', ['content' => 'Success!']);

And the registration hooks would be processed anywhere in your module where you call the \Papal\LogRegister::logIt() function.

Overriding Hooks

It’s possible to override registration hooks in some cases. Say for instance the author of the Homepage module want a particular menu item to display on the home page. Remember our example from above where we saw that the Homepage module uses a VIEW registration hook to display the result of the \Papal\View\Homepage::display() function? Let’s say the author doesn’t want to display the item for /infrastructure from the Infrastructure module. All the author has to do is to place the following within the \Papal\View\Homepage::display() function before the HTML is output:

__register('MENU', "/infrastructure", []);

It’s advisable to not try and override registration hooks from within the global namespace. Rather you should only ever attempt a registration hook override from within your function, which should be in a properly namespaced class.

Conventions

The correct convention for assembling and using a registration hook is to pass an uppercase string for the $hook value, a non-uppercase or mixed-case string for the $trigger value and an associative array for the $args value, even if you only require a single value from $args. You should always pass the value for $hook explicitly though it’s perfectly acceptable to use the result of a function for $trigger and/or $args. Just be sure the function is loaded and available for your registration hook.

If you implement custom registration hooks within your module, please document them in the README.md file so that other developers can easily know that they’re available and how to make use of them.

Content Hooks

In various places through the portal’s templates and views, you may encounter functions similar to the following:

\Papal\Hooks::insert('SIDEBAR_JS_BEFORE');

This is what is referred to as a content hook. Basically a content hook is a simple way to add content to areas of a page. They’re simple to use and they’re even in use within the portal core code itself.

So how do they work? Simple, just call make a registration containing three arguments: a callback, an optional array of args, and an optional priority. The callback will obviously be a string containing a namespaced function name. The args will obviously be the arguments passed to that function. The priority is the order in which the content displays if there are multiple registrations to the same content hook.

As an example, let’s start by defining a function:

namespace Papal;
class HookTest {
  public static function hookDemo() {
    ?>
      <!-- TESTING -->
    <?php
  }
}

Now let’s assemble the registration:

__register('SIDEBAR_JS_BEFORE', 'hookdemo', ['callback' => '\Papal\HookTest::hookDemo']);

The result will be the comment output by the function appearing in the page where the SIDEBAR_JS_BEFORE hook is called using the \Papal\Hooks::insert() function mentioned above.

Now that you have an understanding of how registration hooks work, let’s put them to use and assemble a view:

Assembling views.

1. Directory naming, creating a meta.txt file, and ModuleInfo objects

As stated in the introduction, two parts of a Portal module that absolutely must be present are a parent directory and a file named meta.txt. The directory name must be a proper slug, consisting only of lowercase letters, the digits 0-9, and hyphens. In addition to general information such as the name, description, and author of the module, meta.txt contains functionally relevant values such as the module version and dependencies, as well as activation and deactivation instructions.

The format of meta.txt is very simple, yet strictly enforced. Let’s take a look at an example:

# Copyright 2018 Ringfree Communications Inc.

moduleName:: Demonstration Module
moduleDesc:: Provides "proof of concept" functionality for development and testing purposes.
moduleIcon:: bookmark
authorName:: Kendall Weaver
authorEmail:: kendall@ringfree.com

version:: 1.0.0
license:: WTFPL
section:: none
depends:: papaladmin>=0.0.1,homepage>=0.0.1,CORE>=0.0.1

onActivate:: \Papal\DemoModule::activate
onDeactivate:: \Papal\DemoModule::deactivate

Note the very first line beginning with a # character. This is a comment. Any line beginning with a # will be ignored when the file is evaluated. To keep things simple, inline and multiline commenting are not supported.

The remaining lines adhere to a very simple syntax: the name of the field immediately followed by two instances of the : character, immediately followed by a single space. Failure to strictly ahere to the name:: value paradigm will result in the line not being correctly parsed when the file is evaluated.

When the file is evaluated, what happens “under the hood” is that a \Papal\ModuleInfo object is created and each field is stored as a class-level public variable. Additionally three other class-level public variables are stored: slug being a string containing the name of the directory housing the module code, path being a string containing the file path to the module directory, and active being a boolean representing whether or not the module has been activated. More on this later.

The Fields

  • moduleNameRequired Field – A user-readable name for the module. Feel free to use capitalization, punctuation, numbers, and special characters if they’re necessary to properly name the module.
  • moduleDesc – A user-readable description for the module. Like the name, feel free to take liberties regarding the characters used.
  • moduleIcon – A reference to which icon should be displayed beside the module information on the module administration page. By default the Portal makes use of Font Awesome and supported values for this field are any icon in the solid section of the free set.
  • authorName – The name of the module author. Convention is to use a full, real name.
  • authorEmail – The email address of the module author. Please use a valid email address.
  • versionRequired Field – The module version number. This is used for dependency checking and, as such, it is advisable (though not required) to keep the versioning scheme in line with other modules. Semantic versioning is strongly suggested.
  • license – Presumably all modules written for the Portal will be owned by Ringfree with all rights reserved, however the core itself is fairly application agnostic and could potentially be licensed to other companies and/or open-sourced in the future.
  • section – Simply a means of organizing modules within the module administration interface.
  • depends – A comma separated list of other modules required to be active before this module can be activated. Note that the list also includes comparators and version numbers. Each listed dependency must have a comparator and version number. If a specific version is not required, use >=0 as this should encapsulate all possible versions. Standard comparators are supported: >, <, >=, >=, ==, and !=. Also note that specific version requirements for the Portal itself can be specified by using CORE as the dependency name.
  • onActivate – This allows you to specify a function (with no arguments) to be evaluated when the module is activated. The function can be (and in most cases should be) within the module itself. Note that the namespace and class must be included if applicable. Please do not include parenthesis at the end.
  • onDeactivate – Similar to above, this allows you to specify a function to be evaluated when the module is deactivated. Again no arguments will be passed to the function, the namespace and class must be included, and parenthesis must be omitted.

Note that there are only two required fields: moduleName and version. Also additional arbitrary fields may be specified and they’ll likewise be evaluated into class-level public variables. A potential use case would be making additional metadata available to other modules.

Creating ModuleInfo Objects

The ModuleInfo class exists within the Papal namespace and has a constructor that accepts a single argument: the module slug (the name of the directory where the meta.txt file and the module code is stored). Presuming our above example file exists within a directory named demomodule, the ModuleInfo object would be created as follows:

$x = new \Papal\ModuleInfo('demomodule');

ModuleInfo objects are primarily used to construct the module administration interface, itself part of the papaladmin module. When instantiated, the constructor function explicitly sets the active, path, and slug variables before calling the private parseMeta function which evaluates the data in meta.txt. A missing meta.txt file or the absense of required fields will result in an Exception. Additionally attempting to call the constructor with a reserved value such as CORE will result in an exception. At present CORE is the only reserved value. This page will be updated with any additional reserved values added in the future.

Now that you have a basic understanding of how to put together the meta.txt file, let’s take a look at how the portal handles data:

References, metadata, and the database.

0. Module Development for the Ringfree Portal

The Ringfree Communications Portal, internally designated as “Papal” or “Papal Mainframe” supports extensible functionality by way of installable modules. Module development is a fairly straightforward process and should be accessible to anyone with PHP experience in a capacity beyond the introductory.

The Portal is implemented targeting PHP 7.0, which should be the minimum target version for all module code for the time being. There are and will be no plans to support older 5.x versions of PHP. The Portal uses a fairly standard LAMP application stack with Apache’s mod_rewrite and mod_headers enabled. With minimal configuration, the Portal could be deployed in a number of similar environments such as LEMP, WAMP, etc.

Portal modules, at a minimum, require two components: a directory with a unique name limited to alphanumeric characters and hypens, and a file within that directory named “meta.txt” containing some basic information about the module. Such a module would accomplish nothing other than a demonstration of proof-of-concept, but functionally that’s all that is required to implement a module.

To add functionality to the module, all files within the module directory ending with a .php extension will be evaluated. As such, a module developer has some options regarding how to structure the project. For a simple module it may make sense to place all of the code in one or two PHP files directly beside the meta.txt file in the module directory. For a more complex module it would perhaps be better to organize the code in subdirectories and have a single base level PHP file that includes the appropriate files from the subdirectories. The only “correct” way to approach this is the one that makes the most sense given the scope of the module being worked on.

The Portal’s core code makes use of programming concepts such as namespaces, recursion, inheritance, and static functions. It’s advisable for any module developer to be reasonably familiar with these concepts and to implement them appropriately within their module. In fact some functionality requires classes to exist within specific namespaces (these situations will be pointed out later).

What follows is a guide that will walk you through the development of a module including structuring the meta.txt file, making use of the Portal’s registration features, and implementing a view. To start, let’s take a deeper look at the basic building blocks:

Directory naming, creating a meta.txt file, and ModuleInfo objects.