Upgrading to Joomla 3.8, 3.9, 3.10 and 4.0

Since the release of Joomla 3.8 in September of 2017, the Joomla community has been preparing for two major technical upgrades: moving from PHP 5.6 to 7.x and from Joomla 3.x to Joomla 4.0. PHP 5.6 will cease to get security updates after December 31, 2018, while PHP 7.0 will cease to get security updates after December 3, 2018, so it is imperative that webmasters move their sites to PHP 7.1 or 7.2 in November, 2018. Similarly, the security fixes in Joomla 3.8.13, make it wise to get to at least that software level as of this writing and move to 3.9 and subsequently 3.10 as soon as possible. All of these upgrades have higher than normal testing requirements; this article aims to help with preparation for these migrations.

Joomla 3.8 included a number of changes to provide support for Joomla 7.x and is required in order to upgrade to PHP 7.1 or PHP 7.2.

Moving to Joomla 3.8

Because of the changes to support PHP 7.x, Joomla 3.8 ended up requiring changes to many templates, template frameworks and extensions. Moving early, as I did, required a lot of work, but today most vendors have updated everything so there is not much work required. Here are the steps for upgrading to Joomla 3.8:

• Backup Joomla
• Update your template framework. If there are no updates after September 2017, you will probably be forced to change templates and template frameworks.
• Update your template. If there are not template updates after September 2017, you will probably be forced to change templates.
• Update your extensions.
• Update Joomla.
• Test.

You will probably find that some things are broken. If so, you may need to replace functionality with a different extension, as most extensions that have support have already fixed any problems related to the upgrade to Joomla 3.8 and the related upgrade to PHP 7.x.

Moving to a New PHP Level

Most web hosting firms use CPanel to provide a graphical user interface (GUI) for system administrator. CPanel introduced MultiPHP support that allows each shared host to run a different PHP level and configuration which makes it possible to move to more recent software in an orderly way. If you have a Virtual Private Server (VPS), you have a lot of control over how PHP packages are provisioned, but if you are on a typical shared host you do not. Doing some research on your current configuration before starting. With this in minde, here are the suggested steps to migrate to a higher PHP level:

• Verify that you are on at least Joomla 3.8.
• First do research to identify the PHP modules that you need.
• Look at the PHP requirements for Joomla, and make a list of the PHP packages are listed in Technical Requirements. “Zlib support” means that the Zlib PHP module mod_zlib must be installed. Make a list of all of the modules that Joomla requires. Do not forget mod_ssl for HTTPS support.
• Look at the PHP requirements for the Joomla extensions that you use. Some may require PHP Exif support for image metadata, other Joomla extensions my have other unique requirements.
• If you have a VPS, list the modules currently selected.
• Provision the new PHP release in the EasyApache section of CPanel
• If you have a VPS, use the list of PHP modules that you need to provision the new level of PHP.
• If you do not have a VPS, you will need to call your hosting firm’s technical support to have them confirm that the various modules have been included.
• In Cpanel, set the php.ini file values for the new PHP level to match the values in your current configuration.
• In CPanel, use MultiPHP to change PHP levels.
• Test. If you have problems, turn on debugging in the Joomla configuration to see some of the error messages. It may take two or three tries to get all of the PHP modules that you need.

Moving to Joomla 3.9, 3.10 and 4.0

Moving to Joomla 3.9 required another round of template and template framework updates for all sites along with updates to a number of extensions. Otherwise, this was an uneventful upgrade.

Moving to Joomla 3.10 is planned to be an uneventful upgrade as it is intended to be a transitional release to provide security fix support for two years while site owners and developers fix and test extensions on 4.0. Although 4.0 has some new function, most of the changes are updates to outdated and unsupported dependency libraries; it is likely that a lot of extensions will need maintenance to work with new levels of the various libraries. See the article Potential Backward Compatibility Issues in Joomla 4. In any case, do not move to 4.0 without testing extensively first.

Details

Written by Bruce Moore

Published: 18 November 2018

Created: 15 November 2018

Last Updated: 15 November 2018

Hits: 2168

• Web Hosting

Upgrading a Raspberry Pi from Wheezy to Jessie

I recently realized that one of my Raspberry Pi’s running Network Caller ID (NCID) was running an unsupported version of Debian/Raspian and needed to be upgraded. The first instructions that Google listed omitted some steps that I knew should be included, so I have put together this page with the procedure that I used. The full (and best) document is the official guide on Debian.org.

Backup the Working System

Before doing any upgrade, you should do a backup of the system. In the case of a Raspberry Pi, it is easy to clone the SD, pull the original and upgrade the clone. Since this was a very minimal installation, I chose to backup only the /etc/ncid/ directory to the Subversion server that I use to keep modified configuration files. An ancient saying among system programs was “tape is cheap (no matter how much it costs).” You never need a backup until you do, and when you do, you really need it badly.

The upgrade process will remove packages that do not show up as having dependencies, and some packages may have unresolved dependencies. You may get into a situation where you have not recourse but to fall back and do a painstaking package-by-package upgrade.

Make Sure You Have Enough Free Space

Some upgrade programs are smart enough to check for the necessary free space to complete the upgrade. The Debian approach described here is not, and I know from experience that it will get ugly if you do not have enough free space. Make sure that you have plenty of free space before start; use the command

df -h

to find out how much free space you have available. If you are above 80% or so, you will probably have problems. Keep two windows open and at each step in the process run a df -h command to make sure that you have more space than apt-get updateindicates that it will use.

user@thisrpi /etc/apt/sources.list.d $ df -h Filesystem Size Used Avail Use% Mounted on /dev/root 13G 3.8G 8.4G 31% / devtmpfs 214M 0 214M 0% /dev tmpfs 44M 240K 44M 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 87M 0 87M 0% /run/shm /dev/mmcblk0p5 60M 20M 40M 33% /boot

Make Sure You Have Time

The upgrade process may take a few hours, depending upon which Raspberry Pi model you have, the speed of your connection, and the complexity of your installation. Make sure that you have time to do this, and something else to work on while watching the console messages scroll past.

Upgrade the Current Release

The first step is to upgrade to current maintenance on the old release:

sudo apt-get update sudo apt-get upgrade

The next step will remove some unused dependencies. This sometimes breaks things if you have optional software installed outside the standard installer, and sometimes breaks dependencies even for software that was installed with the installer. Make sure you have a backup. This step is not absolutely necessary, but can reduce the upgrade time in later steps.

sudo apt-get dist-upgrade sudo apt-get autoremove sudo apt-get clean

Upgrade the /etc/apt Configuration Files

Next, upgrade the /etc/apt/sources.list file to change wheezy to jessie, then do the same for all files in the /etc/apt/sources.list.d directory.

Upgrade the System

This is the point of no return; make sure that you have a backup and sufficient free space. To upgrade the basic system, use the commands

sudo apt-get update

Some items may not update; raspberrypi.<collabora.com has been deprecated, so you will need comment it out in /etc/apt/sources.list.d/collabora.list. When you have resolved all repository issues so that you do not get errors, proceed with

sudo apt-get upgrade

When this is complete run the commands

sudo apt-get update sudo apt-get upgrade sudo apt-get dist-upgrade sudo apt-get autoremove

Reboot and Test

At this point, you should be ready to reboot the system. Test all of your applications to make sure that they are working properly–especially those installed outside the installer.

Details

Written by Bruce Moore

Published: 18 September 2018

Created: 18 September 2018

Last Updated: 08 November 2018

Hits: 2452

• Open Source
• Raspberry Pi
• Linux

New Privacy (GDPR) Features in Joomla 3.9

The European Union’s (EU) General Data Protection Regulation (GDPR) went into effect in May, 2018 and has resulted in major changes in privacy administration within the Information Technology world in general and web business operations in particular. Joomla release 3.9 (November, 2019) indroduced a number of changes that immediately provide tools to help with compliance and APIs so that extension developers will be able to easily provide compliance tools.

What is GDPR?

Before talking about the new privacy features in Joomla 3.9, it makes sense to talk give an overview of GDPR. First, neither this article nor Moore Software Services provide legal advice on whether you are subject to GDPR, whether or not your site is compliant, or any other legal advice. This article describes new privacy features in Joomla; whether these are sufficient for your compliance needs is beyond the scope of this article. GDPR became effective in May, 2018 and resulted in a large number of emails where businesses asked customers to confirm consent for data tracking related to email newsletters and other customer relationships. There are many good web articles on the history of GDPR so there is not point in repeating the history here. Suffice it to say that the regulation is overdue and that most businesses have struggled to comply with the basic requirements of the regulation.

Requirements of GDPR

The regulation is complex and has many pages, but is based upon a few simple principles:

• Consent
• Reporting to user of what data a company has pertaining to the user
• Must remove data about a user upon user’s request
• Must report data breaches to users with 72 hours of discovery
• Must maintain records of processing of user data.

These represent significant changes from the way most web businesses have traditionally operated and will require significant work for many firms.

The Penalties for Failure to Comply are Draconian

Failure to comply is fundamentally a bankruptcy issues for most companies and is the greater of

• €20 million
• 4 percent of annual global revenue.

Ignoring GDPR is not an option.

Do US-based Companies Need to Comply with GDPR?

The short answer is yes for most companies and web businesses. If any of your customers are EU citizens and use the site at home or while in the US, you probably need to comply. If any of your customers are non-EU citizens, but use your site while in the EU, you must comply. Consult an attorney. Whether it will be enforced heavily for smaller businesses is an open question, but given that the California Consumer Privacy Act of 2018, the Chicago Personal Data Collection and Protection Ordinance and other US jurisdictions have implemented similar legislation, it is probably a good idea to work toward GDPR compliance; GDPR appears to be the most strict, so complying with it may make it easier for you to comply with the hodgepodge of regulations developing outside the EU.

Joomla 3.9 Helps with GDPR Compliance

The new privacy features and APIs in Joomla help with tracking consent, responding to user requests for information, and maintaining processing records. The sections that follow take a user-interface approach to the new features rather than a functional approach. There are five major user interface additions for the new GDPR privacy functions:

• Privacy Dashboard (under the Users menu)
• Privacy User Action Log (under the Users menu)
• Privacy menu item types (under the Menu menu)
• Privacy plugins (under the Extensions->Plugins menu)
• Privacy Global Configuration options (under the System->Global Configuration menu)

The GDPR Privacy Dashboard is Under the User Menu

The User menu adds new Privacy and User Action Log options as shown in Figure 1. Going to the Privacy menu option shows the dashboard (see Figure 2) where you can get an overview of the information requests an other compliance status items for your site. The Requests option in the dashboard (see Figure 3) shows the number and status of user’s requests for a report on the data pertaining to the user plus a work flow for processing requests.

The most powerful addition in Joomla 3.9 is the addition of API features for extensions to integrate with the core privacy functions. Figure 4 shows the privacy-enabled extensions reporting back what privacy features they have implemented.

The last option in the privacy dashboard is the report on the status of user consents (see Figure 5).

The GDPR User Action Log is Under the User Menu

GDPR requires that you keep a log of how user information is processed. The User Action Log (see Figure 6) under the User menu provides this capability. It will probably be very helpful for problem diagnosis in addition to compliance.

New GDPR End-User Forms are New Menu Item Types

To implement the user interface for the new privacy capabilities, a new menu item category, Privacy (see Figure 7) has been introduced along with three new menu items types (see Figure 8).

GDPR Plugins for Logging

To make the new privacy functions work, Joomla 3.9 adds several new plugins (see Figure 9), one of which requires some configuration. The consent plugin shown in Figure 10 requires administrators to enter a short version of the privacy policy along with a link to an article containing the long version of the privacy policy. You can also set up the user consent to expire, requiring a new consent. This is helpful whenever the privacy policy changes; you set the expiration and then all users will be forced to update their consent.

GDPR Privacy Options in Global Configuration

The final user interface change for the Joomla 3.9 privacy enhancements is a Global Configuration category Privacy that now contains one item for the number of days before a user request for data is escalated to URGENT status.

Conclusions

The privacy extensions in Joomla 3.9 do not provide everything you need for compliance with GDPR and other privacy regulations, but they do provide a way for extensions developers to add capabilities and make it easier for webmasters. Over the next two or three years, extension developers that do not implement privacy features will have a much more difficult time selling their extensions.

Details

Written by Bruce Moore

Published: 15 November 2018

Created: 15 November 2018

Last Updated: 15 November 2018

Hits: 2075

• Web Hosting
• Compliance

Configure Network Caller ID (NCID) to work with an OBi202

If you have an OBi202 (or other ObiHai/Polycom device), you do not need a modem to get Network Caller ID to work with your phone system, at least for reporting purposes, though for automatic hangup, you will still need to attach a modem. In most circumstances, this is not of much use, but if you want to measure the effectiveness of a service like NoMoRoBo, you will need to do this. Because NoMoRoBo hangs up before NCID registers the call via the modem, you will need to capture the call via the Polycom (previously ObiHai) device if you want to identify calls that NoMoRoBo kills before NCID recognizes the call and hangs up.

The obi2ncid gateway provided with NCID can read and parse the syslog from the Polycom device and report the calls that are killed before NCID can recognize the call.

The discussion that follows assumes that you have already installed NCID.

Set up obi2ncid.conf

To set this up, configure the /etc/ncid/obi2ncid.conf file. In most cases you can keep the defaults but in almost all cases you will want to update the identifiers for each line on the ObiHai device to match the extension or number that you are using:

# Defaults: SP1, SP2, SP3, SP4 linesp1 = "Home" linesp2 = "Work" linesp3 = "Fax1" linesp4 = "Fax2"

Configure ObiHai Device for Syslog

Next, you will need to log in to the Polycom device to and change the destination IP address and port for the syslog to match the IP address of your NCID server and the port that the obi2ncid program listens:

Start the obi2ncid Service

Finally, you will need to start the obi2ncid service:

sudo service obi2ncid start

Interpreting the Results

Once you have this set up, you will get additional call information in the NCID client displays if your provider has been blocking calls before NCID registers them:

• For calls that NoMoRoBo hangs up, you will get one entry from the ObiTalk device.
• For calls that NoMoRoBo does not block, you will get one entry from ObiTalk and one from the NCID modem (ncidd).

Details

Written by Bruce Moore

Published: 10 September 2018

Created: 10 September 2018

Last Updated: 10 September 2018

Hits: 2797

• Telephony
• Raspberry Pi
• Productivity