Drupal 8 environment

Project Description

RISKS (2020/02/24):

1) The SOLR search module is still in continuous development and we could not manage to get it to search over multiple site indexes at once yet.  This is necessary before we can continue to start migrating sites. We spend a lot of time currently on getting this to work.

2) There is a strong possibility that I might relocate sometime in 2020/2021 (due to my husband’s work) but Ronel from Corporate Communications has been very involved throughout the whole process of choosing modules, creating individualized modules and testing it. The infrastructure team is also very involved in their part of the setup.

The request was submitted by Géna Thompson on 24 April 2018:

We want to set up a new Drupal 8 web environment where Drupal 8 subsites can be managed. To manage the subsites we want to use a combination of Jenkins and Github instead of AEGIR. We would also like to start off setting up only one subsite on these tools, installing, configuring and theming that Drupal 8 subsite completely so that the functionalities that are currently available in Drupal 7 would be available and so that we can use that particular distribution (as a template/copy) for our other subsites.

Updated: 27/09/2018

The main reason for the upgrade from Drupal 7 to Drupal 8 is security. Since this project started the architecture and our needs have evolved. We’re using a combination of Ansible plays and git (nwu bitbucket) to build the environment. The Environment include AEGIR (to manage Drupal 8 subsites), Drupal 8, Percona, Solr, Memcache, Varnish, and NFS file structure (w/ symlinks) Cloudflare.

Architectural drawing of infrastructure, Architectural drawing of infrastructure dependencies

Project Progress

27%

Overall Project Completion

  • 20%
  • 60%

27%

  • 40%
  • 80%

1. Initial Drupal 8 viability tests Complete 100% Tasks 4 / 4

100%
Description

Install, configure and investigate Drupal 8 and modules to determine if it can replace current Drupal 7 functionalities

Document front-end installation and configuration of Drupal 8

Document front-end installation and configuration modules.

Test Bootstrap theme (Document)

4 of 4 completedTasks

  • Install Drupal 8 on external server (Codaptive server)
  • Install modules on external server (Codaptive server)
  • Document installation and Configuration
  • Test installation

2. Initial tests- Backend Drupal 8 Backend Complete 90% Tasks 4 / 5

90%
Description

Intstall Drupal 8 and modules on a testing environment (LAMP) + Composer + Drush + Drupal Console

4 of 5 completedTasks

  • Install LAMP (Ubuntu Linux, Apache, Mysql and PHP 7.0)
  • Install Composer, Drush & Drupal Console
  • Install Drupal 8
  • Install initial modules using drush
  • Do configuration some Drupal installation with drush

3. Do a test migration of a sub site (news) Complete 97% Tasks 27 / 29

97%
Description

We decide on a custom migration and start experimenting/creating json feeds as the source on news.nwu.ac.za and migration modules to import the content.

27 of 29 completedTasks

  • Determine what is our options regarding our Drupal 8 migratrion - research
  • Do a small test to "migrate" all news content nodes (just titles and bodies) to test site from a json url on news.nwu.ac.za
  • Investigate drupal 'custom migrate modules'
  • Json feed to import users (still not able to bring over roles) - Drupal 7
  • Json feed to import vocabularies - Drupal 7
  • Json feed to import Taxonomies - Drupal 7
  • Json feed on news (to be used for news content migration module) - English - general, potch, mafikeng, vaal (4)
  • Json feed on news (to be used for news content migration module) - Afrikaans - general, potch, vaal (3)
  • Json feed to import Images - News - Drupal 7
  • Json feed - NWU in other Media Drupal 7
  • Json feed to import Experts - Drupal 7
  • Json feed - for import Expert Images - Drupal 7
  • Create Content Type modules (News, NWU in other News & Experts) (3)
  • Create Migration Module for Users (1)
  • Create Migration Modules for Vocabularies and Taxonomy (En) (2)
  • Create migration module for Taxonomy (Afr from csv - translated by CTEXT)
  • Create Migration Modules for Expert (Images, Files & Expert) (3)
  • Create Migration Module for News English - general, potch, mafikeng, vaal | Afrikaans - general, potch, vaal (7)
  • Create Migration Modules for News (Images) (1)
  • Create Migration Modules for NWU in other Media (Images) (1)
  • Create Migration Module for Redirects (1)
  • Mysql queries to change links in body and take H1's out
  • Create a Main Menu Module (and block) that would be used over all the websites
  • Create a Secondary Menu Module (and block) that would be used over all the websites
  • Do Config for LDAP
  • Do Config for CKEditor, IMCE images
  • Create Migrate Module for all the News Views + View Expert Name Module
  • Create and update Migration process documentation
  • Solr Search

4. Determine infrastructure & backend configuration for Drupal 8 multisites Complete 95% Tasks 7 / 8

95%
Description

(3) – Summary of meeting

7 of 8 completedTasks

  • Meet with Eben & Pieter (Architects) - Discuss ideas
  • Meet with Wilhelm & Riaan Marx - Discuss ideas
  • Joint meeting between Architect & Infrastructure (Jacques Oosthuizen, Riaan Stavast, Wilhelm, Eben van Zyl, Riaan Marx, Gena)
  • Meeting with Codaptive consultant regarding infrastructure & backend configuration for Drupal 8 (Jacques Oosthuizen, Riaan Stavast, Eben van Zyl, Gena)
  • NWU architects supplies us with a diagram of the infrastructure layout (Eben van Zyl)
  • Meeting with infrastructure to discuss infrastructure layout and how to proceed (Jacques Oosthuizen, Riaan Stavast, Hannes Kriel, Riaan Marx, Gena)
  • Meeting with assigned infrastructure project members to establish work process (Jacques Oosthuizen, Riaan Stavast, Gena)
  • Infrastructure will be finalised during building process

5. Drupal Standup Meetings (Weekly) (Jacques, Riaan Stavast & Gena & Ronel) Complete 28% Tasks 0 / 2

28%
Description

0 of 2 completedTasks

  • Weekly Meetings 2018/2019

6. Get the new Drupal platform/infrastructure in place Complete 50% Tasks 10 / 28

50%

10 of 28 completedTasks

  • Integration diagram of infrastructure layout of Drupal 8 - Jacques Oosthuizen (Riaan Stavast, Gena Thompson, Ronel Rosher)
  • IPs for Play-Xvshere - Riaan Stavast
  • Resources for each vm on diagram
  • Create account on Ansible-DEV for Gena & Ronel - Jacques & Riaan
  • PHP version plus repo
  • Dev - Build starter play - Aegir environment
  • Dev - Build starter play - Drupal environment
  • Dev - Build starter play - Percona environment
  • Dev - Build starter play - Memcached environment
  • Dev - Build starter play - NFS environment
  • Dev - Build starter play - Varnish environment
  • Dev - Build starter play - Solr environment
  • Test - Build starter play - Aegir environment
  • Test - Build starter play - Drupal environment
  • Test - Build starter play - Percona environment
  • Test - Build starter play - Memcached environment
  • Test - Build starter play - NFS environment
  • Test - Build starter play - Varnish environment
  • Test - Build starter play - Solr environment
  • Test - Build starter play - GIT
  • Production - Build starter play - Aegir environment
  • Production - Build starter play - Drupal environment
  • Production - Build starter play - Percona environment
  • Production - Build starter play - Memcached environment
  • Production - Build starter play - NFS environment
  • Production - Build starter play - Varnish environment
  • Production - Build starter play - Solr environment
  • Production - Build starter play - GIT

7. First Subsite (news.nwu.ac.za) Complete 67% Tasks 8 / 12

67%
Description

8 of 12 completedTasks

  • Development environment - Drupal/AEGIR - Install & Configure
  • Modules & Configuration
  • Test News, Experts and NWU in other Media Content Type Modules (3)
  • Test Expert Migration Modules (Images, Files & Experts) (3)
  • Test Bootstrap NWU theme for Drupal 8
  • Test News Migration Modules (7 +1)
  • Test migration
  • Let users test the new environment (Frontend - Info correct?) (Backend - Can publishers do what they need to do?)
  • Search - Still needs work
  • Testing - Drupal/AEGIR - Install & Configure
  • Production - Drupal/AEGIR - Install & Configure

8. library.nwu.ac.za Complete 69% Tasks 8 / 13

69%
Description

8 of 13 completedTasks

  • Development environment - Drupal/AEGIR - Install & Configure
  • Test Modules & Configuration
  • Test NWU Page Content Type
  • Test Migration Modules (Users, Vocabularies, Taxonomies, Pages)
  • Test Bootstrap NWU theme for Drupal 8
  • Test Webform Custom Module
  • Test Bootstrap Theme
  • Test Migration
  • Ask backend users to test
  • Search - Still needs work
  • Testing/Search environment - Drupal/AEGIR - Install & Configure
  • Production environment - Drupal/AEGIR - Install & Configure

9. commerce.nwu.ac.za Complete 36% Tasks 0 / 14

36%
Description

0 of 14 completedTasks

  • Development environment - Drupal/AEGIR - Install & Configure
  • Test Modules & Configuration
  • Test NWU Page, NWU Image, NWU Staff Content Types
  • Test Migration Modules (Users, Vocabularies, Taxonomies, Pages)
  • Test Bootstrap NWU theme for Drupal 8
  • Test Webform Custom Module
  • Test Migration
  • Test Menu Migration
  • Add menu Blocks - how - php conditions not working - words & wildcard combination
  • Ask backend users to test
  • Search setup for multisite
  • Get a solution for Accordions to work
  • Testing/Search environment - Drupal/AEGIR - Install & Configure
  • Production environment - Drupal/AEGIR - Install & Configure

10. distance.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

11. services.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

12. intranet.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

13. sport.nwu.ac.za Complete 0% Tasks 0 / 1

0%
Description

14. studies.nwu.ac.za Complete 0% Tasks 0 / 1

0%
Description

15. diyservices + diyservices2 Complete 0% Tasks 0 / 0

0%
Description

16. education.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

17. engineering.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

18. health-sciences.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

19. humanities.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

20. law.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

21. natural-science.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

22. theology.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

23. www.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

24. learn.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description