Drupal 8 environment

Project Description

RISKS (2020/02/24):

1) The SOLR search module is still in continuous development and we could not manage to get it to search over multiple site indexes at once yet.  This is necessary before we can continue to start migrating sites. We spend a lot of time currently on getting this to work.

2) There is a strong possibility that I might relocate sometime in 2020/2021 (due to my husband’s work) but Ronel from Corporate Communications has been very involved throughout the whole process of choosing modules, creating individualized modules and testing it. The infrastructure team is also very involved in their part of the setup.

The request was submitted by Géna Thompson on 24 April 2018:

We want to set up a new Drupal 8 web environment where Drupal 8 subsites can be managed. To manage the subsites we want to use a combination of Jenkins and Github instead of AEGIR. We would also like to start off setting up only one subsite on these tools, installing, configuring and theming that Drupal 8 subsite completely so that the functionalities that are currently available in Drupal 7 would be available and so that we can use that particular distribution (as a template/copy) for our other subsites.

Updated: 27/09/2018

The main reason for the upgrade from Drupal 7 to Drupal 8 is security. Since this project started the architecture and our needs have evolved. We’re using a combination of Ansible plays and git (nwu bitbucket) to build the environment. The Environment include AEGIR (to manage Drupal 8 subsites), Drupal 8, Percona, Solr, Memcache, Varnish, and NFS file structure (w/ symlinks) Cloudflare.

Architectural drawing of infrastructure, Architectural drawing of infrastructure dependencies

Project Progress

47%

Project Timing

  • Start
    Sep 27 2018
  • End
    Jul 30 2021

09/27/2018 07/30/2021

70%

Overall Project Completion

  • 20%
  • 60%

47%

  • 40%
  • 80%

1. Initial Drupal 8 viability tests Complete 100% Tasks 4 / 4

100%
Description

Install, configure and investigate Drupal 8 and modules to determine if it can replace current Drupal 7 functionalities

Document front-end installation and configuration of Drupal 8

Document front-end installation and configuration modules.

Test Bootstrap theme (Document)

4 of 4 completedTasks

  • Install Drupal 8 on external server (Codaptive server)
  • Install modules on external server (Codaptive server)
  • Document installation and Configuration
  • Test installation

2. Initial tests- Backend Drupal 8 Backend Complete 100% Tasks 5 / 5

100%
Description

Intstall Drupal 8 and modules on a testing environment (LAMP) + Composer + Drush + Drupal Console

5 of 5 completedTasks

  • Install LAMP (Ubuntu Linux, Apache, Mysql and PHP 7.0)
  • Install Composer, Drush & Drupal Console
  • Install Drupal 8
  • Install initial modules using drush
  • Do configuration some Drupal installation with drush

3. Do a test migration of a sub site (news) Complete 100% Tasks 27 / 29

100%
Description

We decide on a custom migration and start experimenting/creating json feeds as the source on news.nwu.ac.za and migration modules to import the content.

27 of 29 completedTasks

  • Determine what is our options regarding our Drupal 8 migratrion - research
  • Do a small test to "migrate" all news content nodes (just titles and bodies) to test site from a json url on news.nwu.ac.za
  • Investigate drupal 'custom migrate modules'
  • Json feed to import users (still not able to bring over roles) - Drupal 7
  • Json feed to import vocabularies - Drupal 7
  • Json feed to import Taxonomies - Drupal 7
  • Json feed on news (to be used for news content migration module) - English - general, potch, mafikeng, vaal (4)
  • Json feed on news (to be used for news content migration module) - Afrikaans - general, potch, vaal (3)
  • Json feed to import Images - News - Drupal 7
  • Json feed - NWU in other Media Drupal 7
  • Json feed to import Experts - Drupal 7
  • Json feed - for import Expert Images - Drupal 7
  • Create Content Type modules (News, NWU in other News & Experts) (3)
  • Create Migration Module for Users (1)
  • Create Migration Modules for Vocabularies and Taxonomy (En) (2)
  • Create migration module for Taxonomy (Afr from csv - translated by CTEXT)
  • Create Migration Modules for Expert (Images, Files & Expert) (3)
  • Create Migration Module for News English - general, potch, mafikeng, vaal | Afrikaans - general, potch, vaal (7)
  • Create Migration Modules for News (Images) (1)
  • Create Migration Modules for NWU in other Media (Images) (1)
  • Create Migration Module for Redirects (1)
  • Mysql queries to change links in body and take H1's out
  • Create a Main Menu Module (and block) that would be used over all the websites
  • Create a Secondary Menu Module (and block) that would be used over all the websites
  • Do Config for LDAP
  • Do Config for CKEditor, IMCE images
  • Create Migrate Module for all the News Views + View Expert Name Module
  • Create and update Migration process documentation
  • Solr Search

4. Determine infrastructure & backend configuration for Drupal 8 multisites Complete 100% Tasks 7 / 8

100%
Description

(3) – Summary of meeting

7 of 8 completedTasks

  • Meet with Eben & Pieter (Architects) - Discuss ideas
  • Meet with Wilhelm & Riaan Marx - Discuss ideas
  • Joint meeting between Architect & Infrastructure (Jacques Oosthuizen, Riaan Stavast, Wilhelm, Eben van Zyl, Riaan Marx, Gena)
  • Meeting with Codaptive consultant regarding infrastructure & backend configuration for Drupal 8 (Jacques Oosthuizen, Riaan Stavast, Eben van Zyl, Gena)
  • NWU architects supplies us with a diagram of the infrastructure layout (Eben van Zyl)
  • Meeting with infrastructure to discuss infrastructure layout and how to proceed (Jacques Oosthuizen, Riaan Stavast, Hannes Kriel, Riaan Marx, Gena)
  • Meeting with assigned infrastructure project members to establish work process (Jacques Oosthuizen, Riaan Stavast, Gena)
  • Infrastructure will be finalised during building process

5. Drupal Standup Meetings (Weekly) (Jacques, Riaan Stavast & Gena & Ronel) Complete 90% Tasks 0 / 1

90%
Description

0 of 1 completedTasks

  • Weekly Meetings 2018/2019

6. Get the new Drupal platform/infrastructure in place Complete 93% Tasks 10 / 27

93%

10 of 27 completedTasks

  • Integration diagram of infrastructure layout of Drupal 8 - Jacques Oosthuizen (Riaan Stavast, Gena Thompson, Ronel Rosher)
  • IPs for Play-Xvshere - Riaan Stavast
  • Resources for each vm on diagram
  • Create account on Ansible-DEV for Gena & Ronel - Jacques & Riaan
  • PHP version plus repo
  • Dev - Build starter play - Aegir environment
  • Dev - Build starter play - Drupal environment
  • Dev - Build starter play - Percona environment
  • Dev - Build starter play - Memcached environment
  • Dev - Build starter play - NFS environment
  • Dev - Build starter play - Varnish environment
  • Dev - Build starter play - Solr environment
  • Test - Build starter play - Aegir environment
  • Test - Build starter play - Drupal environment
  • Test - Build starter play - Percona environment
  • Test - Build starter play - Memcached environment
  • Test - Build starter play - NFS environment
  • Test - Build starter play - Varnish environment
  • Test - Build starter play - Solr environment
  • Test - Build starter play - GIT
  • Production - Build starter play - Aegir (master + 2 slaves) environment
  • Production - Build starter play - DB environment
  • Production - Build starter play - Memcached environment
  • Production - Build starter play - Haproxy + Varnish environment
  • Production - Build starter play - Solr environment
  • Production - Build starter play - GIT
  • Load balancer on top of Drupal 7 & 8 environment (for Zero-rating to work)

7. First Subsite (news.nwu.ac.za) Complete 79% Tasks 11 / 14

79%
Description

11 of 14 completedTasks

  • Development environment - Drupal/AEGIR - Install & Configure
  • Modules & Configuration
  • Test News, Experts and NWU in other Media Content Type Modules (3)
  • Test Expert Migration Modules (Images, Files & Experts) (3)
  • Test Bootstrap NWU theme for Drupal 8
  • Test News Migration Modules (7 +1)
  • Test migration
  • Let users test the new environment (Frontend - Info correct?) (Backend - Can publishers do what they need to do?)
  • Search - Published procedure on international drupal site as requested (multisite)
  • Testing - Drupal/AEGIR - Install & Configure
  • Production - Drupal/AEGIR - Install & Configure
  • Production - rollback and re-import news and experts / Production - rsync files
  • Production - ensure all export news views work as they should
  • Setup new Google analytics after site has been made live

8. library.nwu.ac.za Complete 86% Tasks 12 / 14

86%
Description

12 of 14 completedTasks

  • Development environment - Drupal/AEGIR - Install & Configure
  • Test Modules & Configuration
  • Test NWU Page Content Type
  • Test Migration Modules (Users, Vocabularies, Taxonomies, Pages)
  • Test Bootstrap NWU theme for Drupal 8
  • Test Webform Custom Module
  • Test Bootstrap Theme
  • Test Migration
  • Ask backend users to test
  • Search
  • Testing/Search environment - Drupal/AEGIR - Install & Configure
  • Production environment - Drupal/AEGIR - Install & Configure
  • Production - talk to Elize van Eldik - any new content?
  • Setup new Google analytics after site has been made live

9. commerce.nwu.ac.za Complete 88% Tasks 14 / 16

88%
Description

14 of 16 completedTasks

  • Development environment - Drupal/AEGIR - Install & Configure
  • Test Modules & Configuration
  • Test NWU Page, NWU Image, NWU Staff Content Types
  • Test Migration Modules (Users, Vocabularies, Taxonomies, Pages)
  • Test Bootstrap NWU theme for Drupal 8
  • Test Webform Custom Module
  • Test Migration
  • Test Menu Migration
  • Add menu Blocks - how - php conditions not working - words & wildcard combination
  • Ask backend users to test
  • Search setup for multisite
  • Get a solution for Accordions to work
  • Testing/Search environment - Drupal/AEGIR - Install & Configure
  • Production environment - Drupal/AEGIR - Install & Configure
  • Production - rollback and re-import pages, staff / Production - rsync files
  • Setup new Google analytics after site has been made live

10. education.nwu.ac.za Complete 76% Tasks 6 / 9

76%
Description

6 of 9 completedTasks

  • Clone base site & verify.
  • Configure Solr Search and Search view
  • Content type modules
  • Migration modules
  • Views modules
  • Menu modules
  • Configure blocks
  • Production - rollback and re-import pages, staff /Production - rsync files
  • Setup new Google analytics after site has been made live

11. engineering.nwu.ac.za Complete 76% Tasks 6 / 9

76%
Description

6 of 9 completedTasks

  • Clone base site & verify.
  • Configure Solr Search and Search view
  • Content type modules
  • Migration modules
  • Views modules
  • Menu modules
  • Configure blocks
  • Production - rollback and re-import pages, staff / Production - rsync files
  • Setup new Google analytics after site has been made live

12. law.nwu.ac.za Complete 68% Tasks 4 / 9

68%
Description

4 of 9 completedTasks

  • Clone base site & verify.
  • Configure Solr Search and Search view
  • Content type modules
  • Migration modules
  • Views modules
  • Menu modules
  • Configure blocks
  • Production - rollback and re-import pages, staff / Production - rsync files
  • Setup new Google analytics after site has been made live

13. theology.nwu.ac.za Complete 34% Tasks 3 / 9

34%
Description

3 of 9 completedTasks

  • Clone base site & verify.
  • Configure Solr Search and Search view
  • Content type modules
  • Migration modules
  • Views modules
  • Menu modules
  • Configure blocks
  • Production - rollback and re-import pages, staff / Production - rsync files
  • Setup new Google analytics after site has been made live

14. diyservices + diyservices2 Complete 23% Tasks 2 / 9

23%
Description

2 of 9 completedTasks

  • Clone base site & verify.
  • Configure Solr Search and Search view
  • Content type modules
  • Migration modules
  • Views modules
  • Configure blocks
  • Production - rollback and re-import pages, staff / Production - rsync files
  • Setup new Google analytics after site has been made live
  • Clone diy to diy2

15. health-sciences.nwu.ac.za Complete 12% Tasks 1 / 9

12%
Description

1 of 9 completedTasks

  • Clone base site & verify.
  • Configure Solr Search and Search view
  • Content type modules
  • Migration modules
  • Views modules
  • Menu blocks
  • Configure blocks
  • Production - rollback and re-import pages, staff / Production - rsync files
  • Setup new Google analytics after site has been made live

16. humanities.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

17. natural-science.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

18. studies.nwu.ac.za Complete 0% Tasks 0 / 1

0%
Description

19. sport.nwu.ac.za Complete 0% Tasks 0 / 1

0%
Description

20. distance.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

21. services.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

22. intranet.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

23. www.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description

24. learn.nwu.ac.za Complete 0% Tasks 0 / 0

0%
Description