Open Stack

Project Description

Collaboration between NWU and UCT to do an Openstack deployment known as African Research Cloud (ARC)

Documents

No documents at this time.

Project Progress

100%

Project Timing

  • Start
    Nov 23 2015
  • End
    Jun 30 2017

11/23/2015 06/30/2017

100%

Overall Project Completion

  • 20%
  • 60%

100%

  • 40%
  • 80%

1. Discovery and Onboarding Complete 100% Tasks 5 / 5

100%
Description

Getting familiar with OpenStack and exploring various ways to deploy and use OpenStack

5 of 5 completedTasks

  • Wipe UCT private cloud
  • Create usable config files to deploy hardware to OpenStack from both UCT and NWU (Completed by: 11/12/2015)
  • Get sandbox environment to test config files for use by NWU
  • Deploy mini Openstack @ NWU (Completed by: 11/12/2015)
  • Add Swift storage to NWU OpenStack (Completed by: 18/12/2015)

2. NWU Swift Deployment to ARC Complete 100% Tasks 7 / 7

100%
Description

7 of 7 completedTasks

  • Formulate a strategy for deployment of Swift to ARC
  • Obtain VLAN and configure networking
  • Testing of VLAN and network components
  • Swift configuration to integrate into ARC
  • Testing of Swift within ARC
  • Enable S3 interface on Swift
  • Architecture revisit due to performance issues on Swift/S3

3. Hardware procurement and commisioning Complete 100% Tasks 7 / 7

100%
Description

7 of 7 completedTasks

  • Tender and procurement of SuperMicro hardware
  • Tender and procurement of server enclosures
  • Site survey of MK1, MK2 and MK-VDK to determine location of cluster nodes
  • Preperation of datacenteres for hardware (Completed by: ??/??/201?)
  • Installation and configuration of networking
  • Commisioning of SuperMicro servers (Completed by: ??/??/201?)
  • Obtain additional hardware to implement a full installation of OpenStack

4. Documentation Complete 100% Tasks 4 / 4

100%
Description

4 of 4 completedTasks

  • Manual / Wiki for installation of MaaS environment
  • Manual / Wiki for OpenStack deployment
  • NWU Swift diagrams
  • ARC diagrams

5. NWU-Genomics Project on ARC Complete 100% Tasks 8 / 8

100%
Description

To demonstrate and test the potential of the ARC two projects have been identified to run in a controlled “POC” environment on the ARC.

One is SKA related, not relevant to our resources.

The other is a Genomics project based on a Galaxy software platform. This is the NWU resource responsibility.

8 of 8 completedTasks

  • Decide on type of Galaxy implementation
  • Build and configure required Galaxy instances (admin node, scheduler, shared strage, compute)
  • Galaxy implementation and configuration
  • Configure bulk data movement
  • Install basic Galaxy tools (FastQC)
  • Run basic tasks/jobs to test Galaxy install
  • Install all tools required for NWU-Genomics pipeline
  • Execute a complete pipeline job

6. Galaxy on CloudMan Complete 100% Tasks 1 / 1

100%
Description

1 of 1 completedTasks

  • Plan of action required to move forward with involvement of UWC/Sanbi (Peter & Kie)

7. ARC Support and Upgrade Complete 100% Tasks 1 / 1

100%
Description

1 of 1 completedTasks

  • Support on current environment and running projects

Project Discussion 4 Responses to Open Stack

  1. OpenStack deployment of infrastructure at NWU into ARC has been completed as far as available resources are concerned. The next step would (possibly) be to add compute nodes at NWU. An architecture revisit will be required to get other parties like UP & Sol Plaatjies involved. The meeting has been brought to ARC board’s attention and scheduling will follow.

    November 14, 2016 at 2:14 pm
    EBEN VAN ZYL
  2. Progress: 22 Aug 2016 –
    Documentation – Team from UCT (Piet & Kie) is starting on support portal
    Storage – S3 API implemented, some performance issues due to current design, busy resolving that
    POC: Galaxy (single instance) – IT support working with potch campus researchers to get a complete pipeline going
    Galaxy on CloudMan – Must enter further discussions via ARC sub-committee on the way forward regarding partners (UWC/SANBI)

    August 22, 2016 at 4:11 pm
    EBEN VAN ZYL
  3. Progress is slow. The ARC went live on 26 Feb (2016). We are however nowhere near implementation of the Swift component within ARC. Further sessions required with Pieter/Hans/Stephan to determine some architecture choices and implementations. Will update after dates have been agreed on. In the process of launching a project within the ARC with Anelda to be part of the ARC POC testing and possible candidate for demonstration at ASAUDIT session in April. Further sessions also required with UCT and possibly external (CERN/Nectar/Rackspace) help required regarding the distributed implementation of OpenStack for ARC. Pieter and Andre (UCT) in duscussions regarding this. Will update with details once sessions are confirmed.

    March 4, 2016 at 3:39 pm
    EBEN VAN ZYL
  4. Tender process completed
    PO generated, awaiting delivery of SuperMicro equipment
    Preparation of datacenters in progress.
    Openstack onboarding in progress.

    November 6, 2015 at 11:26 am
    EBEN VAN ZYL

Leave a Reply