Release QA & Testing

Oxcyon's unique bi-weekly, (pull) updates mean an equally unique approach to quality control and testing. Oxcyon employs a hybrid process which is primarily Agile, but with some waterfall checkpoints regarding quality assurance. . 

First, on the Agile process, and our preferred approach. Traditional  software development methodologies work on the premise that software requirements remain constant throughout the project. But with increase in complexity , the requirements undergo numerous changes and continuously evolve. At times, the customer himself is not sure what he wants. Though iterative model addresses this issue, it's still based on the waterfall model. In Agile methodology   , software is developed in   incremental, rapid cycles. Interactions amongst customers, developers and client are emphasized rather than processes and tools. Agile methodology focuses on responding to change rather than extensive planning.   Incremental testing is used in agile development methods and hence, every release of the project is tested thoroughly. This ensures that any bugs in the system are fixed before the next release.


                                                             


Oxcyon's own Waterfall methodology is essential, due to our virtual updating (layered) technology. Quality testing must be done at each layer of the technology from Oxcyon's core platform (where the pull updates are inherited from). Next, at the individual server level (either on premise or in the cloud), and then finally at the individual portal layer (Home and container of microsites). This hierarchy suggests a waterfall of 'cascading' udpates, which are further rolled out to our clients in an optional, ala carte manner. This allows the clients to select from a larger menu, activating which new functions they may want to leverage locally. Typically in the waterfall model ,software development progress through various phases like Requirements Analysis , Design etc -  sequentially. Oxcyon must consider the waterfall of the update themselves within it's different environments (Dev, Masters, Portals) but using Agile methodology internally to accomplish. quality assurance testing at all cascading levels. To that end, it is not traditional Waterfall methodology with the exception of the QA of these tiers.
 

Developer Activities (Post internal developer testing)
Create install and configuration scripts as needed.	Developer
Remove debugging, Trace and testing code from the software (including disabling assertions).	Developer
Create the releasable software to Internal QC Server for Testing	Developer
Quality Assurance Activities	QA
Verify all source code meets coding standard; run Checkstyle or other style-checker and do manual inspection.	QA
Check that all defects on current defect list have been resolved	Tester
Smoke test/regression test final build	Tester
Verify that every test in the Test Matrix appears in the Test Script.	Tester
Verify all unit tests run automatically on the current build. *	Tester
Verify all system test scripts run automatically on the actual released software. *	Tester
Perform all non-functional system tests manually and document results. *	Tester
Verify Cross Browser Compatibility of all screens (including mobile)	Tester
Validate Global Login & Cross Walks of any existing module	Tester
SEO, ADA, and Speed tests	Tester
Security & Penetration Testing	Tester
Compliance Testing (PCI, HIPAA or other, contingent on function)	Tester
Verify Default Values (Where applicable)	Tester
Verify Data Transfer Recognition (Applicable to new module or fields)	Tester
Verify Default of Search and Index of new data type	Test
Verify installation by installing system on clean machines (VM & Physical stand alone) .	Tester
Have someone not on your team install and run your system without assistance by following your installation directions.	Tester
Install program on machine with older version of program (upgrade install)	Tester
Verify all steps in the deployment plan are completed.	QA
Consistency check:  UI, CSS, User Manual, System Tests, Staged Delivery Plan, and Software must all match!	Analyst
Release Activities
Produce hardcopy of empirical evidence that the release criteria have been met.*	Release team
Schedule Acceptance Test date with customer (and instructor). *	Release team
Virus scan all release media	Release team
Tag and Branch the source code repository. *	Change Control team
Documentation Activities
Verify that User Documentation matches current Release.	Analyst
Create a "Update Notes" text file with installation instructions (where applicable).	Analyst
Write "Known Issues" List.	Analyst
Other Activities
Project Visiblity Calculations are current and accurate.	manager
Create draft features list for next Staged Delivery Plan & Approve Pre-Release Update Notes for distribution (to Clients).	manager
Schedule Project Survival Assessment or Post-Mortem meeting.	manager

Determining what is a local issue vs. a Product Update (to be included in Release)

Oxcyon's production team initially (receives and handles) reported issues (from the client base) via our Issue Management System. Each issue is scrutinizes to determine whether the issue can first be remedied at the the client (site or portal) level, within the Client Console. This can be done by making minor adjustments to code or script at the local site level to achieve desired results. The majority of all reported issues are addressed and remedied in the target site immediately using the client console; generally by an Analyst (assigned to your account). 

If it cannot be resolved at the client site, the issue is then routed by the Analyst to Development (by being tagged wtihin Issue Manager). This issue is reprioritized as a newly identified bug, consideration, module, function or even source code change request.  Bugs are infrequent and are addressed urgently by our development team, and are given the highest priority of any development issue.  Bugs may include a patch that is applied to the site (ahead of the next released synch) as well as a change to the master branch of our repository which is released during the next sync (described below). In this way, a patch may be implemented immediately at the client site, and such changes will remain consistent (upon the next release). 

Change requests to any existing functionality begin with a technical design meeting with an Architect.  The Architect determines whether the change can be integrated into Centralpoint, as an existing option or feature, or if it should be developed as a custom feature in that site.  For any custom features applied to only a singular site, the code is applied so that it is segregated from any future synch of the general Centralpoint release. These unique custom changes would be found under the MyModules folder of any given client site.  All integrated changes must be backward compatible to work with all existing sites on all versions of Centralpoint, and considers attribute review, dependencies (to any other function or module) and default settings available for any new feature. Custom features are developed and tested in a structurally accurate copy of the target site locally, and are transferred to the target site when completed and tested again. 

Integrated changes can be anything from simple property additions to security updates or new module or module 'suites' (A collection of modules typically interdependent on one another to arrive at a more complex result).  All Oxcyon developers develop locally using Git source control.  Once the Architect determines where the change will be applied, the developer applies to change to their local repository.  Simple changes are done in the master branch and are applied to all sites in the next sync.  Testing occurs locally and again on the shared development server after the changes merged into the master branch.  More complicated changes are developed in feature branches.  The developer may begin a project in one sync-cycle and not complete the project until numerous sync-cycles later.  During this process the master (sync) branch is merged into the feature branch after each sync, and the feature branch is merged into the master branch once it is completed and tested locally.  Once a change makes it into the master branch it first triggers a code review with the architect and development team.  After all changes are approved and tested on the shared development server they are deployed to our Uber server for release in the next sync.  The first step in the sync is to update and sync our internal quality control server and test them again.  If all goes well on the quality control server the (Global Release Updates for all clients) update and sync processes will apply the changes to all other sites over the next few days. (Based upon the frequency of each client to 'pull' those updates).   

Your installed server (otherwise known as a Master Management Console or 'Master') will host 3 different types of sites: Development, QC and (any number of) Client Sites (also referred to as Portals, as they (in turn) support n-tiered microsites.  All changes are applied to your site are vis-a-vis the Master Server Update and Sync process(es).  The execution of these processes may be done manually, or automatically (via scheduled taks). 

NOTE: Typically new clients may wish to witness a synch process manually a few times, before becoming familiar (and comfortable) with the synch process, at which time they will automate the synch via scheduled tasks.  The Update process applies the changes (first) to the local Development site on your server for testing before the Sync process applies the changes to the (child) QC Site, and then when approved, the syncronization cascades to your (child) Client Site(s). New modules will be synchronized with a default setting of 'off', requiring you to activate (or turn on) new modules inherited on a per Client Site basis. This allows for (Either Oxcyon or Client) to develop new modular functions, which may be made 'available' to all Client Sites, yet only invoked or activated on (one, some or many) sites yielding a powerful gallery of 'reusable' tools across all Client Sites. 

In many cases our clients choose to configure their client sites in a heirarchy where a private QC/Staging site exists above the public Production site(s) (although not required).  In this approach, you can then use the sync process to apply the changes to your internal QC for testing with real content (for quality assurance and/or testing) before they are deployed to production, also using the sync process.  We offer two options for deployment to production: Sync and Data Sync.  If you do choose to set up a staging site above production as an exact private duplicate of production, this site can be used for testing and applying our changes as well as yours.  You can then execute the Data Sync process from staging to production when staging is adequately tested to move everything (structure and content) from staing to production. In the event Oxcyon is providing managed services (hosting or cloud) Oxcyon manages both the server environment and the 'synchronization' process of each new release (update) in behalf of the client. 

Automated error reporting (performance monitoring, bad link checking, and unusual (anomaly) activity from the baseline released are automatically gathered (and reviewed) after each new release. NOTE: This is an additional layer of testing, which occurs by Oxcyon to monitor the quality of each update or synch. Please see a live list of Centralpoint Evergreen Update Release Notes (representing over 16 years of consistent, bi-weekly updates). 

This includes the following testing at all levels of the technology, PRIOR to any release, for Centralpoint Development (our core product), it's Master (Servers), and the portals (or sites) served by each master. This Quality Assurance testing prior to release, includes but is not limited to: