Getting Started

You can run the integration tests from the basic kbase-ui development environment.

Before running the tests, you must:

build and run kbase-ui
edit the test config file

Build kbase-ui

Although integration tests may be run against any live instance of kbase-ui (locally or remote), it is still necessary to have a local copy of kbase-ui built and run at least once. The kbase-ui build process gathers the tests together and arranges them with the test running javascripts. Running the container using the provided script maps the internal test directory into the local development directory dev/test.

git clone https://github.com/kbase/kbase-ui
cd kbase-ui
make dev-start env=ci build=dev build-image=t

Note that we’ve started the ui container with the CI environment. If we want to test against the local kbase-ui, we can control which environment we are testing against by running kbase-ui against that environment and indicating that to the test script (more on that later.)

Edit the test config file

The integration tests consult a single configuration file for information which cannot be included in the test scripts itself.

After building and running kbase-ui, a sample configuration file is placed at dev/test/integration-tests/config.json. Within this file you must replace token placeholders with the login token for the user kbaseuitest for each environment you wish to test against.

You’ll need the Globus password for kbaseuitest. It is located in the UI Resource document in Google Drive. This document is restricted to current kbase-ui developers only. If you have questions, please ask them on the appropriate channels on KBase Slack (#ui or #kbase_coders are good choices).

Configuration values include core configuration like a user token, the associated username, and associated real name. At present, the only configuration value you need to replace is the token for each environment. This token can be obtained by logging into each environment as the user kbaseuitest and copying the value of the kbase_session cookie field. (E.g. in the browser javascript console, enter document.cookie to expose the current cookie.)

The password for the kbaseuitest account can be found in the UI Developer’s Google Doc, if you have access to it. (And if not, contact a KBase UI developer, e.g. on the #ui slack channel.)

{
    "envDefault": {
        "auth": {
            "username": "kbaseuitest",
            "realname": "KBase UI Test User"
        }
    },
    "envs": [
        {
            "env": "ci",
            "auth": {
                "token": "LOGIN_TOKEN_HERE"
            }
        },
        {
            "env": "next",
            "auth": {
                "token": "LOGIN_TOKEN_HERE"
            }
        },
        {
            "env": "prod",
            "hostPrefix": "narrative",
            "aliases": [
                {
                    "env": "narrative-dev"
                }
            ],
            "auth": {
                "token": "LOGIN_TOKEN_HERE"
            }
        }
    ]
}

Running tests

In a separate terminal in the kbase-ui root directory, enter the following command:

make integration-tests env=ci browser=chrome

or simple

make integration-tests

to use the default env of ci and browser of chrome.

Available values for env are ci, next, narrative-dev, and prod; for browser are chrome, firefox and safari.

NOTE: firefox and safari are currently not supported as their selenium drivers have test-stopping bugs.

NOTE: Due to the variability in test response times, some tests may time out. If this occurs, run the tests one or more additional times until (and if) they succeed. Common conditions for timeout-based test failures are local machine load (the tests are fairly cpu-intensive), network congestion, kbase service latency, and kbase service maintenance.

As the tests are running, the selected browser will probably appear on your computer’s display and you may witness the actions of the test scripts.

After a few moments, the tests will be finished and the results should be similar to what is shown below.

If any tests fail, the test case will indicate as such.

At the time of writing, the test result summary indicates the number of test javascript files which failed, but since we use yaml script files for test cases, and the javascript files don’t contain tests but just execute the test scripts, this statistic is misleading.

erikpearson@Eriks-MBP-2 kbase-ui % make integration-tests
ENV=ci BROWSER=chrome ./node_modules/.bin/grunt integration-tests --env=ci
Running "webdriver:integration" (webdriver) task
TEST ENV        : ci
TEST HOST PREFIX: ci
TEST BROWSER    : chrome

Execution of 4 spec files started at 2020-01-23T18:31:42.218Z

[0-0] TEST ENV        : ci
[0-0] TEST HOST PREFIX: ci
TEST BROWSER    : chrome
[0-0] RUNNING in chrome - /dev/test/integration-tests/specs/merger.js
[0-0] PASSED in chrome - /dev/test/integration-tests/specs/merger.js
[0-1] TEST ENV        : ci
[0-1] TEST HOST PREFIX: ci
[0-1] TEST BROWSER    : chrome
[0-1] RUNNING in chrome - /dev/test/integration-tests/specs/runner.js
[0-1] PASSED in chrome - /dev/test/integration-tests/specs/runner.js
[0-2] TEST ENV        : ci
[0-2] TEST HOST PREFIX: ci
[0-2] TEST BROWSER    : chrome
[0-2] RUNNING in chrome - /dev/test/integration-tests/specs/theSpec.js
[0-2] PASSED in chrome - /dev/test/integration-tests/specs/theSpec.js
[0-3] TEST ENV        : ci
[0-3] TEST HOST PREFIX: ci
[0-3] TEST BROWSER    : chrome
[0-3] RUNNING in chrome - /dev/test/integration-tests/specs/utils.js
[0-3] PASSED in chrome - /dev/test/integration-tests/specs/utils.js

 "dot" Reporter:
..................................

 "spec" Reporter:
------------------------------------------------------------------
[chrome 79.0.3945.130 Mac OS X #0-2] Spec: /Users/erikpearson/work/kbase/sprints/2019/q3/s4/deploy-refactored-ui/kbase-ui/dev/test/integration-tests/specs/theSpec.js
[chrome 79.0.3945.130 Mac OS X #0-2] Running: chrome (v79.0.3945.130) on Mac OS X
[chrome 79.0.3945.130 Mac OS X #0-2] Session ID: a4585417f26f563fd0e1312a4d3bad6d
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] About panel
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ View the about panel directly
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Access the login buttons
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should work
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure main view appears
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should have main panel framing appear when hit the path
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Unauthenticated
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure the main view loads
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ Navigate to catalog main view
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Unauthenticated
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the main view even without auth
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Dashboard with authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ Dashboard should appear when the route is navigated to
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Unauthenticated
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure main view appears
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ Should have main panel framing appear when hit the path
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing data search without authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] No search in place
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should show the no-search alert on each tab
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Searching for something which doesn't exist
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should show a not-found message
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Search for 'sphaeroides''
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should  should reveal 1 or more hits for each tab
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing a media object while authenticated
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ case 1: media object should be displayed
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing feeds with authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should show the feeds main view with a global feed menu item
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing feeds without authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure main view appears
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should have main panel framing appear when hit the path
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing jgi-search without authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing the job browser with authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should show the primary widget
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing the job browser with authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page with the jobbrowser path displayed
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Login plugin - authorized
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Login plugin - unauthorized
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Sidebar nav
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ It should be there
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Sidebar Nav Navigation Tests without Authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ Navigation between items should ... work
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Sidebar Nav Navigation Tests without Authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ Navigation between items with multiple clicks should result in just one instance of a plugin
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Sidebar Nav Navigation Tests without Authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ There should be 7 sidebar items after login
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Sidebar Nav Navigation Tests without Authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ There should be just one sidebar nav item
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Sidebar Nav Navigation Tests
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ It should be there
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure main view appears
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should have main panel framing appear when hit the path
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Unauthenticated
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure main view appears
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should have main panel framing appear when hit the path
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Accessing orgs without authentication
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should get the signin page
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Ensure main view appears
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓ should have main panel framing appear when hit the path
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] Welcome plugin
[chrome 79.0.3945.130 Mac OS X #0-2]    ✓
[chrome 79.0.3945.130 Mac OS X #0-2]
[chrome 79.0.3945.130 Mac OS X #0-2] 34 passing (1m 31.9s)


Spec Files:	 4 passed, 4 total (100% completed) in 00:01:39


Done.

Files

Below are directories and files involved in integration testing.

dev/test - placeholder directory for integration test scripts, code, and configuration; this directory is populated when the ui is built; the files are overlaid from the container via a volume mount.
test/wdio.conf.integration.js - webdriver io test configuration file
src/test - location of script runner files, the configuration file, and canned subtasks; these are copied into the kbase-ui image and subsequently made available locally through a volume mount at /dev/test.
tools/proxy/contents/ssl - empty directory which is populated with test.crt and test.key when make dev-cert is run (and emptied by make rm-dev-cert).
src/client/plugins/PLUGIN/test - location of internal plugin integration test scripts
kbase-ui-plugin-PLUGIN/src/plugin/test - location of external plugin integration test scripts.
- or kbase-ui-plugin-PLUGIN/plugin/test for newer CRA-TS plugins

4. Test Scripts