From 5646e84d44410131038f2dc85a81204a1b9e615f Mon Sep 17 00:00:00 2001 From: myufei Date: Tue, 11 Apr 2017 12:36:35 +0800 Subject: [PATCH 1/4] update content trust test case --- .../9-01-DB-user-push-signed-images.md | 16 +++++++++++--- .../9-02-DB-user-push-unsigned-images.md | 10 ++++++--- .../9-03-DB-user-pull-signed-images.md | 12 ++++++++--- .../9-04-DB-user-pull-unsigned-images.md | 13 +++++++++--- .../9-05-DB-user-delete-signed-images.md | 21 +++++++++++++++---- .../9-11-LDAP-user-push-signed-images.md | 9 +++++--- .../9-12-LDAP-user-push-unsigned-images.md | 10 ++++++--- .../9-13-LDAP-user-pull-signed-images.md | 10 ++++++--- .../9-14-LDAP-user-pull-unsigned-images.md | 10 ++++++--- .../9-15-LDAP-user-delete-signed-images.md | 10 ++++++--- 10 files changed, 90 insertions(+), 31 deletions(-) diff --git a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md index f3590e2ab..7228c20f6 100644 --- a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md @@ -3,21 +3,31 @@ Test 9-01 User push signed images(DB mode) # Purpose: -TBD +To verify user can push images with content trust enabled. # References: User guide # Environment: +* This test requires one Harbor instance is runnning and available. +* A Linux host with Docker CLI installed (Docker client). # Test Steps: -TBD +1. Login UI and create a project. +2. On Docker clinet,run +```sh +export DOCKER_CONTENT_TRUST=1 +export DOCKER_CONTENT_TRUST_SERVER=https://:4443 +``` +and login Harbor. +3. Push an image to the project created in step1. + # Expected Outcome: -TBD +* In step3, Docker client will sign and push the image, a green tick will show in UI. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-02-DB-user-push-unsigned-images.md b/tests/testcases/Group9-Content-trust/9-02-DB-user-push-unsigned-images.md index 3924fb337..aa885c47b 100644 --- a/tests/testcases/Group9-Content-trust/9-02-DB-user-push-unsigned-images.md +++ b/tests/testcases/Group9-Content-trust/9-02-DB-user-push-unsigned-images.md @@ -3,21 +3,25 @@ Test 9-02 User Push unsigned images(DB mode) # Purpose: -TBD +To verify UI will difference unsigned images from signed images. # References: User guide # Environment: +* This test requires one Harbor instance is runnning and available. +* A Linux host with Docker CLI (Docker client) installed. # Test Steps: -TBD +1. Login UI and create a project. +2. Unset DOCKER_CONTENT_TRUST on Docker client and login Harbor. +3. Push a image to project created in step1. # Expected Outcome: -TBD +* A red cross will displayed under signed colume in UI. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md index 6abd644f1..aa591abff 100644 --- a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md @@ -3,21 +3,27 @@ Test 9-03 User pull signed images(DB mode) # Purpose: -TBD +To verify user can pull signed images. # References: User guide # Environment: +* This test requires one Harbor instance is running and available. +* A Linux machine with Docker CLI(Docker client) installed. # Test Steps: +NOTE: +In below test, project X should be replaced by an existing project. -TBD +1. Login UI. +2. On Docker client, set environment DOCKER_CONTENT_TRUST and DOCKER_CONTENT_TRUST_SERVER properly and login Harobr. +3. Pull an image from project X. # Expected Outcome: -TBD +* Image can be pulled successful. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-04-DB-user-pull-unsigned-images.md b/tests/testcases/Group9-Content-trust/9-04-DB-user-pull-unsigned-images.md index 62a764d5e..49d60a6e3 100644 --- a/tests/testcases/Group9-Content-trust/9-04-DB-user-pull-unsigned-images.md +++ b/tests/testcases/Group9-Content-trust/9-04-DB-user-pull-unsigned-images.md @@ -3,21 +3,28 @@ Test 9-04 User pull unsigned images(DB mode) # Purpose: -TBD +To verify whether user can pull unsigned images with content trust enabled. # References: User guide # Environment: +* This test requires one Harbor instance is running and available. +* Harbor is set to authenticate against a local database.The user data is stored in a local database. +* A Linux host with Docker CLI(Docker client) installed. # Test Steps: -TBD +1. Login UI and create a project. +2. On Docker client, unset DOCKER_CONTENT_TRUST and login Harbor. +3. Push an image to the project created in step1. +4. Reset DOCKER_CONTENT_TRUST to 1. +5. Pull the unsigned image. # Expected Outcome: -TBD +* User cannot pull unsigned images with content trust enabled. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md index 0bb61d839..9464fb52b 100644 --- a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md @@ -1,23 +1,36 @@ -Test User delete signed images(DB mode) +Test 9-05 User delete signed images(DB mode) ======= # Purpose: -TBD +To verify whether user can delete signed images. # References: User guide # Environment: +* This test requires one Harbor instance is running and avialable. +* A Linux host with Docker CLI(Docker client) installed. # Test Steps: -TBD +1. Login UI and create a project. +2. On a Docker client,run +```sh +export DOCKER_CONTENT_TRUST=1 +export DOCKER_CONTENT_TRUST_SERVER=https://:4443 +``` +and login Harbor. +3. Push an image to project created in step1. +4. Delete the pushed image. +5. Delete notary tag according to message from UI in step4. +6. Delete the image again. # Expected Outcome: -TBD +* In step4, image cannot be deleted. +* In step6, image can be deleted. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-11-LDAP-user-push-signed-images.md b/tests/testcases/Group9-Content-trust/9-11-LDAP-user-push-signed-images.md index 0d0930940..d1a9855c5 100644 --- a/tests/testcases/Group9-Content-trust/9-11-LDAP-user-push-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-11-LDAP-user-push-signed-images.md @@ -3,21 +3,24 @@ Test 9-11 User push signed images(LDAP mode) # Purpose: -TBD +To verify user can sign and push images(LDAP mode) # References: User guide # Environment: +* This test requires that a Harbor instance is running and available. +* Harbor is set to authenticate against an LDAP or AD server.(auth_mode is set to ldap_auth.) The user data is stored in an LDAP or AD server. +* A Linux host with Docker CLI(Docker client) installed. # Test Steps: -TBD +Same as Test 9-01 except that users are from LDAP/AD. # Expected Outcome: -TBD +* Same as Test 9-01. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-12-LDAP-user-push-unsigned-images.md b/tests/testcases/Group9-Content-trust/9-12-LDAP-user-push-unsigned-images.md index abf196d06..54702daf5 100644 --- a/tests/testcases/Group9-Content-trust/9-12-LDAP-user-push-unsigned-images.md +++ b/tests/testcases/Group9-Content-trust/9-12-LDAP-user-push-unsigned-images.md @@ -3,21 +3,25 @@ Test 9-12 User push unsigned images(LDAP mode) # Purpose: -TBD +To verify UI will difference unsiged images from signed. # References: User guide # Environment: +* This test requires that a Harbor instance is running and available. +* Harbor is set to authenticate against a LDAP or AD server. +* A Linux host with Docker CLI(Docker client)installed. +* A non-admin user that has at least a project as project admin. # Test Steps: -TBD +Same as Test 9-02 except that users are from LDAP/AD. # Expected Outcome: -TBD +* Same as Test 9-02. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-13-LDAP-user-pull-signed-images.md b/tests/testcases/Group9-Content-trust/9-13-LDAP-user-pull-signed-images.md index 6e044229d..7f90c66fd 100644 --- a/tests/testcases/Group9-Content-trust/9-13-LDAP-user-pull-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-13-LDAP-user-pull-signed-images.md @@ -3,21 +3,25 @@ Test 9-13 User pull signed images(LDAP mode) # Purpose: -TBD +To verify whether user can pull signed images with content trust enabled. # References: User guide # Environment: +* This test requires that a Harbor instance is running and available. +* Harbor is set to authenticate against an LDAP or AD server. +* A Linux host with Docker CLI(Docker client)installed. +* A non-admin user that has at least one proejct as project admin. # Test Steps: -TBD +Same as Test 9-03 except that users are from LDAP/AD. # Expected Outcome: -TBD +* Same as Test 9-03. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-14-LDAP-user-pull-unsigned-images.md b/tests/testcases/Group9-Content-trust/9-14-LDAP-user-pull-unsigned-images.md index 563675e14..745eda83c 100644 --- a/tests/testcases/Group9-Content-trust/9-14-LDAP-user-pull-unsigned-images.md +++ b/tests/testcases/Group9-Content-trust/9-14-LDAP-user-pull-unsigned-images.md @@ -3,21 +3,25 @@ Test 9-14 User pull unsigned images(LDAP images) # Purpose: -TBD +To verify whether user can pull unsigned images with content trust enabled. # References: User guide # Environment: +* This test requires that a Harbor instance is running and available. +* Harbor is set to authenticate against a LDAP or AD server. +* A Linux host with Docker CLI(Docker client)installed. +* A non-admin user that has at least one project as project admin. # Test Steps: -TBD +Same as Test 9-04 except that users are from LDAP/AD. # Expected Outcome: -TBD +* Same as Test 9-04. # Possible Problems: None diff --git a/tests/testcases/Group9-Content-trust/9-15-LDAP-user-delete-signed-images.md b/tests/testcases/Group9-Content-trust/9-15-LDAP-user-delete-signed-images.md index 41d19a313..a9c14d0c6 100644 --- a/tests/testcases/Group9-Content-trust/9-15-LDAP-user-delete-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-15-LDAP-user-delete-signed-images.md @@ -3,21 +3,25 @@ Test 9-15 User delete signed images(LDAP mode) # Purpose: -TBD +To verify user can delete signed images with content trust enabled. # References: User guide # Environment: +* This test requries a Harbor istance is running and available. +* Harbor is set authenticate against an LDAP or AD server. +* A Linux host with Docker CLI(Docker client)installed. +* A non-admin user that has at least one project as project admin. # Test Steps: -TBD +Same as Test 9-05 except that users are from LDAP/AD. # Expected Outcome: -TBD +* Same as Test 9-05. # Possible Problems: None From cdc49dd94482aef36397a0e652a7ac0508841069 Mon Sep 17 00:00:00 2001 From: myufei Date: Fri, 14 Apr 2017 14:50:51 +0800 Subject: [PATCH 2/4] update content trust test case --- .../9-01-DB-user-push-signed-images.md | 7 +------ .../9-03-DB-user-pull-signed-images.md | 2 +- .../9-05-DB-user-delete-signed-images.md | 7 +------ 3 files changed, 3 insertions(+), 13 deletions(-) diff --git a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md index 7228c20f6..799ca2df0 100644 --- a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md @@ -16,12 +16,7 @@ User guide # Test Steps: 1. Login UI and create a project. -2. On Docker clinet,run -```sh -export DOCKER_CONTENT_TRUST=1 -export DOCKER_CONTENT_TRUST_SERVER=https://:4443 -``` -and login Harbor. +2. On Docker clinet, follow [Set up notary](../../../docs/use_notary.md)to set up notary and login Harbor. 3. Push an image to the project created in step1. diff --git a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md index aa591abff..d6902ded3 100644 --- a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md @@ -18,7 +18,7 @@ NOTE: In below test, project X should be replaced by an existing project. 1. Login UI. -2. On Docker client, set environment DOCKER_CONTENT_TRUST and DOCKER_CONTENT_TRUST_SERVER properly and login Harobr. +2. On Docker client, follow [Set up notary](../../../docs/use_notary.md) to set up notary and login Harobr. 3. Pull an image from project X. # Expected Outcome: diff --git a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md index 9464fb52b..a44ee6d21 100644 --- a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md @@ -16,12 +16,7 @@ User guide # Test Steps: 1. Login UI and create a project. -2. On a Docker client,run -```sh -export DOCKER_CONTENT_TRUST=1 -export DOCKER_CONTENT_TRUST_SERVER=https://:4443 -``` -and login Harbor. +2. On a Docker client,follow [Set up notary](../../../docs/use_notary.md) to set up notary and login Harbor. 3. Push an image to project created in step1. 4. Delete the pushed image. 5. Delete notary tag according to message from UI in step4. From 57a4b39e0a8957395731e8aa3a62dcff65aa8f80 Mon Sep 17 00:00:00 2001 From: myufei Date: Fri, 14 Apr 2017 14:59:35 +0800 Subject: [PATCH 3/4] update content trust test case --- .../Group9-Content-trust/9-01-DB-user-push-signed-images.md | 2 +- .../Group9-Content-trust/9-03-DB-user-pull-signed-images.md | 2 +- .../Group9-Content-trust/9-05-DB-user-delete-signed-images.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md index 799ca2df0..58da6a4d3 100644 --- a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md @@ -16,7 +16,7 @@ User guide # Test Steps: 1. Login UI and create a project. -2. On Docker clinet, follow [Set up notary](../../../docs/use_notary.md)to set up notary and login Harbor. +2. On Docker clinet, follow [Set up notary](../../../../docs/use_notary.md) to set up notary and login Harbor. 3. Push an image to the project created in step1. diff --git a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md index d6902ded3..33e4e263c 100644 --- a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md @@ -18,7 +18,7 @@ NOTE: In below test, project X should be replaced by an existing project. 1. Login UI. -2. On Docker client, follow [Set up notary](../../../docs/use_notary.md) to set up notary and login Harobr. +2. On Docker client, follow [Set up notary](../../../../docs/use_notary.md) to set up notary and login Harobr. 3. Pull an image from project X. # Expected Outcome: diff --git a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md index a44ee6d21..0ab148f56 100644 --- a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md @@ -16,7 +16,7 @@ User guide # Test Steps: 1. Login UI and create a project. -2. On a Docker client,follow [Set up notary](../../../docs/use_notary.md) to set up notary and login Harbor. +2. On a Docker client,follow [Set up notary](../../../../docs/use_notary.md) to set up notary and login Harbor. 3. Push an image to project created in step1. 4. Delete the pushed image. 5. Delete notary tag according to message from UI in step4. From c7e6cbe3ccb1c6323fda60dbb21a986a768045ed Mon Sep 17 00:00:00 2001 From: myufei Date: Fri, 14 Apr 2017 15:51:31 +0800 Subject: [PATCH 4/4] update test case --- .../9-01-DB-user-push-signed-images.md | 9 ++++++++- .../9-03-DB-user-pull-signed-images.md | 11 ++++++++--- .../9-05-DB-user-delete-signed-images.md | 9 ++++++++- 3 files changed, 24 insertions(+), 5 deletions(-) diff --git a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md index 58da6a4d3..967bba24a 100644 --- a/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-01-DB-user-push-signed-images.md @@ -14,9 +14,16 @@ User guide * A Linux host with Docker CLI installed (Docker client). # Test Steps: +**NOTE:** +In below test, should be replaced by your harbor's ip or FQDN. If you are using a self-signed certificate,make sure to copy the CA root cert into ```/etc/docker/certs.d/``` and ```$HOME/.docker/tls/:4443/``` 1. Login UI and create a project. -2. On Docker clinet, follow [Set up notary](../../../../docs/use_notary.md) to set up notary and login Harbor. +2. On Docker clinet, run +```sh +export DOCKER_CONTENT_TRUST=1 +export DOCKER_CONTNET_TRUST_SERVER=https://:4443 +``` +and login Harbor. 3. Push an image to the project created in step1. diff --git a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md index 33e4e263c..ab8b0a704 100644 --- a/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-03-DB-user-pull-signed-images.md @@ -14,11 +14,16 @@ User guide * A Linux machine with Docker CLI(Docker client) installed. # Test Steps: -NOTE: -In below test, project X should be replaced by an existing project. +**NOTE:** +In below test, project X should be replaced by an existing project and should be replaced by your harbor's ip or FQDN. If you are using a self-signed certificate,make sure to copy the CA root cert into ```/etc/docker/certs.d/``` and ```$HOME/.docker/tls/:4443/``` 1. Login UI. -2. On Docker client, follow [Set up notary](../../../../docs/use_notary.md) to set up notary and login Harobr. +2. On Docker client, run +```sh +export DOCKER_CONTENT_TRUST=1 +export DOCKER_CONTENT_TRUST_SERVER=https://:4443 +``` +and login Harobr. 3. Pull an image from project X. # Expected Outcome: diff --git a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md index 0ab148f56..16867c3cc 100644 --- a/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md +++ b/tests/testcases/Group9-Content-trust/9-05-DB-user-delete-signed-images.md @@ -14,9 +14,16 @@ User guide * A Linux host with Docker CLI(Docker client) installed. # Test Steps: +**NOTE:** +In below test, should be replaced by your harbor's ip or FQDN. If you are using a self-signed certificate,make sure to copy the CA root cert into ```/etc/docker/certs.d/``` and ```$HOME/.docker/tls/:4443/``` 1. Login UI and create a project. -2. On a Docker client,follow [Set up notary](../../../../docs/use_notary.md) to set up notary and login Harbor. +2. On a Docker client,run +```sh +export DOCKER_CONTENT_TRUST=1 +export DOCKER_CONTENT_TRUST_SERVER=https://:4443 +``` +and login Harbor. 3. Push an image to project created in step1. 4. Delete the pushed image. 5. Delete notary tag according to message from UI in step4.