Commit Graph

9939 Commits

Author SHA1 Message Date
Daniel Jiang
038a03b4f6
Add "*" to the claim set in the token for /v2 apis (#14342)
The "*" is used by notary server for permission checking:
84287fd8df/server/server.go (L200)
Hence, we need to add this into the JWT token such that actions like key
rotation can be executed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-03-03 14:17:39 +08:00
Wang Yan
2ecfbbb7bd
fix quota dao codeQL issue (#14295)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-03-03 14:07:43 +08:00
danfengliu
13b5f37fe8
Merge pull request #14123 from danfengliu/cherry-pick-2.1.0-reschedule-docker-login-policy-in-build-base-image
[Cherry-pick 2.1.0] Reschedule docker login policy in base image build process
2021-02-25 14:05:04 +08:00
danfengliu
6517a10ce6 Reschedule docker login policy in base image build process
Signed-off-by: danfengliu <danfengl@vmware.com>
2021-02-25 12:01:06 +08:00
danfengliu
1231fa61db
Merge pull request #14100 from danfengliu/upgrade-docker-to-v20
[Cherry-pick 2.1.0] Fix nightly issues caused by docker upgrade to 20
2021-02-24 15:22:53 +08:00
danfengliu
08d9d286c8 Fix nightly issues caused by docker upgrade to 20
1. Local image should be removed, otherwise docker 20 will not tigger
get manifest request to harbor;
2. E2E image Dockerfile update;
3. Fix nighlty test issue of tag retention, add execution refesh to get
result;
4. Fix nighlty test keyword 'Create An New Project And Go Into
Project' issue that waiting long enough time for list display;
5. Add nightly test case, in GUI, scan result will show if cve id exist in allow list configuration;
6. Move proxy cache test to schdule pipeline, it will save some time for
db pipeline.t p

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-28 16:34:20 +08:00
danfengliu
e034515a93
Merge pull request #14088 from danfengliu/cherry-pick-2.1.0-upgrade-to-python3-in-CI-host
[Cherry-pick 2.1.0] Upgrade python to v3 in git action CI host
2021-01-27 18:03:28 +08:00
danfengliu
af43659b2f Upgrade python to v3 in git action CI host
The original python packaged: in git action host is V2, it should be upgraded to
V3.

Signed-off-by: danfengliu <danfengl@vmware.com>
2021-01-26 21:11:26 +08:00
Wenkai Yin(尹文开)
cc1066f8a9
Merge pull request #14010 from reasonerjt/tokenreview-rm-header-2.1
Remove raw token from header in token review reuqest - 2.1.x
2021-01-14 18:30:36 +08:00
Daniel Jiang
dc95792ee0 Remove raw token from header in token review reuqest
The server to handle token-review may have a limitation for the size of
the header.  When the token is huge the token review may fail.
This commit remove the necessary header to harden the flow.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-14 17:26:08 +08:00
Daniel Jiang
b6de84c571
Update the reg-exp to match v2/catalog api (#13943)
A more strict check is applied such that all requests to
/v2/_catalog/...  will be verified.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-09 12:34:39 +08:00
Wang Yan
b0b19f52d0
update go version to 1.14.13 (#13938)
1, update go version to v1.14.13
2, build binary per PR instead of fetching from storage

Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-08 19:27:18 +08:00
danfengliu
de5445a894
Merge pull request #13908 from danfengliu/fix-issue-of-replication-list-row-locator
Fix issue of wrong xpath for replicaiton row selector
2021-01-08 14:04:55 +08:00
Danfeng Liu (c)
c2dfc23ab8 Fix issue of wrong xpath for replicaiton row selector
Signed-off-by: Danfeng Liu (c) <danfengl@vmware.com>
2021-01-06 16:52:58 +08:00
Daniel Jiang
17215ee505
Remove scanner-pull from system admin's permission (#13903)
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-06 14:53:20 +08:00
Wang Yan
45f743bb73
bump version to v2.1.3 (#13893)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2021-01-05 17:44:49 +08:00
Wenkai Yin(尹文开)
04e3322c9e
Merge pull request #13828 from stonezdj/2.1.3_add_content_type
(cherry-pick) Add content type and length in header
2021-01-05 16:15:02 +08:00
Daniel Jiang
c3c4ccf5d4
Merge pull request #13873 from reasonerjt/token-scope-2.1
Refine the token scope generation - cherrypick to 2.1
2021-01-05 15:14:13 +08:00
stonezdj
b87052ec55 Add content type and length in header
Fixes #13740
Update ManifestExist to return Descriptor instead of digest
For docker 20.10 or containerd, it HEAD the manifest before pull, then
it GET the manifest with digest, add logic to handle this scenario and
correlate the tag between the digest in proxy cache

Signed-off-by: stonezdj <stonezdj@gmail.com>
2021-01-04 17:37:09 +08:00
Daniel Jiang
5e8506831a Refine the token scope generation
This commit directly maps the actoin permission in security context to
the scope generated by the token service in harbor-core.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2021-01-04 13:42:56 +08:00
Qian Deng
591d4eeab8
Merge pull request #13816 from ninjadq/cherry_pick_python_yaml_load_210
Fix pythom yaml load to safe_load
2021-01-04 10:51:17 +08:00
DQ
9f5a85b484 Fix pythom yaml load to safe_load
Signed-off-by: DQ <dengq@vmware.com>
2020-12-21 11:32:55 +08:00
danfengliu
ec2b83dcf3
Merge pull request #13725 from danfengliu/cherry-pick-2.1-image-sample-build-at-local
Build image sample for test in local
2020-12-15 17:19:05 +08:00
danfengliu
5fe6c1ee6f Build image sample for test in local
Change source of most of test image samples from docker-hub to local building ones, so it will cost less docker-hub pull requests.
And some of cases like push cnab, they have to use docker-hub, but image samples in cnab test will cost 17 quotas, in this PR, we
replace those samples, now cnab case will cost 6 quotas.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-11 07:49:42 +00:00
Wang Yan
fcc6751d54
bump up version to v2.1.2 (#13713)
Signed-off-by: Wang Yan <wangyan@vmware.com>
2020-12-09 00:23:44 -08:00
Steven Zou
d5c336dc45
Merge pull request #13711 from steven-zou/fix/js_api_2.1.0
fix(api):update stop job api
2020-12-09 14:36:48 +08:00
Steven Zou
ffc331da7c fix(api):update stop job api
- update StopJob() of basic worker
- update UnSchedule() of basic scheduler
- update the policy store to get more data

fix #13599 , fix #13597

Signed-off-by: Steven Zou <szou@vmware.com>
2020-12-09 13:45:58 +08:00
疯魔慕薇
282459f9b8 Tecent TCR Provider
1. Docker image registry.
2. Helm chart registry.

Signed-off-by: 疯魔慕薇 <kfanjian@gmail.com>
Signed-off-by: fanjiankong <fanjiankong@tencent.com>
2020-12-09 08:42:06 +08:00
danfengliu
e4bffed20b
Merge pull request #13681 from danfengliu/move-proxy-cache-e2e-test-into-a-new-git-action-ci-job
Move proxy cache e2e test into a new CI job
2020-12-04 13:56:20 +08:00
Daniel Jiang
4c01f37ba5
Merge pull request #13686 from reasonerjt/adjust-middleware-v2.1-2nd
Move security/readonly middleware ahead of transaction middleware
2020-12-04 13:33:58 +08:00
danfengliu
6b7f740847 Move proxy cache e2e test into a new CI job
CI job timeout is 60 minutes, but proxy cache case timeout is 20 minutes, once
proxy cache case is timeout, it will reach CI job timeout, then cause CI job terminated
by timeout without running rest of test steps, expecially uploading useful harbor logs,
so move this test into a new clean job for saving time and debugging.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-03 19:36:42 -08:00
Daniel Jiang
a773ef358f Move security/readonly middleware ahead of transaction middleware
In v2.1 security/readonly middleware will query DB by creating new
connection.
If it is put after transaction middleware there's a bigger chance of
deadlock if the concurrent open connections are set too low (#13155)

This commit mitigates that issue.  But we still need work to lower the
connections and better handle the case when http connection is closed.

Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-03 22:45:53 +08:00
Daniel Jiang
41a9d071b6
Merge pull request #13676 from reasonerjt/fix-v2catalog-uri-2.1
Fix the pattern to match v2 catalog URI
2020-12-03 14:13:03 +08:00
Daniel Jiang
c7c409a8e5 Fix the pattern to match v2 catalog URI
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-03 12:55:26 +08:00
Wenkai Yin(尹文开)
2f64eb95b9
Set timezone as UTC for database connection (#13663)
Set timezone as UTC for database connection

Signed-off-by: Wenkai Yin <yinw@vmware.com>
2020-12-03 08:56:18 +08:00
Daniel Jiang
b21f490adc
Merge pull request #13668 from danfengliu/debug_proxy-cache-e2e-test
fix proxy cache e2e test
2020-12-02 20:23:15 +08:00
danfengliu
8c3552993d fix proxy cache e2e test
1. Add print in docker pull api;
2. Move proxy cache API to the front of case list.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-12-02 18:24:26 +08:00
Wenkai Yin(尹文开)
4134df3719
Merge pull request #13665 from reasonerjt/bump-up-goldap-v2.1
[Harbor-v2.1.x]:Bump up go-ldap to v3.2.4
2020-12-02 18:14:25 +08:00
danfengliu
8d88ab26cb
Merge pull request #13635 from danfengliu/cherry-pick-2.1.0-fix-scan-all-python-script
Cherry-pick 2.1.0 Fix scan all python test script issue of missing test step
2020-12-02 10:47:16 +08:00
Daniel Jiang
b6bf23ff1d Bump up go-ldap to v3.2.4
Signed-off-by: Daniel Jiang <jiangd@vmware.com>
2020-12-02 09:57:30 +08:00
He Weiwei
e2bc0a46e0
fix(quota): ignore the fail when getting reference of quota (#13651)
Closes #13387

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-12-01 20:41:31 +08:00
Steven Zou
343e3b0c1e
Merge pull request #13638 from bitsf/fix_redis_sentinel_chart_2.1
fix(redis) redis sentinel for chart cache
2020-12-01 16:57:50 +08:00
Ziming Zhang
86141d1141 fix(redis) redis sentinel for chart cache
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-11-30 19:05:34 +08:00
danfengliu
6f33eed718 Fix scan all python test script issue of missing test step
1. In scan all py-test, missing a important test step for checkin scan all status,
2. which indicate the ending of scan all job.
In swagger, ongoig field is missing in stats struct.

Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-30 17:28:25 +08:00
Daniel Jiang
4976865a1b
Merge pull request #13601 from bitsf/fix_chart_s3_cache
[cherry-pick] fix(chartmuseum) compatible s3 cache fail for 2.1.2
2020-11-29 23:06:49 +08:00
danfengliu
4760afa0e7
Merge pull request #13572 from danfengliu/fix-docker-hub-access-limitation-issue
Add docker access credential to avoid access limitaton by docker-hub
2020-11-29 19:13:38 +08:00
danfengliu
4d501030b6 Add docker access credential to avoid access limitaton by docker-hub
Signed-off-by: danfengliu <danfengl@vmware.com>
2020-11-29 17:20:47 +08:00
Ziming Zhang
06630930d2 fix(chartmuseum) compatible s3 cache fail
Signed-off-by: Ziming Zhang <zziming@vmware.com>
2020-11-25 17:02:33 +08:00
He Weiwei
28fc1d53a4 fix: remove auto re-readable body in BeforeRequest
Remove nop close body in the BeforeRequest helper function.
Middleware must make the request body re-readable itself when it wants
read the body in the middleware.

Closes #13556

Signed-off-by: He Weiwei <hweiwei@vmware.com>
2020-11-24 22:45:22 +08:00
danfengliu
133be175bf
Merge pull request #13574 from danfengliu/cherry-pick-2.1.0-github-actions-update-for-env
ci: update env and path commands in github actions
2020-11-23 21:15:23 +08:00